2024-09-17T04:45:07.709 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T04:45:07.730 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T04:45:07.838 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits} email: pdonnell@redhat.com first_in_suite: false job_id: '7908571' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF15aOCHBTeJqfXVgnwVpCgKMKlvj85bulBQ46Bhec8BtKBJKEtWJ0XoJZ0Wkjb/1i6FQ4Ggl/eKYtRHz+Urkwg= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrynvarVBzUxJ75CW7hv4BHdDyCOq0phXvfDLsmbiX0eBwHonZIECZOFwmecTpHp9/E/EdYh2bXgfFxwHSTPQc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T04:45:07.838 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T04:45:07.839 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T04:45:07.839 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T04:45:07.840 INFO:teuthology.task.internal:Checking packages... 2024-09-17T04:45:07.861 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T04:45:07.861 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T04:45:07.862 INFO:teuthology.packaging:ref: None 2024-09-17T04:45:07.862 INFO:teuthology.packaging:tag: None 2024-09-17T04:45:07.862 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T04:45:07.862 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:45:07.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T04:45:08.005 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:45:08.007 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T04:45:08.014 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T04:45:08.014 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T04:45:08.131 INFO:teuthology.task.internal:Saving configuration 2024-09-17T04:45:08.149 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T04:45:08.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T04:45:08.183 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 04:39:28.751618', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDDsHfi4xDrPoJ+H+xB2+9T1hYcjGjxF3S/XMiSzG3C+'} 2024-09-17T04:45:08.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 04:39:28.750250', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIITun2wG7O2odNb/vcbtbq0w2rihlid5QDHcRtOfjkY'} 2024-09-17T04:45:08.205 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T04:45:08.212 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T04:45:08.212 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T04:45:08.212 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T04:45:08.286 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33fad63ac0>, signals=[15]) 2024-09-17T04:45:08.286 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T04:45:08.292 INFO:teuthology.task.internal:Opening connections... 2024-09-17T04:45:08.293 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T04:45:08.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:45:08.374 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T04:45:08.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:45:08.455 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T04:45:08.461 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-17T04:45:08.466 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-17T04:45:08.466 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="22.04" 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=jammy 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2024-09-17T04:45:08.514 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2024-09-17T04:45:08.515 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T04:45:08.515 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T04:45:08.515 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T04:45:08.515 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T04:45:08.515 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=jammy 2024-09-17T04:45:08.515 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-17T04:45:08.533 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-09-17T04:45:08.538 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-09-17T04:45:08.538 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-09-17T04:45:08.585 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Ubuntu" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="22.04" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_CODENAME=jammy 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:ID=ubuntu 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE=debian 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T04:45:08.586 INFO:teuthology.orchestra.run.smithi104.stdout:UBUNTU_CODENAME=jammy 2024-09-17T04:45:08.587 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-09-17T04:45:08.608 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T04:45:08.617 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T04:45:08.637 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T04:45:08.637 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-17T04:45:08.639 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-09-17T04:45:08.644 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T04:45:08.659 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T04:45:08.660 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-17T04:45:08.687 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-09-17T04:45:08.694 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T04:45:08.755 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T04:45:08.769 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-17T04:45:08.769 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-17T04:45:08.769 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-17T04:45:08.774 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T04:45:08.774 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T04:45:08.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:45:08.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T04:45:08.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T04:45:08.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-17T04:45:08.943 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:45:08.944 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T04:45:08.944 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-17T04:45:08.949 INFO:teuthology.orchestra.run.smithi016.stdout:5.15.0-56-generic 2024-09-17T04:45:08.950 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:45:08.950 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T04:45:08.950 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-17T04:45:08.950 DEBUG:teuthology.orchestra.run.smithi016:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T04:45:08.960 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:45:08.960 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-17T04:45:08.961 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-09-17T04:45:08.966 INFO:teuthology.orchestra.run.smithi104.stdout:5.15.0-56-generic 2024-09-17T04:45:08.966 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:45:08.966 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-17T04:45:08.966 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-17T04:45:08.967 DEBUG:teuthology.orchestra.run.smithi104:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-17T04:45:09.006 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T04:45:09.024 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-17T04:45:09.173 INFO:teuthology.orchestra.run.smithi104.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T04:45:09.173 INFO:teuthology.orchestra.run.smithi104.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T04:45:09.173 INFO:teuthology.orchestra.run.smithi104.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T04:45:09.173 INFO:teuthology.orchestra.run.smithi104.stderr:hint: 2024-09-17T04:45:09.173 INFO:teuthology.orchestra.run.smithi104.stderr:hint: git config --global init.defaultBranch 2024-09-17T04:45:09.174 INFO:teuthology.orchestra.run.smithi104.stderr:hint: 2024-09-17T04:45:09.174 INFO:teuthology.orchestra.run.smithi104.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T04:45:09.174 INFO:teuthology.orchestra.run.smithi104.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T04:45:09.174 INFO:teuthology.orchestra.run.smithi104.stderr:hint: 2024-09-17T04:45:09.174 INFO:teuthology.orchestra.run.smithi104.stderr:hint: git branch -m 2024-09-17T04:45:09.175 INFO:teuthology.orchestra.run.smithi104.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T04:45:09.177 DEBUG:teuthology.orchestra.run.smithi104:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T04:45:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-17T04:45:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-17T04:45:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: git config --global init.defaultBranch 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: 2024-09-17T04:45:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:hint: git branch -m 2024-09-17T04:45:09.179 INFO:teuthology.orchestra.run.smithi016.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-17T04:45:09.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-17T04:45:09.242 DEBUG:teuthology.orchestra.run.smithi104:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T04:45:09.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-17T04:45:09.299 DEBUG:teuthology.orchestra.run.smithi104:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T04:45:09.303 DEBUG:teuthology.orchestra.run.smithi016:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-17T04:45:44.935 INFO:teuthology.orchestra.run.smithi104.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T04:45:44.935 INFO:teuthology.orchestra.run.smithi104.stderr: * [new branch] main -> origin/main 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T04:45:44.972 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T04:45:44.973 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T04:45:44.973 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T04:45:44.973 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T04:45:44.975 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T04:45:44.975 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T04:45:44.976 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T04:45:44.977 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T04:45:44.980 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T04:45:44.981 INFO:teuthology.orchestra.run.smithi104.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T04:45:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-17T04:45:47.027 INFO:teuthology.orchestra.run.smithi016.stderr: * [new branch] main -> origin/main 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190312 -> 20190312 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190416 -> 20190416 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190514 -> 20190514 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190618 -> 20190618 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190717 -> 20190717 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190815 -> 20190815 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20190923 -> 20190923 2024-09-17T04:45:47.069 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20191022 -> 20191022 2024-09-17T04:45:47.070 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20191215 -> 20191215 2024-09-17T04:45:47.070 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200122 -> 20200122 2024-09-17T04:45:47.070 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200316 -> 20200316 2024-09-17T04:45:47.074 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200421 -> 20200421 2024-09-17T04:45:47.074 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200519 -> 20200519 2024-09-17T04:45:47.074 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200619 -> 20200619 2024-09-17T04:45:47.074 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200721 -> 20200721 2024-09-17T04:45:47.074 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200817 -> 20200817 2024-09-17T04:45:47.074 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20200918 -> 20200918 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20201022 -> 20201022 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20201118 -> 20201118 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20201218 -> 20201218 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210208 -> 20210208 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210315 -> 20210315 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210511 -> 20210511 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210716 -> 20210716 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210818 -> 20210818 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20210919 -> 20210919 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20211027 -> 20211027 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20211216 -> 20211216 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220209 -> 20220209 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220310 -> 20220310 2024-09-17T04:45:47.075 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220411 -> 20220411 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220509 -> 20220509 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220610 -> 20220610 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220708 -> 20220708 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220815 -> 20220815 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20220913 -> 20220913 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20221012 -> 20221012 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20221109 -> 20221109 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20221214 -> 20221214 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230117 -> 20230117 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230210 -> 20230210 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230310 -> 20230310 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230404 -> 20230404 2024-09-17T04:45:47.076 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230515 -> 20230515 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230625 -> 20230625 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230804 -> 20230804 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20230919 -> 20230919 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20231030 -> 20231030 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20231111 -> 20231111 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20231211 -> 20231211 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240115 -> 20240115 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240220 -> 20240220 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240312 -> 20240312 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240410 -> 20240410 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240513 -> 20240513 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240610 -> 20240610 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240709 -> 20240709 2024-09-17T04:45:47.077 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240811 -> 20240811 2024-09-17T04:45:47.078 INFO:teuthology.orchestra.run.smithi016.stderr: * [new tag] 20240909 -> 20240909 2024-09-17T04:45:49.586 INFO:teuthology.orchestra.run.smithi104.stderr:Updating files: 30% (1066/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2070/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 69% (2429/3520) Updating files: 69% (2433/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 82% (2887/3520) Updating files: 82% (2902/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T04:45:49.591 INFO:teuthology.orchestra.run.smithi104.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T04:45:49.614 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T04:45:49.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:45:49.764 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T04:45:49.765 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T04:45:49.767 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi104.front.sepia.ceph.com to complete... 2024-09-17T04:45:50.862 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17 04:45:50 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T04:45:50.862 INFO:teuthology.orchestra.run.smithi104.stderr:FINISHED --2024-09-17 04:45:50-- 2024-09-17T04:45:50.862 INFO:teuthology.orchestra.run.smithi104.stderr:Total wall clock time: 1.0s 2024-09-17T04:45:50.862 INFO:teuthology.orchestra.run.smithi104.stderr:Downloaded: 1 files, 43M in 0.8s (52.1 MB/s) 2024-09-17T04:45:50.866 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-17T04:45:50.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T04:45:51.119 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T04:45:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:Updating files: 28% (1004/3520) Updating files: 29% (1021/3520) Updating files: 30% (1056/3520) Updating files: 31% (1092/3520) Updating files: 32% (1127/3520) Updating files: 33% (1162/3520) Updating files: 34% (1197/3520) Updating files: 35% (1232/3520) Updating files: 36% (1268/3520) Updating files: 37% (1303/3520) Updating files: 38% (1338/3520) Updating files: 39% (1373/3520) Updating files: 40% (1408/3520) Updating files: 41% (1444/3520) Updating files: 42% (1479/3520) Updating files: 43% (1514/3520) Updating files: 44% (1549/3520) Updating files: 45% (1584/3520) Updating files: 46% (1620/3520) Updating files: 47% (1655/3520) Updating files: 48% (1690/3520) Updating files: 49% (1725/3520) Updating files: 50% (1760/3520) Updating files: 51% (1796/3520) Updating files: 52% (1831/3520) Updating files: 53% (1866/3520) Updating files: 54% (1901/3520) Updating files: 55% (1936/3520) Updating files: 56% (1972/3520) Updating files: 57% (2007/3520) Updating files: 58% (2042/3520) Updating files: 58% (2055/3520) Updating files: 59% (2077/3520) Updating files: 60% (2112/3520) Updating files: 61% (2148/3520) Updating files: 62% (2183/3520) Updating files: 63% (2218/3520) Updating files: 64% (2253/3520) Updating files: 65% (2288/3520) Updating files: 66% (2324/3520) Updating files: 67% (2359/3520) Updating files: 68% (2394/3520) Updating files: 68% (2426/3520) Updating files: 69% (2429/3520) Updating files: 70% (2464/3520) Updating files: 71% (2500/3520) Updating files: 72% (2535/3520) Updating files: 73% (2570/3520) Updating files: 74% (2605/3520) Updating files: 75% (2640/3520) Updating files: 76% (2676/3520) Updating files: 77% (2711/3520) Updating files: 78% (2746/3520) Updating files: 79% (2781/3520) Updating files: 80% (2816/3520) Updating files: 81% (2852/3520) Updating files: 81% (2884/3520) Updating files: 82% (2887/3520) Updating files: 83% (2922/3520) Updating files: 84% (2957/3520) Updating files: 85% (2992/3520) Updating files: 86% (3028/3520) Updating files: 87% (3063/3520) Updating files: 88% (3098/3520) Updating files: 89% (3133/3520) Updating files: 90% (3168/3520) Updating files: 91% (3204/3520) Updating files: 92% (3239/3520) Updating files: 93% (3274/3520) Updating files: 94% (3309/3520) Updating files: 95% (3344/3520) Updating files: 96% (3380/3520) Updating files: 97% (3415/3520) Updating files: 98% (3450/3520) Updating files: 99% (3485/3520) Updating files: 100% (3520/3520) Updating files: 100% (3520/3520), done. 2024-09-17T04:45:51.944 INFO:teuthology.orchestra.run.smithi016.stdout:HEAD is now at 6c88d9b8 Merge branch 'amd-staging' into 'main' 2024-09-17T04:45:51.968 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T04:45:51.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-17T04:45:52.114 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T04:45:52.114 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-17T04:45:52.116 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-09-17T04:45:52.996 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T04:45:53.000 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T04:45:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17 04:45:52 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-17T04:45:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2024-09-17 04:45:52-- 2024-09-17T04:45:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 1.0s 2024-09-17T04:45:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 43M in 0.8s (51.8 MB/s) 2024-09-17T04:45:53.102 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-17T04:45:53.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo dpkg -i /tmp/linux-image.deb 2024-09-17T04:45:53.203 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T04:45:53.327 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-17T04:45:55.357 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-17T04:45:55.361 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-17T04:45:55.504 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T04:45:56.736 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T04:45:56.822 INFO:teuthology.orchestra.run.smithi104.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:45:59.586 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-17T04:45:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:18.865 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub' 2024-09-17T04:46:18.883 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T04:46:18.914 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-09-17T04:46:19.195 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:19.211 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:19.477 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T04:46:19.484 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T04:46:19.546 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T04:46:19.552 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T04:46:19.680 INFO:teuthology.orchestra.run.smithi104.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T04:46:19.681 INFO:teuthology.orchestra.run.smithi104.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T04:46:19.681 INFO:teuthology.orchestra.run.smithi104.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T04:46:19.735 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-09-17T04:46:19.910 DEBUG:teuthology.orchestra.run.smithi104:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T04:46:19.917 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T04:46:19.917 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T04:46:19.917 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:19.934 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:19.935 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T04:46:19.959 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T04:46:19.960 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T04:46:19.961 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T04:46:19.962 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T04:46:19.976 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T04:46:19.976 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T04:46:19.976 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T04:46:19.976 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T04:46:19.976 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T04:46:19.977 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T04:46:19.978 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T04:46:19.979 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T04:46:19.980 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T04:46:19.981 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T04:46:19.982 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T04:46:19.983 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T04:46:19.984 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T04:46:19.985 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T04:46:19.986 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T04:46:19.987 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T04:46:19.987 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T04:46:19.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T04:46:19.987 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T04:46:19.987 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T04:46:20.025 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T04:46:20.025 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T04:46:20.025 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T04:46:20.025 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T04:46:20.025 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T04:46:20.026 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T04:46:20.027 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T04:46:20.028 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T04:46:20.029 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T04:46:20.030 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T04:46:20.031 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T04:46:20.032 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T04:46:20.033 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T04:46:20.034 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T04:46:20.035 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T04:46:20.036 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T04:46:20.037 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T04:46:20.038 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T04:46:20.039 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T04:46:20.040 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T04:46:20.041 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T04:46:20.042 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T04:46:20.043 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T04:46:20.044 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T04:46:20.045 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T04:46:20.046 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T04:46:20.047 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T04:46:20.048 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T04:46:20.049 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T04:46:20.050 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T04:46:20.051 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T04:46:20.052 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T04:46:20.053 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T04:46:20.054 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T04:46:20.055 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T04:46:20.056 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T04:46:20.057 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T04:46:20.058 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T04:46:20.059 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T04:46:20.060 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T04:46:20.061 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T04:46:20.062 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T04:46:20.063 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T04:46:20.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T04:46:20.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T04:46:20.117 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T04:46:20.117 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T04:46:20.118 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T04:46:20.119 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T04:46:20.120 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T04:46:20.121 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T04:46:20.122 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T04:46:20.123 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T04:46:20.124 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T04:46:20.125 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T04:46:20.126 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T04:46:20.127 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T04:46:20.128 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T04:46:20.129 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T04:46:20.130 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T04:46:20.131 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T04:46:20.132 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T04:46:20.133 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T04:46:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T04:46:20.135 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T04:46:20.136 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T04:46:20.137 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T04:46:20.138 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T04:46:20.139 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T04:46:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T04:46:20.141 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T04:46:20.142 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T04:46:20.143 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T04:46:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T04:46:20.145 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T04:46:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T04:46:20.147 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T04:46:20.148 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T04:46:20.149 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T04:46:20.150 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T04:46:20.151 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T04:46:20.152 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T04:46:20.153 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T04:46:20.154 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T04:46:20.155 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T04:46:20.156 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T04:46:20.157 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T04:46:20.158 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T04:46:20.159 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T04:46:20.160 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T04:46:20.161 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T04:46:20.162 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T04:46:20.163 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T04:46:20.164 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T04:46:20.165 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T04:46:20.166 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T04:46:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T04:46:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T04:46:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T04:46:20.209 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T04:46:20.210 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T04:46:20.211 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T04:46:20.212 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T04:46:20.213 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T04:46:20.214 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T04:46:20.215 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T04:46:20.216 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T04:46:20.217 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T04:46:20.218 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T04:46:20.219 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T04:46:20.220 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T04:46:20.221 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T04:46:20.222 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T04:46:20.223 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T04:46:20.224 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T04:46:20.225 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T04:46:20.226 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T04:46:20.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T04:46:20.227 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T04:46:20.228 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T04:46:20.229 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T04:46:20.230 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T04:46:20.231 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T04:46:20.232 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T04:46:20.233 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T04:46:20.234 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T04:46:20.235 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T04:46:20.236 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T04:46:20.237 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T04:46:20.238 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T04:46:20.239 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T04:46:20.240 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T04:46:20.241 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T04:46:20.242 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T04:46:20.243 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T04:46:20.244 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T04:46:20.245 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T04:46:20.245 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T04:46:20.300 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T04:46:20.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T04:46:20.300 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T04:46:20.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T04:46:20.300 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T04:46:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T04:46:20.302 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T04:46:20.303 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T04:46:20.304 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T04:46:20.305 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T04:46:20.306 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T04:46:20.307 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T04:46:20.308 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T04:46:20.309 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T04:46:20.310 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T04:46:20.311 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T04:46:20.312 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T04:46:20.313 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T04:46:20.314 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T04:46:20.315 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T04:46:20.316 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T04:46:20.317 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T04:46:20.318 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T04:46:20.319 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T04:46:20.320 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T04:46:20.321 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T04:46:20.322 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T04:46:20.323 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T04:46:20.324 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T04:46:20.325 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T04:46:20.326 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T04:46:20.327 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T04:46:20.328 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T04:46:20.329 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T04:46:20.330 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T04:46:20.331 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T04:46:20.332 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T04:46:20.333 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T04:46:20.334 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T04:46:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T04:46:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T04:46:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T04:46:20.338 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T04:46:20.339 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T04:46:20.340 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T04:46:20.341 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T04:46:20.342 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T04:46:20.343 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T04:46:20.344 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T04:46:20.345 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T04:46:20.346 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T04:46:20.347 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T04:46:20.348 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T04:46:20.349 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T04:46:20.350 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T04:46:20.351 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T04:46:20.352 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T04:46:20.353 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T04:46:20.358 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:20.358 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:20.358 DEBUG:teuthology.orchestra.run.smithi104:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T04:46:20.365 INFO:teuthology.orchestra.run.smithi104.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T04:46:20.365 INFO:teuthology.orchestra.run.smithi104.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T04:46:20.365 INFO:teuthology.orchestra.run.smithi104.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T04:46:20.365 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T04:46:20.366 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:20.366 DEBUG:teuthology.orchestra.run.smithi104:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T04:46:20.407 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi104.front.sepia.ceph.com to complete... 2024-09-17T04:46:20.793 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub' 2024-09-17T04:46:20.793 INFO:teuthology.orchestra.run.smithi104.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T04:46:20.806 INFO:teuthology.orchestra.run.smithi104.stderr:Generating grub configuration file ... 2024-09-17T04:46:21.020 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:21.026 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:21.241 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T04:46:21.247 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T04:46:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T04:46:21.294 INFO:teuthology.orchestra.run.smithi104.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T04:46:21.458 INFO:teuthology.orchestra.run.smithi104.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T04:46:21.458 INFO:teuthology.orchestra.run.smithi104.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T04:46:21.458 INFO:teuthology.orchestra.run.smithi104.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T04:46:21.473 INFO:teuthology.orchestra.run.smithi104.stderr:done 2024-09-17T04:46:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub' 2024-09-17T04:46:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T04:46:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-17T04:46:22.610 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-17T04:46:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:22.843 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T04:46:23.251 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T04:46:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T04:46:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T04:46:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T04:46:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T04:46:23.458 INFO:teuthology.orchestra.run.smithi016.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T04:46:23.500 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-17T04:46:23.668 DEBUG:teuthology.orchestra.run.smithi016:> dpkg-deb -c /tmp/linux-image.deb 2024-09-17T04:46:23.677 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-17T04:46:23.677 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-17T04:46:23.677 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:23.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:23.692 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:23.715 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-17T04:46:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-17T04:46:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-17T04:46:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-17T04:46:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-17T04:46:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-17T04:46:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-17T04:46:23.717 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-17T04:46:23.717 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T04:46:23.717 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-17T04:46:23.717 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-17T04:46:23.717 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-17T04:46:23.718 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-17T04:46:23.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-17T04:46:23.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-17T04:46:23.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-17T04:46:23.719 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-17T04:46:23.720 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-17T04:46:23.721 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-17T04:46:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-17T04:46:23.735 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-17T04:46:23.736 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-17T04:46:23.737 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-17T04:46:23.738 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-17T04:46:23.739 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-17T04:46:23.740 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-17T04:46:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-17T04:46:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-17T04:46:23.743 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-17T04:46:23.744 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-17T04:46:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-17T04:46:23.746 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-17T04:46:23.747 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-17T04:46:23.748 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-17T04:46:23.749 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-17T04:46:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-17T04:46:23.751 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-17T04:46:23.752 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-17T04:46:23.752 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-17T04:46:23.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-17T04:46:23.752 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-17T04:46:23.795 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-17T04:46:23.795 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-17T04:46:23.796 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-17T04:46:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-17T04:46:23.798 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-17T04:46:23.799 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-17T04:46:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-17T04:46:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-17T04:46:23.802 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-17T04:46:23.803 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-17T04:46:23.804 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-17T04:46:23.805 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-17T04:46:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-17T04:46:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-17T04:46:23.808 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-17T04:46:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-17T04:46:23.810 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-17T04:46:23.811 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-17T04:46:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-17T04:46:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-17T04:46:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-17T04:46:23.812 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-17T04:46:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-17T04:46:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-17T04:46:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-17T04:46:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-17T04:46:23.814 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-17T04:46:23.815 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-17T04:46:23.815 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-17T04:46:23.815 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-17T04:46:23.815 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-17T04:46:23.816 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-17T04:46:23.816 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-17T04:46:23.817 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-17T04:46:23.818 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-17T04:46:23.819 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-17T04:46:23.820 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-17T04:46:23.821 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-17T04:46:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-17T04:46:23.823 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-17T04:46:23.824 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-17T04:46:23.825 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-17T04:46:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-17T04:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-17T04:46:23.828 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-17T04:46:23.829 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-17T04:46:23.830 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-17T04:46:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-17T04:46:23.832 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-17T04:46:23.833 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-17T04:46:23.834 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-17T04:46:23.835 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-17T04:46:23.836 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-17T04:46:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-17T04:46:23.838 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-17T04:46:23.839 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-17T04:46:23.840 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-17T04:46:23.841 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-17T04:46:23.842 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-17T04:46:23.843 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-17T04:46:23.844 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-17T04:46:23.845 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-17T04:46:23.846 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-17T04:46:23.847 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-17T04:46:23.848 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-17T04:46:23.849 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-17T04:46:23.850 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-17T04:46:23.851 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-17T04:46:23.852 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-17T04:46:23.853 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-17T04:46:23.854 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-17T04:46:23.855 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-17T04:46:23.921 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-17T04:46:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-17T04:46:23.923 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-17T04:46:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-17T04:46:23.925 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-17T04:46:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-17T04:46:23.927 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-17T04:46:23.928 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-17T04:46:23.929 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-17T04:46:23.930 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-17T04:46:23.931 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-17T04:46:23.932 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-17T04:46:23.933 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-17T04:46:23.934 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-17T04:46:23.935 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-17T04:46:23.936 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-17T04:46:23.937 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-17T04:46:23.938 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-17T04:46:23.939 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-17T04:46:23.940 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-17T04:46:23.941 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-17T04:46:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-17T04:46:23.943 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-17T04:46:23.944 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-17T04:46:23.945 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-17T04:46:23.946 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-17T04:46:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-17T04:46:23.948 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-17T04:46:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-17T04:46:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-17T04:46:23.951 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-17T04:46:23.952 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-17T04:46:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-17T04:46:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-17T04:46:23.955 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-17T04:46:23.956 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-17T04:46:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-17T04:46:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-17T04:46:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-17T04:46:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-17T04:46:23.961 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-17T04:46:23.962 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-17T04:46:23.963 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-17T04:46:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-17T04:46:23.965 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-17T04:46:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-17T04:46:23.967 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-17T04:46:23.968 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-17T04:46:23.969 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-17T04:46:23.970 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-17T04:46:23.971 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-17T04:46:23.972 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-17T04:46:23.973 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-17T04:46:23.974 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-17T04:46:23.975 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-17T04:46:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-17T04:46:23.977 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-17T04:46:23.978 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-17T04:46:23.979 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-17T04:46:23.980 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-17T04:46:23.981 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-17T04:46:23.982 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-17T04:46:23.983 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-17T04:46:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-17T04:46:23.985 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-17T04:46:23.986 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-17T04:46:23.987 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-17T04:46:23.988 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-17T04:46:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-17T04:46:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-17T04:46:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-17T04:46:23.992 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-17T04:46:24.031 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-17T04:46:24.032 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-17T04:46:24.033 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-17T04:46:24.034 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-17T04:46:24.035 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-17T04:46:24.036 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-17T04:46:24.037 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-17T04:46:24.038 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-17T04:46:24.039 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-17T04:46:24.040 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-17T04:46:24.041 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-17T04:46:24.042 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-17T04:46:24.043 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-17T04:46:24.044 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-17T04:46:24.045 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-17T04:46:24.046 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-17T04:46:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-17T04:46:24.048 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-17T04:46:24.049 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-17T04:46:24.050 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-17T04:46:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-17T04:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-17T04:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-17T04:46:24.054 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-17T04:46:24.055 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-17T04:46:24.056 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-17T04:46:24.057 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-17T04:46:24.058 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-17T04:46:24.059 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-17T04:46:24.060 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-17T04:46:24.061 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-17T04:46:24.062 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-17T04:46:24.063 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-17T04:46:24.064 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-17T04:46:24.065 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-17T04:46:24.066 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-17T04:46:24.067 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-17T04:46:24.068 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-17T04:46:24.069 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-17T04:46:24.070 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-17T04:46:24.071 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-17T04:46:24.072 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-17T04:46:24.073 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-17T04:46:24.074 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-17T04:46:24.075 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-17T04:46:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-17T04:46:24.077 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-17T04:46:24.078 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-17T04:46:24.079 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-17T04:46:24.083 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:24.083 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:24.083 DEBUG:teuthology.orchestra.run.smithi016:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-17T04:46:24.090 INFO:teuthology.orchestra.run.smithi016.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T04:46:24.090 INFO:teuthology.orchestra.run.smithi016.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T04:46:24.090 INFO:teuthology.orchestra.run.smithi016.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-17T04:46:24.091 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-17T04:46:24.091 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:24.091 DEBUG:teuthology.orchestra.run.smithi016:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-17T04:46:24.134 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-09-17T04:46:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub' 2024-09-17T04:46:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T04:46:24.585 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-17T04:46:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:46:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T04:46:25.062 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T04:46:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T04:46:25.109 INFO:teuthology.orchestra.run.smithi016.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T04:46:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T04:46:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T04:46:25.235 INFO:teuthology.orchestra.run.smithi016.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T04:46:25.243 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-17T04:46:52.637 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-17T04:46:52.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:46:52.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:46:56.317 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T04:46:56.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-17T04:46:56.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:47:11.169 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-09-17T04:47:14.749 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-09-17T04:47:20.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:47:20.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:47:23.229 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-09-17T04:47:23.750 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-17T04:47:23.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:47:35.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:47:35.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:47:53.665 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-09-17T04:48:08.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:48:08.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:48:09.276 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-17T04:48:10.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:48:10.518 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T04:48:10.518 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-09-17T04:48:10.526 INFO:teuthology.orchestra.run.smithi104.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:48:10.528 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:48:10.528 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T04:48:10.528 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T04:48:11.529 DEBUG:teuthology.parallel:result is None 2024-09-17T04:48:23.753 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T04:48:35.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-17T04:48:35.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:48:36.053 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-17T04:48:36.769 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-17T04:48:36.770 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-17T04:48:36.770 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-17T04:48:36.821 INFO:teuthology.orchestra.run.smithi016.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:48:36.821 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-17T04:48:36.822 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T04:48:36.822 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T04:48:37.824 DEBUG:teuthology.parallel:result is None 2024-09-17T04:48:37.824 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T04:48:37.831 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T04:48:37.831 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T04:48:37.835 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T04:48:37.844 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T04:48:37.867 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T04:48:37.901 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T04:48:37.901 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T04:48:37.904 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T04:48:37.955 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T04:48:37.962 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T04:48:37.962 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T04:48:37.965 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T04:48:38.026 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:48:38.047 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:48:38.097 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:48:38.120 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T04:48:38.122 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T04:48:38.131 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T04:48:38.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T04:48:38.134 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T04:48:38.155 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T04:48:38.164 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T04:48:38.164 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T04:48:38.200 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T04:48:38.208 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:48:38.258 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:48:38.309 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:48:38.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T04:48:38.382 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T04:48:38.391 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T04:48:38.443 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:48:38.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T04:48:38.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-17T04:48:38.529 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-09-17T04:48:38.661 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T04:48:38.668 INFO:teuthology.task.internal:Starting timer... 2024-09-17T04:48:38.668 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T04:48:38.819 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T04:48:38.854 DEBUG:teuthology.task.selinux:Excluding smithi016: OS 'ubuntu' does not support SELinux 2024-09-17T04:48:38.854 DEBUG:teuthology.task.selinux:Excluding smithi104: OS 'ubuntu' does not support SELinux 2024-09-17T04:48:38.854 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T04:48:38.854 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T04:48:38.854 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T04:48:38.854 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T04:48:38.886 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T04:48:38.898 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T04:48:38.898 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi104.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T04:57:05.742 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com')] 2024-09-17T04:57:05.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-17T04:57:05.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:57:05.825 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-17T04:57:05.940 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-17T04:57:05.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:57:05.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T04:57:06.022 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-17T04:57:06.167 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-17T04:57:06.167 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T04:57:06.178 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T04:57:06.178 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T04:57:06.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:57:06.182 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T04:57:06.182 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T04:57:06.220 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T04:57:06.220 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Command line: ntpd -gq 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: ---------------------------------------------------- 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: corporation. Support and training for ntp-4 are 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: available at https://www.nwtime.org/support 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: ---------------------------------------------------- 2024-09-17T04:57:06.221 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: proto: precision = 0.044 usec (-24) 2024-09-17T04:57:06.222 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: basedate set to 2022-02-04 2024-09-17T04:57:06.222 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: gps base set to 2022-02-06 (week 2196) 2024-09-17T04:57:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T04:57:06.222 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T04:57:06.223 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T04:57:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listen normally on 3 ens1f0 172.21.15.16:123 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listen normally on 4 lo [::1]:123 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1564%4]:123 2024-09-17T04:57:06.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:06 ntpd[18481]: Listening on routing socket on fd #22 for interface updates 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Command line: ntpd -gq 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: ---------------------------------------------------- 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: corporation. Support and training for ntp-4 are 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: available at https://www.nwtime.org/support 2024-09-17T04:57:06.248 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: ---------------------------------------------------- 2024-09-17T04:57:06.249 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: proto: precision = 0.041 usec (-24) 2024-09-17T04:57:06.249 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: basedate set to 2022-02-04 2024-09-17T04:57:06.249 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: gps base set to 2022-02-06 (week 2196) 2024-09-17T04:57:06.249 INFO:teuthology.orchestra.run.smithi104.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T04:57:06.250 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T04:57:06.250 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T04:57:06.250 INFO:teuthology.orchestra.run.smithi104.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T04:57:06.251 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T04:57:06.251 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T04:57:06.251 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T04:57:06.251 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listen normally on 3 enp3s0f1 172.21.15.104:123 2024-09-17T04:57:06.251 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listen normally on 4 lo [::1]:123 2024-09-17T04:57:06.252 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce89%5]:123 2024-09-17T04:57:06.252 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:06 ntpd[18466]: Listening on routing socket on fd #22 for interface updates 2024-09-17T04:57:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:07.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:07 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:07.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:07 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:08.252 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:08.253 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:08 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:09.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:09.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:09 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:09.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:09 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:09.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:09 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:09.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:09 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:09.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:09 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:09.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:09 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:09.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:09 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:10.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:10.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:10.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:10.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:10 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:11 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:11.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:11 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:11 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:11 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:11.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:11 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:11.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:11 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:11.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:11 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:12.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:12.250 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:12 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:13.224 INFO:teuthology.orchestra.run.smithi016.stdout:17 Sep 04:57:13 ntpd[18481]: ntpd: time slew +0.002328 s 2024-09-17T04:57:13.224 INFO:teuthology.orchestra.run.smithi016.stdout:ntpd: time slew +0.002328s 2024-09-17T04:57:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T04:57:13.225 INFO:teuthology.orchestra.run.smithi016.stderr:17 Sep 04:57:13 ntpd[18481]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T04:57:13.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:13.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:13.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T04:57:13.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T04:57:13.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T04:57:13.248 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T04:57:13.249 INFO:teuthology.orchestra.run.smithi104.stdout:17 Sep 04:57:13 ntpd[18466]: ntpd: time slew +0.003171 s 2024-09-17T04:57:13.249 INFO:teuthology.orchestra.run.smithi104.stdout:ntpd: time slew +0.003171s 2024-09-17T04:57:13.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T04:57:13.249 INFO:teuthology.orchestra.run.smithi104.stderr:17 Sep 04:57:13 ntpd[18466]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T04:57:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T04:57:13.338 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================== 2024-09-17T04:57:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.427 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T04:57:13.427 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2024-09-17T04:57:13.427 INFO:teuthology.orchestra.run.smithi104.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.428 INFO:teuthology.orchestra.run.smithi104.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.428 INFO:teuthology.orchestra.run.smithi104.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.428 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T04:57:13.429 INFO:teuthology.run_tasks:Running task install... 2024-09-17T04:57:13.437 DEBUG:teuthology.task.install:project ceph 2024-09-17T04:57:13.437 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T04:57:13.437 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T04:57:13.437 INFO:teuthology.task.install:Using flavor: default 2024-09-17T04:57:13.443 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T04:57:13.443 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T04:57:13.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-key list | grep Ceph 2024-09-17T04:57:13.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-key list | grep Ceph 2024-09-17T04:57:13.552 INFO:teuthology.orchestra.run.smithi016.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T04:57:13.572 INFO:teuthology.orchestra.run.smithi104.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T04:57:13.605 INFO:teuthology.orchestra.run.smithi016.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T04:57:13.605 INFO:teuthology.orchestra.run.smithi016.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T04:57:13.606 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T04:57:13.606 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T04:57:13.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:57:13.622 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T04:57:13.623 INFO:teuthology.orchestra.run.smithi104.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T04:57:13.623 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T04:57:13.624 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T04:57:13.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:57:13.753 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T04:57:13.753 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:57:13.764 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T04:57:13.765 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:57:13.866 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:57:13.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T04:57:13.877 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:57:13.877 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T04:57:13.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo apt-get update 2024-09-17T04:57:13.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo apt-get update 2024-09-17T04:57:14.077 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T04:57:14.077 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T04:57:14.083 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T04:57:14.100 INFO:teuthology.orchestra.run.smithi016.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T04:57:14.128 INFO:teuthology.orchestra.run.smithi016.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T04:57:14.133 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T04:57:14.148 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T04:57:14.154 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T04:57:14.170 INFO:teuthology.orchestra.run.smithi016.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T04:57:14.177 INFO:teuthology.orchestra.run.smithi104.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T04:57:14.192 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T04:57:14.198 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T04:57:14.219 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T04:57:14.219 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T04:57:14.224 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T04:57:14.240 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T04:57:14.281 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T04:57:14.362 INFO:teuthology.orchestra.run.smithi104.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T04:57:14.696 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 29.2 kB in 1s (51.1 kB/s) 2024-09-17T04:57:14.813 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 29.2 kB in 1s (42.6 kB/s) 2024-09-17T04:57:15.770 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-17T04:57:15.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:57:15.853 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-09-17T04:57:15.864 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-17T04:57:15.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:57:15.945 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-09-17T04:57:16.051 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-17T04:57:16.051 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-17T04:57:16.128 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2024-09-17T04:57:16.128 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2024-09-17T04:57:16.245 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T04:57:16.246 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T04:57:16.246 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T04:57:16.246 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T04:57:16.246 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T04:57:16.247 INFO:teuthology.orchestra.run.smithi016.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T04:57:16.247 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T04:57:16.248 INFO:teuthology.orchestra.run.smithi016.stdout:The following additional packages will be installed: 2024-09-17T04:57:16.248 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T04:57:16.248 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T04:57:16.248 INFO:teuthology.orchestra.run.smithi016.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T04:57:16.248 INFO:teuthology.orchestra.run.smithi016.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T04:57:16.248 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T04:57:16.250 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout:Suggested packages: 2024-09-17T04:57:16.251 INFO:teuthology.orchestra.run.smithi016.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout:Recommended packages: 2024-09-17T04:57:16.252 INFO:teuthology.orchestra.run.smithi016.stdout: btrfs-tools 2024-09-17T04:57:16.320 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T04:57:16.320 INFO:teuthology.orchestra.run.smithi104.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T04:57:16.320 INFO:teuthology.orchestra.run.smithi104.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T04:57:16.320 INFO:teuthology.orchestra.run.smithi104.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T04:57:16.321 INFO:teuthology.orchestra.run.smithi104.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T04:57:16.321 INFO:teuthology.orchestra.run.smithi104.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T04:57:16.321 INFO:teuthology.orchestra.run.smithi104.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T04:57:16.321 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2024-09-17T04:57:16.322 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T04:57:16.322 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T04:57:16.322 INFO:teuthology.orchestra.run.smithi104.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T04:57:16.322 INFO:teuthology.orchestra.run.smithi104.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T04:57:16.322 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T04:57:16.323 INFO:teuthology.orchestra.run.smithi104.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T04:57:16.323 INFO:teuthology.orchestra.run.smithi104.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T04:57:16.323 INFO:teuthology.orchestra.run.smithi104.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T04:57:16.323 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T04:57:16.324 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2024-09-17T04:57:16.325 INFO:teuthology.orchestra.run.smithi104.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout:Recommended packages: 2024-09-17T04:57:16.326 INFO:teuthology.orchestra.run.smithi104.stdout: btrfs-tools 2024-09-17T04:57:16.412 INFO:teuthology.orchestra.run.smithi016.stdout:The following NEW packages will be installed: 2024-09-17T04:57:16.412 INFO:teuthology.orchestra.run.smithi016.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T04:57:16.412 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T04:57:16.412 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T04:57:16.412 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T04:57:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T04:57:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T04:57:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T04:57:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T04:57:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T04:57:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T04:57:16.415 INFO:teuthology.orchestra.run.smithi016.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T04:57:16.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T04:57:16.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T04:57:16.416 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T04:57:16.416 INFO:teuthology.orchestra.run.smithi016.stdout: socat unzip xmlstarlet zip 2024-09-17T04:57:16.478 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2024-09-17T04:57:16.478 INFO:teuthology.orchestra.run.smithi104.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T04:57:16.479 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T04:57:16.479 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T04:57:16.479 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T04:57:16.481 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T04:57:16.482 INFO:teuthology.orchestra.run.smithi104.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T04:57:16.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T04:57:16.483 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T04:57:16.483 INFO:teuthology.orchestra.run.smithi104.stdout: socat unzip xmlstarlet zip 2024-09-17T04:57:16.565 INFO:teuthology.orchestra.run.smithi016.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T04:57:16.565 INFO:teuthology.orchestra.run.smithi016.stdout:Need to get 239 MB of archives. 2024-09-17T04:57:16.566 INFO:teuthology.orchestra.run.smithi016.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T04:57:16.566 INFO:teuthology.orchestra.run.smithi016.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T04:57:16.612 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T04:57:16.628 INFO:teuthology.orchestra.run.smithi104.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-17T04:57:16.628 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 239 MB of archives. 2024-09-17T04:57:16.628 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T04:57:16.628 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T04:57:16.676 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T04:57:16.694 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T04:57:16.737 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T04:57:16.740 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T04:57:16.758 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T04:57:16.796 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T04:57:16.804 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T04:57:16.808 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T04:57:16.814 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T04:57:16.815 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T04:57:16.818 INFO:teuthology.orchestra.run.smithi016.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T04:57:16.819 INFO:teuthology.orchestra.run.smithi016.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T04:57:16.827 INFO:teuthology.orchestra.run.smithi016.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T04:57:16.937 INFO:teuthology.orchestra.run.smithi016.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T04:57:16.999 INFO:teuthology.orchestra.run.smithi016.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T04:57:17.000 INFO:teuthology.orchestra.run.smithi016.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T04:57:17.001 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T04:57:17.009 INFO:teuthology.orchestra.run.smithi016.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T04:57:17.028 INFO:teuthology.orchestra.run.smithi016.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T04:57:17.042 INFO:teuthology.orchestra.run.smithi016.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T04:57:17.094 INFO:teuthology.orchestra.run.smithi016.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T04:57:17.095 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T04:57:17.095 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T04:57:17.105 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T04:57:17.155 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T04:57:17.269 INFO:teuthology.orchestra.run.smithi016.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T04:57:17.280 INFO:teuthology.orchestra.run.smithi016.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T04:57:17.304 INFO:teuthology.orchestra.run.smithi016.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T04:57:17.312 INFO:teuthology.orchestra.run.smithi016.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T04:57:17.314 INFO:teuthology.orchestra.run.smithi016.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T04:57:17.316 INFO:teuthology.orchestra.run.smithi016.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T04:57:17.331 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T04:57:17.335 INFO:teuthology.orchestra.run.smithi016.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T04:57:17.338 INFO:teuthology.orchestra.run.smithi016.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T04:57:17.341 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T04:57:17.344 INFO:teuthology.orchestra.run.smithi016.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T04:57:17.351 INFO:teuthology.orchestra.run.smithi016.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T04:57:17.370 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T04:57:17.378 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T04:57:17.380 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T04:57:17.381 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T04:57:17.398 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T04:57:17.401 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T04:57:17.406 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T04:57:17.412 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T04:57:17.428 INFO:teuthology.orchestra.run.smithi016.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T04:57:17.430 INFO:teuthology.orchestra.run.smithi016.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T04:57:17.432 INFO:teuthology.orchestra.run.smithi016.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T04:57:17.434 INFO:teuthology.orchestra.run.smithi016.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T04:57:17.435 INFO:teuthology.orchestra.run.smithi016.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T04:57:17.435 INFO:teuthology.orchestra.run.smithi016.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T04:57:17.436 INFO:teuthology.orchestra.run.smithi016.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T04:57:17.437 INFO:teuthology.orchestra.run.smithi016.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T04:57:17.437 INFO:teuthology.orchestra.run.smithi016.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T04:57:17.455 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T04:57:17.490 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T04:57:17.492 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T04:57:17.494 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T04:57:17.495 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T04:57:17.496 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T04:57:17.496 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T04:57:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T04:57:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T04:57:17.498 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T04:57:17.512 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T04:57:17.514 INFO:teuthology.orchestra.run.smithi016.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T04:57:17.514 INFO:teuthology.orchestra.run.smithi016.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T04:57:17.564 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T04:57:17.564 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T04:57:17.567 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T04:57:17.568 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T04:57:17.575 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T04:57:17.575 INFO:teuthology.orchestra.run.smithi104.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T04:57:17.582 INFO:teuthology.orchestra.run.smithi016.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T04:57:17.596 INFO:teuthology.orchestra.run.smithi016.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T04:57:17.596 INFO:teuthology.orchestra.run.smithi016.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T04:57:17.598 INFO:teuthology.orchestra.run.smithi016.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T04:57:17.598 INFO:teuthology.orchestra.run.smithi016.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T04:57:17.599 INFO:teuthology.orchestra.run.smithi016.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T04:57:17.599 INFO:teuthology.orchestra.run.smithi016.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T04:57:17.600 INFO:teuthology.orchestra.run.smithi016.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T04:57:17.601 INFO:teuthology.orchestra.run.smithi016.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T04:57:17.621 INFO:teuthology.orchestra.run.smithi104.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T04:57:17.657 INFO:teuthology.orchestra.run.smithi104.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T04:57:17.657 INFO:teuthology.orchestra.run.smithi104.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T04:57:17.660 INFO:teuthology.orchestra.run.smithi104.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T04:57:17.660 INFO:teuthology.orchestra.run.smithi104.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T04:57:17.661 INFO:teuthology.orchestra.run.smithi104.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T04:57:17.661 INFO:teuthology.orchestra.run.smithi104.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T04:57:17.662 INFO:teuthology.orchestra.run.smithi104.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T04:57:17.662 INFO:teuthology.orchestra.run.smithi104.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T04:57:17.670 INFO:teuthology.orchestra.run.smithi016.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T04:57:17.672 INFO:teuthology.orchestra.run.smithi016.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T04:57:17.674 INFO:teuthology.orchestra.run.smithi016.stdout:Get:52 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T04:57:17.677 INFO:teuthology.orchestra.run.smithi016.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T04:57:17.678 INFO:teuthology.orchestra.run.smithi016.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T04:57:17.712 INFO:teuthology.orchestra.run.smithi016.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T04:57:17.738 INFO:teuthology.orchestra.run.smithi104.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T04:57:17.739 INFO:teuthology.orchestra.run.smithi104.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T04:57:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T04:57:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T04:57:17.763 INFO:teuthology.orchestra.run.smithi016.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T04:57:17.764 INFO:teuthology.orchestra.run.smithi016.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T04:57:17.764 INFO:teuthology.orchestra.run.smithi016.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T04:57:17.765 INFO:teuthology.orchestra.run.smithi016.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T04:57:17.766 INFO:teuthology.orchestra.run.smithi016.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T04:57:17.768 INFO:teuthology.orchestra.run.smithi016.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T04:57:17.820 INFO:teuthology.orchestra.run.smithi104.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T04:57:17.821 INFO:teuthology.orchestra.run.smithi104.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T04:57:17.825 INFO:teuthology.orchestra.run.smithi104.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T04:57:17.825 INFO:teuthology.orchestra.run.smithi104.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T04:57:17.826 INFO:teuthology.orchestra.run.smithi104.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T04:57:17.826 INFO:teuthology.orchestra.run.smithi104.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T04:57:17.828 INFO:teuthology.orchestra.run.smithi104.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T04:57:17.829 INFO:teuthology.orchestra.run.smithi104.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T04:57:17.839 INFO:teuthology.orchestra.run.smithi016.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T04:57:17.840 INFO:teuthology.orchestra.run.smithi016.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T04:57:17.841 INFO:teuthology.orchestra.run.smithi016.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T04:57:17.851 INFO:teuthology.orchestra.run.smithi016.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T04:57:17.852 INFO:teuthology.orchestra.run.smithi016.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T04:57:17.853 INFO:teuthology.orchestra.run.smithi016.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T04:57:17.901 INFO:teuthology.orchestra.run.smithi104.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T04:57:17.902 INFO:teuthology.orchestra.run.smithi104.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T04:57:17.912 INFO:teuthology.orchestra.run.smithi104.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T04:57:17.914 INFO:teuthology.orchestra.run.smithi104.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T04:57:17.914 INFO:teuthology.orchestra.run.smithi104.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T04:57:17.915 INFO:teuthology.orchestra.run.smithi104.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T04:57:17.921 INFO:teuthology.orchestra.run.smithi016.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T04:57:17.923 INFO:teuthology.orchestra.run.smithi016.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T04:57:17.984 INFO:teuthology.orchestra.run.smithi104.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T04:57:18.032 INFO:teuthology.orchestra.run.smithi016.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T04:57:18.032 INFO:teuthology.orchestra.run.smithi016.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T04:57:18.034 INFO:teuthology.orchestra.run.smithi016.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T04:57:18.034 INFO:teuthology.orchestra.run.smithi016.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T04:57:18.035 INFO:teuthology.orchestra.run.smithi016.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T04:57:18.093 INFO:teuthology.orchestra.run.smithi104.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T04:57:18.093 INFO:teuthology.orchestra.run.smithi104.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T04:57:18.095 INFO:teuthology.orchestra.run.smithi104.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T04:57:18.095 INFO:teuthology.orchestra.run.smithi104.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T04:57:18.096 INFO:teuthology.orchestra.run.smithi104.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T04:57:18.107 INFO:teuthology.orchestra.run.smithi016.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T04:57:18.109 INFO:teuthology.orchestra.run.smithi016.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T04:57:18.109 INFO:teuthology.orchestra.run.smithi016.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T04:57:18.167 INFO:teuthology.orchestra.run.smithi104.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T04:57:18.169 INFO:teuthology.orchestra.run.smithi104.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T04:57:18.170 INFO:teuthology.orchestra.run.smithi104.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T04:57:18.179 INFO:teuthology.orchestra.run.smithi016.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T04:57:18.180 INFO:teuthology.orchestra.run.smithi016.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T04:57:18.180 INFO:teuthology.orchestra.run.smithi016.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T04:57:18.181 INFO:teuthology.orchestra.run.smithi016.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T04:57:18.181 INFO:teuthology.orchestra.run.smithi016.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T04:57:18.182 INFO:teuthology.orchestra.run.smithi016.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T04:57:18.232 INFO:teuthology.orchestra.run.smithi104.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T04:57:18.234 INFO:teuthology.orchestra.run.smithi104.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T04:57:18.240 INFO:teuthology.orchestra.run.smithi104.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T04:57:18.240 INFO:teuthology.orchestra.run.smithi104.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T04:57:18.241 INFO:teuthology.orchestra.run.smithi104.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T04:57:18.241 INFO:teuthology.orchestra.run.smithi104.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T04:57:18.242 INFO:teuthology.orchestra.run.smithi104.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T04:57:18.243 INFO:teuthology.orchestra.run.smithi104.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T04:57:18.249 INFO:teuthology.orchestra.run.smithi016.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T04:57:18.250 INFO:teuthology.orchestra.run.smithi016.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T04:57:18.253 INFO:teuthology.orchestra.run.smithi016.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T04:57:18.288 INFO:teuthology.orchestra.run.smithi104.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T04:57:18.310 INFO:teuthology.orchestra.run.smithi104.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T04:57:18.311 INFO:teuthology.orchestra.run.smithi104.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T04:57:18.314 INFO:teuthology.orchestra.run.smithi104.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T04:57:18.331 INFO:teuthology.orchestra.run.smithi016.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T04:57:18.333 INFO:teuthology.orchestra.run.smithi016.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T04:57:18.333 INFO:teuthology.orchestra.run.smithi016.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T04:57:18.336 INFO:teuthology.orchestra.run.smithi016.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T04:57:18.339 INFO:teuthology.orchestra.run.smithi016.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T04:57:18.339 INFO:teuthology.orchestra.run.smithi016.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T04:57:18.341 INFO:teuthology.orchestra.run.smithi016.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T04:57:18.344 INFO:teuthology.orchestra.run.smithi016.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T04:57:18.344 INFO:teuthology.orchestra.run.smithi016.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T04:57:18.371 INFO:teuthology.orchestra.run.smithi016.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T04:57:18.391 INFO:teuthology.orchestra.run.smithi104.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T04:57:18.393 INFO:teuthology.orchestra.run.smithi104.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T04:57:18.393 INFO:teuthology.orchestra.run.smithi104.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T04:57:18.397 INFO:teuthology.orchestra.run.smithi104.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T04:57:18.400 INFO:teuthology.orchestra.run.smithi104.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T04:57:18.400 INFO:teuthology.orchestra.run.smithi104.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T04:57:18.401 INFO:teuthology.orchestra.run.smithi104.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T04:57:18.412 INFO:teuthology.orchestra.run.smithi016.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T04:57:18.417 INFO:teuthology.orchestra.run.smithi016.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T04:57:18.418 INFO:teuthology.orchestra.run.smithi016.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T04:57:18.430 INFO:teuthology.orchestra.run.smithi016.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T04:57:18.443 INFO:teuthology.orchestra.run.smithi016.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T04:57:18.446 INFO:teuthology.orchestra.run.smithi016.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T04:57:18.473 INFO:teuthology.orchestra.run.smithi104.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T04:57:18.477 INFO:teuthology.orchestra.run.smithi104.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T04:57:18.478 INFO:teuthology.orchestra.run.smithi104.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T04:57:18.493 INFO:teuthology.orchestra.run.smithi016.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T04:57:18.494 INFO:teuthology.orchestra.run.smithi016.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T04:57:18.495 INFO:teuthology.orchestra.run.smithi016.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T04:57:18.495 INFO:teuthology.orchestra.run.smithi016.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T04:57:18.496 INFO:teuthology.orchestra.run.smithi016.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T04:57:18.497 INFO:teuthology.orchestra.run.smithi016.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T04:57:18.504 INFO:teuthology.orchestra.run.smithi016.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T04:57:18.554 INFO:teuthology.orchestra.run.smithi104.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T04:57:18.554 INFO:teuthology.orchestra.run.smithi104.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T04:57:18.555 INFO:teuthology.orchestra.run.smithi104.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T04:57:18.555 INFO:teuthology.orchestra.run.smithi104.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T04:57:18.557 INFO:teuthology.orchestra.run.smithi104.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T04:57:18.558 INFO:teuthology.orchestra.run.smithi104.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T04:57:18.565 INFO:teuthology.orchestra.run.smithi104.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T04:57:18.575 INFO:teuthology.orchestra.run.smithi016.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T04:57:18.575 INFO:teuthology.orchestra.run.smithi016.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T04:57:18.577 INFO:teuthology.orchestra.run.smithi016.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T04:57:18.635 INFO:teuthology.orchestra.run.smithi104.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T04:57:18.636 INFO:teuthology.orchestra.run.smithi104.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T04:57:18.638 INFO:teuthology.orchestra.run.smithi104.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T04:57:18.656 INFO:teuthology.orchestra.run.smithi016.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T04:57:18.717 INFO:teuthology.orchestra.run.smithi104.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T04:57:19.467 INFO:teuthology.orchestra.run.smithi104.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T04:57:19.613 INFO:teuthology.orchestra.run.smithi104.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T04:57:19.618 INFO:teuthology.orchestra.run.smithi104.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T04:57:19.627 INFO:teuthology.orchestra.run.smithi104.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T04:57:19.678 INFO:teuthology.orchestra.run.smithi104.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T04:57:19.870 INFO:teuthology.orchestra.run.smithi104.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T04:57:20.327 INFO:teuthology.orchestra.run.smithi016.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T04:57:20.591 INFO:teuthology.orchestra.run.smithi016.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T04:57:20.591 INFO:teuthology.orchestra.run.smithi016.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T04:57:20.634 INFO:teuthology.orchestra.run.smithi104.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T04:57:20.635 INFO:teuthology.orchestra.run.smithi104.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T04:57:20.652 INFO:teuthology.orchestra.run.smithi104.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T04:57:20.741 INFO:teuthology.orchestra.run.smithi104.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T04:57:20.762 INFO:teuthology.orchestra.run.smithi104.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T04:57:20.766 INFO:teuthology.orchestra.run.smithi104.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T04:57:21.900 INFO:teuthology.orchestra.run.smithi016.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T04:57:21.901 INFO:teuthology.orchestra.run.smithi016.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T04:57:21.903 INFO:teuthology.orchestra.run.smithi016.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T04:57:21.904 INFO:teuthology.orchestra.run.smithi016.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T04:57:21.904 INFO:teuthology.orchestra.run.smithi016.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T04:57:21.905 INFO:teuthology.orchestra.run.smithi016.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T04:57:22.439 INFO:teuthology.orchestra.run.smithi016.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T04:57:22.672 INFO:teuthology.orchestra.run.smithi104.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T04:57:22.797 INFO:teuthology.orchestra.run.smithi104.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T04:57:22.798 INFO:teuthology.orchestra.run.smithi104.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T04:57:23.122 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 239 MB in 6s (39.8 MB/s) 2024-09-17T04:57:23.270 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T04:57:23.338 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T04:57:23.341 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T04:57:23.419 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T04:57:23.479 INFO:teuthology.orchestra.run.smithi104.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T04:57:23.480 INFO:teuthology.orchestra.run.smithi104.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T04:57:23.482 INFO:teuthology.orchestra.run.smithi104.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T04:57:23.482 INFO:teuthology.orchestra.run.smithi104.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T04:57:23.482 INFO:teuthology.orchestra.run.smithi104.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T04:57:23.483 INFO:teuthology.orchestra.run.smithi104.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T04:57:23.768 INFO:teuthology.orchestra.run.smithi104.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T04:57:23.888 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T04:57:23.902 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T04:57:23.947 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T04:57:24.275 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T04:57:24.290 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T04:57:24.333 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T04:57:24.421 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 239 MB in 7s (32.9 MB/s) 2024-09-17T04:57:24.564 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T04:57:24.628 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-17T04:57:24.631 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T04:57:24.680 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T04:57:24.694 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T04:57:24.710 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T04:57:24.778 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T04:57:24.982 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T04:57:25.003 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T04:57:25.041 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T04:57:25.065 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T04:57:25.080 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T04:57:25.123 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T04:57:25.406 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T04:57:25.420 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T04:57:25.444 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T04:57:25.458 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T04:57:25.493 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T04:57:25.498 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T04:57:25.842 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T04:57:25.856 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T04:57:25.943 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T04:57:25.980 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T04:57:25.995 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T04:57:26.046 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:57:26.296 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T04:57:26.311 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T04:57:26.354 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T04:57:26.575 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T04:57:26.590 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T04:57:26.633 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:57:26.774 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T04:57:26.789 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T04:57:26.841 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T04:57:26.944 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T04:57:26.960 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T04:57:27.020 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:57:27.277 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T04:57:27.292 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T04:57:27.344 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:57:27.365 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T04:57:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T04:57:27.423 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T04:57:27.759 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librados2. 2024-09-17T04:57:27.774 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:27.780 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T04:57:27.795 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T04:57:27.817 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:27.847 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:57:28.267 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T04:57:28.270 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libnbd0. 2024-09-17T04:57:28.282 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T04:57:28.285 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T04:57:28.320 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T04:57:28.325 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:57:28.625 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librbd1. 2024-09-17T04:57:28.641 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:28.684 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:28.845 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T04:57:28.860 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T04:57:28.903 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T04:57:29.256 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librados2. 2024-09-17T04:57:29.271 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:29.323 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:29.387 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T04:57:29.402 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:29.436 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:29.714 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rados. 2024-09-17T04:57:29.729 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:29.764 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:29.867 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libnbd0. 2024-09-17T04:57:29.882 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T04:57:29.934 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T04:57:30.092 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T04:57:30.107 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:30.150 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:30.237 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librbd1. 2024-09-17T04:57:30.251 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:30.295 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:30.402 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T04:57:30.418 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:30.460 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:30.765 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T04:57:30.779 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:30.780 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T04:57:30.795 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:30.830 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:30.831 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:31.141 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T04:57:31.156 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T04:57:31.158 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rados. 2024-09-17T04:57:31.173 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:31.200 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T04:57:31.225 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:31.477 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T04:57:31.492 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:31.535 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:31.569 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T04:57:31.584 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:31.628 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:31.880 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T04:57:31.895 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:31.913 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T04:57:31.929 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T04:57:31.938 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:31.972 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T04:57:32.258 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T04:57:32.266 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T04:57:32.274 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T04:57:32.281 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:32.308 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T04:57:32.332 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:32.569 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T04:57:32.585 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T04:57:32.619 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T04:57:32.686 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T04:57:32.700 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T04:57:32.769 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T04:57:32.972 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua5.1. 2024-09-17T04:57:32.988 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T04:57:33.030 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T04:57:33.071 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T04:57:33.086 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:33.129 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:33.350 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-any. 2024-09-17T04:57:33.366 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T04:57:33.409 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T04:57:33.458 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T04:57:33.473 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T04:57:33.516 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T04:57:33.611 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package zip. 2024-09-17T04:57:33.627 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T04:57:33.661 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T04:57:33.802 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T04:57:33.817 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T04:57:33.852 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T04:57:33.981 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package unzip. 2024-09-17T04:57:33.996 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T04:57:34.031 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T04:57:34.221 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T04:57:34.236 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T04:57:34.288 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T04:57:34.468 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package luarocks. 2024-09-17T04:57:34.484 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T04:57:34.527 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T04:57:34.625 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package lua5.1. 2024-09-17T04:57:34.639 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T04:57:34.683 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T04:57:34.918 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package librgw2. 2024-09-17T04:57:34.932 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:34.984 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:35.011 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package lua-any. 2024-09-17T04:57:35.025 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T04:57:35.069 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T04:57:35.363 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package zip. 2024-09-17T04:57:35.378 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T04:57:35.430 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T04:57:35.662 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T04:57:35.678 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:35.720 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:35.783 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package unzip. 2024-09-17T04:57:35.796 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T04:57:35.833 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T04:57:36.006 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T04:57:36.022 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T04:57:36.056 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T04:57:36.228 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package luarocks. 2024-09-17T04:57:36.242 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T04:57:36.286 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T04:57:36.359 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T04:57:36.375 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:36.417 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:36.711 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package librgw2. 2024-09-17T04:57:36.724 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:36.761 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:36.871 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-common. 2024-09-17T04:57:36.887 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:36.929 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:37.380 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T04:57:37.393 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:37.438 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:37.766 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T04:57:37.779 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T04:57:37.824 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T04:57:38.135 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T04:57:38.151 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:38.194 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:38.351 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-base. 2024-09-17T04:57:38.368 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:38.419 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:38.605 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-common. 2024-09-17T04:57:38.619 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:38.663 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:39.063 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T04:57:39.078 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T04:57:39.121 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T04:57:39.625 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T04:57:39.641 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T04:57:39.674 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T04:57:39.944 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T04:57:39.960 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T04:57:40.002 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T04:57:40.110 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-base. 2024-09-17T04:57:40.124 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:40.177 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:40.322 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T04:57:40.338 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T04:57:40.380 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T04:57:40.666 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T04:57:40.682 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T04:57:40.733 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T04:57:40.748 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T04:57:40.763 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T04:57:40.806 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T04:57:41.011 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T04:57:41.026 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T04:57:41.061 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T04:57:41.109 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T04:57:41.123 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T04:57:41.167 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T04:57:41.355 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-portend. 2024-09-17T04:57:41.371 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T04:57:41.413 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T04:57:41.528 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T04:57:41.543 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T04:57:41.578 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T04:57:41.716 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T04:57:41.732 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T04:57:41.783 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T04:57:41.839 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T04:57:41.854 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T04:57:41.898 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T04:57:42.086 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T04:57:42.101 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T04:57:42.144 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T04:57:42.200 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T04:57:42.215 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T04:57:42.259 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T04:57:42.508 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T04:57:42.524 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T04:57:42.561 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T04:57:42.572 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T04:57:42.576 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T04:57:42.619 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T04:57:42.884 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T04:57:42.900 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T04:57:42.914 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-portend. 2024-09-17T04:57:42.929 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T04:57:42.942 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T04:57:42.964 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T04:57:43.245 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T04:57:43.258 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T04:57:43.261 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T04:57:43.274 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T04:57:43.303 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T04:57:43.316 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T04:57:43.564 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-mako. 2024-09-17T04:57:43.579 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T04:57:43.614 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T04:57:43.636 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T04:57:43.651 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T04:57:43.730 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T04:57:43.951 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T04:57:43.967 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T04:57:44.009 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T04:57:44.057 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T04:57:44.071 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T04:57:44.115 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T04:57:44.329 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T04:57:44.344 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T04:57:44.387 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T04:57:44.459 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T04:57:44.474 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T04:57:44.518 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T04:57:44.689 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-webob. 2024-09-17T04:57:44.705 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T04:57:44.747 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T04:57:44.820 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T04:57:44.834 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T04:57:44.878 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T04:57:45.101 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T04:57:45.117 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T04:57:45.154 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T04:57:45.173 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-mako. 2024-09-17T04:57:45.188 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T04:57:45.223 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T04:57:45.420 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T04:57:45.436 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T04:57:45.478 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T04:57:45.543 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T04:57:45.558 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T04:57:45.601 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T04:57:45.781 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-paste. 2024-09-17T04:57:45.797 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T04:57:45.847 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T04:57:45.904 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T04:57:45.919 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T04:57:45.962 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T04:57:46.185 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T04:57:46.200 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T04:57:46.234 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T04:57:46.290 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webob. 2024-09-17T04:57:46.297 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T04:57:46.340 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T04:57:46.529 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T04:57:46.545 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T04:57:46.587 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T04:57:46.668 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T04:57:46.675 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T04:57:46.712 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T04:57:46.873 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T04:57:46.889 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T04:57:46.931 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T04:57:46.970 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T04:57:46.978 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T04:57:47.021 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T04:57:47.285 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T04:57:47.301 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T04:57:47.332 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-paste. 2024-09-17T04:57:47.339 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T04:57:47.343 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T04:57:47.407 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T04:57:47.698 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T04:57:47.713 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T04:57:47.726 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T04:57:47.734 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T04:57:47.756 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T04:57:47.776 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T04:57:48.051 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-idna. 2024-09-17T04:57:48.067 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T04:57:48.079 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T04:57:48.087 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T04:57:48.100 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T04:57:48.129 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T04:57:48.399 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T04:57:48.406 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T04:57:48.440 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T04:57:48.487 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T04:57:48.502 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T04:57:48.545 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T04:57:48.768 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T04:57:48.776 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T04:57:48.818 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T04:57:48.865 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-requests. 2024-09-17T04:57:48.881 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T04:57:48.923 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T04:57:49.173 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T04:57:49.180 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T04:57:49.218 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T04:57:49.231 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T04:57:49.234 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T04:57:49.534 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T04:57:50.493 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-idna. 2024-09-17T04:57:50.500 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T04:57:50.550 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T04:57:50.554 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T04:57:50.570 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:50.612 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:50.928 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T04:57:50.936 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T04:57:51.017 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T04:57:51.019 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T04:57:51.033 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:51.076 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:51.387 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T04:57:51.402 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T04:57:51.406 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-requests. 2024-09-17T04:57:51.414 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T04:57:51.437 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T04:57:51.456 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T04:57:51.700 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T04:57:51.713 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T04:57:51.756 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T04:57:51.809 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T04:57:51.817 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T04:57:51.859 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T04:57:52.101 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T04:57:52.116 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:52.167 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:52.195 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T04:57:52.203 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:52.270 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T04:57:52.620 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:52.662 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:52.675 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T04:57:52.683 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:52.725 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:53.045 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T04:57:53.052 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T04:57:53.086 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T04:57:53.174 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T04:57:53.189 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:53.232 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:53.355 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T04:57:53.363 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T04:57:53.405 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T04:57:53.752 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T04:57:53.758 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:53.800 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:54.127 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph. 2024-09-17T04:57:54.143 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:54.185 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:54.194 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T04:57:54.202 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:54.244 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:54.504 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T04:57:54.520 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:54.562 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:54.848 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T04:57:54.856 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:54.882 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T04:57:54.898 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:54.898 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:54.974 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:55.369 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephadm. 2024-09-17T04:57:55.385 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:55.427 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:55.718 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph. 2024-09-17T04:57:55.725 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:55.755 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T04:57:55.771 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T04:57:55.776 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:55.813 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T04:57:56.153 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T04:57:56.161 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:56.199 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T04:57:56.203 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:56.215 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T04:57:56.257 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T04:57:56.573 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T04:57:56.580 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:56.623 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:57.026 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephadm. 2024-09-17T04:57:57.033 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:57:57.109 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:57.279 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-babel. 2024-09-17T04:57:57.293 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T04:57:57.362 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T04:57:57.478 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T04:57:57.485 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T04:57:57.537 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T04:57:57.690 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T04:57:57.705 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T04:57:57.748 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T04:57:57.865 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T04:57:57.872 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T04:57:57.923 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T04:57:58.101 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T04:57:58.117 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:58.160 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:58.530 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T04:57:58.546 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T04:57:58.588 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T04:57:58.874 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-routes. 2024-09-17T04:57:58.890 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T04:57:58.932 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T04:57:58.970 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-babel. 2024-09-17T04:57:58.975 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T04:57:59.020 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T04:57:59.302 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T04:57:59.314 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T04:57:59.318 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:59.320 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T04:57:59.360 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:57:59.364 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T04:57:59.734 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T04:57:59.740 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:57:59.784 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:00.078 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T04:58:00.084 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T04:58:00.128 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T04:58:00.414 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-routes. 2024-09-17T04:58:00.421 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T04:58:00.464 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T04:58:00.851 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T04:58:00.857 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:00.901 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:01.824 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T04:58:01.839 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T04:58:01.874 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T04:58:02.349 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T04:58:02.364 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T04:58:02.496 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T04:58:02.858 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T04:58:02.873 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T04:58:02.916 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T04:58:03.080 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T04:58:03.088 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T04:58:03.139 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T04:58:03.236 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T04:58:03.252 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T04:58:03.294 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T04:58:03.642 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T04:58:03.649 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T04:58:03.777 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T04:58:04.036 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T04:58:04.051 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:04.094 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:04.142 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T04:58:04.149 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T04:58:04.192 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T04:58:04.502 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T04:58:04.510 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T04:58:04.552 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T04:58:05.156 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T04:58:05.172 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T04:58:05.214 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T04:58:05.419 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T04:58:05.428 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:05.492 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T04:58:05.502 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:05.508 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T04:58:05.542 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T04:58:05.879 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T04:58:05.895 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T04:58:05.937 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T04:58:06.390 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T04:58:06.407 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T04:58:06.448 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T04:58:06.505 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T04:58:06.513 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T04:58:06.555 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T04:58:06.811 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T04:58:06.827 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T04:58:06.866 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T04:58:06.875 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T04:58:06.877 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T04:58:06.924 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T04:58:07.269 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T04:58:07.278 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T04:58:07.280 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T04:58:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T04:58:07.319 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T04:58:07.347 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T04:58:07.613 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T04:58:07.622 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T04:58:07.663 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T04:58:07.725 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T04:58:07.742 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T04:58:07.791 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T04:58:07.917 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T04:58:07.928 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T04:58:07.966 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T04:58:08.130 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T04:58:08.147 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T04:58:08.189 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T04:58:08.328 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T04:58:08.337 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T04:58:08.378 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T04:58:08.789 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T04:58:08.799 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T04:58:08.838 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T04:58:08.869 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T04:58:08.884 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:08.927 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:09.125 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T04:58:09.135 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T04:58:09.167 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T04:58:09.247 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T04:58:09.262 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T04:58:09.305 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T04:58:09.600 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T04:58:09.616 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T04:58:09.649 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T04:58:09.855 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T04:58:09.864 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:09.905 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:09.936 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package jq. 2024-09-17T04:58:09.952 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T04:58:09.994 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T04:58:10.216 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T04:58:10.225 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T04:58:10.230 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package socat. 2024-09-17T04:58:10.247 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T04:58:10.266 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T04:58:10.288 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T04:58:10.636 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T04:58:10.644 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T04:58:10.685 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T04:58:10.842 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T04:58:10.858 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T04:58:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T04:58:10.955 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package jq. 2024-09-17T04:58:10.963 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T04:58:11.004 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T04:58:11.245 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-test. 2024-09-17T04:58:11.257 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package socat. 2024-09-17T04:58:11.261 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:11.266 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T04:58:11.307 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T04:58:11.312 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:11.602 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T04:58:11.611 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T04:58:11.652 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T04:58:12.030 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-test. 2024-09-17T04:58:12.039 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:12.080 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:13.450 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T04:58:13.466 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:13.516 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:13.879 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T04:58:13.895 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:13.937 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:14.001 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T04:58:14.011 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:14.042 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:14.299 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T04:58:14.316 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T04:58:14.356 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T04:58:14.413 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T04:58:14.423 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:14.454 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T04:58:14.643 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:14.684 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:14.757 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T04:58:14.767 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T04:58:14.799 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T04:58:14.986 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T04:58:15.003 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:15.026 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T04:58:15.036 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:15.045 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:15.076 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:15.273 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T04:58:15.289 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:15.322 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:15.345 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T04:58:15.355 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T04:58:15.395 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:15.625 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T04:58:15.641 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T04:58:15.673 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T04:58:15.682 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:15.683 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T04:58:15.722 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:16.025 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T04:58:16.035 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T04:58:16.047 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T04:58:16.064 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T04:58:16.075 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T04:58:16.112 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T04:58:16.404 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T04:58:16.414 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T04:58:16.442 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T04:58:16.454 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T04:58:16.458 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T04:58:16.500 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T04:58:16.808 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T04:58:16.818 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T04:58:16.858 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T04:58:16.897 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package pkg-config. 2024-09-17T04:58:16.913 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T04:58:16.946 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T04:58:17.191 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T04:58:17.208 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T04:58:17.254 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package pkg-config. 2024-09-17T04:58:17.264 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T04:58:17.266 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T04:58:17.304 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T04:58:17.565 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T04:58:17.574 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T04:58:17.615 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T04:58:17.669 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T04:58:17.685 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T04:58:17.727 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T04:58:18.013 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T04:58:18.029 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T04:58:18.068 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T04:58:18.071 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T04:58:18.077 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T04:58:18.117 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T04:58:18.437 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T04:58:18.447 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T04:58:18.483 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T04:58:18.487 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T04:58:18.500 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T04:58:18.550 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T04:58:18.807 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T04:58:18.816 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T04:58:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T04:58:18.928 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T04:58:18.945 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T04:58:18.994 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T04:58:19.134 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T04:58:19.144 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T04:58:19.201 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T04:58:19.339 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-py. 2024-09-17T04:58:19.356 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T04:58:19.397 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T04:58:19.528 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-py. 2024-09-17T04:58:19.538 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T04:58:19.578 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T04:58:19.726 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T04:58:19.742 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T04:58:19.784 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T04:58:19.906 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T04:58:19.916 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T04:58:19.956 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T04:58:20.221 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T04:58:20.237 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T04:58:20.271 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T04:58:20.410 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T04:58:20.419 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T04:58:20.460 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T04:58:20.607 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-toml. 2024-09-17T04:58:20.624 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T04:58:20.665 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T04:58:20.993 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T04:58:21.010 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T04:58:21.012 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-toml. 2024-09-17T04:58:21.022 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T04:58:21.051 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T04:58:21.062 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T04:58:21.365 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T04:58:21.374 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T04:58:21.405 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T04:58:21.414 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T04:58:21.422 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T04:58:21.463 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T04:58:21.733 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T04:58:21.750 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T04:58:21.776 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T04:58:21.786 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T04:58:21.790 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T04:58:21.826 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T04:58:22.079 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T04:58:22.089 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T04:58:22.129 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T04:58:22.578 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package radosgw. 2024-09-17T04:58:22.594 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:22.636 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:22.865 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package radosgw. 2024-09-17T04:58:22.874 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:22.915 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:23.615 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T04:58:23.631 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:23.673 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:23.760 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T04:58:23.770 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T04:58:23.810 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:23.960 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T04:58:24.100 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T04:58:24.236 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T04:58:24.362 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T04:58:24.364 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T04:58:24.499 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T04:58:24.631 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T04:58:24.763 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T04:58:25.089 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T04:58:25.148 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T04:58:25.720 INFO:teuthology.orchestra.run.smithi016.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T04:58:25.765 INFO:teuthology.orchestra.run.smithi016.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T04:58:25.767 INFO:teuthology.orchestra.run.smithi104.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T04:58:25.813 INFO:teuthology.orchestra.run.smithi104.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-17T04:58:25.814 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:25.859 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:26.333 INFO:teuthology.orchestra.run.smithi016.stdout:Adding system user cephadm....done 2024-09-17T04:58:26.385 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user cephadm....done 2024-09-17T04:58:26.486 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T04:58:26.589 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T04:58:26.866 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T04:58:26.888 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T04:58:27.125 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T04:58:27.181 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T04:58:27.251 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T04:58:27.307 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T04:58:27.519 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T04:58:27.575 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T04:58:27.787 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T04:58:27.884 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T04:58:27.937 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T04:58:27.993 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T04:58:28.272 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T04:58:28.312 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T04:58:28.658 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T04:58:28.681 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T04:58:28.934 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T04:58:28.949 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T04:58:29.194 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T04:58:29.225 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T04:58:29.470 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T04:58:29.543 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T04:58:29.763 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T04:58:29.886 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T04:58:30.162 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T04:58:30.172 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T04:58:30.447 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T04:58:30.448 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T04:58:30.716 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:30.748 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:30.992 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T04:58:31.024 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T04:58:31.101 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T04:58:31.150 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T04:58:31.202 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T04:58:31.328 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T04:58:31.612 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T04:58:31.613 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T04:58:31.761 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T04:58:31.763 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T04:58:32.028 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T04:58:32.040 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T04:58:32.154 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T04:58:32.431 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T04:58:32.493 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T04:58:32.528 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T04:58:32.609 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T04:58:32.735 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T04:58:32.816 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T04:58:32.844 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T04:58:32.934 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T04:58:33.059 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T04:58:33.111 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T04:58:33.237 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T04:58:33.436 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T04:58:33.522 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T04:58:33.561 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T04:58:33.839 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T04:58:33.855 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T04:58:34.131 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T04:58:34.158 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T04:58:34.283 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T04:58:34.449 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T04:58:34.550 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T04:58:34.968 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T04:58:35.260 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T04:58:35.278 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T04:58:35.429 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T04:58:35.528 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T04:58:35.580 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T04:58:35.670 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T04:58:35.805 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T04:58:36.164 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T04:58:36.181 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T04:58:36.574 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T04:58:36.665 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T04:58:36.858 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T04:58:36.925 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T04:58:37.243 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T04:58:37.318 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T04:58:37.544 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T04:58:37.627 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T04:58:37.896 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T04:58:37.962 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T04:58:38.255 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T04:58:38.313 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T04:58:38.523 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T04:58:38.581 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T04:58:38.649 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:38.740 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:38.967 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T04:58:39.050 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T04:58:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T04:58:39.376 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T04:58:40.598 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T04:58:40.615 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T04:58:40.849 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:58:40.867 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:58:40.967 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T04:58:41.001 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T04:58:41.218 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T04:58:41.286 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T04:58:41.343 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T04:58:41.395 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T04:58:41.469 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T04:58:41.496 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T04:58:41.779 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T04:58:41.814 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T04:58:42.047 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T04:58:42.082 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T04:58:42.315 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:58:42.367 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:58:42.458 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T04:58:42.492 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T04:58:42.758 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T04:58:42.811 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T04:58:43.018 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T04:58:43.078 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T04:58:43.285 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T04:58:43.354 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T04:58:43.553 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T04:58:43.631 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T04:58:43.837 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T04:58:43.906 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T04:58:44.121 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T04:58:44.174 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T04:58:44.256 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T04:58:44.300 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T04:58:44.557 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T04:58:44.602 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T04:58:44.699 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T04:58:44.752 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T04:58:44.976 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T04:58:45.045 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T04:58:45.285 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T04:58:45.347 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T04:58:45.569 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T04:58:45.614 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T04:58:45.695 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T04:58:45.715 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T04:58:45.816 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T04:58:45.821 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T04:58:45.909 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T04:58:45.921 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T04:58:46.176 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T04:58:46.181 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T04:58:46.298 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T04:58:46.302 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T04:58:46.683 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T04:58:46.687 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T04:58:46.934 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T04:58:46.947 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T04:58:47.060 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T04:58:47.098 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T04:58:47.328 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:58:47.382 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T04:58:47.454 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T04:58:47.508 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T04:58:47.789 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T04:58:47.834 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T04:58:48.106 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T04:58:48.152 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T04:58:48.509 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T04:58:48.522 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T04:58:48.635 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T04:58:48.648 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T04:58:48.911 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T04:58:48.916 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T04:58:49.192 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T04:58:49.212 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T04:58:49.482 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T04:58:49.493 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T04:58:49.859 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T04:58:49.879 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T04:58:50.160 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T04:58:50.180 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T04:58:50.513 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T04:58:50.549 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T04:58:50.638 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:50.675 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:50.764 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:50.801 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:50.915 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T04:58:50.927 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T04:58:51.218 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:51.264 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:51.344 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:51.389 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:51.470 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T04:58:51.508 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T04:58:51.809 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:51.938 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:51.943 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:52.064 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:52.086 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:52.198 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:52.291 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T04:58:52.291 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T04:58:52.417 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T04:58:52.417 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T04:58:52.929 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.041 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.080 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.150 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.206 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.276 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.332 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.435 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:53.458 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T04:58:53.561 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T04:58:54.812 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:54.840 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:54.937 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:54.949 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.058 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.072 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.176 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.190 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.291 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.302 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.425 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.427 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:55.936 INFO:teuthology.orchestra.run.smithi016.stdout:Adding group ceph....done 2024-09-17T04:58:55.989 INFO:teuthology.orchestra.run.smithi104.stdout:Adding group ceph....done 2024-09-17T04:58:56.357 INFO:teuthology.orchestra.run.smithi016.stdout:Adding system user ceph....done 2024-09-17T04:58:56.390 INFO:teuthology.orchestra.run.smithi104.stdout:Adding system user ceph....done 2024-09-17T04:58:56.559 INFO:teuthology.orchestra.run.smithi104.stdout:Setting system user ceph properties....done 2024-09-17T04:58:56.568 INFO:teuthology.orchestra.run.smithi016.stdout:Setting system user ceph properties....done 2024-09-17T04:58:56.579 INFO:teuthology.orchestra.run.smithi104.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T04:58:56.592 INFO:teuthology.orchestra.run.smithi016.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T04:58:56.723 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T04:58:56.747 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T04:58:57.133 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T04:58:57.183 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T04:58:57.786 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:57.875 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:58.011 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:58:58.011 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:58:58.073 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:58:58.074 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T04:58:58.629 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:58.726 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:58.755 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:58.852 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:58:59.292 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:58:59.293 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:58:59.406 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:58:59.407 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T04:59:00.023 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:00.070 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:00.351 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T04:59:00.453 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T04:59:01.034 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:01.147 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:01.256 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T04:59:01.257 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T04:59:01.376 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T04:59:01.377 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T04:59:01.886 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:02.049 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:02.098 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T04:59:02.098 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T04:59:02.269 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T04:59:02.269 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T04:59:02.720 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:02.933 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:03.149 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T04:59:03.149 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T04:59:03.287 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T04:59:03.287 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T04:59:03.780 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:03.906 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:03.927 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:04.049 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:04.052 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:04.196 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:04.257 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T04:59:04.258 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T04:59:04.414 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T04:59:04.414 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T04:59:04.883 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.025 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.055 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.160 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.198 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.302 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.332 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.475 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T04:59:05.672 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T04:59:05.772 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T04:59:05.862 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T04:59:05.925 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T04:59:05.970 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T04:59:06.147 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T04:59:07.117 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T04:59:07.406 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T04:59:07.795 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T04:59:07.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T04:59:07.876 INFO:teuthology.orchestra.run.smithi104.stdout:Reading package lists... 2024-09-17T04:59:08.059 INFO:teuthology.orchestra.run.smithi104.stdout:Building dependency tree... 2024-09-17T04:59:08.059 INFO:teuthology.orchestra.run.smithi104.stdout:Reading state information... 2024-09-17T04:59:08.232 INFO:teuthology.orchestra.run.smithi016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T04:59:08.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T04:59:08.246 INFO:teuthology.orchestra.run.smithi104.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T04:59:08.246 INFO:teuthology.orchestra.run.smithi104.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T04:59:08.247 INFO:teuthology.orchestra.run.smithi104.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T04:59:08.247 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T04:59:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T04:59:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T04:59:08.247 INFO:teuthology.orchestra.run.smithi104.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T04:59:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T04:59:08.248 INFO:teuthology.orchestra.run.smithi104.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T04:59:08.248 INFO:teuthology.orchestra.run.smithi104.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T04:59:08.250 INFO:teuthology.orchestra.run.smithi104.stdout:The following additional packages will be installed: 2024-09-17T04:59:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T04:59:08.250 INFO:teuthology.orchestra.run.smithi104.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T04:59:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T04:59:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T04:59:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T04:59:08.251 INFO:teuthology.orchestra.run.smithi104.stdout:Suggested packages: 2024-09-17T04:59:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T04:59:08.251 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T04:59:08.252 INFO:teuthology.orchestra.run.smithi104.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T04:59:08.304 INFO:teuthology.orchestra.run.smithi104.stdout:The following NEW packages will be installed: 2024-09-17T04:59:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T04:59:08.305 INFO:teuthology.orchestra.run.smithi104.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T04:59:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T04:59:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T04:59:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T04:59:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T04:59:08.306 INFO:teuthology.orchestra.run.smithi104.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T04:59:08.307 INFO:teuthology.orchestra.run.smithi104.stdout:The following packages will be upgraded: 2024-09-17T04:59:08.308 INFO:teuthology.orchestra.run.smithi104.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T04:59:08.315 INFO:teuthology.orchestra.run.smithi016.stdout:Reading package lists... 2024-09-17T04:59:08.364 INFO:teuthology.orchestra.run.smithi104.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T04:59:08.364 INFO:teuthology.orchestra.run.smithi104.stdout:Need to get 55.8 MB of archives. 2024-09-17T04:59:08.364 INFO:teuthology.orchestra.run.smithi104.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T04:59:08.364 INFO:teuthology.orchestra.run.smithi104.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T04:59:08.497 INFO:teuthology.orchestra.run.smithi104.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T04:59:08.500 INFO:teuthology.orchestra.run.smithi104.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T04:59:08.502 INFO:teuthology.orchestra.run.smithi016.stdout:Building dependency tree... 2024-09-17T04:59:08.503 INFO:teuthology.orchestra.run.smithi016.stdout:Reading state information... 2024-09-17T04:59:08.528 INFO:teuthology.orchestra.run.smithi104.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T04:59:08.528 INFO:teuthology.orchestra.run.smithi104.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T04:59:08.529 INFO:teuthology.orchestra.run.smithi104.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T04:59:08.536 INFO:teuthology.orchestra.run.smithi104.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T04:59:08.537 INFO:teuthology.orchestra.run.smithi104.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T04:59:08.538 INFO:teuthology.orchestra.run.smithi104.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T04:59:08.539 INFO:teuthology.orchestra.run.smithi104.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T04:59:08.540 INFO:teuthology.orchestra.run.smithi104.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T04:59:08.541 INFO:teuthology.orchestra.run.smithi104.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T04:59:08.549 INFO:teuthology.orchestra.run.smithi104.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T04:59:08.550 INFO:teuthology.orchestra.run.smithi104.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T04:59:08.550 INFO:teuthology.orchestra.run.smithi104.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T04:59:08.566 INFO:teuthology.orchestra.run.smithi104.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T04:59:08.694 INFO:teuthology.orchestra.run.smithi016.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T04:59:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T04:59:08.695 INFO:teuthology.orchestra.run.smithi016.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T04:59:08.695 INFO:teuthology.orchestra.run.smithi016.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T04:59:08.695 INFO:teuthology.orchestra.run.smithi016.stdout:The following additional packages will be installed: 2024-09-17T04:59:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T04:59:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T04:59:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T04:59:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T04:59:08.696 INFO:teuthology.orchestra.run.smithi016.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T04:59:08.697 INFO:teuthology.orchestra.run.smithi016.stdout:Suggested packages: 2024-09-17T04:59:08.697 INFO:teuthology.orchestra.run.smithi016.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T04:59:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T04:59:08.698 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T04:59:08.754 INFO:teuthology.orchestra.run.smithi016.stdout:The following NEW packages will be installed: 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T04:59:08.755 INFO:teuthology.orchestra.run.smithi016.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T04:59:08.756 INFO:teuthology.orchestra.run.smithi016.stdout:The following packages will be upgraded: 2024-09-17T04:59:08.757 INFO:teuthology.orchestra.run.smithi016.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T04:59:08.775 INFO:teuthology.orchestra.run.smithi104.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T04:59:08.777 INFO:teuthology.orchestra.run.smithi104.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T04:59:08.778 INFO:teuthology.orchestra.run.smithi104.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T04:59:08.780 INFO:teuthology.orchestra.run.smithi104.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T04:59:08.781 INFO:teuthology.orchestra.run.smithi104.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T04:59:08.785 INFO:teuthology.orchestra.run.smithi104.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T04:59:08.786 INFO:teuthology.orchestra.run.smithi104.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T04:59:08.793 INFO:teuthology.orchestra.run.smithi104.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T04:59:08.794 INFO:teuthology.orchestra.run.smithi104.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T04:59:08.814 INFO:teuthology.orchestra.run.smithi104.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T04:59:08.815 INFO:teuthology.orchestra.run.smithi104.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T04:59:08.825 INFO:teuthology.orchestra.run.smithi104.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T04:59:08.844 INFO:teuthology.orchestra.run.smithi104.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T04:59:08.848 INFO:teuthology.orchestra.run.smithi104.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T04:59:08.849 INFO:teuthology.orchestra.run.smithi104.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T04:59:08.849 INFO:teuthology.orchestra.run.smithi104.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T04:59:08.850 INFO:teuthology.orchestra.run.smithi104.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T04:59:08.861 INFO:teuthology.orchestra.run.smithi104.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T04:59:08.863 INFO:teuthology.orchestra.run.smithi104.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T04:59:08.943 INFO:teuthology.orchestra.run.smithi016.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-17T04:59:08.943 INFO:teuthology.orchestra.run.smithi016.stdout:Need to get 55.8 MB of archives. 2024-09-17T04:59:08.943 INFO:teuthology.orchestra.run.smithi016.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T04:59:08.943 INFO:teuthology.orchestra.run.smithi016.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T04:59:09.007 INFO:teuthology.orchestra.run.smithi104.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T04:59:09.007 INFO:teuthology.orchestra.run.smithi104.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T04:59:09.007 INFO:teuthology.orchestra.run.smithi104.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T04:59:09.007 INFO:teuthology.orchestra.run.smithi104.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T04:59:09.008 INFO:teuthology.orchestra.run.smithi104.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T04:59:09.008 INFO:teuthology.orchestra.run.smithi104.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T04:59:09.020 INFO:teuthology.orchestra.run.smithi104.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T04:59:09.021 INFO:teuthology.orchestra.run.smithi104.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T04:59:09.405 INFO:teuthology.orchestra.run.smithi104.stdout:Preconfiguring packages ... 2024-09-17T04:59:09.557 INFO:teuthology.orchestra.run.smithi104.stdout:Fetched 55.8 MB in 1s (79.2 MB/s) 2024-09-17T04:59:09.563 INFO:teuthology.orchestra.run.smithi016.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T04:59:09.579 INFO:teuthology.orchestra.run.smithi016.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T04:59:09.703 INFO:teuthology.orchestra.run.smithi016.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T04:59:09.704 INFO:teuthology.orchestra.run.smithi016.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T04:59:09.705 INFO:teuthology.orchestra.run.smithi016.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T04:59:09.733 INFO:teuthology.orchestra.run.smithi016.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T04:59:09.735 INFO:teuthology.orchestra.run.smithi016.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T04:59:09.738 INFO:teuthology.orchestra.run.smithi016.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T04:59:09.741 INFO:teuthology.orchestra.run.smithi016.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T04:59:09.741 INFO:teuthology.orchestra.run.smithi016.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T04:59:09.743 INFO:teuthology.orchestra.run.smithi016.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T04:59:09.802 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T04:59:09.806 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T04:59:09.809 INFO:teuthology.orchestra.run.smithi016.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T04:59:09.809 INFO:teuthology.orchestra.run.smithi016.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T04:59:09.809 INFO:teuthology.orchestra.run.smithi016.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T04:59:09.889 INFO:teuthology.orchestra.run.smithi016.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T04:59:09.931 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T04:59:10.563 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T04:59:10.577 INFO:teuthology.orchestra.run.smithi016.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T04:59:10.579 INFO:teuthology.orchestra.run.smithi016.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T04:59:10.580 INFO:teuthology.orchestra.run.smithi016.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T04:59:10.582 INFO:teuthology.orchestra.run.smithi016.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T04:59:10.583 INFO:teuthology.orchestra.run.smithi016.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T04:59:10.587 INFO:teuthology.orchestra.run.smithi016.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T04:59:10.589 INFO:teuthology.orchestra.run.smithi016.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T04:59:10.634 INFO:teuthology.orchestra.run.smithi016.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T04:59:10.634 INFO:teuthology.orchestra.run.smithi016.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T04:59:10.672 INFO:teuthology.orchestra.run.smithi016.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T04:59:10.716 INFO:teuthology.orchestra.run.smithi016.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T04:59:10.729 INFO:teuthology.orchestra.run.smithi016.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T04:59:10.803 INFO:teuthology.orchestra.run.smithi016.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T04:59:10.806 INFO:teuthology.orchestra.run.smithi016.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T04:59:10.807 INFO:teuthology.orchestra.run.smithi016.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T04:59:10.808 INFO:teuthology.orchestra.run.smithi016.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T04:59:10.808 INFO:teuthology.orchestra.run.smithi016.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T04:59:10.829 INFO:teuthology.orchestra.run.smithi016.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T04:59:10.831 INFO:teuthology.orchestra.run.smithi016.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T04:59:11.312 INFO:teuthology.orchestra.run.smithi016.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T04:59:11.313 INFO:teuthology.orchestra.run.smithi016.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T04:59:11.313 INFO:teuthology.orchestra.run.smithi016.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T04:59:11.313 INFO:teuthology.orchestra.run.smithi016.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T04:59:11.313 INFO:teuthology.orchestra.run.smithi016.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T04:59:11.313 INFO:teuthology.orchestra.run.smithi016.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T04:59:11.330 INFO:teuthology.orchestra.run.smithi016.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T04:59:11.331 INFO:teuthology.orchestra.run.smithi016.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T04:59:11.637 INFO:teuthology.orchestra.run.smithi104.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T04:59:11.732 INFO:teuthology.orchestra.run.smithi016.stdout:Preconfiguring packages ... 2024-09-17T04:59:11.906 INFO:teuthology.orchestra.run.smithi016.stdout:Fetched 55.8 MB in 3s (21.7 MB/s) 2024-09-17T04:59:11.918 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package flex. 2024-09-17T04:59:11.988 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T04:59:11.991 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T04:59:12.118 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T04:59:12.194 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T04:59:12.198 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T04:59:12.346 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T04:59:12.653 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T04:59:12.856 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T04:59:12.995 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T04:59:13.238 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T04:59:13.834 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T04:59:13.838 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T04:59:13.984 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T04:59:14.101 INFO:teuthology.orchestra.run.smithi016.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T04:59:14.384 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T04:59:14.425 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package flex. 2024-09-17T04:59:14.500 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143291 files and directories currently installed.) 2024-09-17T04:59:14.504 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T04:59:14.519 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T04:59:14.541 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T04:59:14.826 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T04:59:15.061 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T04:59:15.230 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T04:59:15.244 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package bison. 2024-09-17T04:59:15.317 INFO:teuthology.orchestra.run.smithi104.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T04:59:15.320 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T04:59:15.363 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T04:59:15.664 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T04:59:15.731 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T04:59:15.746 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T04:59:15.789 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T04:59:16.142 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package dump. 2024-09-17T04:59:16.157 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T04:59:16.184 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T04:59:16.187 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T04:59:16.200 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T04:59:16.312 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T04:59:16.512 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package indent. 2024-09-17T04:59:16.527 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:16.570 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T04:59:16.726 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T04:59:16.806 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T04:59:16.822 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T04:59:16.852 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T04:59:16.881 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T04:59:17.186 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T04:59:17.227 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T04:59:17.243 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T04:59:17.285 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T04:59:17.520 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package bison. 2024-09-17T04:59:17.546 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T04:59:17.562 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T04:59:17.598 INFO:teuthology.orchestra.run.smithi016.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143364 files and directories currently installed.) 2024-09-17T04:59:17.602 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T04:59:17.604 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T04:59:17.647 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T04:59:17.908 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T04:59:17.924 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T04:59:17.958 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:18.064 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T04:59:18.080 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T04:59:18.114 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T04:59:18.210 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T04:59:18.226 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T04:59:18.268 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:18.434 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package dump. 2024-09-17T04:59:18.451 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T04:59:18.492 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T04:59:18.521 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T04:59:18.537 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T04:59:18.579 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T04:59:18.871 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package indent. 2024-09-17T04:59:18.888 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:18.933 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T04:59:18.938 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T04:59:18.949 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T04:59:18.990 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T04:59:19.274 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T04:59:19.290 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T04:59:19.340 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T04:59:19.744 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T04:59:19.761 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T04:59:19.811 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T04:59:20.072 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T04:59:20.089 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T04:59:20.139 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T04:59:20.285 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T04:59:20.301 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T04:59:20.335 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:20.575 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T04:59:20.592 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T04:59:20.596 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T04:59:20.612 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T04:59:20.650 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:20.654 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:20.953 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T04:59:20.970 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T04:59:20.973 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T04:59:20.989 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T04:59:21.019 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:21.031 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:21.289 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T04:59:21.306 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T04:59:21.347 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T04:59:21.401 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T04:59:21.417 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T04:59:21.459 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T04:59:21.659 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T04:59:21.676 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T04:59:21.709 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T04:59:21.779 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T04:59:21.795 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T04:59:21.828 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T04:59:22.182 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T04:59:22.198 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:22.611 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:23.002 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T04:59:23.017 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:23.060 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:23.162 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T04:59:23.179 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T04:59:23.220 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:23.354 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T04:59:23.370 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:23.404 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:23.490 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T04:59:23.506 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T04:59:23.548 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:23.640 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T04:59:23.656 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T04:59:23.698 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T04:59:23.884 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T04:59:23.901 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T04:59:23.942 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:24.218 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T04:59:24.234 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T04:59:24.262 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T04:59:24.277 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T04:59:24.279 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T04:59:24.320 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T04:59:24.629 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package modemmanager. 2024-09-17T04:59:24.646 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T04:59:24.656 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T04:59:24.673 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T04:59:24.687 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:24.714 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T04:59:25.185 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T04:59:25.201 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:25.251 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:25.344 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package network-manager. 2024-09-17T04:59:25.360 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T04:59:25.402 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T04:59:25.662 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T04:59:25.679 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:25.729 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:26.018 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package ppp. 2024-09-17T04:59:26.034 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T04:59:26.048 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T04:59:26.065 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:26.107 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:26.396 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T04:59:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T04:59:26.435 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T04:59:26.484 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T04:59:26.722 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T04:59:26.737 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T04:59:26.780 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T04:59:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T04:59:26.979 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T04:59:27.013 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T04:59:27.033 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T04:59:27.048 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T04:59:27.091 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T04:59:27.333 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package modemmanager. 2024-09-17T04:59:27.350 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T04:59:27.377 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T04:59:27.391 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:27.392 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T04:59:27.435 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T04:59:27.891 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T04:59:27.906 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:27.949 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:28.031 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package network-manager. 2024-09-17T04:59:28.048 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T04:59:28.098 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T04:59:28.512 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T04:59:28.526 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T04:59:28.588 INFO:teuthology.orchestra.run.smithi104.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T04:59:28.618 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-common (238) ... 2024-09-17T04:59:29.055 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package ppp. 2024-09-17T04:59:29.072 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T04:59:29.110 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T04:59:29.125 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:29.171 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:29.424 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T04:59:29.752 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T04:59:29.767 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T04:59:29.810 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T04:59:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T04:59:30.079 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T04:59:30.138 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T04:59:30.365 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql. 2024-09-17T04:59:30.378 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T04:59:30.407 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T04:59:30.421 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql (14+238) ... 2024-09-17T04:59:30.423 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T04:59:30.457 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T04:59:30.673 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T04:59:30.688 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T04:59:30.731 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T04:59:30.736 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T04:59:30.753 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:30.975 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T04:59:30.990 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T04:59:31.033 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T04:59:31.316 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T04:59:31.319 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T04:59:31.332 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T04:59:31.334 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T04:59:31.369 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T04:59:31.385 INFO:teuthology.orchestra.run.smithi016.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T04:59:31.415 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-common (238) ... 2024-09-17T04:59:31.638 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package tcl. 2024-09-17T04:59:31.653 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T04:59:31.700 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T04:59:31.873 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T04:59:31.889 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T04:59:31.934 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:31.999 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T04:59:32.014 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T04:59:32.057 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T04:59:32.411 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T04:59:32.426 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T04:59:32.461 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T04:59:32.764 INFO:teuthology.orchestra.run.smithi104.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T04:59:32.779 INFO:teuthology.orchestra.run.smithi104.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T04:59:32.832 INFO:teuthology.orchestra.run.smithi104.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T04:59:33.134 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T04:59:33.151 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql. 2024-09-17T04:59:33.167 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T04:59:33.209 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql (14+238) ... 2024-09-17T04:59:33.321 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T04:59:33.439 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T04:59:33.453 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T04:59:33.469 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T04:59:33.503 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T04:59:33.598 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:33.706 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T04:59:33.722 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T04:59:33.725 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T04:59:33.764 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T04:59:33.851 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T04:59:33.977 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:34.100 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T04:59:34.117 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T04:59:34.158 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T04:59:34.428 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package tcl. 2024-09-17T04:59:34.445 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T04:59:34.491 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T04:59:34.789 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T04:59:34.806 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T04:59:34.839 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T04:59:34.931 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T04:59:35.032 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T04:59:35.202 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T04:59:35.219 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T04:59:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T04:59:35.410 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T04:59:35.410 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T04:59:35.621 INFO:teuthology.orchestra.run.smithi016.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T04:59:35.638 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T04:59:35.689 INFO:teuthology.orchestra.run.smithi016.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T04:59:35.966 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T04:59:36.110 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T04:59:36.202 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T04:59:36.236 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T04:59:36.328 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T04:59:36.345 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T04:59:36.446 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T04:59:36.454 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:36.564 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T04:59:36.580 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T04:59:36.681 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T04:59:36.690 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:36.782 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:36.824 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T04:59:36.950 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T04:59:37.688 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T04:59:37.736 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T04:59:37.815 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T04:59:37.830 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T04:59:38.181 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T04:59:38.181 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T04:59:38.293 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T04:59:38.358 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:38.501 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T04:59:38.627 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T04:59:38.753 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T04:59:38.879 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T04:59:38.968 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T04:59:39.032 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T04:59:39.156 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T04:59:39.266 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T04:59:39.324 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T04:59:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T04:59:39.425 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:39.476 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T04:59:39.543 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T04:59:39.586 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T04:59:39.736 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T04:59:39.737 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T04:59:39.862 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T04:59:39.863 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:39.989 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T04:59:40.194 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout:which means they are currently configured by ifupdown: 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout:- enp3s0f1 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T04:59:40.432 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T04:59:40.564 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T04:59:40.853 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T04:59:40.967 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T04:59:41.013 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T04:59:41.108 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T04:59:41.173 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:41.299 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T04:59:41.402 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T04:59:41.424 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T04:59:41.542 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T04:59:41.662 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T04:59:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T04:59:41.822 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T04:59:41.948 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T04:59:42.139 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:42.158 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T04:59:42.274 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:42.284 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:42.383 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T04:59:42.393 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T04:59:42.509 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-common (238) ... 2024-09-17T04:59:42.528 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T04:59:42.654 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T04:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T04:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T04:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:which means they are currently configured by ifupdown: 2024-09-17T04:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:- ens1f0 2024-09-17T04:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T04:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T04:59:43.250 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T04:59:43.388 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T04:59:43.450 INFO:teuthology.orchestra.run.smithi104.stdout:Adding user postgres to group ssl-cert 2024-09-17T04:59:43.707 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T04:59:43.707 INFO:teuthology.orchestra.run.smithi104.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T04:59:43.797 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T04:59:43.972 INFO:teuthology.orchestra.run.smithi104.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T04:59:43.972 INFO:teuthology.orchestra.run.smithi104.stdout:Removing obsolete dictionary files: 2024-09-17T04:59:44.243 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T04:59:44.725 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T04:59:44.965 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T04:59:45.116 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T04:59:45.202 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:45.242 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T04:59:45.352 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-common (238) ... 2024-09-17T04:59:45.667 INFO:teuthology.orchestra.run.smithi104.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T04:59:45.688 INFO:teuthology.orchestra.run.smithi104.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T04:59:45.737 INFO:teuthology.orchestra.run.smithi104.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T04:59:45.737 INFO:teuthology.orchestra.run.smithi104.stdout:This user must also own the server process. 2024-09-17T04:59:45.737 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T04:59:45.737 INFO:teuthology.orchestra.run.smithi104.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T04:59:45.737 INFO:teuthology.orchestra.run.smithi104.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout:The default text search configuration will be set to "english". 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout:Data page checksums are disabled. 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout:creating subdirectories ... ok 2024-09-17T04:59:45.738 INFO:teuthology.orchestra.run.smithi104.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T04:59:45.762 INFO:teuthology.orchestra.run.smithi104.stdout:selecting default max_connections ... 100 2024-09-17T04:59:45.796 INFO:teuthology.orchestra.run.smithi104.stdout:selecting default shared_buffers ... 128MB 2024-09-17T04:59:45.810 INFO:teuthology.orchestra.run.smithi104.stdout:selecting default time zone ... Etc/UTC 2024-09-17T04:59:45.811 INFO:teuthology.orchestra.run.smithi104.stdout:creating configuration files ... ok 2024-09-17T04:59:45.973 INFO:teuthology.orchestra.run.smithi104.stdout:running bootstrap script ... ok 2024-09-17T04:59:46.277 INFO:teuthology.orchestra.run.smithi016.stdout:Adding user postgres to group ssl-cert 2024-09-17T04:59:46.535 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T04:59:46.535 INFO:teuthology.orchestra.run.smithi016.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T04:59:46.577 INFO:teuthology.orchestra.run.smithi104.stdout:performing post-bootstrap initialization ... ok 2024-09-17T04:59:46.786 INFO:teuthology.orchestra.run.smithi016.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T04:59:46.786 INFO:teuthology.orchestra.run.smithi016.stdout:Removing obsolete dictionary files: 2024-09-17T04:59:47.166 INFO:teuthology.orchestra.run.smithi104.stdout:syncing data to disk ... ok 2024-09-17T04:59:47.572 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T04:59:47.688 INFO:teuthology.orchestra.run.smithi104.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T04:59:48.303 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T04:59:48.657 INFO:teuthology.orchestra.run.smithi016.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T04:59:48.675 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:This user must also own the server process. 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:The default text search configuration will be set to "english". 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:Data page checksums are disabled. 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T04:59:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T04:59:48.734 INFO:teuthology.orchestra.run.smithi016.stdout:creating subdirectories ... ok 2024-09-17T04:59:48.734 INFO:teuthology.orchestra.run.smithi016.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T04:59:48.758 INFO:teuthology.orchestra.run.smithi016.stdout:selecting default max_connections ... 100 2024-09-17T04:59:48.793 INFO:teuthology.orchestra.run.smithi016.stdout:selecting default shared_buffers ... 128MB 2024-09-17T04:59:48.807 INFO:teuthology.orchestra.run.smithi016.stdout:selecting default time zone ... Etc/UTC 2024-09-17T04:59:48.808 INFO:teuthology.orchestra.run.smithi016.stdout:creating configuration files ... ok 2024-09-17T04:59:48.982 INFO:teuthology.orchestra.run.smithi016.stdout:running bootstrap script ... ok 2024-09-17T04:59:49.730 INFO:teuthology.orchestra.run.smithi016.stdout:performing post-bootstrap initialization ... ok 2024-09-17T04:59:50.659 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:50.776 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T04:59:50.819 INFO:teuthology.orchestra.run.smithi016.stdout:syncing data to disk ... ok 2024-09-17T04:59:50.877 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up postgresql (14+238) ... 2024-09-17T04:59:51.003 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:51.129 INFO:teuthology.orchestra.run.smithi104.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:51.335 INFO:teuthology.orchestra.run.smithi016.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T04:59:51.391 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T04:59:51.391 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T04:59:52.173 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T04:59:52.257 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T04:59:52.559 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T04:59:52.711 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T04:59:54.316 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:54.321 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T04:59:54.430 INFO:teuthology.orchestra.run.smithi104.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T04:59:54.442 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T04:59:54.543 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up postgresql (14+238) ... 2024-09-17T04:59:54.644 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T04:59:54.770 INFO:teuthology.orchestra.run.smithi016.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T04:59:54.984 INFO:teuthology.orchestra.run.smithi104.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T04:59:54.991 DEBUG:teuthology.parallel:result is None 2024-09-17T04:59:55.041 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T04:59:55.041 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T04:59:55.823 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T04:59:55.915 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T04:59:56.250 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T04:59:56.436 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T04:59:57.528 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T04:59:57.662 INFO:teuthology.orchestra.run.smithi016.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T04:59:58.222 INFO:teuthology.orchestra.run.smithi016.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T04:59:58.227 DEBUG:teuthology.parallel:result is None 2024-09-17T04:59:58.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:59:58.380 DEBUG:teuthology.orchestra.run.smithi016:> dpkg-query -W -f '${Version}' ceph 2024-09-17T04:59:58.402 INFO:teuthology.orchestra.run.smithi016.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:59:58.403 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:59:58.403 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T04:59:58.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T04:59:58.547 DEBUG:teuthology.orchestra.run.smithi104:> dpkg-query -W -f '${Version}' ceph 2024-09-17T04:59:58.571 INFO:teuthology.orchestra.run.smithi104.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:59:58.571 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T04:59:58.572 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T04:59:58.574 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T04:59:58.574 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:59:58.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T04:59:58.594 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:59:58.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T04:59:58.635 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T04:59:58.635 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:59:58.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T04:59:58.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T04:59:58.718 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:59:58.718 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T04:59:58.738 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T04:59:58.802 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T04:59:58.803 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:59:58.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T04:59:58.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T04:59:58.886 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:59:58.886 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T04:59:58.907 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T04:59:58.969 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T04:59:58.969 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:59:58.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T04:59:58.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T04:59:59.054 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:59:59.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T04:59:59.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T04:59:59.138 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T04:59:59.220 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T04:59:59.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-09-17T04:59:59.224 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /var/log/ceph 2024-09-17T04:59:59.244 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T04:59:59.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T04:59:59.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T04:59:59.311 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T04:59:59.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T04:59:59.353 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T04:59:59.384 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T04:59:59.384 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T04:59:59.384 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T04:59:59.384 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:59:59.384 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T04:59:59.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T04:59:59.513 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T04:59:59.514 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T04:59:59.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T04:59:59.620 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T04:59:59.620 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T04:59:59.620 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908571', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF15aOCHBTeJqfXVgnwVpCgKMKlvj85bulBQ46Bhec8BtKBJKEtWJ0XoJZ0Wkjb/1i6FQ4Ggl/eKYtRHz+Urkwg=', 'smithi104.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrynvarVBzUxJ75CW7hv4BHdDyCOq0phXvfDLsmbiX0eBwHonZIECZOFwmecTpHp9/E/EdYh2bXgfFxwHSTPQc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T04:59:59.621 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T04:59:59.624 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T04:59:59.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T04:59:59.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T04:59:59.693 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T04:59:59.694 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T04:59:59.743 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T04:59:59.743 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-17T04:59:59.795 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T04:59:59.795 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:59:59.795 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 888 Links: 1 2024-09-17T04:59:59.795 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:59:59.796 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-17 04:53:15.605260363 +0000 2024-09-17T04:59:59.796 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-17 04:53:15.341262720 +0000 2024-09-17T04:59:59.796 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-17 04:53:15.341262720 +0000 2024-09-17T04:59:59.796 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-17 04:53:15.341262720 +0000 2024-09-17T04:59:59.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T04:59:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-17T04:59:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-17T04:59:59.856 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000248879 s, 2.1 MB/s 2024-09-17T04:59:59.858 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T04:59:59.908 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-17T04:59:59.959 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T04:59:59.959 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T04:59:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 903 Links: 1 2024-09-17T04:59:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T04:59:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-17 04:53:16.169255326 +0000 2024-09-17T04:59:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-17 04:53:15.893257791 +0000 2024-09-17T04:59:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-17 04:53:15.893257791 +0000 2024-09-17T04:59:59.960 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-17 04:53:15.893257791 +0000 2024-09-17T04:59:59.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T05:00:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-17T05:00:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-17T05:00:00.022 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000248492 s, 2.1 MB/s 2024-09-17T05:00:00.024 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T05:00:00.077 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-17T05:00:00.127 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T05:00:00.127 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:00:00.127 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 923 Links: 1 2024-09-17T05:00:00.127 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:00:00.127 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-17 04:53:16.445252860 +0000 2024-09-17T05:00:00.128 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-17 04:53:16.441252895 +0000 2024-09-17T05:00:00.128 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-17 04:53:16.441252895 +0000 2024-09-17T05:00:00.128 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-17 04:53:16.441252895 +0000 2024-09-17T05:00:00.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T05:00:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-17T05:00:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-17T05:00:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000233002 s, 2.2 MB/s 2024-09-17T05:00:00.190 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T05:00:00.240 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 939 Links: 1 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-17 04:53:17.313245106 +0000 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-17 04:53:17.021247714 +0000 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-17 04:53:17.021247714 +0000 2024-09-17T05:00:00.291 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-17 04:53:17.021247714 +0000 2024-09-17T05:00:00.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T05:00:00.353 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-17T05:00:00.353 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-17T05:00:00.353 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000228325 s, 2.2 MB/s 2024-09-17T05:00:00.354 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T05:00:00.404 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:00.404 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:00.404 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T05:00:00.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T05:00:00.413 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-09-17T05:00:00.463 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T05:00:00.463 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:00:00.463 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 887 Links: 1 2024-09-17T05:00:00.463 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:00:00.463 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-17 04:53:03.111856997 +0000 2024-09-17T05:00:00.464 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-17 04:53:02.863859212 +0000 2024-09-17T05:00:00.464 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-17 04:53:02.863859212 +0000 2024-09-17T05:00:00.464 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-17 04:53:02.863859212 +0000 2024-09-17T05:00:00.464 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T05:00:00.528 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-17T05:00:00.529 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-17T05:00:00.529 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000411537 s, 1.2 MB/s 2024-09-17T05:00:00.530 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T05:00:00.584 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 904 Links: 1 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-17 04:53:03.687851851 +0000 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-17 04:53:03.411854317 +0000 2024-09-17T05:00:00.634 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-17 04:53:03.411854317 +0000 2024-09-17T05:00:00.635 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-17 04:53:03.411854317 +0000 2024-09-17T05:00:00.635 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T05:00:00.696 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-17T05:00:00.696 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-17T05:00:00.696 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.00038093 s, 1.3 MB/s 2024-09-17T05:00:00.698 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T05:00:00.753 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-09-17T05:00:00.802 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 920 Links: 1 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-17 04:53:04.291846457 +0000 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-17 04:53:03.995849102 +0000 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-17 04:53:03.995849102 +0000 2024-09-17T05:00:00.803 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-17 04:53:03.995849102 +0000 2024-09-17T05:00:00.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T05:00:00.864 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-17T05:00:00.864 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-17T05:00:00.865 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000373723 s, 1.4 MB/s 2024-09-17T05:00:00.866 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T05:00:00.921 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-09-17T05:00:00.975 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T05:00:00.975 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T05:00:00.976 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 937 Links: 1 2024-09-17T05:00:00.976 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T05:00:00.976 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-17 04:53:04.591843778 +0000 2024-09-17T05:00:00.976 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-17 04:53:04.587843814 +0000 2024-09-17T05:00:00.976 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-17 04:53:04.587843814 +0000 2024-09-17T05:00:00.976 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-17 04:53:04.587843814 +0000 2024-09-17T05:00:00.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T05:00:01.036 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-17T05:00:01.037 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-17T05:00:01.037 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000391591 s, 1.3 MB/s 2024-09-17T05:00:01.039 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T05:00:01.093 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:01.093 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T05:00:01.093 INFO:tasks.ceph:Generating config... 2024-09-17T05:00:01.095 INFO:tasks.ceph:[client] pid file = 2024-09-17T05:00:01.096 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-17T05:00:01.096 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T05:00:01.096 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T05:00:01.096 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T05:00:01.096 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T05:00:01.096 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T05:00:01.097 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T05:00:01.097 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T05:00:01.097 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T05:00:01.097 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-17T05:00:01.097 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-17T05:00:01.097 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T05:00:01.098 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T05:00:01.099 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T05:00:01.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T05:00:01.159 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T05:00:01.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T05:00:01.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:00:01.258 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.104'), ('mon.c', '[v2:172.21.15.104:3301,v1:172.21.15.104:6790]')] 2024-09-17T05:00:01.259 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.104,[v2:172.21.15.104:3301,v1:172.21.15.104:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T05:00:01.260 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:01.260 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T05:00:01.307 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.104 --addv c '[v2:172.21.15.104:3301,v1:172.21.15.104:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:01.411 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:01.411 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 23ee29f8-0841-48bc-a01a-8eb97d561cfc 2024-09-17T05:00:01.411 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-17T05:00:01.411 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-09-17T05:00:01.411 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 23ee29f8-0841-48bc-a01a-8eb97d561cfc 2024-09-17T05:00:01.411 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-09-17T05:00:01.415354+0000 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-17T05:00:01.415354+0000 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 17 (quincy) 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] mon.c 2024-09-17T05:00:01.412 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T05:00:01.415 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T05:00:01.423 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 23ee29f8-0841-48bc-a01a-8eb97d561cfc... 2024-09-17T05:00:01.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T05:00:01.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T05:00:01.509 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-09-17T05:00:01.510 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-09-17T05:00:01.511 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.512 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.104,[v2:172.21.15.104:3301,v1:172.21.15.104:6790]" 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 23ee29f8-0841-48bc-a01a-8eb97d561cfc 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-09-17T05:00:01.513 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-09-17T05:00:01.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2024-09-17T05:00:01.515 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-17T05:00:01.516 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-09-17T05:00:01.517 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.518 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: mon down mkfs grace = 300 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-09-17T05:00:01.519 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-17T05:00:01.520 INFO:teuthology.orchestra.run.smithi104.stdout:[global] 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: chdir = "" 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: auth supported = cephx 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: filestore xattr use omap = true 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: mon clock drift allowed = 1.000 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: osd crush chooseleaf type = 0 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: auth debug = true 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on old message = true 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on bug = true 2024-09-17T05:00:01.521 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon pg warn max object skew = 0 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default size = 2 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow primary affinity = true 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd allow pg remap = true 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on legacy crush tunables = false 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on crush straw calc version zero = false 2024-09-17T05:00:01.522 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on no sortbitwise = false 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on osd down out interval zero = false 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon warn on too few osds = false 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon_allow_pool_size_one = true 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: osd default data pool replay window = 5 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon allow pool delete = true 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: mon cluster log file level = debug 2024-09-17T05:00:01.523 INFO:teuthology.orchestra.run.smithi104.stdout: debug asserts on shutdown = true 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: mon health detail to clog = false 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: mon host = "172.21.15.16,172.21.15.104,[v2:172.21.15.104:3301,v1:172.21.15.104:6790]" 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: ms die on skipped message = False 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: fsid = 23ee29f8-0841-48bc-a01a-8eb97d561cfc 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout:[osd] 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: osd journal size = 100 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub load threshold = 5.0 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: osd scrub max interval = 600 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: osd mclock profile = high_recovery_ops 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: osd recover clone overlap = true 2024-09-17T05:00:01.524 INFO:teuthology.orchestra.run.smithi104.stdout: osd recovery max chunk = 1048576 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug shutdown = true 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug op order = true 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug verify stray on activate = true 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug trim objects = true 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd open classes on start = true 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug pg log writeout = true 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd deep scrub update digest min age = 30 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: osd map max advance = 10 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.525 INFO:teuthology.orchestra.run.smithi104.stdout: journal zero on create = true 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: filestore ondisk finisher threads = 3 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: filestore apply finisher threads = 3 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: bdev debug aio = true 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: osd debug misdirected ops = true 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore block size = 96636764160 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: bluestore fsck on mount = True 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: debug bluefs = 20 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: debug bluestore = 20 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: debug osd = 20 2024-09-17T05:00:01.526 INFO:teuthology.orchestra.run.smithi104.stdout: debug rocksdb = 10 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd full ratio = 0.9 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd nearfull ratio = 0.8 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: osd failsafe full ratio = 0.95 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: osd objectstore = bluestore 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr] 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: debug mgr = 20 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:00:01.527 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mgr/telemetry/nag = false 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout:[mon] 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: debug ms = 1 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: debug mon = 20 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: debug paxos = 20 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: debug auth = 20 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mon data avail warn = 5 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mon mgr mkfs grace = 240 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min pgs per osd = 4 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd reporter subtree level = osd 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mon osd prime pg temp = true 2024-09-17T05:00:01.528 INFO:teuthology.orchestra.run.smithi104.stdout: mon reweight min bytes per osd = 10 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: # 1m isn't quite enough 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: mon_down_mkfs_grace = 2m 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: mon_warn_on_filestore_osds = false 2024-09-17T05:00:01.529 INFO:teuthology.orchestra.run.smithi104.stdout: mon down mkfs grace = 300 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[client] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: rgw cache enabled = true 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable ops log = true 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: rgw enable usage log = true 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout: pid file = "" 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.a] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.b] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mon.c] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.a] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.c] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.b] 2024-09-17T05:00:01.530 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.d] 2024-09-17T05:00:01.532 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T05:00:01.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T05:00:01.643 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T05:00:01.643 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:01.643 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T05:00:01.652 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:01.652 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T05:00:01.706 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:00:01.706 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:01.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T05:00:01.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:00:01.783 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:01.783 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:01.838 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:00:01.839 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:01.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T05:00:01.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T05:00:01.874 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:01.874 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:01.926 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T05:00:01.927 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T05:00:01.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T05:00:01.993 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T05:00:01.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T05:00:02.070 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T05:00:02.088 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T05:00:02.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T05:00:02.154 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T05:00:02.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T05:00:02.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T05:00:02.288 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T05:00:02.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T05:00:02.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T05:00:02.382 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T05:00:02.387 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T05:00:02.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T05:00:02.514 INFO:teuthology.orchestra.run.smithi104.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T05:00:02.519 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T05:00:02.538 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T05:00:02.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T05:00:02.592 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T05:00:02.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T05:00:02.662 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T05:00:02.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T05:00:02.731 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T05:00:02.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T05:00:02.798 INFO:teuthology.orchestra.run.smithi104.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T05:00:02.815 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T05:00:02.816 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi104.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T05:00:02.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T05:00:02.836 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:02.836 INFO:tasks.ceph:role: osd.0 2024-09-17T05:00:02.836 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:00:02.836 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T05:00:02.897 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:02.898 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:02.922 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-17T05:00:02.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-17T05:00:02.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T05:00:02.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T05:00:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:03.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:03.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T05:00:03.118 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:03.118 INFO:tasks.ceph:role: osd.1 2024-09-17T05:00:03.118 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:00:03.118 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T05:00:03.177 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:03.178 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:03.202 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-17T05:00:03.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-17T05:00:03.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T05:00:03.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T05:00:03.330 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:03.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:03.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T05:00:03.393 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:03.393 INFO:tasks.ceph:role: osd.2 2024-09-17T05:00:03.394 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:00:03.394 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T05:00:03.458 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:03.459 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:03.483 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-17T05:00:03.486 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-17T05:00:03.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T05:00:03.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T05:00:03.615 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:03.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:03.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T05:00:03.678 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:03.678 INFO:tasks.ceph:role: osd.3 2024-09-17T05:00:03.678 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:00:03.678 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:03.738 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:03.739 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:03.762 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-09-17T05:00:03.765 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-09-17T05:00:03.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T05:00:03.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T05:00:03.890 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:03.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:03.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:04.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:04.159+0000 7f88243a27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T05:00:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:04.159+0000 7f88243a27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T05:00:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:04.159+0000 7f88243a27c0 -1 bdev(0x557522a42a00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:04.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:04.163+0000 7f88243a27c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T05:00:07.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T05:00:07.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:07.527+0000 7fbe02e397c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T05:00:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:07.527+0000 7fbe02e397c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T05:00:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:07.527+0000 7fbe02e397c0 -1 bdev(0x56452b36ea00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:07.527+0000 7fbe02e397c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T05:00:10.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T05:00:10.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:10.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:10.895+0000 7f1e598697c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T05:00:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:10.895+0000 7f1e598697c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T05:00:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:10.895+0000 7f1e598697c0 -1 bdev(0x55cdb930ca00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:10.899+0000 7f1e598697c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T05:00:14.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T05:00:14.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:14.299+0000 7f98624297c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T05:00:14.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:14.303+0000 7f98624297c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T05:00:14.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:14.303+0000 7f98624297c0 -1 bdev(0x563711dc2a00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:14.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:14.303+0000 7f98624297c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T05:00:17.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T05:00:17.599 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T05:00:17.621 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:17.621 INFO:tasks.ceph:role: osd.4 2024-09-17T05:00:17.622 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:00:17.622 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T05:00:17.690 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:17.690 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:17.691 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:18.090 INFO:teuthology.orchestra.run.smithi104.stdout:Discarding blocks...Done. 2024-09-17T05:00:18.093 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-09-17T05:00:18.093 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T05:00:18.189 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T05:00:18.207 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:18.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:18.208 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T05:00:18.274 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:18.274 INFO:tasks.ceph:role: osd.5 2024-09-17T05:00:18.274 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:00:18.274 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T05:00:18.348 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:18.348 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:18.348 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:18.348 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:18.349 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:18.349 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:18.349 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:18.349 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:18.349 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:18.349 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:18.750 INFO:teuthology.orchestra.run.smithi104.stdout:Discarding blocks...Done. 2024-09-17T05:00:18.753 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-09-17T05:00:18.753 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T05:00:18.903 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T05:00:18.919 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:18.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:18.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T05:00:18.988 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:18.988 INFO:tasks.ceph:role: osd.6 2024-09-17T05:00:18.988 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:00:18.988 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T05:00:19.060 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:19.061 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:19.486 INFO:teuthology.orchestra.run.smithi104.stdout:Discarding blocks...Done. 2024-09-17T05:00:19.489 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-09-17T05:00:19.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T05:00:19.623 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T05:00:19.639 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:19.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:19.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T05:00:19.701 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T05:00:19.701 INFO:tasks.ceph:role: osd.7 2024-09-17T05:00:19.701 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:00:19.702 DEBUG:teuthology.orchestra.run.smithi104:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout: = sunit=0 swidth=0 blks 2024-09-17T05:00:19.768 INFO:teuthology.orchestra.run.smithi104.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T05:00:19.769 INFO:teuthology.orchestra.run.smithi104.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T05:00:19.769 INFO:teuthology.orchestra.run.smithi104.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T05:00:19.769 INFO:teuthology.orchestra.run.smithi104.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T05:00:20.134 INFO:teuthology.orchestra.run.smithi104.stdout:Discarding blocks...Done. 2024-09-17T05:00:20.137 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi104.front.sepia.ceph.com -o noatime 2024-09-17T05:00:20.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T05:00:20.218 DEBUG:teuthology.orchestra.run.smithi104:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T05:00:20.278 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T05:00:20.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:00:20.279 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:20.429 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:20.426+0000 7f7f2b3757c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T05:00:20.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:20.430+0000 7f7f2b3757c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T05:00:20.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:20.430+0000 7f7f2b3757c0 -1 bdev(0x55e471ecea00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:20.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:20.430+0000 7f7f2b3757c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T05:00:23.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T05:00:23.750 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:23.863 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:23.862+0000 7f07b462b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T05:00:23.864 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:23.862+0000 7f07b462b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T05:00:23.864 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:23.862+0000 7f07b462b7c0 -1 bdev(0x55737284aa00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:23.864 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:23.862+0000 7f07b462b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T05:00:27.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T05:00:27.118 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:27.231 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:27.230+0000 7f940f25b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T05:00:27.232 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:27.230+0000 7f940f25b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T05:00:27.232 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:27.230+0000 7f940f25b7c0 -1 bdev(0x556fbc918a00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:27.232 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:27.230+0000 7f940f25b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T05:00:29.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:00:29.626 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:00:30.477 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T05:00:30.497 DEBUG:teuthology.orchestra.run.smithi104:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:30.600 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:30.598+0000 7f96b97b27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T05:00:30.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:30.598+0000 7f96b97b27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T05:00:30.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:30.598+0000 7f96b97b27c0 -1 bdev(0x555d87b3ea00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T05:00:30.601 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:30.598+0000 7f96b97b27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T05:00:33.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T05:00:33.893 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T05:00:33.898 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:33.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T05:00:33.920 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:33.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T05:00:33.987 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:33.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T05:00:34.051 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:34.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T05:00:34.119 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:34.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T05:00:34.187 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:34.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T05:00:34.255 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:34.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T05:00:34.326 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T05:00:34.347 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.348 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T05:00:34.415 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.415 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T05:00:34.482 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.482 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T05:00:34.550 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.550 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T05:00:34.618 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T05:00:34.677 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.678 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T05:00:34.741 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:34.742 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T05:00:34.751 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:34.751 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T05:00:34.803 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.803 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T05:00:34.811 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:34.811 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T05:00:34.862 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T05:00:34.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T05:00:34.865 DEBUG:teuthology.orchestra.run.smithi104:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg== 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg== 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg== 2024-09-17T05:00:34.910 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg== 2024-09-17T05:00:34.911 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw== 2024-09-17T05:00:34.912 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2024-09-17T05:00:34.913 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ== 2024-09-17T05:00:34.924 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.y] 2024-09-17T05:00:34.924 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg== 2024-09-17T05:00:34.924 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.a] 2024-09-17T05:00:34.924 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg== 2024-09-17T05:00:34.924 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.c] 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg== 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.0] 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng== 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.1] 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg== 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.2] 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg== 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.3] 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q== 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout:[mgr.x] 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g== 2024-09-17T05:00:34.925 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.b] 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA== 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout:[mds.d] 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg== 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.4] 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg== 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.5] 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ== 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.6] 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w== 2024-09-17T05:00:34.926 INFO:teuthology.orchestra.run.smithi104.stdout:[osd.7] 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA== 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout:[client.0] 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout:[client.2] 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg== 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout:[client.1] 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw== 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout:[client.3] 2024-09-17T05:00:34.927 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ== 2024-09-17T05:00:34.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:00:34.966 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:00:35.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.044 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.258 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.395 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:00:35.463 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T05:00:35.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.530 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.602 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T05:00:35.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.672 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.878 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T05:00:35.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:35.947 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.017 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.088 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.159 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T05:00:36.226 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T05:00:36.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T05:00:36.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:00:36.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T05:00:36.389 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T05:00:36.408 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:00:36.531 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T05:00:36.550 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T05:00:36.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T05:00:36.731 DEBUG:teuthology.orchestra.run.smithi104:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T05:00:36.751 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:36.754 DEBUG:teuthology.orchestra.run.smithi104:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T05:00:36.802 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T05:00:36.802 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T05:00:36.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T05:00:36.805 INFO:tasks.ceph.mon.a:Started 2024-09-17T05:00:36.806 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T05:00:36.806 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T05:00:36.845 INFO:tasks.ceph.mon.b:Started 2024-09-17T05:00:36.845 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T05:00:36.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T05:00:36.848 INFO:tasks.ceph.mon.c:Started 2024-09-17T05:00:36.848 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T05:00:36.848 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T05:00:36.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T05:00:36.852 INFO:tasks.ceph.mgr.y:Started 2024-09-17T05:00:36.852 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T05:00:36.852 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T05:00:36.855 INFO:tasks.ceph.mgr.x:Started 2024-09-17T05:00:36.855 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T05:00:36.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T05:00:36.858 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T05:00:36.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T05:00:36.906 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T05:00:36.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T05:00:36.958 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T05:00:36.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T05:00:37.005 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T05:00:37.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T05:00:37.049 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T05:00:37.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T05:00:37.093 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T05:00:37.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T05:00:37.137 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T05:00:37.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T05:00:37.181 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T05:00:37.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T05:00:37.226 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T05:00:37.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T05:00:37.278 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T05:00:37.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T05:00:37.326 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T05:00:37.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T05:00:37.377 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T05:00:37.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T05:00:37.426 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T05:00:37.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T05:00:37.473 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T05:00:37.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T05:00:37.521 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T05:00:37.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T05:00:37.570 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T05:00:37.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T05:00:37.622 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T05:00:37.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T05:00:37.669 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T05:00:37.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T05:00:37.714 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T05:00:37.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T05:00:37.761 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T05:00:37.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T05:00:37.809 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T05:00:37.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T05:00:37.857 DEBUG:tasks.ceph:set 22 configs 2024-09-17T05:00:37.858 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi016') 2024-09-17T05:00:42.607 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-17T05:00:42.704 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.202+0000 7f45b6a00640 1 -- 172.21.15.16:0/3817976761 <== mon.2 v2:172.21.15.104:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) ==== 109+0+0 (secure 0 0 0) 0x7f45b001e8a0 con 0x7f45a0003bb0 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 -- 172.21.15.16:0/3817976761 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f45a0003bb0 msgr2=0x7f45c01489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 --2- 172.21.15.16:0/3817976761 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f45a0003bb0 0x7f45c01489d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f45b0003eb0 tx=0x7f45b0014830 comp rx=0 tx=0).stop 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 -- 172.21.15.16:0/3817976761 shutdown_connections 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 --2- 172.21.15.16:0/3817976761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45a0007720 0x7f45c00cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 --2- 172.21.15.16:0/3817976761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45a0006c50 0x7f45c00cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 --2- 172.21.15.16:0/3817976761 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f45a0003bb0 0x7f45c01489d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 -- 172.21.15.16:0/3817976761 >> 172.21.15.16:0/3817976761 conn(0x7f45c0073ca0 msgr2=0x7f45c01eb130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 -- 172.21.15.16:0/3817976761 shutdown_connections 2024-09-17T05:00:43.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.206+0000 7f45bfe00640 1 -- 172.21.15.16:0/3817976761 wait complete. 2024-09-17T05:00:43.224 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.246+0000 7f717de00640 1 -- 172.21.15.16:0/35352583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f7178074f50 con 0x7f71580065f0 2024-09-17T05:00:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.246+0000 7f7174c00640 1 -- 172.21.15.16:0/35352583 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v21)=0 v21) ==== 123+0+0 (secure 0 0 0) 0x7f717001cb10 con 0x7f71580065f0 2024-09-17T05:00:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f7174c00640 1 -- 172.21.15.16:0/35352583 <== mon.0 v2:172.21.15.16:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f717001d8e0 con 0x7f71580065f0 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 -- 172.21.15.16:0/35352583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71580065f0 msgr2=0x7f7178155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 --2- 172.21.15.16:0/35352583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71580065f0 0x7f7178155320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7158006160 tx=0x7f7170001af0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 -- 172.21.15.16:0/35352583 shutdown_connections 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 --2- 172.21.15.16:0/35352583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71580065f0 0x7f7178155320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 --2- 172.21.15.16:0/35352583 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7158004b20 0x7f717815f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 --2- 172.21.15.16:0/35352583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7158004020 0x7f71781524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 -- 172.21.15.16:0/35352583 >> 172.21.15.16:0/35352583 conn(0x7f7178073ca0 msgr2=0x7f717814ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 -- 172.21.15.16:0/35352583 shutdown_connections 2024-09-17T05:00:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.250+0000 7f717de00640 1 -- 172.21.15.16:0/35352583 wait complete. 2024-09-17T05:00:43.269 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.269 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.269 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.269 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.269 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.269 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 -- 172.21.15.16:0/3804549019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64007d70 msgr2=0x7fcf64008180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 --2- 172.21.15.16:0/3804549019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64007d70 0x7fcf64008180 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcf70006bf0 tx=0x7fcf7001d790 comp rx=0 tx=0).stop 2024-09-17T05:00:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 -- 172.21.15.16:0/3804549019 shutdown_connections 2024-09-17T05:00:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 --2- 172.21.15.16:0/3804549019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64007d70 0x7fcf64008180 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 -- 172.21.15.16:0/3804549019 >> 172.21.15.16:0/3804549019 conn(0x7fcf840b2dd0 msgr2=0x7fcf840b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 -- 172.21.15.16:0/3804549019 shutdown_connections 2024-09-17T05:00:43.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.442+0000 7fcf8ae00640 1 -- 172.21.15.16:0/3804549019 wait complete. 2024-09-17T05:00:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 Processor -- start 2024-09-17T05:00:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 -- start start 2024-09-17T05:00:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64003b10 0x7fcf841589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 0x7fcf84157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf64008870 0x7fcf840b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf840c61c0 con 0x7fcf64007d70 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcf840c6040 con 0x7fcf64003b10 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.446+0000 7fcf8ae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcf840c6340 con 0x7fcf64008870 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 0x7fcf84157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 0x7fcf84157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32926/0 (socket says 172.21.15.16:32926) 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 -- 172.21.15.16:0/1663571102 learned_addr learned my addr 172.21.15.16:0/1663571102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83e00640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64003b10 0x7fcf841589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf88e00640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf64008870 0x7fcf840b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 -- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf64008870 msgr2=0x7fcf840b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf64008870 0x7fcf840b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 -- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64003b10 msgr2=0x7fcf841589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64003b10 0x7fcf841589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 -- 172.21.15.16:0/1663571102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf84158340 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf83400640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 0x7fcf84157e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001e030 tx=0x7fcf700a6b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700bf070 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf70003b20 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf840b4250 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700ac040 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf840b47f0 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf700c2070 con 0x7fcf64007d70 2024-09-17T05:00:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf700c4a80 con 0x7fcf64007d70 2024-09-17T05:00:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.450+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf84073f30 con 0x7fcf64007d70 2024-09-17T05:00:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.454+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcf70017560 con 0x7fcf64007d70 2024-09-17T05:00:43.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.514+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fcf84074b80 con 0x7fcf64007d70 2024-09-17T05:00:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf7be00640 1 -- 172.21.15.16:0/1663571102 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fcf700c4cd0 con 0x7fcf64007d70 2024-09-17T05:00:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 msgr2=0x7fcf84157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 0x7fcf84157e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf7001e030 tx=0x7fcf700a6b30 comp rx=0 tx=0).stop 2024-09-17T05:00:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 shutdown_connections 2024-09-17T05:00:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf64008870 0x7fcf840b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf64007d70 0x7fcf84157e00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 --2- 172.21.15.16:0/1663571102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf64003b10 0x7fcf841589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 >> 172.21.15.16:0/1663571102 conn(0x7fcf840b2dd0 msgr2=0x7fcf840ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 shutdown_connections 2024-09-17T05:00:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.518+0000 7fcf8ae00640 1 -- 172.21.15.16:0/1663571102 wait complete. 2024-09-17T05:00:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 -- 172.21.15.16:0/3431329885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44007d30 msgr2=0x7eff44008140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 --2- 172.21.15.16:0/3431329885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44007d30 0x7eff44008140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff50006bf0 tx=0x7eff5001d830 comp rx=0 tx=0).stop 2024-09-17T05:00:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 -- 172.21.15.16:0/3431329885 shutdown_connections 2024-09-17T05:00:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 --2- 172.21.15.16:0/3431329885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44007d30 0x7eff44008140 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 -- 172.21.15.16:0/3431329885 >> 172.21.15.16:0/3431329885 conn(0x7eff640b2dd0 msgr2=0x7eff640b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 -- 172.21.15.16:0/3431329885 shutdown_connections 2024-09-17T05:00:43.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 -- 172.21.15.16:0/3431329885 wait complete. 2024-09-17T05:00:43.538 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.538 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 Processor -- start 2024-09-17T05:00:43.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.538+0000 7eff6b200640 1 -- start start 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44003b10 0x7eff641f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff44007d30 0x7eff641f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 0x7eff641f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7eff64074150 con 0x7eff44008830 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff64073fd0 con 0x7eff44003b10 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7eff640742d0 con 0x7eff44007d30 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff69200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 0x7eff641f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff44007d30 0x7eff641f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff69200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 0x7eff641f9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32932/0 (socket says 172.21.15.16:32932) 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff69200640 1 -- 172.21.15.16:0/527437976 learned_addr learned my addr 172.21.15.16:0/527437976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:43.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63e00640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44003b10 0x7eff641f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 -- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44003b10 msgr2=0x7eff641f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44003b10 0x7eff641f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 -- 172.21.15.16:0/527437976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 msgr2=0x7eff641f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 0x7eff641f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 -- 172.21.15.16:0/527437976 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff640ca510 con 0x7eff44007d30 2024-09-17T05:00:43.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff69200640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 0x7eff641f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff63400640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff44007d30 0x7eff641f8e40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff440036b0 tx=0x7eff500042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500ac040 con 0x7eff44007d30 2024-09-17T05:00:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff50003600 con 0x7eff44007d30 2024-09-17T05:00:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50003a30 con 0x7eff44007d30 2024-09-17T05:00:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff640ca990 con 0x7eff44007d30 2024-09-17T05:00:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7eff640caf50 con 0x7eff44007d30 2024-09-17T05:00:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff500c1030 con 0x7eff44007d30 2024-09-17T05:00:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff500bab60 con 0x7eff44007d30 2024-09-17T05:00:43.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.542+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff640ca6e0 con 0x7eff44007d30 2024-09-17T05:00:43.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.546+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7eff50017580 con 0x7eff44007d30 2024-09-17T05:00:43.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.606+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7eff6415f040 con 0x7eff44007d30 2024-09-17T05:00:43.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff5be00640 1 -- 172.21.15.16:0/527437976 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v23)=0 v23) ==== 119+0+0 (secure 0 0 0) 0x7eff500ba430 con 0x7eff44007d30 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff44007d30 msgr2=0x7eff641f8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff44007d30 0x7eff641f8e40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7eff440036b0 tx=0x7eff500042f0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 shutdown_connections 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff44008830 0x7eff641f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7eff44007d30 0x7eff641f8e40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 --2- 172.21.15.16:0/527437976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff44003b10 0x7eff641f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 >> 172.21.15.16:0/527437976 conn(0x7eff640b2dd0 msgr2=0x7eff6406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 shutdown_connections 2024-09-17T05:00:43.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.614+0000 7eff6b200640 1 -- 172.21.15.16:0/527437976 wait complete. 2024-09-17T05:00:43.633 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.633 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.633 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.633 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.633 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.633 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.634 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.634 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.634 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.634 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-17T05:00:43.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config dump 2024-09-17T05:00:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 -- 172.21.15.16:0/3035146143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 msgr2=0x7f752015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 --2- 172.21.15.16:0/3035146143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752015cab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7510009900 tx=0x7f751001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 -- 172.21.15.16:0/3035146143 shutdown_connections 2024-09-17T05:00:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 --2- 172.21.15.16:0/3035146143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75200d1580 0x7f75200d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 --2- 172.21.15.16:0/3035146143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752015cab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 -- 172.21.15.16:0/3035146143 >> 172.21.15.16:0/3035146143 conn(0x7f75200d2270 msgr2=0x7f75200d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 -- 172.21.15.16:0/3035146143 shutdown_connections 2024-09-17T05:00:43.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.830+0000 7f7526a00640 1 -- 172.21.15.16:0/3035146143 wait complete. 2024-09-17T05:00:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 Processor -- start 2024-09-17T05:00:43.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 -- start start 2024-09-17T05:00:43.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752006fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75200d0010 0x7f75201f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75200d1580 0x7f752006e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f752016a530 con 0x7f75200cc8c0 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f752016a3b0 con 0x7f75200d1580 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f752016a6b0 con 0x7f75200d0010 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752006fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752006fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32946/0 (socket says 172.21.15.16:32946) 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 -- 172.21.15.16:0/50919914 learned_addr learned my addr 172.21.15.16:0/50919914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7517e00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75200d0010 0x7f75201f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7524a00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75200d1580 0x7f752006e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 -- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75200d0010 msgr2=0x7f75201f9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75200d0010 0x7f75201f9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 -- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75200d1580 msgr2=0x7f752006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75200d1580 0x7f752006e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 -- 172.21.15.16:0/50919914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75201fa260 con 0x7f75200cc8c0 2024-09-17T05:00:43.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751fe00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752006fc70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7510009850 tx=0x7f75100a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75100b2040 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75100043b0 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510005290 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7520156960 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7520150700 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f75100bd060 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.834+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7510003ad0 con 0x7f75200cc8c0 2024-09-17T05:00:43.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.838+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752015cab0 con 0x7f75200cc8c0 2024-09-17T05:00:43.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.838+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f75201fa260 con 0x7f75200cc8c0 2024-09-17T05:00:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f75200d04c0 con 0x7f75200cc8c0 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f751cc00640 1 -- 172.21.15.16:0/50919914 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f75100164d0 con 0x7f75200cc8c0 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:00:43.901 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:00:43.902 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:00:43.903 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:00:43.903 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:00:43.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 msgr2=0x7f752006fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752006fc70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7510009850 tx=0x7f75100a5730 comp rx=0 tx=0).stop 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 shutdown_connections 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75200d1580 0x7f752006e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75200d0010 0x7f75201f9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 --2- 172.21.15.16:0/50919914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75200cc8c0 0x7f752006fc70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 >> 172.21.15.16:0/50919914 conn(0x7f75200d2270 msgr2=0x7f7520151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 shutdown_connections 2024-09-17T05:00:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:43.901+0000 7f7526a00640 1 -- 172.21.15.16:0/50919914 wait complete. 2024-09-17T05:00:43.921 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T05:00:43.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 -- 172.21.15.16:0/1279674206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc0cce00 msgr2=0x7ff3dc0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/1279674206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc0cce00 0x7ff3dc0cd210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc0ccbc0 tx=0x7ff3cc01c800 comp rx=0 tx=0).stop 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 -- 172.21.15.16:0/1279674206 shutdown_connections 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/1279674206 >> v2:172.21.15.104:3300/0 conn(0x7ff3dc073dc0 0x7ff3dc0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/1279674206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc0cce00 0x7ff3dc0cd210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 -- 172.21.15.16:0/1279674206 >> 172.21.15.16:0/1279674206 conn(0x7ff3dc0b2dd0 msgr2=0x7ff3dc0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 -- 172.21.15.16:0/1279674206 shutdown_connections 2024-09-17T05:00:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.077+0000 7ff3e0a00640 1 -- 172.21.15.16:0/1279674206 wait complete. 2024-09-17T05:00:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 Processor -- start 2024-09-17T05:00:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 -- start start 2024-09-17T05:00:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3dc073dc0 0x7ff3dc0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff3dc0cce00 0x7ff3dc0b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 0x7ff3dc0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc168d50 con 0x7ff3dc15f040 2024-09-17T05:00:44.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc168bd0 con 0x7ff3dc073dc0 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3e0a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff3dc168ed0 con 0x7ff3dc0cce00 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3da800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 0x7ff3dc0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3da800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 0x7ff3dc0b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32966/0 (socket says 172.21.15.16:32966) 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.081+0000 7ff3da800640 1 -- 172.21.15.16:0/2567684758 learned_addr learned my addr 172.21.15.16:0/2567684758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d3e00640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff3dc0cce00 0x7ff3dc0b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d9e00640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3dc073dc0 0x7ff3dc0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3da800640 1 -- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff3dc0cce00 msgr2=0x7ff3dc0b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3da800640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff3dc0cce00 0x7ff3dc0b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3da800640 1 -- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3dc073dc0 msgr2=0x7ff3dc0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3da800640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3dc073dc0 0x7ff3dc0b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3da800640 1 -- 172.21.15.16:0/2567684758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc1f9680 con 0x7ff3dc15f040 2024-09-17T05:00:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3da800640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 0x7ff3dc0b6a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400cae0 tx=0x7ff3d400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4016020 con 0x7ff3dc15f040 2024-09-17T05:00:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3d400ab40 con 0x7ff3dc15f040 2024-09-17T05:00:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3dc14e190 con 0x7ff3dc15f040 2024-09-17T05:00:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400ce20 con 0x7ff3dc15f040 2024-09-17T05:00:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3dc15ccc0 con 0x7ff3dc15f040 2024-09-17T05:00:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff3d4019020 con 0x7ff3dc15f040 2024-09-17T05:00:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff3d401b9d0 con 0x7ff3dc15f040 2024-09-17T05:00:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.085+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3dc15cf20 con 0x7ff3dc15f040 2024-09-17T05:00:44.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.089+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff3d4020030 con 0x7ff3dc15f040 2024-09-17T05:00:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.153+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff3dc0b48c0 con 0x7ff3dc15f040 2024-09-17T05:00:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3d2000640 1 -- 172.21.15.16:0/2567684758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff3dc074710 con 0x7ff3dc15f040 2024-09-17T05:00:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 msgr2=0x7ff3dc0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 0x7ff3dc0b6a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400cae0 tx=0x7ff3d400cbd0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 shutdown_connections 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3dc15f040 0x7ff3dc0b6a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff3dc0cce00 0x7ff3dc0b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 --2- 172.21.15.16:0/2567684758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3dc073dc0 0x7ff3dc0b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 >> 172.21.15.16:0/2567684758 conn(0x7ff3dc0b2dd0 msgr2=0x7ff3dc15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 shutdown_connections 2024-09-17T05:00:44.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:44.157+0000 7ff3e0a00640 1 -- 172.21.15.16:0/2567684758 wait complete. 2024-09-17T05:00:44.178 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T05:00:44.178 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T05:00:44.179 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:44.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T05:00:44.199 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:44.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T05:00:44.259 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:44.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T05:00:44.319 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:00:44.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T05:00:44.383 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:44.383 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T05:00:44.402 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:44.403 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T05:00:44.466 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:44.466 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T05:00:44.526 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:00:44.526 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T05:00:44.585 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 0b146e62-e5a9-4933-a5f7-95e1c4231514 0 2024-09-17T05:00:44.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 -- 172.21.15.104:0/444632363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec815c490 msgr2=0x7feec8073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 --2- 172.21.15.104:0/444632363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec815c490 0x7feec8073dc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009840 tx=0x7feeb001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 -- 172.21.15.104:0/444632363 shutdown_connections 2024-09-17T05:00:44.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 --2- 172.21.15.104:0/444632363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec815c490 0x7feec8073dc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 -- 172.21.15.104:0/444632363 >> 172.21.15.104:0/444632363 conn(0x7feec80b2dd0 msgr2=0x7feec80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:44.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 -- 172.21.15.104:0/444632363 shutdown_connections 2024-09-17T05:00:44.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.777+0000 7feecf400640 1 -- 172.21.15.104:0/444632363 wait complete. 2024-09-17T05:00:44.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 Processor -- start 2024-09-17T05:00:44.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 -- start start 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 0x7feec8147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec8074300 0x7feec8151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feec815c490 0x7feec81521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feec816b220 con 0x7feec8074300 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feec816b0a0 con 0x7feec806f5f0 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecf400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7feec816b3a0 con 0x7feec815c490 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 0x7feec8147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 0x7feec8147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47892/0 (socket says 172.21.15.104:47892) 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 -- 172.21.15.104:0/2892447339 learned_addr learned my addr 172.21.15.104:0/2892447339 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feebfe00640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec8074300 0x7feec8151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecd400640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feec815c490 0x7feec81521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 -- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feec815c490 msgr2=0x7feec81521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feec815c490 0x7feec81521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 -- 172.21.15.104:0/2892447339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec8074300 msgr2=0x7feec8151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec8074300 0x7feec8151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 -- 172.21.15.104:0/2892447339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec80c8260 con 0x7feec806f5f0 2024-09-17T05:00:44.784 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.781+0000 7feecca00640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 0x7feec8147320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c7f0 tx=0x7feeb800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8017070 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb800a640 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec80c8500 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb800aa70 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec80c8ac0 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feeb801a070 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7feeb801c780 con 0x7feec806f5f0 2024-09-17T05:00:44.785 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec8073dc0 con 0x7feec806f5f0 2024-09-17T05:00:44.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.785+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7feeb8021020 con 0x7feec806f5f0 2024-09-17T05:00:44.855 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.853+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0b146e62-e5a9-4933-a5f7-95e1c4231514", "id": 0} v 0) -- 0x7feec8074b80 con 0x7feec806f5f0 2024-09-17T05:00:44.859 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feec5600640 1 -- 172.21.15.104:0/2892447339 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0b146e62-e5a9-4933-a5f7-95e1c4231514", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7feec8074710 con 0x7feec806f5f0 2024-09-17T05:00:44.859 INFO:teuthology.orchestra.run.smithi104.stdout:0 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 msgr2=0x7feec8147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 0x7feec8147320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c7f0 tx=0x7feeb800ccb0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 shutdown_connections 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feec815c490 0x7feec81521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec8074300 0x7feec8151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 --2- 172.21.15.104:0/2892447339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feec806f5f0 0x7feec8147320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 >> 172.21.15.104:0/2892447339 conn(0x7feec80b2dd0 msgr2=0x7feec815a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 shutdown_connections 2024-09-17T05:00:44.861 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:44.861+0000 7feecf400640 1 -- 172.21.15.104:0/2892447339 wait complete. 2024-09-17T05:00:44.879 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new e77bd243-cf7e-422c-8f72-6b796df25a4e 1 2024-09-17T05:00:45.065 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 -- 172.21.15.104:0/1019466099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 msgr2=0x7fd8f00cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 --2- 172.21.15.104:0/1019466099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f00cb760 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009900 tx=0x7fd8e401c820 comp rx=0 tx=0).stop 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 -- 172.21.15.104:0/1019466099 shutdown_connections 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 --2- 172.21.15.104:0/1019466099 >> v2:172.21.15.16:3300/0 conn(0x7fd8f006ce90 0x7fd8f006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 --2- 172.21.15.104:0/1019466099 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f00cb760 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 -- 172.21.15.104:0/1019466099 >> 172.21.15.104:0/1019466099 conn(0x7fd8f00cd980 msgr2=0x7fd8f00cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 -- 172.21.15.104:0/1019466099 shutdown_connections 2024-09-17T05:00:45.066 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.065+0000 7fd8f8800640 1 -- 172.21.15.104:0/1019466099 wait complete. 2024-09-17T05:00:45.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 Processor -- start 2024-09-17T05:00:45.067 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 -- start start 2024-09-17T05:00:45.071 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f006ce90 0x7fd8f01f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.071 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 0x7fd8f00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f015eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f015d250 con 0x7fd8f006ce90 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f015d0d0 con 0x7fd8f006ff20 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f015d3d0 con 0x7fd8f00cb350 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f015eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 0x7fd8f00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f015eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:38580/0 (socket says 172.21.15.104:38580) 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 0x7fd8f00c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47900/0 (socket says 172.21.15.104:47900) 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 -- 172.21.15.104:0/2007881482 learned_addr learned my addr 172.21.15.104:0/2007881482 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:45.072 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f5e00640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f006ce90 0x7fd8f01f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 -- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 msgr2=0x7fd8f00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 0x7fd8f00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 -- 172.21.15.104:0/2007881482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f006ce90 msgr2=0x7fd8f01f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f006ce90 0x7fd8f01f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 -- 172.21.15.104:0/2007881482 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f00ca050 con 0x7fd8f00cb350 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f5400640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 0x7fd8f00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f6800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f015eca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e9a0 tx=0x7fd8ec00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec004d30 con 0x7fd8f00cb350 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8ec00cac0 con 0x7fd8f00cb350 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f00ca560 con 0x7fd8f00cb350 2024-09-17T05:00:45.073 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec005020 con 0x7fd8f00cb350 2024-09-17T05:00:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f00c7dc0 con 0x7fd8f00cb350 2024-09-17T05:00:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8ec0051c0 con 0x7fd8f00cb350 2024-09-17T05:00:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.069+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd8ec01a400 con 0x7fd8f00cb350 2024-09-17T05:00:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.073+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f006d2a0 con 0x7fd8f00cb350 2024-09-17T05:00:45.074 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.073+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd8ec016030 con 0x7fd8f00cb350 2024-09-17T05:00:45.142 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.141+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e77bd243-cf7e-422c-8f72-6b796df25a4e", "id": 1} v 0) -- 0x7fd8f00c4430 con 0x7fd8f00cb350 2024-09-17T05:00:45.147 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.145+0000 7fd8ea000640 1 -- 172.21.15.104:0/2007881482 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e77bd243-cf7e-422c-8f72-6b796df25a4e", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd8ec011470 con 0x7fd8f00cb350 2024-09-17T05:00:45.147 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:00:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 msgr2=0x7fd8f015eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f015eca0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e9a0 tx=0x7fd8ec00ee60 comp rx=0 tx=0).stop 2024-09-17T05:00:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 shutdown_connections 2024-09-17T05:00:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8f00cb350 0x7fd8f015eca0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8f006ff20 0x7fd8f00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.148 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 --2- 172.21.15.104:0/2007881482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f006ce90 0x7fd8f01f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 >> 172.21.15.104:0/2007881482 conn(0x7fd8f00cd980 msgr2=0x7fd8f00b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 shutdown_connections 2024-09-17T05:00:45.149 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.149+0000 7fd8f8800640 1 -- 172.21.15.104:0/2007881482 wait complete. 2024-09-17T05:00:45.168 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new ac3ac825-4660-4f82-91bf-9fb14fc1765c 2 2024-09-17T05:00:45.261 INFO:tasks.ceph.mgr.x.smithi104.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T05:00:45.261 INFO:tasks.ceph.mgr.x.smithi104.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T05:00:45.262 INFO:tasks.ceph.mgr.x.smithi104.stderr: from numpy import show_config as show_numpy_config 2024-09-17T05:00:45.324 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 -- 172.21.15.104:0/272711993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 msgr2=0x7f6de40af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 --2- 172.21.15.104:0/272711993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40af4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009900 tx=0x7f6dd001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 -- 172.21.15.104:0/272711993 shutdown_connections 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 --2- 172.21.15.104:0/272711993 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40afa00 0x7f6de415f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 --2- 172.21.15.104:0/272711993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40af4c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 -- 172.21.15.104:0/272711993 >> 172.21.15.104:0/272711993 conn(0x7f6de40cd980 msgr2=0x7f6de40cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 -- 172.21.15.104:0/272711993 shutdown_connections 2024-09-17T05:00:45.325 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 -- 172.21.15.104:0/272711993 wait complete. 2024-09-17T05:00:45.326 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 Processor -- start 2024-09-17T05:00:45.326 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.325+0000 7f6dec200640 1 -- start start 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de40afa00 0x7f6de41fcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 0x7f6de4153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4071ab0 con 0x7f6de40afa00 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4071930 con 0x7f6de4155c10 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6de4071c30 con 0x7f6de40ce4d0 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 0x7f6de4153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.329 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dea200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 0x7f6de4153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:38592/0 (socket says 172.21.15.104:38592) 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 -- 172.21.15.104:0/979125083 learned_addr learned my addr 172.21.15.104:0/979125083 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de9800640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de40afa00 0x7f6de41fcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 -- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 msgr2=0x7f6de40b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 -- 172.21.15.104:0/979125083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de40afa00 msgr2=0x7f6de41fcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de40afa00 0x7f6de41fcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 -- 172.21.15.104:0/979125083 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de41fc7a0 con 0x7f6de40ce4d0 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dea200640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:45.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de8e00640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 0x7f6de4153370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009850 tx=0x7f6dd0005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:45.331 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00b2040 con 0x7f6de40ce4d0 2024-09-17T05:00:45.331 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dd0004240 con 0x7f6de40ce4d0 2024-09-17T05:00:45.331 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0005030 con 0x7f6de40ce4d0 2024-09-17T05:00:45.331 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de4166420 con 0x7f6de40ce4d0 2024-09-17T05:00:45.331 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de40b3fa0 con 0x7f6de40ce4d0 2024-09-17T05:00:45.331 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6dd00046d0 con 0x7f6de40ce4d0 2024-09-17T05:00:45.332 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f6dd0004de0 con 0x7f6de40ce4d0 2024-09-17T05:00:45.332 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.329+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de4163de0 con 0x7f6de40ce4d0 2024-09-17T05:00:45.333 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.333+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6dd00164d0 con 0x7f6de40ce4d0 2024-09-17T05:00:45.401 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.401+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac3ac825-4660-4f82-91bf-9fb14fc1765c", "id": 2} v 0) -- 0x7f6de415f310 con 0x7f6de40ce4d0 2024-09-17T05:00:45.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.405+0000 7f6de1600640 1 -- 172.21.15.104:0/979125083 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac3ac825-4660-4f82-91bf-9fb14fc1765c", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f6dd0005580 con 0x7f6de40ce4d0 2024-09-17T05:00:45.407 INFO:teuthology.orchestra.run.smithi104.stdout:2 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 msgr2=0x7f6de4153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 0x7f6de4153370 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009850 tx=0x7f6dd0005760 comp rx=0 tx=0).stop 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 shutdown_connections 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6de4155c10 0x7f6de40b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6de40ce4d0 0x7f6de4153370 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 --2- 172.21.15.104:0/979125083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de40afa00 0x7f6de41fcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 >> 172.21.15.104:0/979125083 conn(0x7f6de40cd980 msgr2=0x7f6de4154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 shutdown_connections 2024-09-17T05:00:45.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.409+0000 7f6dec200640 1 -- 172.21.15.104:0/979125083 wait complete. 2024-09-17T05:00:45.427 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 465db35d-f042-4b72-bec6-79f3208dd66a 3 2024-09-17T05:00:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 -- 172.21.15.104:0/3542207350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a18004810 msgr2=0x7f1a18004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.580 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 --2- 172.21.15.104:0/3542207350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a18004810 0x7f1a18004c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009b10 tx=0x7f1a2001d230 comp rx=0 tx=0).stop 2024-09-17T05:00:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 -- 172.21.15.104:0/3542207350 shutdown_connections 2024-09-17T05:00:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 --2- 172.21.15.104:0/3542207350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a18004810 0x7f1a18004c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 -- 172.21.15.104:0/3542207350 >> 172.21.15.104:0/3542207350 conn(0x7f1a340b42d0 msgr2=0x7f1a340b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 -- 172.21.15.104:0/3542207350 shutdown_connections 2024-09-17T05:00:45.581 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 -- 172.21.15.104:0/3542207350 wait complete. 2024-09-17T05:00:45.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 Processor -- start 2024-09-17T05:00:45.582 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.581+0000 7f1a39000640 1 -- start start 2024-09-17T05:00:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18004810 0x7f1a341e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 0x7f1a341e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a34159840 0x7f1a341f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340d08d0 con 0x7f1a18004810 2024-09-17T05:00:45.584 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a340b2b20 con 0x7f1a341592f0 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1a341600f0 con 0x7f1a34159840 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 0x7f1a341e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a32800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a34159840 0x7f1a341f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 0x7f1a341e94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47936/0 (socket says 172.21.15.104:47936) 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 -- 172.21.15.104:0/1956574437 learned_addr learned my addr 172.21.15.104:0/1956574437 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31e00640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18004810 0x7f1a341e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 -- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a34159840 msgr2=0x7f1a341f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.585 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a34159840 0x7f1a341f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 -- 172.21.15.104:0/1956574437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18004810 msgr2=0x7f1a341e8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18004810 0x7f1a341e8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 -- 172.21.15.104:0/1956574437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a340c6840 con 0x7f1a341592f0 2024-09-17T05:00:45.586 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a32800640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a34159840 0x7f1a341f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a31400640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 0x7f1a341e94f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a2001d060 tx=0x7f1a200050a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a200bb030 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a20004900 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20004d50 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a341f7690 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a340c6de0 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a200be030 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1a200b7c60 con 0x7f1a341592f0 2024-09-17T05:00:45.587 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.585+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a341605e0 con 0x7f1a341592f0 2024-09-17T05:00:45.589 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.589+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1a20016f40 con 0x7f1a341592f0 2024-09-17T05:00:45.658 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.657+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "465db35d-f042-4b72-bec6-79f3208dd66a", "id": 3} v 0) -- 0x7f1a340716b0 con 0x7f1a341592f0 2024-09-17T05:00:45.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.661+0000 7f1a2a000640 1 -- 172.21.15.104:0/1956574437 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "465db35d-f042-4b72-bec6-79f3208dd66a", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1a200b77d0 con 0x7f1a341592f0 2024-09-17T05:00:45.663 INFO:teuthology.orchestra.run.smithi104.stdout:3 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 msgr2=0x7f1a341e94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 0x7f1a341e94f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a2001d060 tx=0x7f1a200050a0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 shutdown_connections 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a34159840 0x7f1a341f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a341592f0 0x7f1a341e94f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 --2- 172.21.15.104:0/1956574437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a18004810 0x7f1a341e8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.664 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 >> 172.21.15.104:0/1956574437 conn(0x7f1a340b42d0 msgr2=0x7f1a3415b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.665 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 shutdown_connections 2024-09-17T05:00:45.665 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.665+0000 7f1a39000640 1 -- 172.21.15.104:0/1956574437 wait complete. 2024-09-17T05:00:45.683 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new e3bbd5f6-7043-4a0b-95b6-4b576d2e1994 4 2024-09-17T05:00:45.835 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.833+0000 7efe88200640 1 -- 172.21.15.104:0/3178598664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 msgr2=0x7efe80073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.833+0000 7efe88200640 1 --2- 172.21.15.104:0/3178598664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe80073dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe800710d0 tx=0x7efe6c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 -- 172.21.15.104:0/3178598664 shutdown_connections 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 --2- 172.21.15.104:0/3178598664 >> v2:172.21.15.104:3300/0 conn(0x7efe800c6450 0x7efe80074300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 --2- 172.21.15.104:0/3178598664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe80073dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 --2- 172.21.15.104:0/3178598664 >> v2:172.21.15.16:3300/0 conn(0x7efe8006f5f0 0x7efe8015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 -- 172.21.15.104:0/3178598664 >> 172.21.15.104:0/3178598664 conn(0x7efe800b2dd0 msgr2=0x7efe800b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 -- 172.21.15.104:0/3178598664 shutdown_connections 2024-09-17T05:00:45.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 -- 172.21.15.104:0/3178598664 wait complete. 2024-09-17T05:00:45.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 Processor -- start 2024-09-17T05:00:45.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 -- start start 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe8006f5f0 0x7efe80157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe800c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.837+0000 7efe88200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe800c6450 0x7efe800b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe88200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe80160a40 con 0x7efe800c6450 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe88200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe801608c0 con 0x7efe8006f5f0 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe88200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efe80160bc0 con 0x7efe800c6040 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe85800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe8006f5f0 0x7efe80157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.841 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe800c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe85800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe8006f5f0 0x7efe80157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47958/0 (socket says 172.21.15.104:47958) 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe800c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:38620/0 (socket says 172.21.15.104:38620) 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe85800640 1 -- 172.21.15.104:0/802313213 learned_addr learned my addr 172.21.15.104:0/802313213 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe86200640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe800c6450 0x7efe800b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 -- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe8006f5f0 msgr2=0x7efe80157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe8006f5f0 0x7efe80157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 -- 172.21.15.104:0/802313213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe800c6450 msgr2=0x7efe800b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe800c6450 0x7efe800b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 -- 172.21.15.104:0/802313213 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe800b43b0 con 0x7efe800c6040 2024-09-17T05:00:45.842 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe84e00640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe800c8ef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0ac000 tx=0x7efe6c005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0bb070 con 0x7efe800c6040 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe6c004320 con 0x7efe800c6040 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe800b4690 con 0x7efe800c6040 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0b2040 con 0x7efe800c6040 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efe80157e70 con 0x7efe800c6040 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe6c0be070 con 0x7efe800c6040 2024-09-17T05:00:45.843 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7efe6c0c0950 con 0x7efe800c6040 2024-09-17T05:00:45.844 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.841+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8015c490 con 0x7efe800c6040 2024-09-17T05:00:45.844 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.845+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7efe6c0d8860 con 0x7efe800c6040 2024-09-17T05:00:45.867 INFO:tasks.ceph.mgr.y.smithi016.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T05:00:45.867 INFO:tasks.ceph.mgr.y.smithi016.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T05:00:45.868 INFO:tasks.ceph.mgr.y.smithi016.stderr: from numpy import show_config as show_numpy_config 2024-09-17T05:00:45.913 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.913+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e3bbd5f6-7043-4a0b-95b6-4b576d2e1994", "id": 4} v 0) -- 0x7efe80074b80 con 0x7efe800c6040 2024-09-17T05:00:45.917 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe7d600640 1 -- 172.21.15.104:0/802313213 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3bbd5f6-7043-4a0b-95b6-4b576d2e1994", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7efe6c0c6030 con 0x7efe800c6040 2024-09-17T05:00:45.917 INFO:teuthology.orchestra.run.smithi104.stdout:4 2024-09-17T05:00:45.918 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 msgr2=0x7efe800c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe800c8ef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0ac000 tx=0x7efe6c005790 comp rx=0 tx=0).stop 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 shutdown_connections 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe800c6450 0x7efe800b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe800c6040 0x7efe800c8ef0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 --2- 172.21.15.104:0/802313213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe8006f5f0 0x7efe80157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 >> 172.21.15.104:0/802313213 conn(0x7efe800b2dd0 msgr2=0x7efe800b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 shutdown_connections 2024-09-17T05:00:45.919 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:45.917+0000 7efe88200640 1 -- 172.21.15.104:0/802313213 wait complete. 2024-09-17T05:00:45.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 967db2c5-8b0e-4551-8a02-77c8628fda1d 5 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 -- 172.21.15.104:0/3578763867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6580744d0 msgr2=0x7fe65806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 --2- 172.21.15.104:0/3578763867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6580744d0 0x7fe65806e0d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe650009c80 tx=0x7fe65001cda0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 -- 172.21.15.104:0/3578763867 shutdown_connections 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 --2- 172.21.15.104:0/3578763867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6580744d0 0x7fe65806e0d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 --2- 172.21.15.104:0/3578763867 >> v2:172.21.15.104:3300/0 conn(0x7fe65806f5f0 0x7fe65815c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 -- 172.21.15.104:0/3578763867 >> 172.21.15.104:0/3578763867 conn(0x7fe6580b2dd0 msgr2=0x7fe6580b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 -- 172.21.15.104:0/3578763867 shutdown_connections 2024-09-17T05:00:46.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 -- 172.21.15.104:0/3578763867 wait complete. 2024-09-17T05:00:46.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 Processor -- start 2024-09-17T05:00:46.094 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 -- start start 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.093+0000 7fe65e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe65806f5f0 0x7fe65814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6580c6040 0x7fe658153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe658154160 0x7fe658164d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe65816a470 con 0x7fe65806f5f0 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe65816a2f0 con 0x7fe6580c6040 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe65816a5f0 con 0x7fe658154160 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6580c6040 0x7fe658153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe658154160 0x7fe658164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6580c6040 0x7fe658153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47986/0 (socket says 172.21.15.104:47986) 2024-09-17T05:00:46.097 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64fe00640 1 -- 172.21.15.104:0/490538563 learned_addr learned my addr 172.21.15.104:0/490538563 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657400640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe65806f5f0 0x7fe65814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 -- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6580c6040 msgr2=0x7fe658153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6580c6040 0x7fe658153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 -- 172.21.15.104:0/490538563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe65806f5f0 msgr2=0x7fe65814eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe65806f5f0 0x7fe65814eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 -- 172.21.15.104:0/490538563 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe658154570 con 0x7fe658154160 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657400640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe65806f5f0 0x7fe65814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe657e00640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe658154160 0x7fe658164d10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6500ac000 tx=0x7fe650005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6500bc070 con 0x7fe658154160 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe650004000 con 0x7fe658154160 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65815a530 con 0x7fe658154160 2024-09-17T05:00:46.098 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6500b2040 con 0x7fe658154160 2024-09-17T05:00:46.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe658165d90 con 0x7fe658154160 2024-09-17T05:00:46.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe6500bf070 con 0x7fe658154160 2024-09-17T05:00:46.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fe6500c16c0 con 0x7fe658154160 2024-09-17T05:00:46.099 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.097+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65815a7a0 con 0x7fe658154160 2024-09-17T05:00:46.100 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.101+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe658154570 con 0x7fe658154160 2024-09-17T05:00:46.171 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.169+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "967db2c5-8b0e-4551-8a02-77c8628fda1d", "id": 5} v 0) -- 0x7fe6580c8210 con 0x7fe658154160 2024-09-17T05:00:46.176 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe64f400640 1 -- 172.21.15.104:0/490538563 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "967db2c5-8b0e-4551-8a02-77c8628fda1d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fe650016990 con 0x7fe658154160 2024-09-17T05:00:46.176 INFO:teuthology.orchestra.run.smithi104.stdout:5 2024-09-17T05:00:46.177 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe658154160 msgr2=0x7fe658164d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe658154160 0x7fe658164d10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe6500ac000 tx=0x7fe650005af0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 shutdown_connections 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe658154160 0x7fe658164d10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6580c6040 0x7fe658153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 --2- 172.21.15.104:0/490538563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe65806f5f0 0x7fe65814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 >> 172.21.15.104:0/490538563 conn(0x7fe6580b2dd0 msgr2=0x7fe6580d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 shutdown_connections 2024-09-17T05:00:46.178 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.177+0000 7fe65e200640 1 -- 172.21.15.104:0/490538563 wait complete. 2024-09-17T05:00:46.197 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8 6 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 -- 172.21.15.104:0/306712781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 msgr2=0x7f588c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 --2- 172.21.15.104:0/306712781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c06e3b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5884009c80 tx=0x7f588401cda0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 -- 172.21.15.104:0/306712781 shutdown_connections 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 --2- 172.21.15.104:0/306712781 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c06e3b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 --2- 172.21.15.104:0/306712781 >> v2:172.21.15.16:3300/0 conn(0x7f588c0744d0 0x7f588c0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 -- 172.21.15.104:0/306712781 >> 172.21.15.104:0/306712781 conn(0x7f588c0b2dd0 msgr2=0x7f588c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 -- 172.21.15.104:0/306712781 shutdown_connections 2024-09-17T05:00:46.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.349+0000 7f5891200640 1 -- 172.21.15.104:0/306712781 wait complete. 2024-09-17T05:00:46.353 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 Processor -- start 2024-09-17T05:00:46.353 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 -- start start 2024-09-17T05:00:46.355 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.355 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c0cce00 0x7f588c0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.355 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c06ce90 0x7f588c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f588c168a60 con 0x7f588c06ce90 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1688e0 con 0x7f588c0cce00 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5891200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f588c168be0 con 0x7f588c06dfa0 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5889e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5883e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c0cce00 0x7f588c0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5889e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c0ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.104:38642/0 (socket says 172.21.15.104:38642) 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f588a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c06ce90 0x7f588c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.353+0000 7f5889e00640 1 -- 172.21.15.104:0/1950756465 learned_addr learned my addr 172.21.15.104:0/1950756465 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5889e00640 1 -- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c0cce00 msgr2=0x7f588c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5889e00640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c0cce00 0x7f588c0b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5889e00640 1 -- 172.21.15.104:0/1950756465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c06ce90 msgr2=0x7f588c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.356 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5889e00640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c06ce90 0x7f588c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5889e00640 1 -- 172.21.15.104:0/1950756465 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c06d7e0 con 0x7f588c06dfa0 2024-09-17T05:00:46.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f588a800640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c06ce90 0x7f588c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:46.357 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5889e00640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c0ca040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f587c001ec0 tx=0x7f587c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c017070 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f587c003600 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c003a50 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c0cad00 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c0b4200 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f587c01a070 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f587c003bf0 con 0x7f588c06dfa0 2024-09-17T05:00:46.358 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.357+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588c06e3b0 con 0x7f588c06dfa0 2024-09-17T05:00:46.359 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.361+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f587c021020 con 0x7f588c06dfa0 2024-09-17T05:00:46.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.429+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8", "id": 6} v 0) -- 0x7f588c0b2730 con 0x7f588c06dfa0 2024-09-17T05:00:46.434 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5882000640 1 -- 172.21.15.104:0/1950756465 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f588c06d7e0 con 0x7f588c06dfa0 2024-09-17T05:00:46.434 INFO:teuthology.orchestra.run.smithi104.stdout:6 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 msgr2=0x7f588c0ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c0ca040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f587c001ec0 tx=0x7f587c00c890 comp rx=0 tx=0).stop 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 shutdown_connections 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c06ce90 0x7f588c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f588c0cce00 0x7f588c0b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 --2- 172.21.15.104:0/1950756465 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c06dfa0 0x7f588c0ca040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 >> 172.21.15.104:0/1950756465 conn(0x7f588c0b2dd0 msgr2=0x7f588c0b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.433+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 shutdown_connections 2024-09-17T05:00:46.435 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.437+0000 7f5891200640 1 -- 172.21.15.104:0/1950756465 wait complete. 2024-09-17T05:00:46.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo ceph --cluster ceph osd new 53764c31-8173-4f18-8367-b1f777813351 7 2024-09-17T05:00:46.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 -- 172.21.15.104:0/3604322309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600d02d0 msgr2=0x7ff1600cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 --2- 172.21.15.104:0/3604322309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600d02d0 0x7ff1600cf140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff14c009900 tx=0x7ff14c01c750 comp rx=0 tx=0).stop 2024-09-17T05:00:46.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 -- 172.21.15.104:0/3604322309 shutdown_connections 2024-09-17T05:00:46.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 --2- 172.21.15.104:0/3604322309 >> v2:172.21.15.16:3300/0 conn(0x7ff1600cf680 0x7ff1600b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 --2- 172.21.15.104:0/3604322309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600d02d0 0x7ff1600cf140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.606 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 -- 172.21.15.104:0/3604322309 >> 172.21.15.104:0/3604322309 conn(0x7ff1600b42d0 msgr2=0x7ff1600b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 -- 172.21.15.104:0/3604322309 shutdown_connections 2024-09-17T05:00:46.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.605+0000 7ff167400640 1 -- 172.21.15.104:0/3604322309 wait complete. 2024-09-17T05:00:46.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 Processor -- start 2024-09-17T05:00:46.608 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 -- start start 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 0x7ff16014ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1600d02d0 0x7ff1601f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff160159f70 0x7ff1601f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff160169e20 con 0x7ff1600d02d0 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff160169ca0 con 0x7ff1600cf680 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff167400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff160169fa0 con 0x7ff160159f70 2024-09-17T05:00:46.610 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 0x7ff16014ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.611 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff165400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff160159f70 0x7ff1601f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.611 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 0x7ff16014ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:48018/0 (socket says 172.21.15.104:48018) 2024-09-17T05:00:46.611 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff15fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1600d02d0 0x7ff1601f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.611 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 -- 172.21.15.104:0/2928863929 learned_addr learned my addr 172.21.15.104:0/2928863929 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-17T05:00:46.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 -- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff160159f70 msgr2=0x7ff1601f9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff160159f70 0x7ff1601f9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 -- 172.21.15.104:0/2928863929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1600d02d0 msgr2=0x7ff1601f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1600d02d0 0x7ff1601f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 -- 172.21.15.104:0/2928863929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1601fa890 con 0x7ff1600cf680 2024-09-17T05:00:46.612 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff15fe00640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1600d02d0 0x7ff1601f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff164a00640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 0x7ff16014ecc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c3c0 tx=0x7ff15400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.609+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154017070 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1540053c0 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154003860 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff160157a70 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1601fa5b0 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff15401a070 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff154011cc0 con 0x7ff1600cf680 2024-09-17T05:00:46.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1600b2100 con 0x7ff1600cf680 2024-09-17T05:00:46.614 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.613+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff154021020 con 0x7ff1600cf680 2024-09-17T05:00:46.684 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.685+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd new", "uuid": "53764c31-8173-4f18-8367-b1f777813351", "id": 7} v 0) -- 0x7ff160076350 con 0x7ff1600cf680 2024-09-17T05:00:46.688 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff15cc00640 1 -- 172.21.15.104:0/2928863929 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "53764c31-8173-4f18-8367-b1f777813351", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff1600cfa90 con 0x7ff1600cf680 2024-09-17T05:00:46.688 INFO:teuthology.orchestra.run.smithi104.stdout:7 2024-09-17T05:00:46.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 msgr2=0x7ff16014ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.689 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 0x7ff16014ecc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c3c0 tx=0x7ff15400c880 comp rx=0 tx=0).stop 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 shutdown_connections 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff160159f70 0x7ff1601f9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1600d02d0 0x7ff1601f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 --2- 172.21.15.104:0/2928863929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1600cf680 0x7ff16014ecc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 >> 172.21.15.104:0/2928863929 conn(0x7ff1600b42d0 msgr2=0x7ff1600d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 shutdown_connections 2024-09-17T05:00:46.690 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:00:46.689+0000 7ff167400640 1 -- 172.21.15.104:0/2928863929 wait complete. 2024-09-17T05:00:46.708 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T05:00:46.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T05:00:46.711 INFO:tasks.ceph.osd.0:Started 2024-09-17T05:00:46.711 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T05:00:46.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T05:00:46.714 INFO:tasks.ceph.osd.1:Started 2024-09-17T05:00:46.714 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T05:00:46.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T05:00:46.717 INFO:tasks.ceph.osd.2:Started 2024-09-17T05:00:46.717 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T05:00:46.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T05:00:46.721 INFO:tasks.ceph.osd.3:Started 2024-09-17T05:00:46.721 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T05:00:46.722 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T05:00:46.725 INFO:tasks.ceph.osd.4:Started 2024-09-17T05:00:46.725 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T05:00:46.725 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T05:00:46.728 INFO:tasks.ceph.osd.5:Started 2024-09-17T05:00:46.728 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T05:00:46.728 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T05:00:46.731 INFO:tasks.ceph.osd.6:Started 2024-09-17T05:00:46.731 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T05:00:46.731 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T05:00:46.735 INFO:tasks.ceph.osd.7:Started 2024-09-17T05:00:46.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:00:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 -- 172.21.15.16:0/31254278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f00b30b0 msgr2=0x7f15f00d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 --2- 172.21.15.16:0/31254278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f00b30b0 0x7f15f00d0d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009c80 tx=0x7f15e801cc80 comp rx=0 tx=0).stop 2024-09-17T05:00:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 -- 172.21.15.16:0/31254278 shutdown_connections 2024-09-17T05:00:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 --2- 172.21.15.16:0/31254278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f00b30b0 0x7f15f00d0d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 --2- 172.21.15.16:0/31254278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f015c6f0 0x7f15f0160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 -- 172.21.15.16:0/31254278 >> 172.21.15.16:0/31254278 conn(0x7f15f00cf380 msgr2=0x7f15f00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 -- 172.21.15.16:0/31254278 shutdown_connections 2024-09-17T05:00:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 -- 172.21.15.16:0/31254278 wait complete. 2024-09-17T05:00:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 Processor -- start 2024-09-17T05:00:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.929+0000 7f15f6400640 1 -- start start 2024-09-17T05:00:46.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 0x7f15f00cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15f015c6f0 0x7f15f0158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 0x7f15f0156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0169eb0 con 0x7f15f01608d0 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0169d30 con 0x7f15f015c6f0 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f15f016a030 con 0x7f15f00b30b0 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 0x7f15f0156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 0x7f15f00cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 0x7f15f00cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:53250/0 (socket says 172.21.15.16:53250) 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 -- 172.21.15.16:0/471777741 learned_addr learned my addr 172.21.15.16:0/471777741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 0x7f15f0156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33018/0 (socket says 172.21.15.16:33018) 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15e6a00640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15f015c6f0 0x7f15f0158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 -- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15f015c6f0 msgr2=0x7f15f0158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15f015c6f0 0x7f15f0158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 -- 172.21.15.16:0/471777741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 msgr2=0x7f15f0156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 0x7f15f0156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 -- 172.21.15.16:0/471777741 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f00cc8b0 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15efe00640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 0x7f15f0156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15ef400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 0x7f15f00cc370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15e000e7c0 tx=0x7f15e000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e000ccc0 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e000ce80 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005280 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f006f280 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f0158390 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f15e00047b0 con 0x7f15f00b30b0 2024-09-17T05:00:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f15e0021440 con 0x7f15f00b30b0 2024-09-17T05:00:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.933+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f0160390 con 0x7f15f00b30b0 2024-09-17T05:00:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.937+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f15f00cc8b0 con 0x7f15f00b30b0 2024-09-17T05:00:46.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.997+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15f0161170 con 0x7f15f00b30b0 2024-09-17T05:00:46.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:46.997+0000 7f15e7e00640 1 -- 172.21.15.16:0/471777741 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f15e0004aa0 con 0x7f15f00b30b0 2024-09-17T05:00:46.999 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:00:47.000 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:00:46.688430+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:00:47.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 msgr2=0x7f15f00cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 0x7f15f00cc370 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15e000e7c0 tx=0x7f15e000ec80 comp rx=0 tx=0).stop 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 shutdown_connections 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15f01608d0 0x7f15f0156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15f015c6f0 0x7f15f0158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 --2- 172.21.15.16:0/471777741 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15f00b30b0 0x7f15f00cc370 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 >> 172.21.15.16:0/471777741 conn(0x7f15f00cf380 msgr2=0x7f15f0155db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 shutdown_connections 2024-09-17T05:00:47.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:00:47.001+0000 7f15f6400640 1 -- 172.21.15.16:0/471777741 wait complete. 2024-09-17T05:00:47.019 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T05:00:47.019 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T05:00:47.826 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 -- 172.21.15.16:0/3599554549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0d1580 msgr2=0x7f12fc0d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 --2- 172.21.15.16:0/3599554549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0d1580 0x7f12fc0d1990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009840 tx=0x7f12e801c820 comp rx=0 tx=0).stop 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 -- 172.21.15.16:0/3599554549 shutdown_connections 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 --2- 172.21.15.16:0/3599554549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0d1580 0x7f12fc0d1990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 --2- 172.21.15.16:0/3599554549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12fc0cc8c0 0x7f12fc15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 -- 172.21.15.16:0/3599554549 >> 172.21.15.16:0/3599554549 conn(0x7f12fc0d2270 msgr2=0x7f12fc0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 -- 172.21.15.16:0/3599554549 shutdown_connections 2024-09-17T05:00:47.976 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.973+0000 7f1301a00640 1 -- 172.21.15.16:0/3599554549 wait complete. 2024-09-17T05:00:47.977 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 Processor -- start 2024-09-17T05:00:47.978 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 -- start start 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 0x7f12fc0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12fc0d0010 0x7f12fc0c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12fc0d1580 0x7f12fc139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc16a530 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc16a3b0 con 0x7f12fc0d0010 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f1301a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc16a6b0 con 0x7f12fc0d1580 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 0x7f12fc0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 0x7f12fc0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59490/0 (socket says 172.21.15.16:59490) 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 -- 172.21.15.16:0/953663074 learned_addr learned my addr 172.21.15.16:0/953663074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12fa000640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12fc0d0010 0x7f12fc0c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12fb400640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12fc0d1580 0x7f12fc139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:47.981 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 -- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12fc0d1580 msgr2=0x7f12fc139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12fc0d1580 0x7f12fc139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 -- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12fc0d0010 msgr2=0x7f12fc0c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12fc0d0010 0x7f12fc0c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.977+0000 7f12faa00640 1 -- 172.21.15.16:0/953663074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc0c8690 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f12faa00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 0x7f12fc0c5da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00e790 tx=0x7f12ec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00eea0 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12ec00cce0 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004290 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc0c8970 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.982 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc1591d0 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.983 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12fc0c8bb0 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.983 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 2) ==== 48655+0+0 (secure 0 0 0) 0x7f12ec003600 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.983 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.981+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f12ec04f760 con 0x7f12fc0cc8c0 2024-09-17T05:00:47.985 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:47.985+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f12fc0c8bb0 con 0x7f12fc0cc8c0 2024-09-17T05:00:48.048 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.045+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12fc15cab0 con 0x7f12fc0cc8c0 2024-09-17T05:00:48.048 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f12f2a00640 1 -- 172.21.15.16:0/953663074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f12ec013c20 con 0x7f12fc0cc8c0 2024-09-17T05:00:48.049 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-17T05:00:48.049 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:00:46.688430+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 msgr2=0x7f12fc0c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 0x7f12fc0c5da0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00e790 tx=0x7f12ec00ec50 comp rx=0 tx=0).stop 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 shutdown_connections 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12fc0d1580 0x7f12fc139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12fc0d0010 0x7f12fc0c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 --2- 172.21.15.16:0/953663074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12fc0cc8c0 0x7f12fc0c5da0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 >> 172.21.15.16:0/953663074 conn(0x7f12fc0d2270 msgr2=0x7f12fc151890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 shutdown_connections 2024-09-17T05:00:48.050 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:48.049+0000 7f1301a00640 1 -- 172.21.15.16:0/953663074 wait complete. 2024-09-17T05:00:48.067 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T05:00:48.225 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-09-17T05:00:48.225+0000 7f6da27b67c0 -1 Falling back to public interface 2024-09-17T05:00:48.422 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-17T05:00:48.421+0000 7fbae8a3b7c0 -1 Falling back to public interface 2024-09-17T05:00:48.465 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-09-17T05:00:48.465+0000 7fda99800640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T05:00:48.465 INFO:tasks.ceph.mgr.x.smithi104.stderr:2024-09-17T05:00:48.465+0000 7fda99800640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T05:00:48.485 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-09-17T05:00:48.485+0000 7f9c2fe2b7c0 -1 Falling back to public interface 2024-09-17T05:00:48.501 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-09-17T05:00:48.501+0000 7fde0ab3c7c0 -1 Falling back to public interface 2024-09-17T05:00:48.665 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-09-17T05:00:48.665+0000 7eff8493c7c0 -1 Falling back to public interface 2024-09-17T05:00:48.698 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-17T05:00:48.697+0000 7f70152f07c0 -1 Falling back to public interface 2024-09-17T05:00:48.698 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-17T05:00:48.697+0000 7fa5f00227c0 -1 Falling back to public interface 2024-09-17T05:00:48.750 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-17T05:00:48.749+0000 7fca39f677c0 -1 Falling back to public interface 2024-09-17T05:00:52.461 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-09-17T05:00:52.461+0000 7f6da27b67c0 -1 osd.5 0 log_to_monitors true 2024-09-17T05:00:52.681 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-09-17T05:00:52.681+0000 7fde0ab3c7c0 -1 osd.7 0 log_to_monitors true 2024-09-17T05:00:52.733 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-17T05:00:52.729+0000 7fbae8a3b7c0 -1 osd.3 0 log_to_monitors true 2024-09-17T05:00:52.995 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-17T05:00:52.993+0000 7fca39f677c0 -1 osd.0 0 log_to_monitors true 2024-09-17T05:00:53.009 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-09-17T05:00:53.009+0000 7f9c2fe2b7c0 -1 osd.4 0 log_to_monitors true 2024-09-17T05:00:53.096 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-09-17T05:00:53.097+0000 7eff8493c7c0 -1 osd.6 0 log_to_monitors true 2024-09-17T05:00:53.344 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-17T05:00:53.341+0000 7fa5f00227c0 -1 osd.1 0 log_to_monitors true 2024-09-17T05:00:53.448 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-17T05:00:53.445+0000 7f70152f07c0 -1 osd.2 0 log_to_monitors true 2024-09-17T05:00:54.451 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-09-17T05:00:54.449+0000 7f6d9e000640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T05:00:54.477 INFO:tasks.ceph.osd.5.smithi104.stderr:2024-09-17T05:00:54.477+0000 7f6d93600640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:54.874 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 -- 172.21.15.16:0/1086409719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42600cf300 msgr2=0x7f4260155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 --2- 172.21.15.16:0/1086409719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42600cf300 0x7f4260155300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4260073c20 tx=0x7f424c01c770 comp rx=0 tx=0).stop 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 -- 172.21.15.16:0/1086409719 shutdown_connections 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 --2- 172.21.15.16:0/1086409719 >> v2:172.21.15.104:3300/0 conn(0x7f4260155840 0x7f42600cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 --2- 172.21.15.16:0/1086409719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42600cf300 0x7f4260155300 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4260073c20 tx=0x7f424c01c770 comp rx=0 tx=0).stop 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 -- 172.21.15.16:0/1086409719 >> 172.21.15.16:0/1086409719 conn(0x7f42600cdb20 msgr2=0x7f42600cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:55.107 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 -- 172.21.15.16:0/1086409719 shutdown_connections 2024-09-17T05:00:55.109 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.105+0000 7f4265a00640 1 -- 172.21.15.16:0/1086409719 wait complete. 2024-09-17T05:00:55.110 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f4265a00640 1 Processor -- start 2024-09-17T05:00:55.110 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f4265a00640 1 -- start start 2024-09-17T05:00:55.112 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f4265a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 0x7f4260162090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:55.112 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f4265a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4260155840 0x7f4260206290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:55.113 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f4265a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260152420 0x7f42600b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:55.113 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f4265a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4260170220 con 0x7f4260155840 2024-09-17T05:00:55.113 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.109+0000 7f425ea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 0x7f4260162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:55.113 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 0x7f4260162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:44046/0 (socket says 172.21.15.16:44046) 2024-09-17T05:00:55.113 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 -- 172.21.15.16:0/2539047238 learned_addr learned my addr 172.21.15.16:0/2539047238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:00:55.113 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42601700a0 con 0x7f4260152420 2024-09-17T05:00:55.114 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f42601703a0 con 0x7f42600ceef0 2024-09-17T05:00:55.114 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425f400640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260152420 0x7f42600b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:55.114 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425e000640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4260155840 0x7f4260206290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 -- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260152420 msgr2=0x7f42600b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260152420 0x7f42600b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 -- 172.21.15.16:0/2539047238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4260155840 msgr2=0x7f4260206290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4260155840 0x7f4260206290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42600b6d80 con 0x7f42600ceef0 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425e000640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4260155840 0x7f4260206290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425f400640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260152420 0x7f42600b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:00:55.115 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f425ea00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 0x7f4260162090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f425000c400 tx=0x7f425000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:55.116 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250017070 con 0x7f42600ceef0 2024-09-17T05:00:55.116 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4250003600 con 0x7f42600ceef0 2024-09-17T05:00:55.116 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250003a10 con 0x7f42600ceef0 2024-09-17T05:00:55.116 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42601615b0 con 0x7f42600ceef0 2024-09-17T05:00:55.116 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.113+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4260161b70 con 0x7f42600ceef0 2024-09-17T05:00:55.117 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.117+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42600cc3a0 con 0x7f42600ceef0 2024-09-17T05:00:55.120 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.117+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f425001a070 con 0x7f42600ceef0 2024-09-17T05:00:55.121 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.121+0000 7f4256a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4240073240 0x7f4240075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:00:55.121 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.121+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7f42500926b0 con 0x7f42600ceef0 2024-09-17T05:00:55.121 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.121+0000 7f425e000640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4240073240 0x7f4240075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:00:55.122 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.121+0000 7f425e000640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4240073240 0x7f4240075750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f424c0af000 tx=0x7f424c01afb0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:00:55.126 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.125+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f42600cbee0 con 0x7f42600ceef0 2024-09-17T05:00:55.445 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.445+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42601531f0 con 0x7f42600ceef0 2024-09-17T05:00:55.447 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.445+0000 7f4256a00640 1 -- 172.21.15.16:0/2539047238 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7555 (secure 0 0 0) 0x7f425005dee0 con 0x7f42600ceef0 2024-09-17T05:00:55.447 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-17T05:00:55.447 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":13,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:00:54.527247+0000","last_up_change":"2024-09-17T05:00:54.527247+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:00:55.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4240073240 msgr2=0x7f4240075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:55.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4240073240 0x7f4240075750 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f424c0af000 tx=0x7f424c01afb0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 msgr2=0x7f4260162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:00:55.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 0x7f4260162090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f425000c400 tx=0x7f425000c8c0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 shutdown_connections 2024-09-17T05:00:55.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4240073240 0x7f4240075750 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260152420 0x7f42600b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4260155840 0x7f4260206290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 --2- 172.21.15.16:0/2539047238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f42600ceef0 0x7f4260162090 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:00:55.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 >> 172.21.15.16:0/2539047238 conn(0x7f42600cdb20 msgr2=0x7f426006f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:00:55.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 shutdown_connections 2024-09-17T05:00:55.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:00:55.449+0000 7f4265a00640 1 -- 172.21.15.16:0/2539047238 wait complete. 2024-09-17T05:00:55.479 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-17T05:00:56.363 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-17T05:00:56.360+0000 7f7010200640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T05:00:56.402 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-17T05:00:56.400+0000 7fca34e00640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T05:00:56.410 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-09-17T05:00:56.408+0000 7f7006200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:56.422 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-09-17T05:00:56.421+0000 7fde05a00640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T05:00:56.432 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-17T05:00:56.428+0000 7fbae3800640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T05:00:56.438 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-09-17T05:00:56.436+0000 7fca2ae00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:56.444 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-09-17T05:00:56.445+0000 7eff80200640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T05:00:56.453 INFO:tasks.ceph.osd.7.smithi104.stderr:2024-09-17T05:00:56.453+0000 7fddfba00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:56.454 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-09-17T05:00:56.453+0000 7f9c2ae00640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T05:00:56.466 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-09-17T05:00:56.464+0000 7fbad9800640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:56.475 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-17T05:00:56.472+0000 7fa5eae00640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T05:00:56.477 INFO:tasks.ceph.osd.6.smithi104.stderr:2024-09-17T05:00:56.477+0000 7eff75800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:56.492 INFO:tasks.ceph.osd.4.smithi104.stderr:2024-09-17T05:00:56.493+0000 7f9c20400640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:56.494 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-09-17T05:00:56.492+0000 7fa5e0e00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T05:00:59.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:00:59.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:01:02.285 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 -- 172.21.15.16:0/1696757183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 msgr2=0x7f20340c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 --2- 172.21.15.16:0/1696757183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c9850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f20340ce800 tx=0x7f202c01cb80 comp rx=0 tx=0).stop 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 -- 172.21.15.16:0/1696757183 shutdown_connections 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 --2- 172.21.15.16:0/1696757183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c9850 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 --2- 172.21.15.16:0/1696757183 >> v2:172.21.15.104:3300/0 conn(0x7f20340d1c90 0x7f203415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 -- 172.21.15.16:0/1696757183 >> 172.21.15.16:0/1696757183 conn(0x7f20340d2270 msgr2=0x7f20340d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:02.451 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 -- 172.21.15.16:0/1696757183 shutdown_connections 2024-09-17T05:01:02.452 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.448+0000 7f203ae00640 1 -- 172.21.15.16:0/1696757183 wait complete. 2024-09-17T05:01:02.453 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 Processor -- start 2024-09-17T05:01:02.453 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 -- start start 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20340cc8c0 0x7f20340b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20341535f0 0x7f20341f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2034169f50 con 0x7f20340c9440 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2034169dd0 con 0x7f20340cc8c0 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f203416a0d0 con 0x7f20341535f0 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46622/0 (socket says 172.21.15.16:46622) 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 -- 172.21.15.16:0/792373030 learned_addr learned my addr 172.21.15.16:0/792373030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:02.455 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2038e00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20341535f0 0x7f20341f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.456 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033400640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20340cc8c0 0x7f20340b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 -- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20341535f0 msgr2=0x7f20341f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20341535f0 0x7f20341f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 -- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20340cc8c0 msgr2=0x7f20340b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20340cc8c0 0x7f20340b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 -- 172.21.15.16:0/792373030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20340c6d70 con 0x7f20340c9440 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f2033e00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c77e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f202400c3c0 tx=0x7f202400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20240068a0 con 0x7f20340c9440 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2024006b30 con 0x7f20340c9440 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20340b7b20 con 0x7f20340c9440 2024-09-17T05:01:02.457 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024007020 con 0x7f20340c9440 2024-09-17T05:01:02.458 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.452+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20340c6fe0 con 0x7f20340c9440 2024-09-17T05:01:02.458 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.456+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20340c9850 con 0x7f20340c9440 2024-09-17T05:01:02.458 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.456+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2024019020 con 0x7f20340c9440 2024-09-17T05:01:02.460 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.456+0000 7f202be00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2014073200 0x7f2014075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.460 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.456+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f2024093510 con 0x7f20340c9440 2024-09-17T05:01:02.460 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.456+0000 7f2033400640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2014073200 0x7f2014075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.461 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.460+0000 7f2033400640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2014073200 0x7f2014075710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2034155b30 tx=0x7f202000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:02.480 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.476+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f202405e1a0 con 0x7f20340c9440 2024-09-17T05:01:02.656 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.652+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20340d0cc0 con 0x7f20340c9440 2024-09-17T05:01:02.657 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f202be00640 1 -- 172.21.15.16:0/792373030 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12884 (secure 0 0 0) 0x7f2024062bd0 con 0x7f20340c9440 2024-09-17T05:01:02.657 INFO:teuthology.misc.health.smithi016.stdout: 2024-09-17T05:01:02.657 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":18,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:00.432442+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:02.659 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2014073200 msgr2=0x7f2014075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.659 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2014073200 0x7f2014075710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2034155b30 tx=0x7f202000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.659 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 msgr2=0x7f20340c77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c77e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f202400c3c0 tx=0x7f202400c880 comp rx=0 tx=0).stop 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 shutdown_connections 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2014073200 0x7f2014075710 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20341535f0 0x7f20341f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20340cc8c0 0x7f20340b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 --2- 172.21.15.16:0/792373030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20340c9440 0x7f20340c77e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 >> 172.21.15.16:0/792373030 conn(0x7f20340d2270 msgr2=0x7f20340ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 shutdown_connections 2024-09-17T05:01:02.660 INFO:teuthology.misc.health.smithi016.stderr:2024-09-17T05:01:02.656+0000 7f203ae00640 1 -- 172.21.15.16:0/792373030 wait complete. 2024-09-17T05:01:02.678 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T05:01:02.678 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T05:01:02.678 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:01:02.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:01:02.682 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:01:02.682 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:01:02.682 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:01:02.685 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:01:02.685 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:01:02.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:01:02.688 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:01:02.688 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:01:02.689 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:01:02.691 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:01:02.692 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T05:01:02.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:02.797 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:01:02.797 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:01:02.805 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:01:02.805 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:01:02.808 INFO:tasks.ceph.mds.b.smithi104.stderr:2024-09-17T05:01:02.808+0000 7f5c97400640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-09-17T05:01:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 -- 172.21.15.16:0/3563781957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 msgr2=0x7f9128073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 --2- 172.21.15.16:0/3563781957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 0x7f9128073650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9120009c80 tx=0x7f912001cd10 comp rx=0 tx=0).stop 2024-09-17T05:01:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 -- 172.21.15.16:0/3563781957 shutdown_connections 2024-09-17T05:01:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 --2- 172.21.15.16:0/3563781957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 0x7f9128073650 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 --2- 172.21.15.16:0/3563781957 >> v2:172.21.15.104:3300/0 conn(0x7f91280b2f50 0x7f912815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 -- 172.21.15.16:0/3563781957 >> 172.21.15.16:0/3563781957 conn(0x7f912806f900 msgr2=0x7f91280b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 -- 172.21.15.16:0/3563781957 shutdown_connections 2024-09-17T05:01:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 -- 172.21.15.16:0/3563781957 wait complete. 2024-09-17T05:01:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.856+0000 7f912da00640 1 Processor -- start 2024-09-17T05:01:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- start start 2024-09-17T05:01:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 0x7f91280c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 0x7f91280c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f912815c0a0 0x7f9128142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9128168d30 con 0x7f91280b2f50 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9128168bb0 con 0x7f912815c0a0 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9128168eb0 con 0x7f9128073240 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 0x7f91280c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 0x7f91280c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46644/0 (socket says 172.21.15.16:46644) 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 -- 172.21.15.16:0/683327406 learned_addr learned my addr 172.21.15.16:0/683327406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f9127400640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f912815c0a0 0x7f9128142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f9126a00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 0x7f91280c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 -- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 msgr2=0x7f91280c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 0x7f91280c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 -- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f912815c0a0 msgr2=0x7f9128142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f912815c0a0 0x7f9128142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 -- 172.21.15.16:0/683327406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91280c5400 con 0x7f91280b2f50 2024-09-17T05:01:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911fe00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 0x7f91280c4ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f911000c3c0 tx=0x7f911000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110017070 con 0x7f91280b2f50 2024-09-17T05:01:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91100053c0 con 0x7f91280b2f50 2024-09-17T05:01:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110003860 con 0x7f91280b2f50 2024-09-17T05:01:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91280c5910 con 0x7f91280b2f50 2024-09-17T05:01:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91280caef0 con 0x7f91280b2f50 2024-09-17T05:01:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.860+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9128073650 con 0x7f91280b2f50 2024-09-17T05:01:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.864+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f911001a070 con 0x7f91280b2f50 2024-09-17T05:01:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.864+0000 7f911ea00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9104073200 0x7f9104075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.864+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f9110093130 con 0x7f91280b2f50 2024-09-17T05:01:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.864+0000 7f9126a00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9104073200 0x7f9104075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.864+0000 7f9126a00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9104073200 0x7f9104075710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9118009b40 tx=0x7f91180091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:02.868+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f911005de40 con 0x7f91280b2f50 2024-09-17T05:01:03.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91280c5630 con 0x7f91280b2f50 2024-09-17T05:01:03.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f911ea00640 1 -- 172.21.15.16:0/683327406 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f9110062870 con 0x7f91280b2f50 2024-09-17T05:01:03.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:03.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-17T05:01:02:809049+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T05:01:03.082 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-17T05:01:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9104073200 msgr2=0x7f9104075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9104073200 0x7f9104075710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9118009b40 tx=0x7f91180091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 msgr2=0x7f91280c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 0x7f91280c4ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f911000c3c0 tx=0x7f911000c880 comp rx=0 tx=0).stop 2024-09-17T05:01:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 shutdown_connections 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9104073200 0x7f9104075710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f912815c0a0 0x7f9128142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91280b2f50 0x7f91280c4ec0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 --2- 172.21.15.16:0/683327406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9128073240 0x7f91280c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 >> 172.21.15.16:0/683327406 conn(0x7f912806f900 msgr2=0x7f91280cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 shutdown_connections 2024-09-17T05:01:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:03.080+0000 7f912da00640 1 -- 172.21.15.16:0/683327406 wait complete. 2024-09-17T05:01:05.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 -- 172.21.15.16:0/1909335231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 msgr2=0x7f970c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 --2- 172.21.15.16:0/1909335231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c15bb60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f970c06e960 tx=0x7f970001c830 comp rx=0 tx=0).stop 2024-09-17T05:01:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 -- 172.21.15.16:0/1909335231 shutdown_connections 2024-09-17T05:01:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 --2- 172.21.15.16:0/1909335231 >> v2:172.21.15.104:3300/0 conn(0x7f970c15c0a0 0x7f970c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 --2- 172.21.15.16:0/1909335231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c15bb60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 -- 172.21.15.16:0/1909335231 >> 172.21.15.16:0/1909335231 conn(0x7f970c06f900 msgr2=0x7f970c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 -- 172.21.15.16:0/1909335231 shutdown_connections 2024-09-17T05:01:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 -- 172.21.15.16:0/1909335231 wait complete. 2024-09-17T05:01:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.272+0000 7f9713800640 1 Processor -- start 2024-09-17T05:01:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 -- start start 2024-09-17T05:01:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f970c15c0a0 0x7f970c1fa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f970c15f150 0x7f970c1608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f970c169010 con 0x7f970c0b2f50 2024-09-17T05:01:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f970c168e90 con 0x7f970c15c0a0 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f970c169190 con 0x7f970c15f150 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46658/0 (socket says 172.21.15.16:46658) 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 -- 172.21.15.16:0/2284929825 learned_addr learned my addr 172.21.15.16:0/2284929825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9711800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f970c15f150 0x7f970c1608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f970be00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f970c15c0a0 0x7f970c1fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 -- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f970c15f150 msgr2=0x7f970c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f970c15f150 0x7f970c1608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 -- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f970c15c0a0 msgr2=0x7f970c1fa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f970c15c0a0 0x7f970c1fa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 -- 172.21.15.16:0/2284929825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c1605b0 con 0x7f970c0b2f50 2024-09-17T05:01:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9710e00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c143af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97000b0040 tx=0x7f97000a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000b2040 con 0x7f970c0b2f50 2024-09-17T05:01:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9700004340 con 0x7f970c0b2f50 2024-09-17T05:01:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000044f0 con 0x7f970c0b2f50 2024-09-17T05:01:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c0b1dc0 con 0x7f970c0b2f50 2024-09-17T05:01:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.276+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c1fae20 con 0x7f970c0b2f50 2024-09-17T05:01:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.280+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c15bb60 con 0x7f970c0b2f50 2024-09-17T05:01:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.280+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9700005290 con 0x7f970c0b2f50 2024-09-17T05:01:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.280+0000 7f9708c00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96ec073200 0x7f96ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.280+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f970012dfe0 con 0x7f970c0b2f50 2024-09-17T05:01:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.280+0000 7f970be00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96ec073200 0x7f96ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.284+0000 7f970be00640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96ec073200 0x7f96ec075710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f970c15a930 tx=0x7f96f800b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:05.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.284+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f970c1605b0 con 0x7f970c0b2f50 2024-09-17T05:01:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.496+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f970c15cdf0 con 0x7f970c0b2f50 2024-09-17T05:01:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.496+0000 7f9708c00640 1 -- 172.21.15.16:0/2284929825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7f97000164b0 con 0x7f970c0b2f50 2024-09-17T05:01:05.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:05.501 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"btime":"2024-09-17T05:01:03:810382+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4390,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T05:01:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96ec073200 msgr2=0x7f96ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96ec073200 0x7f96ec075710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f970c15a930 tx=0x7f96f800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 msgr2=0x7f970c143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c143af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97000b0040 tx=0x7f97000a57e0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 shutdown_connections 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96ec073200 0x7f96ec075710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f970c15f150 0x7f970c1608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f970c15c0a0 0x7f970c1fa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 --2- 172.21.15.16:0/2284929825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970c0b2f50 0x7f970c143af0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 >> 172.21.15.16:0/2284929825 conn(0x7f970c06f900 msgr2=0x7f970c1537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 shutdown_connections 2024-09-17T05:01:05.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.500+0000 7f9713800640 1 -- 172.21.15.16:0/2284929825 wait complete. 2024-09-17T05:01:05.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 -- 172.21.15.16:0/468146645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b40073240 msgr2=0x7f9b40073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 --2- 172.21.15.16:0/468146645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b40073240 0x7f9b40073650 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9b40071ce0 tx=0x7f9b3801caf0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 -- 172.21.15.16:0/468146645 shutdown_connections 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 --2- 172.21.15.16:0/468146645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b40073240 0x7f9b40073650 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 --2- 172.21.15.16:0/468146645 >> v2:172.21.15.104:3300/0 conn(0x7f9b400b2f50 0x7f9b4015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 -- 172.21.15.16:0/468146645 >> 172.21.15.16:0/468146645 conn(0x7f9b4006f900 msgr2=0x7f9b400b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 -- 172.21.15.16:0/468146645 shutdown_connections 2024-09-17T05:01:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 -- 172.21.15.16:0/468146645 wait complete. 2024-09-17T05:01:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 Processor -- start 2024-09-17T05:01:05.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.680+0000 7f9b47400640 1 -- start start 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b40073240 0x7f9b4014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b400b2f50 0x7f9b401f9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 0x7f9b401f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40168c70 con 0x7f9b4015c0a0 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b40168af0 con 0x7f9b400b2f50 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9b40168df0 con 0x7f9b40073240 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 0x7f9b401f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 0x7f9b401f9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46668/0 (socket says 172.21.15.16:46668) 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 -- 172.21.15.16:0/1492267125 learned_addr learned my addr 172.21.15.16:0/1492267125 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b44a00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b40073240 0x7f9b4014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b3fe00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b400b2f50 0x7f9b401f9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 -- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b40073240 msgr2=0x7f9b4014c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b40073240 0x7f9b4014c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 -- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b400b2f50 msgr2=0x7f9b401f9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b400b2f50 0x7f9b401f9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 -- 172.21.15.16:0/1492267125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4014cec0 con 0x7f9b4015c0a0 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b44a00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b40073240 0x7f9b4014c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b3fe00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b400b2f50 0x7f9b401f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b45400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 0x7f9b401f9cb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b38098700 tx=0x7f9b38003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380b2040 con 0x7f9b4015c0a0 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b380bb070 con 0x7f9b4015c0a0 2024-09-17T05:01:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b40165a80 con 0x7f9b4015c0a0 2024-09-17T05:01:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38004330 con 0x7f9b4015c0a0 2024-09-17T05:01:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b40164950 con 0x7f9b4015c0a0 2024-09-17T05:01:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.684+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b401657f0 con 0x7f9b4015c0a0 2024-09-17T05:01:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.688+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b380044d0 con 0x7f9b4015c0a0 2024-09-17T05:01:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.688+0000 7f9b3cc00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9b20073230 0x7f9b20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.688+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f9b3812e780 con 0x7f9b4015c0a0 2024-09-17T05:01:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.688+0000 7f9b44a00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9b20073230 0x7f9b20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.688+0000 7f9b44a00640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9b20073230 0x7f9b20075740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009b90 tx=0x7f9b340091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.692+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9b38016770 con 0x7f9b4015c0a0 2024-09-17T05:01:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.868+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9b40164700 con 0x7f9b4015c0a0 2024-09-17T05:01:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.868+0000 7f9b3cc00640 1 -- 172.21.15.16:0/1492267125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f9b380f9b70 con 0x7f9b4015c0a0 2024-09-17T05:01:05.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:05.873 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:01:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:01:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:01:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:01:05.873 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:01:05.873 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9b20073230 msgr2=0x7f9b20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9b20073230 0x7f9b20075740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009b90 tx=0x7f9b340091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 msgr2=0x7f9b401f9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 0x7f9b401f9cb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9b38098700 tx=0x7f9b38003f40 comp rx=0 tx=0).stop 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 shutdown_connections 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9b20073230 0x7f9b20075740 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4015c0a0 0x7f9b401f9cb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b400b2f50 0x7f9b401f9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 --2- 172.21.15.16:0/1492267125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b40073240 0x7f9b4014c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 >> 172.21.15.16:0/1492267125 conn(0x7f9b4006f900 msgr2=0x7f9b4015e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 shutdown_connections 2024-09-17T05:01:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:05.872+0000 7f9b47400640 1 -- 172.21.15.16:0/1492267125 wait complete. 2024-09-17T05:01:05.894 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:01:05.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 -- 172.21.15.16:0/2263732846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4208160670 msgr2=0x7f42081670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 --2- 172.21.15.16:0/2263732846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4208160670 0x7f42081670e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42080cd440 tx=0x7f420001cb60 comp rx=0 tx=0).stop 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 -- 172.21.15.16:0/2263732846 shutdown_connections 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 --2- 172.21.15.16:0/2263732846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4208160670 0x7f42081670e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 --2- 172.21.15.16:0/2263732846 >> v2:172.21.15.104:3300/0 conn(0x7f420806db30 0x7f420815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 -- 172.21.15.16:0/2263732846 >> 172.21.15.16:0/2263732846 conn(0x7f42080cd980 msgr2=0x7f42080cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 -- 172.21.15.16:0/2263732846 shutdown_connections 2024-09-17T05:01:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.052+0000 7f420ea00640 1 -- 172.21.15.16:0/2263732846 wait complete. 2024-09-17T05:01:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 Processor -- start 2024-09-17T05:01:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 -- start start 2024-09-17T05:01:06.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420806db30 0x7f42080cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 0x7f42081f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4208160670 0x7f420813e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4208071180 con 0x7f420815d320 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f420815d940 con 0x7f420806db30 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420ea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4208169e40 con 0x7f4208160670 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 0x7f42081f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 0x7f42081f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46686/0 (socket says 172.21.15.16:46686) 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 -- 172.21.15.16:0/2525595993 learned_addr learned my addr 172.21.15.16:0/2525595993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420d600640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420806db30 0x7f42080cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420e000640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4208160670 0x7f420813e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 -- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4208160670 msgr2=0x7f420813e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4208160670 0x7f420813e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 -- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420806db30 msgr2=0x7f42080cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420806db30 0x7f42080cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.056+0000 7f420cc00640 1 -- 172.21.15.16:0/2525595993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42080b4b10 con 0x7f420815d320 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f420cc00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 0x7f42081f9d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c3a0 tx=0x7f41f400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4017070 con 0x7f420815d320 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41f4003600 con 0x7f420815d320 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4003a50 con 0x7f420815d320 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42080b4df0 con 0x7f420815d320 2024-09-17T05:01:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42080b53b0 con 0x7f420815d320 2024-09-17T05:01:06.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420815cde0 con 0x7f420815d320 2024-09-17T05:01:06.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f41f401a070 con 0x7f420815d320 2024-09-17T05:01:06.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f4205600640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f41e8073230 0x7f41e8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:06.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.060+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f41f4093100 con 0x7f420815d320 2024-09-17T05:01:06.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.064+0000 7f420d600640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f41e8073230 0x7f41e8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:06.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.064+0000 7f420d600640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f41e8073230 0x7f41e8075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41fc004630 tx=0x7f41fc00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:06.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.068+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41f405de10 con 0x7f420815d320 2024-09-17T05:01:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.240+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f420815f960 con 0x7f420815d320 2024-09-17T05:01:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.820+0000 7f4205600640 1 -- 172.21.15.16:0/2525595993 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f41f4062840 con 0x7f420815d320 2024-09-17T05:01:06.825 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f41e8073230 msgr2=0x7f41e8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f41e8073230 0x7f41e8075740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f41fc004630 tx=0x7f41fc00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 msgr2=0x7f42081f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 0x7f42081f9d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41f400c3a0 tx=0x7f41f400c860 comp rx=0 tx=0).stop 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 shutdown_connections 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f41e8073230 0x7f41e8075740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4208160670 0x7f420813e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f420815d320 0x7f42081f9d30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 --2- 172.21.15.16:0/2525595993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420806db30 0x7f42080cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:06.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 >> 172.21.15.16:0/2525595993 conn(0x7f42080cd980 msgr2=0x7f420815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:06.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 shutdown_connections 2024-09-17T05:01:06.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:06.824+0000 7f420ea00640 1 -- 172.21.15.16:0/2525595993 wait complete. 2024-09-17T05:01:06.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-17T05:01:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 -- 172.21.15.16:0/2015095401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600ccff0 msgr2=0x7fcb6006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 --2- 172.21.15.16:0/2015095401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600ccff0 0x7fcb6006d3b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009bc0 tx=0x7fcb5801cb10 comp rx=0 tx=0).stop 2024-09-17T05:01:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 -- 172.21.15.16:0/2015095401 shutdown_connections 2024-09-17T05:01:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 --2- 172.21.15.16:0/2015095401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600ccff0 0x7fcb6006d3b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 --2- 172.21.15.16:0/2015095401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb60159fc0 0x7fcb60160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 -- 172.21.15.16:0/2015095401 >> 172.21.15.16:0/2015095401 conn(0x7fcb6006f9c0 msgr2=0x7fcb600b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 -- 172.21.15.16:0/2015095401 shutdown_connections 2024-09-17T05:01:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.024+0000 7fcb5f200640 1 -- 172.21.15.16:0/2015095401 wait complete. 2024-09-17T05:01:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 Processor -- start 2024-09-17T05:01:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 -- start start 2024-09-17T05:01:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb600cddd0 0x7fcb601587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 0x7fcb60165540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb601fc5e0 0x7fcb60074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6016a0e0 con 0x7fcb60159fc0 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60169f60 con 0x7fcb601fc5e0 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcb6016a260 con 0x7fcb600cddd0 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 0x7fcb60165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 0x7fcb60165540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57248/0 (socket says 172.21.15.16:57248) 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 -- 172.21.15.16:0/842360885 learned_addr learned my addr 172.21.15.16:0/842360885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5e800640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb601fc5e0 0x7fcb60074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5de00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb600cddd0 0x7fcb601587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 -- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb600cddd0 msgr2=0x7fcb601587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb600cddd0 0x7fcb601587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 -- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb601fc5e0 msgr2=0x7fcb60074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb601fc5e0 0x7fcb60074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 -- 172.21.15.16:0/842360885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb600731b0 con 0x7fcb60159fc0 2024-09-17T05:01:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5de00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb600cddd0 0x7fcb601587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb57e00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 0x7fcb60165540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c3a0 tx=0x7fcb4800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48017070 con 0x7fcb60159fc0 2024-09-17T05:01:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb48003600 con 0x7fcb60159fc0 2024-09-17T05:01:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb60073490 con 0x7fcb60159fc0 2024-09-17T05:01:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48003a30 con 0x7fcb60159fc0 2024-09-17T05:01:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.028+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb60073a50 con 0x7fcb60159fc0 2024-09-17T05:01:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.032+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6006d3b0 con 0x7fcb60159fc0 2024-09-17T05:01:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.032+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb4801a070 con 0x7fcb60159fc0 2024-09-17T05:01:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.032+0000 7fcb56000640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073200 0x7fcb3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.032+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7fcb48093640 con 0x7fcb60159fc0 2024-09-17T05:01:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.036+0000 7fcb5de00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073200 0x7fcb3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.036+0000 7fcb5de00640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073200 0x7fcb3c075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000bbd0 tx=0x7fcb5000b5b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.036+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb48021020 con 0x7fcb60159fc0 2024-09-17T05:01:07.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.216+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fcb601fcb30 con 0x7fcb60159fc0 2024-09-17T05:01:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.828+0000 7fcb56000640 1 -- 172.21.15.16:0/842360885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7fcb4805e2d0 con 0x7fcb60159fc0 2024-09-17T05:01:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073200 msgr2=0x7fcb3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073200 0x7fcb3c075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000bbd0 tx=0x7fcb5000b5b0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 msgr2=0x7fcb60165540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 0x7fcb60165540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800c3a0 tx=0x7fcb4800c860 comp rx=0 tx=0).stop 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 shutdown_connections 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073200 0x7fcb3c075710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb601fc5e0 0x7fcb60074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb60159fc0 0x7fcb60165540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 --2- 172.21.15.16:0/842360885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb600cddd0 0x7fcb601587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:07.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 >> 172.21.15.16:0/842360885 conn(0x7fcb6006f9c0 msgr2=0x7fcb600cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 shutdown_connections 2024-09-17T05:01:07.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:07.832+0000 7fcb5f200640 1 -- 172.21.15.16:0/842360885 wait complete. 2024-09-17T05:01:07.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:01:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 -- 172.21.15.16:0/3064424806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600744d0 msgr2=0x7fa66006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 --2- 172.21.15.16:0/3064424806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600744d0 0x7fa66006e0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa6600cec50 tx=0x7fa65801cb80 comp rx=0 tx=0).stop 2024-09-17T05:01:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 -- 172.21.15.16:0/3064424806 shutdown_connections 2024-09-17T05:01:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 --2- 172.21.15.16:0/3064424806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600744d0 0x7fa66006e0d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 --2- 172.21.15.16:0/3064424806 >> v2:172.21.15.104:3300/0 conn(0x7fa66006f5f0 0x7fa66015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 -- 172.21.15.16:0/3064424806 >> 172.21.15.16:0/3064424806 conn(0x7fa6600b2dd0 msgr2=0x7fa6600b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 -- 172.21.15.16:0/3064424806 shutdown_connections 2024-09-17T05:01:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 -- 172.21.15.16:0/3064424806 wait complete. 2024-09-17T05:01:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.048+0000 7fa667200640 1 Processor -- start 2024-09-17T05:01:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- start start 2024-09-17T05:01:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa66006f5f0 0x7fa66014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 0x7fa660150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa66006e040 0x7fa660150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa66016a390 con 0x7fa6600c6040 2024-09-17T05:01:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa66016a210 con 0x7fa66006e040 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa66016a510 con 0x7fa66006f5f0 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 0x7fa660150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 0x7fa660150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57268/0 (socket says 172.21.15.16:57268) 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 -- 172.21.15.16:0/1268829845 learned_addr learned my addr 172.21.15.16:0/1268829845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa65fe00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa66006f5f0 0x7fa66014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa665200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa66006e040 0x7fa660150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 -- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa66006f5f0 msgr2=0x7fa66014cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa66006f5f0 0x7fa66014cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 -- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa66006e040 msgr2=0x7fa660150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa66006e040 0x7fa660150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 -- 172.21.15.16:0/1268829845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa660153e90 con 0x7fa6600c6040 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa65fe00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa66006f5f0 0x7fa66014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa665200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa66006e040 0x7fa660150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa657e00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 0x7fa660150870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c400 tx=0x7fa64800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648017070 con 0x7fa6600c6040 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa648003600 con 0x7fa6600c6040 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa660154170 con 0x7fa6600c6040 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648003a50 con 0x7fa6600c6040 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6601559f0 con 0x7fa6600c6040 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.052+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66015c490 con 0x7fa6600c6040 2024-09-17T05:01:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.056+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa64801a070 con 0x7fa6600c6040 2024-09-17T05:01:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.056+0000 7fa65cc00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa63c073230 0x7fa63c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.056+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fa6480934b0 con 0x7fa6600c6040 2024-09-17T05:01:08.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.056+0000 7fa65fe00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa63c073230 0x7fa63c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:08.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.056+0000 7fa65fe00640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa63c073230 0x7fa63c075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa650009ac0 tx=0x7fa6500091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.060+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa64805e520 con 0x7fa6600c6040 2024-09-17T05:01:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.272+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa660074b80 con 0x7fa6600c6040 2024-09-17T05:01:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.832+0000 7fa65cc00640 1 -- 172.21.15.16:0/1268829845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fa64805de20 con 0x7fa6600c6040 2024-09-17T05:01:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa63c073230 msgr2=0x7fa63c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa63c073230 0x7fa63c075740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa650009ac0 tx=0x7fa6500091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 msgr2=0x7fa660150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 0x7fa660150870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa64800c400 tx=0x7fa64800c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 shutdown_connections 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa63c073230 0x7fa63c075740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa66006e040 0x7fa660150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6600c6040 0x7fa660150870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 --2- 172.21.15.16:0/1268829845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa66006f5f0 0x7fa66014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 >> 172.21.15.16:0/1268829845 conn(0x7fa6600b2dd0 msgr2=0x7fa66015b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 shutdown_connections 2024-09-17T05:01:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:08.836+0000 7fa667200640 1 -- 172.21.15.16:0/1268829845 wait complete. 2024-09-17T05:01:08.903 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-17T05:01:08.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-17T05:01:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 -- 172.21.15.16:0/356918310 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 msgr2=0x7f155c06dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 --2- 172.21.15.16:0/356918310 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 0x7f155c06dfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f154c009900 tx=0x7f154c01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 -- 172.21.15.16:0/356918310 shutdown_connections 2024-09-17T05:01:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 --2- 172.21.15.16:0/356918310 >> v2:172.21.15.104:3300/0 conn(0x7f155c06e4e0 0x7f155c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 --2- 172.21.15.16:0/356918310 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 0x7f155c06dfa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 -- 172.21.15.16:0/356918310 >> 172.21.15.16:0/356918310 conn(0x7f155c0d0050 msgr2=0x7f155c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 -- 172.21.15.16:0/356918310 shutdown_connections 2024-09-17T05:01:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.064+0000 7f1561000640 1 -- 172.21.15.16:0/356918310 wait complete. 2024-09-17T05:01:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 Processor -- start 2024-09-17T05:01:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 -- start start 2024-09-17T05:01:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 0x7f155c144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 0x7f155c0b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f155c15f4e0 0x7f155c0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f155c074b00 con 0x7f155c06e4e0 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f155c074c80 con 0x7f155c15f4e0 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f155c074e00 con 0x7f155c0b4620 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 0x7f155c144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 0x7f155c144aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57272/0 (socket says 172.21.15.16:57272) 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 -- 172.21.15.16:0/1444929738 learned_addr learned my addr 172.21.15.16:0/1444929738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1551400640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 0x7f155c0b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f155a800640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f155c15f4e0 0x7f155c0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:09.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 -- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 msgr2=0x7f155c0b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 0x7f155c0b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 -- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f155c15f4e0 msgr2=0x7f155c0b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f155c15f4e0 0x7f155c0b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 -- 172.21.15.16:0/1444929738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155c144fe0 con 0x7f155c06e4e0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1559e00640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 0x7f155c144aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f154c009850 tx=0x7f154c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0b2040 con 0x7f155c06e4e0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f154c005320 con 0x7f155c06e4e0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155c14fb70 con 0x7f155c06e4e0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c0046d0 con 0x7f155c06e4e0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.068+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f155c154d30 con 0x7f155c06e4e0 2024-09-17T05:01:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.072+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c0cbd40 con 0x7f155c06e4e0 2024-09-17T05:01:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.072+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f154c004870 con 0x7f155c06e4e0 2024-09-17T05:01:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.072+0000 7f1552a00640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1538073200 0x7f1538075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.072+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f154c12e190 con 0x7f155c06e4e0 2024-09-17T05:01:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.072+0000 7f1551400640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1538073200 0x7f1538075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:09.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.072+0000 7f1551400640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1538073200 0x7f1538075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f155c158d80 tx=0x7f15440091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.076+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f155c0cbd40 con 0x7f155c06e4e0 2024-09-17T05:01:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.260+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f155c15efa0 con 0x7f155c06e4e0 2024-09-17T05:01:09.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.848+0000 7f1552a00640 1 -- 172.21.15.16:0/1444929738 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f154c0bc060 con 0x7f155c06e4e0 2024-09-17T05:01:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1538073200 msgr2=0x7f1538075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1538073200 0x7f1538075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f155c158d80 tx=0x7f15440091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 msgr2=0x7f155c144aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 0x7f155c144aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f154c009850 tx=0x7f154c005de0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 shutdown_connections 2024-09-17T05:01:09.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1538073200 0x7f1538075710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f155c15f4e0 0x7f155c0b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f155c0b4620 0x7f155c0b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 --2- 172.21.15.16:0/1444929738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f155c06e4e0 0x7f155c144aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 >> 172.21.15.16:0/1444929738 conn(0x7f155c0d0050 msgr2=0x7f155c159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 shutdown_connections 2024-09-17T05:01:09.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:09.852+0000 7f1561000640 1 -- 172.21.15.16:0/1444929738 wait complete. 2024-09-17T05:01:09.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:01:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 -- 172.21.15.16:0/1722969842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c06dfa0 msgr2=0x7f6c2c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 --2- 172.21.15.16:0/1722969842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c06dfa0 0x7f6c2c06e3b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009840 tx=0x7f6c2001c820 comp rx=0 tx=0).stop 2024-09-17T05:01:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 -- 172.21.15.16:0/1722969842 shutdown_connections 2024-09-17T05:01:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 --2- 172.21.15.16:0/1722969842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c06dfa0 0x7f6c2c06e3b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 -- 172.21.15.16:0/1722969842 >> 172.21.15.16:0/1722969842 conn(0x7f6c2c0d00c0 msgr2=0x7f6c2c0d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 -- 172.21.15.16:0/1722969842 shutdown_connections 2024-09-17T05:01:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.044+0000 7f6c33600640 1 -- 172.21.15.16:0/1722969842 wait complete. 2024-09-17T05:01:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 Processor -- start 2024-09-17T05:01:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 -- start start 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c06dfa0 0x7f6c2c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c2c073cc0 0x7f6c2c1f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 0x7f6c2c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c169fc0 con 0x7f6c2c15c690 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c169e40 con 0x7f6c2c06dfa0 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c33600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c16a140 con 0x7f6c2c073cc0 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 0x7f6c2c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 0x7f6c2c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57290/0 (socket says 172.21.15.16:57290) 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 -- 172.21.15.16:0/39215166 learned_addr learned my addr 172.21.15.16:0/39215166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c30c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c06dfa0 0x7f6c2c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c2be00640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c2c073cc0 0x7f6c2c1f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 -- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c2c073cc0 msgr2=0x7f6c2c1f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c2c073cc0 0x7f6c2c1f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 -- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c06dfa0 msgr2=0x7f6c2c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c06dfa0 0x7f6c2c15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 -- 172.21.15.16:0/39215166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c150d10 con 0x7f6c2c15c690 2024-09-17T05:01:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.048+0000 7f6c31600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 0x7f6c2c152270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c730 tx=0x7f6c2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24003b80 con 0x7f6c2c15c690 2024-09-17T05:01:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c24003d20 con 0x7f6c2c15c690 2024-09-17T05:01:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c150ff0 con 0x7f6c2c15c690 2024-09-17T05:01:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400aa50 con 0x7f6c2c15c690 2024-09-17T05:01:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1515b0 con 0x7f6c2c15c690 2024-09-17T05:01:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c06e3b0 con 0x7f6c2c15c690 2024-09-17T05:01:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c2400abf0 con 0x7f6c2c15c690 2024-09-17T05:01:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c28c00640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c0c073200 0x7f6c0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f6c24094060 con 0x7f6c2c15c690 2024-09-17T05:01:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.052+0000 7f6c30c00640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c0c073200 0x7f6c0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.056+0000 7f6c30c00640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c0c073200 0x7f6c0c075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c200af000 tx=0x7f6c2001b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:10.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.056+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6c2405e860 con 0x7f6c2c15c690 2024-09-17T05:01:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:10.236+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6c2c0adf90 con 0x7f6c2c15c690 2024-09-17T05:01:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.864+0000 7f6c28c00640 1 -- 172.21.15.16:0/39215166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f6c24063290 con 0x7f6c2c15c690 2024-09-17T05:01:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-09-17T05:01:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c0c073200 msgr2=0x7f6c0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c0c073200 0x7f6c0c075710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6c200af000 tx=0x7f6c2001b060 comp rx=0 tx=0).stop 2024-09-17T05:01:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 msgr2=0x7f6c2c152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 0x7f6c2c152270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c730 tx=0x7f6c2400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:01:11.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 shutdown_connections 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c0c073200 0x7f6c0c075710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2c15c690 0x7f6c2c152270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c2c073cc0 0x7f6c2c1f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 --2- 172.21.15.16:0/39215166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c2c06dfa0 0x7f6c2c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.868+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 >> 172.21.15.16:0/39215166 conn(0x7f6c2c0d00c0 msgr2=0x7f6c2c068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.872+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 shutdown_connections 2024-09-17T05:01:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:11.872+0000 7f6c33600640 1 -- 172.21.15.16:0/39215166 wait complete. 2024-09-17T05:01:11.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 -- 172.21.15.16:0/3000647155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c3c0 msgr2=0x7fc6840d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 --2- 172.21.15.16:0/3000647155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c3c0 0x7fc6840d0090 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc684074400 tx=0x7fc67401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 -- 172.21.15.16:0/3000647155 shutdown_connections 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 --2- 172.21.15.16:0/3000647155 >> v2:172.21.15.104:3300/0 conn(0x7fc6840d05d0 0x7fc6840d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 --2- 172.21.15.16:0/3000647155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c3c0 0x7fc6840d0090 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 -- 172.21.15.16:0/3000647155 >> 172.21.15.16:0/3000647155 conn(0x7fc6840d2180 msgr2=0x7fc6840d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 -- 172.21.15.16:0/3000647155 shutdown_connections 2024-09-17T05:01:12.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.068+0000 7fc683200640 1 -- 172.21.15.16:0/3000647155 wait complete. 2024-09-17T05:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 Processor -- start 2024-09-17T05:01:12.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 -- start start 2024-09-17T05:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 0x7fc68414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc68415c3c0 0x7fc68414e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:12.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc68415feb0 0x7fc6841fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc684169cf0 con 0x7fc6840d05d0 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc684169b70 con 0x7fc68415feb0 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc683200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc684169e70 con 0x7fc68415c3c0 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 0x7fc68414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 0x7fc68414dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57302/0 (socket says 172.21.15.16:57302) 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 -- 172.21.15.16:0/173020280 learned_addr learned my addr 172.21.15.16:0/173020280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc67be00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc68415c3c0 0x7fc68414e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc682800640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc68415feb0 0x7fc6841fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 -- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc68415c3c0 msgr2=0x7fc68414e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc68415c3c0 0x7fc68414e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 -- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc68415feb0 msgr2=0x7fc6841fc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc68415feb0 0x7fc6841fc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc681e00640 1 -- 172.21.15.16:0/173020280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68415a8d0 con 0x7fc6840d05d0 2024-09-17T05:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.072+0000 7fc67be00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc68415c3c0 0x7fc68414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc681e00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 0x7fc68414dfb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6740194d0 tx=0x7fc6740a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:12.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6740bb070 con 0x7fc6840d05d0 2024-09-17T05:01:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6740044b0 con 0x7fc6840d05d0 2024-09-17T05:01:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6740b2040 con 0x7fc6840d05d0 2024-09-17T05:01:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68415abb0 con 0x7fc6840d05d0 2024-09-17T05:01:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68415b170 con 0x7fc6840d05d0 2024-09-17T05:01:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6840d09e0 con 0x7fc6840d05d0 2024-09-17T05:01:12.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.076+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc6740052d0 con 0x7fc6840d05d0 2024-09-17T05:01:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.080+0000 7fc67a000640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073230 0x7fc660075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:12.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.080+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7fc67412e100 con 0x7fc6840d05d0 2024-09-17T05:01:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.080+0000 7fc67be00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073230 0x7fc660075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:12.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.080+0000 7fc67be00640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073230 0x7fc660075740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc66c005fd0 tx=0x7fc66c005f20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:12.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.080+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc6740c4080 con 0x7fc6840d05d0 2024-09-17T05:01:12.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.256+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc6840b61c0 con 0x7fc6840d05d0 2024-09-17T05:01:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.868+0000 7fc67a000640 1 -- 172.21.15.16:0/173020280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7fc6740164b0 con 0x7fc6840d05d0 2024-09-17T05:01:12.871 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-09-17T05:01:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.872+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073230 msgr2=0x7fc660075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.872+0000 7fc683200640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073230 0x7fc660075740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc66c005fd0 tx=0x7fc66c005f20 comp rx=0 tx=0).stop 2024-09-17T05:01:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.872+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 msgr2=0x7fc68414dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.872+0000 7fc683200640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 0x7fc68414dfb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6740194d0 tx=0x7fc6740a57f0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 shutdown_connections 2024-09-17T05:01:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073230 0x7fc660075740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc68415feb0 0x7fc6841fc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc68415c3c0 0x7fc68414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 --2- 172.21.15.16:0/173020280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6840d05d0 0x7fc68414dfb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 >> 172.21.15.16:0/173020280 conn(0x7fc6840d2180 msgr2=0x7fc6840cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 shutdown_connections 2024-09-17T05:01:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:12.876+0000 7fc683200640 1 -- 172.21.15.16:0/173020280 wait complete. 2024-09-17T05:01:12.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 -- 172.21.15.16:0/4027750815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 msgr2=0x7f6f040739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 --2- 172.21.15.16:0/4027750815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f040739c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6f040d08e0 tx=0x7f6efc01cd80 comp rx=0 tx=0).stop 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 -- 172.21.15.16:0/4027750815 shutdown_connections 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 --2- 172.21.15.16:0/4027750815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f040739c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 --2- 172.21.15.16:0/4027750815 >> v2:172.21.15.104:3300/0 conn(0x7f6f0415a820 0x7f6f040cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 --2- 172.21.15.16:0/4027750815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f040ca6b0 0x7f6f04074be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.092+0000 7f6f0c200640 1 -- 172.21.15.16:0/4027750815 >> 172.21.15.16:0/4027750815 conn(0x7f6f040b2250 msgr2=0x7f6f040d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 -- 172.21.15.16:0/4027750815 shutdown_connections 2024-09-17T05:01:13.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 -- 172.21.15.16:0/4027750815 wait complete. 2024-09-17T05:01:13.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 Processor -- start 2024-09-17T05:01:13.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 -- start start 2024-09-17T05:01:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f040ca6b0 0x7f6f0415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f0414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f0415a820 0x7f6f0413e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.096+0000 7f6f0c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0415ebd0 con 0x7f6f040cb1d0 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f0c200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0415ea50 con 0x7f6f0415a820 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f0c200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0415ed50 con 0x7f6f040ca6b0 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f0414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f0414c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57326/0 (socket says 172.21.15.16:57326) 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 -- 172.21.15.16:0/3770446007 learned_addr learned my addr 172.21.15.16:0/3770446007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f09800640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f040ca6b0 0x7f6f0415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f0a200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f0415a820 0x7f6f0413e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:13.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 -- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f040ca6b0 msgr2=0x7f6f0415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:13.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f040ca6b0 0x7f6f0415af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 -- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f0415a820 msgr2=0x7f6f0413e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f0415a820 0x7f6f0413e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 -- 172.21.15.16:0/3770446007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0413eaa0 con 0x7f6f040cb1d0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f08e00640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f0414c7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000e7d0 tx=0x7f6ef000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000ccc0 con 0x7f6f040cb1d0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ef0010430 con 0x7f6f040cb1d0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0010860 con 0x7f6f040cb1d0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0414e700 con 0x7f6f040cb1d0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f041f7140 con 0x7f6f040cb1d0 2024-09-17T05:01:13.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0414ec20 con 0x7f6f040cb1d0 2024-09-17T05:01:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.100+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6ef0010a00 con 0x7f6f040cb1d0 2024-09-17T05:01:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.104+0000 7f6f01600640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6ee4073200 0x7f6ee4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.104+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f6ef0094fc0 con 0x7f6f040cb1d0 2024-09-17T05:01:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.104+0000 7f6f09800640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6ee4073200 0x7f6ee4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.104+0000 7f6f09800640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6ee4073200 0x7f6ee4075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009ac0 tx=0x7f6ef80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:13.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.108+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f0414ec20 con 0x7f6f040cb1d0 2024-09-17T05:01:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.320+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f6f040739c0 con 0x7f6f040cb1d0 2024-09-17T05:01:13.819 INFO:tasks.ceph.mds.d.smithi104.stderr:2024-09-17T05:01:13.816+0000 7f3c6ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:01:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.888+0000 7f6f01600640 1 -- 172.21.15.16:0/3770446007 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f6ef005f5d0 con 0x7f6f040cb1d0 2024-09-17T05:01:13.893 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-09-17T05:01:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6ee4073200 msgr2=0x7f6ee4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6ee4073200 0x7f6ee4075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009ac0 tx=0x7f6ef80091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 msgr2=0x7f6f0414c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f0414c7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000e7d0 tx=0x7f6ef000ec90 comp rx=0 tx=0).stop 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 shutdown_connections 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6ee4073200 0x7f6ee4075710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f0415a820 0x7f6f0413e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f040cb1d0 0x7f6f0414c7f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 --2- 172.21.15.16:0/3770446007 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f040ca6b0 0x7f6f0415af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 >> 172.21.15.16:0/3770446007 conn(0x7f6f040b2250 msgr2=0x7f6f040afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 shutdown_connections 2024-09-17T05:01:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:13.892+0000 7f6f0c200640 1 -- 172.21.15.16:0/3770446007 wait complete. 2024-09-17T05:01:13.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 -- 172.21.15.16:0/1850716823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 msgr2=0x7f4b1000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1850716823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b1000b9d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009e90 tx=0x7f4b2001d5f0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 -- 172.21.15.16:0/1850716823 shutdown_connections 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1850716823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b1000b9d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 -- 172.21.15.16:0/1850716823 >> 172.21.15.16:0/1850716823 conn(0x7f4b2806f900 msgr2=0x7f4b280b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 -- 172.21.15.16:0/1850716823 shutdown_connections 2024-09-17T05:01:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.080+0000 7f4b2f800640 1 -- 172.21.15.16:0/1850716823 wait complete. 2024-09-17T05:01:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 Processor -- start 2024-09-17T05:01:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 -- start start 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b2814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b280cbe70 0x7f4b281f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b281f55a0 0x7f4b2815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2815bef0 con 0x7f4b10005360 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2815bd70 con 0x7f4b281f55a0 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2f800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2815c070 con 0x7f4b280cbe70 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b2814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b2814eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57338/0 (socket says 172.21.15.16:57338) 2024-09-17T05:01:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 -- 172.21.15.16:0/1672095329 learned_addr learned my addr 172.21.15.16:0/1672095329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b27e00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b280cbe70 0x7f4b281f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2d800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b281f55a0 0x7f4b2815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 -- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b280cbe70 msgr2=0x7f4b281f9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b280cbe70 0x7f4b281f9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 -- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b281f55a0 msgr2=0x7f4b2815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b281f55a0 0x7f4b2815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.084+0000 7f4b2ce00640 1 -- 172.21.15.16:0/1672095329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b281e67b0 con 0x7f4b10005360 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b2ce00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b2814eed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c003ed0 tx=0x7f4b1c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c015070 con 0x7f4b10005360 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b1c011840 con 0x7f4b10005360 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c011c70 con 0x7f4b10005360 2024-09-17T05:01:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281e6a90 con 0x7f4b10005360 2024-09-17T05:01:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281f8c90 con 0x7f4b10005360 2024-09-17T05:01:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2815f310 con 0x7f4b10005360 2024-09-17T05:01:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b1c018070 con 0x7f4b10005360 2024-09-17T05:01:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b24c00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b08073200 0x7f4b08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.088+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4b1c095060 con 0x7f4b10005360 2024-09-17T05:01:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.092+0000 7f4b27e00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b08073200 0x7f4b08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.092+0000 7f4b27e00640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b08073200 0x7f4b08075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b14001020 tx=0x7f4b140067b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:14.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.092+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b1c05f770 con 0x7f4b10005360 2024-09-17T05:01:14.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.304+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b280cc2e0 con 0x7f4b10005360 2024-09-17T05:01:14.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.304+0000 7f4b24c00640 1 -- 172.21.15.16:0/1672095329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5222 (secure 0 0 0) 0x7f4b1c0641a0 con 0x7f4b10005360 2024-09-17T05:01:14.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:14.309 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"btime":"2024-09-17T05:01:13:886815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:13.324651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:14.309 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b08073200 msgr2=0x7f4b08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b08073200 0x7f4b08075710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4b14001020 tx=0x7f4b140067b0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 msgr2=0x7f4b2814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b2814eed0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c003ed0 tx=0x7f4b1c001640 comp rx=0 tx=0).stop 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 shutdown_connections 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b08073200 0x7f4b08075710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b281f55a0 0x7f4b2815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b280cbe70 0x7f4b281f9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 --2- 172.21.15.16:0/1672095329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b10005360 0x7f4b2814eed0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 >> 172.21.15.16:0/1672095329 conn(0x7f4b2806f900 msgr2=0x7f4b2806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 shutdown_connections 2024-09-17T05:01:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.308+0000 7f4b2f800640 1 -- 172.21.15.16:0/1672095329 wait complete. 2024-09-17T05:01:14.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 -- 172.21.15.16:0/395692973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840c9440 msgr2=0x7fe5840c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 --2- 172.21.15.16:0/395692973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840c9440 0x7fe5840c9850 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009c80 tx=0x7fe57c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 -- 172.21.15.16:0/395692973 shutdown_connections 2024-09-17T05:01:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 --2- 172.21.15.16:0/395692973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840c9440 0x7fe5840c9850 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 --2- 172.21.15.16:0/395692973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5840d1c90 0x7fe58415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 -- 172.21.15.16:0/395692973 >> 172.21.15.16:0/395692973 conn(0x7fe5840d2270 msgr2=0x7fe5840d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 -- 172.21.15.16:0/395692973 shutdown_connections 2024-09-17T05:01:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.484+0000 7fe58a000640 1 -- 172.21.15.16:0/395692973 wait complete. 2024-09-17T05:01:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 Processor -- start 2024-09-17T05:01:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 -- start start 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5840c9440 0x7fe5840b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 0x7fe584150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5841f9fb0 0x7fe5841fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe584169fa0 con 0x7fe5840cc8c0 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe584169e20 con 0x7fe5840c9440 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe58a000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe58416a120 con 0x7fe5841f9fb0 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 0x7fe584150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 0x7fe584150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57356/0 (socket says 172.21.15.16:57356) 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 -- 172.21.15.16:0/3861008881 learned_addr learned my addr 172.21.15.16:0/3861008881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe583e00640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5841f9fb0 0x7fe5841fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe583400640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5840c9440 0x7fe5840b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 -- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5841f9fb0 msgr2=0x7fe5841fa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5841f9fb0 0x7fe5841fa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 -- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5840c9440 msgr2=0x7fe5840b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5840c9440 0x7fe5840b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe57be00640 1 -- 172.21.15.16:0/3861008881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe584151450 con 0x7fe5840cc8c0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.488+0000 7fe583400640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5840c9440 0x7fe5840b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57be00640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 0x7fe584150aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c9a0 tx=0x7fe56c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c007bb0 con 0x7fe5840cc8c0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe56c007e40 con 0x7fe5840cc8c0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00f6b0 con 0x7fe5840cc8c0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe584151730 con 0x7fe5840cc8c0 2024-09-17T05:01:14.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5840c6f70 con 0x7fe5840cc8c0 2024-09-17T05:01:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe584151c50 con 0x7fe5840cc8c0 2024-09-17T05:01:14.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe56c00f850 con 0x7fe5840cc8c0 2024-09-17T05:01:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57b400640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe560073200 0x7fe560075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.492+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe56c094d80 con 0x7fe5840cc8c0 2024-09-17T05:01:14.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.496+0000 7fe583400640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe560073200 0x7fe560075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.496+0000 7fe583400640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe560073200 0x7fe560075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5840b1bc0 tx=0x7fe574008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:14.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.496+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe56c014030 con 0x7fe5840cc8c0 2024-09-17T05:01:14.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.672+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5840c9850 con 0x7fe5840cc8c0 2024-09-17T05:01:14.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe57b400640 1 -- 172.21.15.16:0/3861008881 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7fe56c05f9c0 con 0x7fe5840cc8c0 2024-09-17T05:01:14.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:14.679 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe560073200 msgr2=0x7fe560075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe560073200 0x7fe560075710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5840b1bc0 tx=0x7fe574008040 comp rx=0 tx=0).stop 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 msgr2=0x7fe584150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 0x7fe584150aa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00c9a0 tx=0x7fe56c00ce60 comp rx=0 tx=0).stop 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 shutdown_connections 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe560073200 0x7fe560075710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5841f9fb0 0x7fe5841fa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5840cc8c0 0x7fe584150aa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 --2- 172.21.15.16:0/3861008881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5840c9440 0x7fe5840b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 >> 172.21.15.16:0/3861008881 conn(0x7fe5840d2270 msgr2=0x7fe5841531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:14.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 shutdown_connections 2024-09-17T05:01:14.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.676+0000 7fe58a000640 1 -- 172.21.15.16:0/3861008881 wait complete. 2024-09-17T05:01:14.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3899800640 1 -- 172.21.15.16:0/1215318429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878004810 msgr2=0x7f3878004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3899800640 1 --2- 172.21.15.16:0/1215318429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3878004810 0x7f3878004c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f387c001940 tx=0x7f387c01eff0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3893400640 1 -- 172.21.15.16:0/1215318429 reap_dead start 2024-09-17T05:01:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3899800640 1 -- 172.21.15.16:0/1215318429 shutdown_connections 2024-09-17T05:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3899800640 1 -- 172.21.15.16:0/1215318429 >> 172.21.15.16:0/1215318429 conn(0x7f38940d0860 msgr2=0x7f38940d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3899800640 1 -- 172.21.15.16:0/1215318429 shutdown_connections 2024-09-17T05:01:14.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.856+0000 7f3899800640 1 -- 172.21.15.16:0/1215318429 wait complete. 2024-09-17T05:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 Processor -- start 2024-09-17T05:01:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 -- start start 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3878004810 0x7f3894154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38941f9b10 0x7f38941e9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 0x7f389414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38940d03c0 con 0x7f38941e99a0 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38940b5a80 con 0x7f38941f9b10 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3899800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f38940b4640 con 0x7f3878004810 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 0x7f389414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 0x7f389414ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57370/0 (socket says 172.21.15.16:57370) 2024-09-17T05:01:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 -- 172.21.15.16:0/569205276 learned_addr learned my addr 172.21.15.16:0/569205276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3892a00640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3878004810 0x7f3894154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f388a000640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38941f9b10 0x7f38941e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 -- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3878004810 msgr2=0x7f3894154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3878004810 0x7f3894154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 -- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38941f9b10 msgr2=0x7f38941e9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.860+0000 7f3893400640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38941f9b10 0x7f38941e9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f3893400640 1 -- 172.21.15.16:0/569205276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38941552a0 con 0x7f38941e99a0 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388a000640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38941f9b10 0x7f38941e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f3893400640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 0x7f389414ea80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3894072b00 tx=0x7f388c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c004b30 con 0x7f38941e99a0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f388c004cd0 con 0x7f38941e99a0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c003be0 con 0x7f38941e99a0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38941557b0 con 0x7f38941e99a0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38941f9310 con 0x7f38941e99a0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3894155520 con 0x7f38941e99a0 2024-09-17T05:01:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f388c004030 con 0x7f38941e99a0 2024-09-17T05:01:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388b400640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3870073200 0x7f3870075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.864+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f388c094d80 con 0x7f38941e99a0 2024-09-17T05:01:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.868+0000 7f3892a00640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3870073200 0x7f3870075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:14.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.868+0000 7f3892a00640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3870073200 0x7f3870075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3884001ec0 tx=0x7f3884000fa0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:14.872+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f388c05f400 con 0x7f38941e99a0 2024-09-17T05:01:15.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.044+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38940cf320 con 0x7f38941e99a0 2024-09-17T05:01:15.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f388b400640 1 -- 172.21.15.16:0/569205276 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f388c063e30 con 0x7f38941e99a0 2024-09-17T05:01:15.052 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:15.053 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3870073200 msgr2=0x7f3870075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3870073200 0x7f3870075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3884001ec0 tx=0x7f3884000fa0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 msgr2=0x7f389414ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 0x7f389414ea80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3894072b00 tx=0x7f388c00f3c0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 shutdown_connections 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3870073200 0x7f3870075710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38941e99a0 0x7f389414ea80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38941f9b10 0x7f38941e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 --2- 172.21.15.16:0/569205276 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3878004810 0x7f3894154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 >> 172.21.15.16:0/569205276 conn(0x7f38940d0860 msgr2=0x7f389406e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 shutdown_connections 2024-09-17T05:01:15.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.048+0000 7f3899800640 1 -- 172.21.15.16:0/569205276 wait complete. 2024-09-17T05:01:15.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 -- 172.21.15.16:0/2529737960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0901607f0 msgr2=0x7fb0900b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 --2- 172.21.15.16:0/2529737960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0901607f0 0x7fb0900b1bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb078009840 tx=0x7fb07801c790 comp rx=0 tx=0).stop 2024-09-17T05:01:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 -- 172.21.15.16:0/2529737960 shutdown_connections 2024-09-17T05:01:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 --2- 172.21.15.16:0/2529737960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0901607f0 0x7fb0900b1bb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 -- 172.21.15.16:0/2529737960 >> 172.21.15.16:0/2529737960 conn(0x7fb0900cecc0 msgr2=0x7fb0900cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 -- 172.21.15.16:0/2529737960 shutdown_connections 2024-09-17T05:01:15.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.243+0000 7fb096e00640 1 -- 172.21.15.16:0/2529737960 wait complete. 2024-09-17T05:01:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 Processor -- start 2024-09-17T05:01:15.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 -- start start 2024-09-17T05:01:15.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 0x7fb09006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb090152240 0x7fb090155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb090166f20 0x7fb090151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb09016b150 con 0x7fb09015a300 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb09016afd0 con 0x7fb090166f20 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb09016b2d0 con 0x7fb090152240 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 0x7fb09006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 0x7fb09006dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57380/0 (socket says 172.21.15.16:57380) 2024-09-17T05:01:15.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 -- 172.21.15.16:0/1624209682 learned_addr learned my addr 172.21.15.16:0/1624209682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb087e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb090152240 0x7fb090155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb094e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb090166f20 0x7fb090151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 -- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb090152240 msgr2=0x7fb090155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb090152240 0x7fb090155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 -- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb090166f20 msgr2=0x7fb090151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb090166f20 0x7fb090151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 -- 172.21.15.16:0/1624209682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb090166c90 con 0x7fb09015a300 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb087e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb090152240 0x7fb090155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb094e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb090166f20 0x7fb090151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08fe00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 0x7fb09006dfa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c420 tx=0x7fb08000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:15.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080017070 con 0x7fb09015a300 2024-09-17T05:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0800053c0 con 0x7fb09015a300 2024-09-17T05:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080003860 con 0x7fb09015a300 2024-09-17T05:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0901fbff0 con 0x7fb09015a300 2024-09-17T05:01:15.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.247+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0901541f0 con 0x7fb09015a300 2024-09-17T05:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.251+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0900d1ce0 con 0x7fb09015a300 2024-09-17T05:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.251+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb08001a070 con 0x7fb09015a300 2024-09-17T05:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.251+0000 7fb08cc00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb06c073200 0x7fb06c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.251+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb080093980 con 0x7fb09015a300 2024-09-17T05:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.251+0000 7fb087e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb06c073200 0x7fb06c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.251+0000 7fb087e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb06c073200 0x7fb06c075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0780af000 tx=0x7fb07801afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.255+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb08005e5c0 con 0x7fb09015a300 2024-09-17T05:01:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.467+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0900cf530 con 0x7fb09015a300 2024-09-17T05:01:15.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.467+0000 7fb08cc00640 1 -- 172.21.15.16:0/1624209682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5222 (secure 0 0 0) 0x7fb08005dec0 con 0x7fb09015a300 2024-09-17T05:01:15.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:15.472 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"btime":"2024-09-17T05:01:13:886815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:13.324651+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:15.472 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.467+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb06c073200 msgr2=0x7fb06c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.467+0000 7fb096e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb06c073200 0x7fb06c075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0780af000 tx=0x7fb07801afd0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.467+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 msgr2=0x7fb09006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.467+0000 7fb096e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 0x7fb09006dfa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c420 tx=0x7fb08000c8e0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 shutdown_connections 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb06c073200 0x7fb06c075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb090166f20 0x7fb090151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb090152240 0x7fb090155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 --2- 172.21.15.16:0/1624209682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb09015a300 0x7fb09006dfa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 >> 172.21.15.16:0/1624209682 conn(0x7fb0900cecc0 msgr2=0x7fb0900701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 shutdown_connections 2024-09-17T05:01:15.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.471+0000 7fb096e00640 1 -- 172.21.15.16:0/1624209682 wait complete. 2024-09-17T05:01:15.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 -- 172.21.15.16:0/3492700900 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 msgr2=0x7f91dc0c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 --2- 172.21.15.16:0/3492700900 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0c9850 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009c80 tx=0x7f91d401cda0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 -- 172.21.15.16:0/3492700900 shutdown_connections 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 --2- 172.21.15.16:0/3492700900 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0c9850 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 --2- 172.21.15.16:0/3492700900 >> v2:172.21.15.104:3300/0 conn(0x7f91dc0d1c90 0x7f91dc15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 -- 172.21.15.16:0/3492700900 >> 172.21.15.16:0/3492700900 conn(0x7f91dc0d2270 msgr2=0x7f91dc0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 -- 172.21.15.16:0/3492700900 shutdown_connections 2024-09-17T05:01:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.647+0000 7f91e1c00640 1 -- 172.21.15.16:0/3492700900 wait complete. 2024-09-17T05:01:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 Processor -- start 2024-09-17T05:01:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 -- start start 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 0x7f91dc0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91dc152ff0 0x7f91dc0b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc169f00 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc169d80 con 0x7f91dc152ff0 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91e1c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc16a080 con 0x7f91dc0c9440 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91d3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 0x7f91dc0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91d3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 0x7f91dc0b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57384/0 (socket says 172.21.15.16:57384) 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.651+0000 7f91d3e00640 1 -- 172.21.15.16:0/1735421621 learned_addr learned my addr 172.21.15.16:0/1735421621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91daa00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91db400640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91dc152ff0 0x7f91dc0b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d3e00640 1 -- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 msgr2=0x7f91dc0712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d3e00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d3e00640 1 -- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91dc152ff0 msgr2=0x7f91dc0b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d3e00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91dc152ff0 0x7f91dc0b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d3e00640 1 -- 172.21.15.16:0/1735421621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc071b50 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91daa00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91db400640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91dc152ff0 0x7f91dc0b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:15.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d3e00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 0x7f91dc0b1080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e7f0 tx=0x7f91c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c400ccc0 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91c400ce80 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc0b05a0 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4005280 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc0b15c0 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc0c9850 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.655+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91c4005420 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.659+0000 7f91d2a00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91b8073230 0x7f91b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.659+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f91c4094f80 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.659+0000 7f91daa00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91b8073230 0x7f91b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.659+0000 7f91daa00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91b8073230 0x7f91b8075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000f70 tx=0x7f91cc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:15.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.663+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f91c405fc50 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.835+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91dc0718e0 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91d2a00640 1 -- 172.21.15.16:0/1735421621 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f91c405f550 con 0x7f91dc0cc8c0 2024-09-17T05:01:15.843 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:15.843 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91b8073230 msgr2=0x7f91b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91b8073230 0x7f91b8075740 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f91cc000f70 tx=0x7f91cc009040 comp rx=0 tx=0).stop 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 msgr2=0x7f91dc0b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 0x7f91dc0b1080 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f91c400e7f0 tx=0x7f91c400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 shutdown_connections 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91b8073230 0x7f91b8075740 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91dc152ff0 0x7f91dc0b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc0cc8c0 0x7f91dc0b1080 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 --2- 172.21.15.16:0/1735421621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91dc0c9440 0x7f91dc0712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 >> 172.21.15.16:0/1735421621 conn(0x7f91dc0d2270 msgr2=0x7f91dc0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 shutdown_connections 2024-09-17T05:01:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:15.839+0000 7f91e1c00640 1 -- 172.21.15.16:0/1735421621 wait complete. 2024-09-17T05:01:16.268 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:01:08.278850+0000', 'modified': '2024-09-17T05:01:13.324651+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4390}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4390': {'gid': 4390, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6836/636271305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6835', 'nonce': 636271305}, {'type': 'v1', 'addr': '172.21.15.104:6836', 'nonce': 636271305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4390, 'qdb_cluster': [4390]} 2024-09-17T05:01:16.268 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:01:16.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:01:16.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 -- 172.21.15.16:0/1522014015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284159f70 msgr2=0x7f92840d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 --2- 172.21.15.16:0/1522014015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284159f70 0x7f92840d1b80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9274009840 tx=0x7f927401c6d0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 -- 172.21.15.16:0/1522014015 shutdown_connections 2024-09-17T05:01:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 --2- 172.21.15.16:0/1522014015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9284159f70 0x7f92840d1b80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 -- 172.21.15.16:0/1522014015 >> 172.21.15.16:0/1522014015 conn(0x7f92840b42d0 msgr2=0x7f92840b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 -- 172.21.15.16:0/1522014015 shutdown_connections 2024-09-17T05:01:16.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 -- 172.21.15.16:0/1522014015 wait complete. 2024-09-17T05:01:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 Processor -- start 2024-09-17T05:01:16.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.427+0000 7f928a400640 1 -- start start 2024-09-17T05:01:16.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92840d02d0 0x7f9284147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9284159f70 0x7f92840b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 0x7f92840b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f928416b040 con 0x7f928414b8b0 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f928416aec0 con 0x7f92840d02d0 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f928416b1c0 con 0x7f9284159f70 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 0x7f92840b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92840d02d0 0x7f9284147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 0x7f92840b72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57404/0 (socket says 172.21.15.16:57404) 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 -- 172.21.15.16:0/2324963132 learned_addr learned my addr 172.21.15.16:0/2324963132 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f927aa00640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9284159f70 0x7f92840b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 -- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9284159f70 msgr2=0x7f92840b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9284159f70 0x7f92840b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 -- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92840d02d0 msgr2=0x7f9284147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92840d02d0 0x7f9284147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 -- 172.21.15.16:0/2324963132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9284157c30 con 0x7f928414b8b0 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f9283e00640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 0x7f92840b72e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c760 tx=0x7f927c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f927be00640 1 -- 172.21.15.16:0/2324963132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c003b80 con 0x7f928414b8b0 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f927be00640 1 -- 172.21.15.16:0/2324963132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f927c003d20 con 0x7f928414b8b0 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f927be00640 1 -- 172.21.15.16:0/2324963132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c00aab0 con 0x7f928414b8b0 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9284158130 con 0x7f928414b8b0 2024-09-17T05:01:16.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.431+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92841586f0 con 0x7f928414b8b0 2024-09-17T05:01:16.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f927be00640 1 -- 172.21.15.16:0/2324963132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f927c00ac50 con 0x7f928414b8b0 2024-09-17T05:01:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f927be00640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9260073200 0x7f9260075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f9291c73000 1 Processor -- start 2024-09-17T05:01:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f927be00640 1 -- 172.21.15.16:0/2324963132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f927c094290 con 0x7f928414b8b0 2024-09-17T05:01:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f9291c73000 1 -- start start 2024-09-17T05:01:16.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f9283400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9260073200 0x7f9260075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.435+0000 7f9283400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9260073200 0x7f9260075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f92740af000 tx=0x7f927401af10 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ade6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95adebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e95976ca0 0x555e95adf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e959b3410 con 0x555e9598d950 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e958f0b40 con 0x555e95976ca0 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x555e958dff70 con 0x555e9559d460 2024-09-17T05:01:16.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95adebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95adebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57420/0 (socket says 172.21.15.16:57420) 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 -- 172.21.15.16:0/1835640982 learned_addr learned my addr 172.21.15.16:0/1835640982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9283e00640 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e95976ca0 0x555e95adf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9283400640 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ade6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 -- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 msgr2=0x555e95ade6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ade6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 -- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e95976ca0 msgr2=0x555e95adf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e95976ca0 0x555e95adf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 -- 172.21.15.16:0/1835640982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e958a75b0 con 0x555e9598d950 2024-09-17T05:01:16.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f927aa00640 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95adebe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f926c001710 tx=0x7f926c001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3e91d76eb82fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9273e00640 1 -- 172.21.15.16:0/1835640982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0052e0 con 0x555e9598d950 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9273e00640 1 -- 172.21.15.16:0/1835640982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f926c018070 con 0x555e9598d950 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9273e00640 1 -- 172.21.15.16:0/1835640982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c00fa10 con 0x555e9598d950 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- 172.21.15.16:0/1835640982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 msgr2=0x555e95adebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95adebe0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f926c001710 tx=0x7f926c001bd0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- 172.21.15.16:0/1835640982 shutdown_connections 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e95976ca0 0x555e95adf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95adebe0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 --2- 172.21.15.16:0/1835640982 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ade6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- 172.21.15.16:0/1835640982 >> 172.21.15.16:0/1835640982 conn(0x555e95a63d90 msgr2=0x555e95a93690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- 172.21.15.16:0/1835640982 shutdown_connections 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- 172.21.15.16:0/1835640982 wait complete. 2024-09-17T05:01:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 -- start start 2024-09-17T05:01:16.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.439+0000 7f9291c73000 1 Processor -- start 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ae5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 0x555e95ae5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95ae63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e9587a5f0 con 0x555e95976ca0 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e958e3d80 con 0x555e9598d950 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x555e95886520 con 0x555e9559d460 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 10 client.0 _ll_register_callbacks cb 0x555e9588a830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 0x555e95ae5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 0x555e95ae5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57424/0 (socket says 172.21.15.16:57424) 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 -- 172.21.15.16:0/282030727 learned_addr learned my addr 172.21.15.16:0/282030727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9283400640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ae5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9283e00640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95ae63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 -- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 msgr2=0x555e95ae5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ae5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 -- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 msgr2=0x555e95ae63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95ae63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 -- 172.21.15.16:0/282030727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9587e190 con 0x555e95976ca0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f927aa00640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 0x555e95ae5e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f926c015040 tx=0x7f926c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5837ed7421849d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9270c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9270c00640 1 -- 172.21.15.16:0/282030727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c021070 con 0x555e95976ca0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9270c00640 1 -- 172.21.15.16:0/282030727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f926c003600 con 0x555e95976ca0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e95878110 con 0x555e95976ca0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9270c00640 1 -- 172.21.15.16:0/282030727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f926c003800 con 0x555e95976ca0 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 10 client.4503 fetch_fsmap learned FSMap version 7 2024-09-17T05:01:16.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x555e95964f10 con 0x555e95976ca0 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9270c00640 1 -- 172.21.15.16:0/282030727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555e95878110 con 0x555e95976ca0 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9270c00640 1 -- 172.21.15.16:0/282030727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f926c0039a0 con 0x555e95976ca0 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 10 client.4503 fetch_fsmap finished waiting for FSMap version 7 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 10 client.4503 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 20 client.4503 populate_metadata read hostname 'smithi016' 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e955827d0 0x555e95aefd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 4 client.4503 mds_command: new command op to 4390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 --> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e958f2a90 con 0x555e955827d0 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.443+0000 7f9283400640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e955827d0 0x555e95aefd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9283400640 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e955827d0 0x555e95aefd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583948105070a75d server_cookie=47e8a652565d5a0f in_seq=0 out_seq=0 2024-09-17T05:01:16.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9270c00640 10 client.4503 ms_handle_connect on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9270c00640 1 -- 172.21.15.16:0/282030727 <== mds.0 v2:172.21.15.104:6835/636271305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e958f2a90 con 0x555e955827d0 2024-09-17T05:01:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9270c00640 10 client.4503 handle_command_reply: tid=0 2024-09-17T05:01:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 client.4503 shutdown 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 msgr2=0x555e95ae5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 0x555e95ae5e60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f926c015040 tx=0x7f926c00f7b0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 shutdown_connections 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e955827d0 0x555e95aefd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95ae63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e95976ca0 0x555e95ae5e60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 --2- 172.21.15.16:0/282030727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e9559d460 0x555e95ae5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9270c00640 0 client.4503 ms_handle_reset on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 >> 172.21.15.16:0/282030727 conn(0x555e95a63d90 msgr2=0x555e95a9a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 shutdown_connections 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 1 -- 172.21.15.16:0/282030727 wait complete. 2024-09-17T05:01:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.447+0000 7f9291c73000 20 client.4503 trim_cache size 0 max 16384 2024-09-17T05:01:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 Processor -- start 2024-09-17T05:01:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 -- start start 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 0x555e95b50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b50b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95b510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9284206a40 con 0x555e9559d460 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9284205a40 con 0x555e9598d950 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9284204da0 con 0x555e95976ca0 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 0x555e95b50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 0x555e95b50630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57434/0 (socket says 172.21.15.16:57434) 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 -- 172.21.15.16:0/3790866883 learned_addr learned my addr 172.21.15.16:0/3790866883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283e00640 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95b510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f927aa00640 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b50b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 -- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 msgr2=0x555e95b50b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b50b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 -- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 msgr2=0x555e95b510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95b510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 -- 172.21.15.16:0/3790866883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9593bdb0 con 0x555e9559d460 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f927aa00640 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b50b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9283400640 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 0x555e95b50630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92740b0040 tx=0x7f92740b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81445f155aa3a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9273400640 1 -- 172.21.15.16:0/3790866883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004100 con 0x555e9559d460 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9273400640 1 -- 172.21.15.16:0/3790866883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9274005130 con 0x555e9559d460 2024-09-17T05:01:16.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.467+0000 7f9273400640 1 -- 172.21.15.16:0/3790866883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274005500 con 0x555e9559d460 2024-09-17T05:01:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- 172.21.15.16:0/3790866883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 msgr2=0x555e95b50630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 0x555e95b50630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92740b0040 tx=0x7f92740b9000 comp rx=0 tx=0).stop 2024-09-17T05:01:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- 172.21.15.16:0/3790866883 shutdown_connections 2024-09-17T05:01:16.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9598d950 0x555e95b510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b50b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- 172.21.15.16:0/3790866883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9559d460 0x555e95b50630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- 172.21.15.16:0/3790866883 >> 172.21.15.16:0/3790866883 conn(0x555e95a63d90 msgr2=0x555e95a92cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- 172.21.15.16:0/3790866883 shutdown_connections 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- 172.21.15.16:0/3790866883 wait complete. 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- start start 2024-09-17T05:01:16.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 Processor -- start 2024-09-17T05:01:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9559d460 0x555e95b54670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b54bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95b550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555e958e3d80 con 0x555e9598d950 2024-09-17T05:01:16.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555e95964a80 con 0x555e9559d460 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x555e958e3090 con 0x555e95976ca0 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9291c73000 10 client.0 _ll_register_callbacks cb 0x7f927c001850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95b550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95b550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57436/0 (socket says 172.21.15.16:57436) 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 -- 172.21.15.16:0/4033460537 learned_addr learned my addr 172.21.15.16:0/4033460537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283400640 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9559d460 0x555e95b54670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 -- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 msgr2=0x555e95b54bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b54bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 -- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9559d460 msgr2=0x555e95b54670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9559d460 0x555e95b54670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.471+0000 7f9283e00640 1 -- 172.21.15.16:0/4033460537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e958a6ab0 con 0x555e9598d950 2024-09-17T05:01:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9283e00640 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95b550f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f927c05e810 tx=0x7f927c001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668209b22c90607 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 1 -- 172.21.15.16:0/4033460537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c05dff0 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 1 -- 172.21.15.16:0/4033460537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f927c082de0 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9274005130 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 1 -- 172.21.15.16:0/4033460537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f927c05e390 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 10 client.4509 fetch_fsmap learned FSMap version 7 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x555e95885070 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 1 -- 172.21.15.16:0/4033460537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9274005130 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 1 -- 172.21.15.16:0/4033460537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f927c05fb00 con 0x555e9598d950 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 10 client.4509 fetch_fsmap finished waiting for FSMap version 7 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 10 client.4509 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 20 client.4509 populate_metadata read hostname 'smithi016' 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e95570830 0x555e95b3c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 4 client.4509 mds_command: new command op to 4390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:01:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 --> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] -- command(tid 0: {"prefix": "status"}) -- 0x555e958ee540 con 0x555e95570830 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9283400640 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e95570830 0x555e95b3c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9283400640 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e95570830 0x555e95b3c930 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eabb131394dedec server_cookie=f394f5faa2f215e6 in_seq=0 out_seq=0 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 10 client.4509 ms_handle_connect on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 1 -- 172.21.15.16:0/4033460537 <== mds.0 v2:172.21.15.104:6835/636271305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x555e958ee540 con 0x555e95570830 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 10 client.4509 handle_command_reply: tid=0 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 client.4509 shutdown 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 msgr2=0x555e95b550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95b550f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f927c05e810 tx=0x7f927c001410 comp rx=0 tx=0).stop 2024-09-17T05:01:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 shutdown_connections 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x555e95570830 0x555e95b3c930 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555e9598d950 0x555e95b550f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555e95976ca0 0x555e95b54bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9270c00640 0 client.4509 ms_handle_reset on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 --2- 172.21.15.16:0/4033460537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e9559d460 0x555e95b54670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 >> 172.21.15.16:0/4033460537 conn(0x555e95a63d90 msgr2=0x555e95aef6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 shutdown_connections 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 1 -- 172.21.15.16:0/4033460537 wait complete. 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f9291c73000 20 client.4509 trim_cache size 0 max 16384 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:01:16.483 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4390, 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 7.6440720539999996, 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 7, 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 13.670609156999999 2024-09-17T05:01:16.484 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:01:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9260073200 msgr2=0x7f9260075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9260073200 0x7f9260075710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f92740af000 tx=0x7f927401af10 comp rx=0 tx=0).stop 2024-09-17T05:01:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 msgr2=0x7f92840b72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 0x7f92840b72e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c760 tx=0x7f927c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:01:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 shutdown_connections 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9260073200 0x7f9260075710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f928414b8b0 0x7f92840b72e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9284159f70 0x7f92840b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 --2- 172.21.15.16:0/2324963132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92840d02d0 0x7f9284147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 >> 172.21.15.16:0/2324963132 conn(0x7f92840b42d0 msgr2=0x7f928415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 shutdown_connections 2024-09-17T05:01:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.475+0000 7f928a400640 1 -- 172.21.15.16:0/2324963132 wait complete. 2024-09-17T05:01:16.500 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:01:16.500 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:16.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:01:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 -- 172.21.15.16:0/165739573 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f611415c270 msgr2=0x7f611415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 --2- 172.21.15.16:0/165739573 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f611415c270 0x7f611415c680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6100009870 tx=0x7f610001c7d0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 -- 172.21.15.16:0/165739573 shutdown_connections 2024-09-17T05:01:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 --2- 172.21.15.16:0/165739573 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f611415c270 0x7f611415c680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 -- 172.21.15.16:0/165739573 >> 172.21.15.16:0/165739573 conn(0x7f611406f900 msgr2=0x7f61140b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 -- 172.21.15.16:0/165739573 shutdown_connections 2024-09-17T05:01:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.655+0000 7f611c200640 1 -- 172.21.15.16:0/165739573 wait complete. 2024-09-17T05:01:16.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 Processor -- start 2024-09-17T05:01:16.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- start start 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 0x7f61141438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61140b2f50 0x7f6114164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611415c270 0x7f61141652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6114169e50 con 0x7f6114073240 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6114169cd0 con 0x7f611415c270 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6114169fd0 con 0x7f61140b2f50 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 0x7f61141438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 0x7f61141438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57450/0 (socket says 172.21.15.16:57450) 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 -- 172.21.15.16:0/2031351142 learned_addr learned my addr 172.21.15.16:0/2031351142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6118e00640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61140b2f50 0x7f6114164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611a200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611415c270 0x7f61141652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 -- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61140b2f50 msgr2=0x7f6114164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61140b2f50 0x7f6114164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 -- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611415c270 msgr2=0x7f61141652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611415c270 0x7f61141652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 -- 172.21.15.16:0/2031351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61141510c0 con 0x7f6114073240 2024-09-17T05:01:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6119800640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 0x7f61141438f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f610800c410 tx=0x7f610800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108017070 con 0x7f6114073240 2024-09-17T05:01:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61080053c0 con 0x7f6114073240 2024-09-17T05:01:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108003860 con 0x7f6114073240 2024-09-17T05:01:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6114143e30 con 0x7f6114073240 2024-09-17T05:01:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611415d770 con 0x7f6114073240 2024-09-17T05:01:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.659+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f611415c680 con 0x7f6114073240 2024-09-17T05:01:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.663+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f610801a070 con 0x7f6114073240 2024-09-17T05:01:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.663+0000 7f6111600640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60f4073230 0x7f60f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.663+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f61080939c0 con 0x7f6114073240 2024-09-17T05:01:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.663+0000 7f6118e00640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60f4073230 0x7f60f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:16.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.663+0000 7f6118e00640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60f4073230 0x7f60f4075740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61000af000 tx=0x7f610001b010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:16.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.667+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f610805e690 con 0x7f6114073240 2024-09-17T05:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f611406e130 con 0x7f6114073240 2024-09-17T05:01:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f6111600640 1 -- 172.21.15.16:0/2031351142 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f610805df90 con 0x7f6114073240 2024-09-17T05:01:16.886 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60f4073230 msgr2=0x7f60f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60f4073230 0x7f60f4075740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61000af000 tx=0x7f610001b010 comp rx=0 tx=0).stop 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 msgr2=0x7f61141438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 0x7f61141438f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f610800c410 tx=0x7f610800c8d0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 shutdown_connections 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60f4073230 0x7f60f4075740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f611415c270 0x7f61141652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61140b2f50 0x7f6114164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 --2- 172.21.15.16:0/2031351142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6114073240 0x7f61141438f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 >> 172.21.15.16:0/2031351142 conn(0x7f611406f900 msgr2=0x7f611415bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 shutdown_connections 2024-09-17T05:01:16.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:16.883+0000 7f611c200640 1 -- 172.21.15.16:0/2031351142 wait complete. 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:01:16.909 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-17T05:01:16.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:16.910 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:01:16.919 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42416sec preferred_lft 42416sec 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:16.920 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:01:16.921 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:16.922 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:01:16.922 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:17.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:17.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:17.083 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:01:17.102 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:01:17.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:17.104 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:01:17.178 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:01:17.178 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:01:17.178 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:01:17.179 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:01:17.179 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:01:17.179 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:01:17.179 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:17.179 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:17.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:01:17.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:01:17.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:01:17.179 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:17.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:17.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:17.322 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:01:17.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:17.374 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-17T05:01:17.425 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:17.426 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:17.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:17.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-17T05:01:17.426 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:17.555 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-17T05:01:17.556 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:17.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:17.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:17.728 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:17.728 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:17.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:01:17.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:01:17.728 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:17.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:17.814 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-17T05:01:17.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:17.814 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:17.823 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-17T05:01:17.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:17.823 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:17.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-17T05:01:17.933 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T05:01:17.986 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/run/lock 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T05:01:17.987 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/fs/nfsd 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/run/rpc_pipefs 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T05:01:17.988 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T05:01:17.989 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T05:01:17.989 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-17T05:01:17.989 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:17.989 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:17.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:17.990 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-17T05:01:18.041 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T05:01:18.041 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-17T05:01:18.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-17T05:01:18.107 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T05:01:18.158 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-17T05:01:18.158 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/run/lock 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-17T05:01:18.159 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/fs/nfsd 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/run/rpc_pipefs 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T05:01:18.160 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T05:01:18.161 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-17T05:01:18.161 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:18.161 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:18.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:18.162 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-17T05:01:18.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-09-17T05:01:18.195+0000 7f81a0323580 -1 init, newargv = 0x56179dbf3c40 newargc=15 2024-09-17T05:01:18.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[34134]: starting ceph client 2024-09-17T05:01:18.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[34134]: starting fuse 2024-09-17T05:01:19.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-17T05:01:19.332 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/run/lock 2024-09-17T05:01:19.388 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/fs/nfsd 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/run/rpc_pipefs 2024-09-17T05:01:19.389 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:19.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:19.391 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-17T05:01:19.442 INFO:teuthology.orchestra.run.smithi016.stdout:44 2024-09-17T05:01:19.442 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-17T05:01:19.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-17T05:01:19.443 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-09-17T05:01:19.444 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:01:19.574 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.34134.asok 2024-09-17T05:01:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:19.586 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34134.asok 2024-09-17T05:01:19.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:19.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34134.asok status 2024-09-17T05:01:19.771 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:01:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-17T05:01:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T05:01:19.771 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "34134", 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4521, 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:01:19.772 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4521 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3213273627 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-17T05:01:19.773 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3213273627 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4521 172.21.15.16:0/3213273627", 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "172.21.15.16:0/3213273627", 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 7, 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 26, 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-17T05:01:19.774 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:01:19.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:19.787 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-17T05:01:19.787 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:19.796 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-17T05:01:19.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:19.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:19.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:19.858 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-17T05:01:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:19.990 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-09-17T05:01:19.990 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-17T05:01:19.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:19.997 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:19.997 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:20.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:01:20.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 -- 172.21.15.16:0/3692190162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef8074a20 msgr2=0x7fbef815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 --2- 172.21.15.16:0/3692190162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef8074a20 0x7fbef815efe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009900 tx=0x7fbee801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 -- 172.21.15.16:0/3692190162 shutdown_connections 2024-09-17T05:01:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 --2- 172.21.15.16:0/3692190162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef8074a20 0x7fbef815efe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 -- 172.21.15.16:0/3692190162 >> 172.21.15.16:0/3692190162 conn(0x7fbef80b2250 msgr2=0x7fbef80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 -- 172.21.15.16:0/3692190162 shutdown_connections 2024-09-17T05:01:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.207+0000 7fbefe000640 1 -- 172.21.15.16:0/3692190162 wait complete. 2024-09-17T05:01:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 Processor -- start 2024-09-17T05:01:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 -- start start 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef8074a20 0x7fbef8153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbef8074e30 0x7fbef8158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 0x7fbef815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbef8169d80 con 0x7fbef815f520 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbef8169c00 con 0x7fbef8074a20 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbef8169f00 con 0x7fbef8074e30 2024-09-17T05:01:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 0x7fbef815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 0x7fbef815a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44900/0 (socket says 172.21.15.16:44900) 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 -- 172.21.15.16:0/3022129482 learned_addr learned my addr 172.21.15.16:0/3022129482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7400640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef8074a20 0x7fbef8153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 -- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbef8074e30 msgr2=0x7fbef8158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbef8074e30 0x7fbef8158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 -- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef8074a20 msgr2=0x7fbef8153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef8074a20 0x7fbef8153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 -- 172.21.15.16:0/3022129482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef80ab090 con 0x7fbef815f520 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7400640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef8074a20 0x7fbef8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbef7e00640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 0x7fbef815a760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbef000c730 tx=0x7fbef000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0003ba0 con 0x7fbef815f520 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef0003d60 con 0x7fbef815f520 2024-09-17T05:01:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0005330 con 0x7fbef815f520 2024-09-17T05:01:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef81f9fe0 con 0x7fbef815f520 2024-09-17T05:01:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.211+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef81fa5a0 con 0x7fbef815f520 2024-09-17T05:01:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.215+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef81f9e00 con 0x7fbef815f520 2024-09-17T05:01:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.215+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbef00056e0 con 0x7fbef815f520 2024-09-17T05:01:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.215+0000 7fbeef400640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbed4073200 0x7fbed4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.215+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbef0094550 con 0x7fbef815f520 2024-09-17T05:01:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.215+0000 7fbef7400640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbed4073200 0x7fbed4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.219+0000 7fbef7400640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbed4073200 0x7fbed4075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbee80af000 tx=0x7fbee801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.219+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbef81f9e00 con 0x7fbef815f520 2024-09-17T05:01:20.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbef815efe0 con 0x7fbef815f520 2024-09-17T05:01:20.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbeef400640 1 -- 172.21.15.16:0/3022129482 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fbef005f230 con 0x7fbef815f520 2024-09-17T05:01:20.411 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbed4073200 msgr2=0x7fbed4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbed4073200 0x7fbed4075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbee80af000 tx=0x7fbee801b120 comp rx=0 tx=0).stop 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 msgr2=0x7fbef815a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 0x7fbef815a760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbef000c730 tx=0x7fbef000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 shutdown_connections 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbed4073200 0x7fbed4075710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbef815f520 0x7fbef815a760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbef8074e30 0x7fbef8158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 --2- 172.21.15.16:0/3022129482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef8074a20 0x7fbef8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.407+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 >> 172.21.15.16:0/3022129482 conn(0x7fbef80b2250 msgr2=0x7fbef815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.411+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 shutdown_connections 2024-09-17T05:01:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.411+0000 7fbefe000640 1 -- 172.21.15.16:0/3022129482 wait complete. 2024-09-17T05:01:20.433 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:20.433 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:20.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 -- 172.21.15.16:0/301525421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80780744d0 msgr2=0x7f807806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 --2- 172.21.15.16:0/301525421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80780744d0 0x7f807806e0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8070009840 tx=0x7f807001c740 comp rx=0 tx=0).stop 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 -- 172.21.15.16:0/301525421 shutdown_connections 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 --2- 172.21.15.16:0/301525421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80780744d0 0x7f807806e0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 -- 172.21.15.16:0/301525421 >> 172.21.15.16:0/301525421 conn(0x7f80780b2dd0 msgr2=0x7f80780b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 -- 172.21.15.16:0/301525421 shutdown_connections 2024-09-17T05:01:20.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.643+0000 7f807d200640 1 -- 172.21.15.16:0/301525421 wait complete. 2024-09-17T05:01:20.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 Processor -- start 2024-09-17T05:01:20.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 -- start start 2024-09-17T05:01:20.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f807806f5f0 0x7f80781fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f807815c490 0x7f8078167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 0x7f8078073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f807816b2d0 con 0x7f80780c96e0 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f807816b150 con 0x7f807815c490 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f807d200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f807816b450 con 0x7f807806f5f0 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 0x7f8078073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 0x7f8078073dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44902/0 (socket says 172.21.15.16:44902) 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 -- 172.21.15.16:0/229075154 learned_addr learned my addr 172.21.15.16:0/229075154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8075e00640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f807806f5f0 0x7f80781fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f806d400640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f807815c490 0x7f8078167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 -- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f807806f5f0 msgr2=0x7f80781fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f807806f5f0 0x7f80781fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 -- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f807815c490 msgr2=0x7f8078167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f807815c490 0x7f8078167ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 -- 172.21.15.16:0/229075154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80780c9e10 con 0x7f80780c96e0 2024-09-17T05:01:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.647+0000 7f8076800640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 0x7f8078073dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f807001cc20 tx=0x7f8070005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:20.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700040f0 con 0x7f80780c96e0 2024-09-17T05:01:20.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8070004290 con 0x7f80780c96e0 2024-09-17T05:01:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700046d0 con 0x7f80780c96e0 2024-09-17T05:01:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80780ca0f0 con 0x7f80780c96e0 2024-09-17T05:01:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80780ca6b0 con 0x7f80780c96e0 2024-09-17T05:01:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80780c9b80 con 0x7f80780c96e0 2024-09-17T05:01:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8070004870 con 0x7f80780c96e0 2024-09-17T05:01:20.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f806ea00640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8054073200 0x7f8054075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:20.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.651+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f807012e490 con 0x7f80780c96e0 2024-09-17T05:01:20.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.655+0000 7f8075e00640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8054073200 0x7f8054075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:20.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.655+0000 7f8075e00640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8054073200 0x7f8054075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8068001bf0 tx=0x7f8068001a20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:20.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.659+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f80700c1030 con 0x7f80780c96e0 2024-09-17T05:01:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f807806e0d0 con 0x7f80780c96e0 2024-09-17T05:01:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f806ea00640 1 -- 172.21.15.16:0/229075154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f80700ac030 con 0x7f80780c96e0 2024-09-17T05:01:20.851 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:01:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8054073200 msgr2=0x7f8054075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8054073200 0x7f8054075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8068001bf0 tx=0x7f8068001a20 comp rx=0 tx=0).stop 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 msgr2=0x7f8078073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 0x7f8078073dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f807001cc20 tx=0x7f8070005780 comp rx=0 tx=0).stop 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 shutdown_connections 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8054073200 0x7f8054075710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80780c96e0 0x7f8078073dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f807815c490 0x7f8078167ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 --2- 172.21.15.16:0/229075154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f807806f5f0 0x7f80781fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 >> 172.21.15.16:0/229075154 conn(0x7f80780b2dd0 msgr2=0x7f80780b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 shutdown_connections 2024-09-17T05:01:20.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:20.847+0000 7f807d200640 1 -- 172.21.15.16:0/229075154 wait complete. 2024-09-17T05:01:20.874 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-17T05:01:20.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:20.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-17T05:01:20.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[34134]: fuse finished with error 0 and tester_r 0 2024-09-17T05:01:20.926 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:01:20.969 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:01:20.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:20.969 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-17T05:01:21.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 -- 172.21.15.16:0/3275735030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee40004810 msgr2=0x7fee40004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 --2- 172.21.15.16:0/3275735030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee40004810 0x7fee40004c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c310 tx=0x7fee4401cf20 comp rx=0 tx=0).stop 2024-09-17T05:01:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 -- 172.21.15.16:0/3275735030 shutdown_connections 2024-09-17T05:01:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 --2- 172.21.15.16:0/3275735030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee40004810 0x7fee40004c20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 -- 172.21.15.16:0/3275735030 >> 172.21.15.16:0/3275735030 conn(0x7fee5c0d10c0 msgr2=0x7fee5c0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 -- 172.21.15.16:0/3275735030 shutdown_connections 2024-09-17T05:01:21.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.231+0000 7fee61000640 1 -- 172.21.15.16:0/3275735030 wait complete. 2024-09-17T05:01:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 Processor -- start 2024-09-17T05:01:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 -- start start 2024-09-17T05:01:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee40004810 0x7fee5c14f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 0x7fee5c0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee5c0ca990 0x7fee5c0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0b5a20 con 0x7fee5c14fe80 2024-09-17T05:01:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c0b4920 con 0x7fee5c0ca990 2024-09-17T05:01:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c0b5ba0 con 0x7fee40004810 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 0x7fee5c0ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 0x7fee5c0ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44924/0 (socket says 172.21.15.16:44924) 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 -- 172.21.15.16:0/504819516 learned_addr learned my addr 172.21.15.16:0/504819516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee59e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee40004810 0x7fee5c14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 -- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee40004810 msgr2=0x7fee5c14f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee40004810 0x7fee5c14f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee5a800640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee5c0ca990 0x7fee5c0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 -- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee5c0ca990 msgr2=0x7fee5c0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee5c0ca990 0x7fee5c0cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 -- 172.21.15.16:0/504819516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c1ec460 con 0x7fee5c14fe80 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee59e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee40004810 0x7fee5c14f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee5a800640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee5c0ca990 0x7fee5c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee53e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 0x7fee5c0ca450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee4401b620 tx=0x7fee44001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440a7e20 con 0x7fee5c14fe80 2024-09-17T05:01:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee44001500 con 0x7fee5c14fe80 2024-09-17T05:01:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee440064e0 con 0x7fee5c14fe80 2024-09-17T05:01:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5c1ec740 con 0x7fee5c14fe80 2024-09-17T05:01:21.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.235+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5c0cb2e0 con 0x7fee5c14fe80 2024-09-17T05:01:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.239+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5c0b6100 con 0x7fee5c14fe80 2024-09-17T05:01:21.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.239+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fee44003600 con 0x7fee5c14fe80 2024-09-17T05:01:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.239+0000 7fee52000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee38073230 0x7fee38075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.239+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fee4412dee0 con 0x7fee5c14fe80 2024-09-17T05:01:21.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.239+0000 7fee59e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee38073230 0x7fee38075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.239+0000 7fee59e00640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee38073230 0x7fee38075740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006660 tx=0x7fee4c006100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:21.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.243+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fee440bf030 con 0x7fee5c14fe80 2024-09-17T05:01:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.423+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee5c0cb5d0 con 0x7fee5c14fe80 2024-09-17T05:01:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.423+0000 7fee52000640 1 -- 172.21.15.16:0/504819516 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7fee44016cd0 con 0x7fee5c14fe80 2024-09-17T05:01:21.428 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:21.428 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee38073230 msgr2=0x7fee38075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee38073230 0x7fee38075740 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fee4c006660 tx=0x7fee4c006100 comp rx=0 tx=0).stop 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 msgr2=0x7fee5c0ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 0x7fee5c0ca450 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fee4401b620 tx=0x7fee44001d00 comp rx=0 tx=0).stop 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 shutdown_connections 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee38073230 0x7fee38075740 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee5c0ca990 0x7fee5c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee5c14fe80 0x7fee5c0ca450 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 --2- 172.21.15.16:0/504819516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee40004810 0x7fee5c14f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 >> 172.21.15.16:0/504819516 conn(0x7fee5c0d10c0 msgr2=0x7fee5c06d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 shutdown_connections 2024-09-17T05:01:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.427+0000 7fee61000640 1 -- 172.21.15.16:0/504819516 wait complete. 2024-09-17T05:01:21.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 -- 172.21.15.16:0/1744627869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 msgr2=0x7f4f90073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 --2- 172.21.15.16:0/1744627869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90073950 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009bc0 tx=0x7f4f8801cb10 comp rx=0 tx=0).stop 2024-09-17T05:01:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 -- 172.21.15.16:0/1744627869 shutdown_connections 2024-09-17T05:01:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 --2- 172.21.15.16:0/1744627869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90073950 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 -- 172.21.15.16:0/1744627869 >> 172.21.15.16:0/1744627869 conn(0x7f4f9006f900 msgr2=0x7f4f900b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 -- 172.21.15.16:0/1744627869 shutdown_connections 2024-09-17T05:01:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.603+0000 7f4f97600640 1 -- 172.21.15.16:0/1744627869 wait complete. 2024-09-17T05:01:21.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 Processor -- start 2024-09-17T05:01:21.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 -- start start 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f900b2f50 0x7f4f9014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9015c270 0x7f4f900cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90169c00 con 0x7f4f9015c680 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f90169a80 con 0x7f4f9015c270 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f97600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4f90169d80 con 0x7f4f900b2f50 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44948/0 (socket says 172.21.15.16:44948) 2024-09-17T05:01:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 -- 172.21.15.16:0/892437677 learned_addr learned my addr 172.21.15.16:0/892437677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f87e00640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9015c270 0x7f4f900cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f94c00640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f900b2f50 0x7f4f9014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 -- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f900b2f50 msgr2=0x7f4f9014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f900b2f50 0x7f4f9014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 -- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9015c270 msgr2=0x7f4f900cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9015c270 0x7f4f900cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 -- 172.21.15.16:0/892437677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f901fc1a0 con 0x7f4f9015c680 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.607+0000 7f4f95600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90150760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f88019720 tx=0x7f4f880a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f880b2040 con 0x7f4f9015c680 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f88004d40 con 0x7f4f9015c680 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88004320 con 0x7f4f9015c680 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f901fc480 con 0x7f4f9015c680 2024-09-17T05:01:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f901fca40 con 0x7f4f9015c680 2024-09-17T05:01:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f900adb40 con 0x7f4f9015c680 2024-09-17T05:01:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4f880be060 con 0x7f4f9015c680 2024-09-17T05:01:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f8d600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f6c073200 0x7f4f6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4f8812e1a0 con 0x7f4f9015c680 2024-09-17T05:01:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.611+0000 7f4f94c00640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f6c073200 0x7f4f6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.615+0000 7f4f94c00640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f6c073200 0x7f4f6c075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4f800046c0 tx=0x7f4f80009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.615+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4f88016790 con 0x7f4f9015c680 2024-09-17T05:01:21.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.791+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f901fbf30 con 0x7f4f9015c680 2024-09-17T05:01:21.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.791+0000 7f4f8d600640 1 -- 172.21.15.16:0/892437677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f4f880f9020 con 0x7f4f9015c680 2024-09-17T05:01:21.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:21.797 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:21.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f6c073200 msgr2=0x7f4f6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f6c073200 0x7f4f6c075710 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4f800046c0 tx=0x7f4f80009040 comp rx=0 tx=0).stop 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 msgr2=0x7f4f90150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90150760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4f88019720 tx=0x7f4f880a5a00 comp rx=0 tx=0).stop 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 shutdown_connections 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f6c073200 0x7f4f6c075710 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9015c680 0x7f4f90150760 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f9015c270 0x7f4f900cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 --2- 172.21.15.16:0/892437677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f900b2f50 0x7f4f9014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 >> 172.21.15.16:0/892437677 conn(0x7f4f9006f900 msgr2=0x7f4f900adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 shutdown_connections 2024-09-17T05:01:21.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.795+0000 7f4f97600640 1 -- 172.21.15.16:0/892437677 wait complete. 2024-09-17T05:01:21.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:01:21.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 -- 172.21.15.16:0/3676431795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c004810 msgr2=0x7f447c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 --2- 172.21.15.16:0/3676431795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c004810 0x7f447c004c20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44800016f0 tx=0x7f448001c750 comp rx=0 tx=0).stop 2024-09-17T05:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 -- 172.21.15.16:0/3676431795 shutdown_connections 2024-09-17T05:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 --2- 172.21.15.16:0/3676431795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f447c004810 0x7f447c004c20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 -- 172.21.15.16:0/3676431795 >> 172.21.15.16:0/3676431795 conn(0x7f44980b2250 msgr2=0x7f44980d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 -- 172.21.15.16:0/3676431795 shutdown_connections 2024-09-17T05:01:21.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.975+0000 7f449e000640 1 -- 172.21.15.16:0/3676431795 wait complete. 2024-09-17T05:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 Processor -- start 2024-09-17T05:01:21.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 -- start start 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f447c004810 0x7f4498140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 0x7f449815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f449815d560 0x7f4498142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f449815f160 con 0x7f44980c9bf0 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f449815efe0 con 0x7f447c004810 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f449e000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f449815f2e0 con 0x7f449815d560 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f448ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 0x7f449815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f448ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 0x7f449815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44970/0 (socket says 172.21.15.16:44970) 2024-09-17T05:01:21.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.979+0000 7f448ea00640 1 -- 172.21.15.16:0/3919895313 learned_addr learned my addr 172.21.15.16:0/3919895313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f4497400640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f447c004810 0x7f4498140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f4497e00640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f449815d560 0x7f4498142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448ea00640 1 -- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f449815d560 msgr2=0x7f4498142f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448ea00640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f449815d560 0x7f4498142f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448ea00640 1 -- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f447c004810 msgr2=0x7f4498140ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448ea00640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f447c004810 0x7f4498140ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448ea00640 1 -- 172.21.15.16:0/3919895313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44980b35b0 con 0x7f44980c9bf0 2024-09-17T05:01:21.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448ea00640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 0x7f449815d020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f448001ae50 tx=0x7f4480006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480003890 con 0x7f44980c9bf0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4480003b20 con 0x7f44980c9bf0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44980b5fb0 con 0x7f44980c9bf0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44800a55b0 con 0x7f44980c9bf0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44980b32d0 con 0x7f44980c9bf0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44980b5d20 con 0x7f44980c9bf0 2024-09-17T05:01:21.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.983+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f44800a5750 con 0x7f44980c9bf0 2024-09-17T05:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.987+0000 7f448fe00640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4474073200 0x7f4474075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.987+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f448012dde0 con 0x7f44980c9bf0 2024-09-17T05:01:21.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.987+0000 7f4497400640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4474073200 0x7f4474075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:21.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.987+0000 7f4497400640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4474073200 0x7f4474075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f449806bbd0 tx=0x7f44880091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:21.991+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f44980b35b0 con 0x7f44980c9bf0 2024-09-17T05:01:22.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.199+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f449815bd40 con 0x7f44980c9bf0 2024-09-17T05:01:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.919+0000 7f448fe00640 1 -- 172.21.15.16:0/3919895313 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f44800b3070 con 0x7f44980c9bf0 2024-09-17T05:01:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4474073200 msgr2=0x7f4474075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4474073200 0x7f4474075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f449806bbd0 tx=0x7f44880091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 msgr2=0x7f449815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 0x7f449815d020 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f448001ae50 tx=0x7f4480006950 comp rx=0 tx=0).stop 2024-09-17T05:01:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 shutdown_connections 2024-09-17T05:01:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4474073200 0x7f4474075710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f449815d560 0x7f4498142f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44980c9bf0 0x7f449815d020 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 --2- 172.21.15.16:0/3919895313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f447c004810 0x7f4498140ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 >> 172.21.15.16:0/3919895313 conn(0x7f44980b2250 msgr2=0x7f44980ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 shutdown_connections 2024-09-17T05:01:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:22.923+0000 7f449e000640 1 -- 172.21.15.16:0/3919895313 wait complete. 2024-09-17T05:01:22.952 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-17T05:01:22.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:01:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 -- 172.21.15.16:0/2941601292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 msgr2=0x7f5e48155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 --2- 172.21.15.16:0/2941601292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48155500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009900 tx=0x7f5e3801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 -- 172.21.15.16:0/2941601292 shutdown_connections 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 --2- 172.21.15.16:0/2941601292 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e480afbd0 0x7f5e4815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 --2- 172.21.15.16:0/2941601292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48155500 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 -- 172.21.15.16:0/2941601292 >> 172.21.15.16:0/2941601292 conn(0x7f5e480cd980 msgr2=0x7f5e480cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 -- 172.21.15.16:0/2941601292 shutdown_connections 2024-09-17T05:01:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 -- 172.21.15.16:0/2941601292 wait complete. 2024-09-17T05:01:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 Processor -- start 2024-09-17T05:01:23.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.115+0000 7f5e4de00640 1 -- start start 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e480afbd0 0x7f5e481f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e48155a40 0x7f5e48152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e480718f0 con 0x7f5e480ce4d0 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e48071770 con 0x7f5e480afbd0 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5e48071a70 con 0x7f5e48155a40 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44986/0 (socket says 172.21.15.16:44986) 2024-09-17T05:01:23.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 -- 172.21.15.16:0/3124444387 learned_addr learned my addr 172.21.15.16:0/3124444387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e46a00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e480afbd0 0x7f5e481f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e47400640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e48155a40 0x7f5e48152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 -- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e48155a40 msgr2=0x7f5e48152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e48155a40 0x7f5e48152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 -- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e480afbd0 msgr2=0x7f5e481f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e480afbd0 0x7f5e481f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 -- 172.21.15.16:0/3124444387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e481f74d0 con 0x7f5e480ce4d0 2024-09-17T05:01:23.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3fe00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48160b30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000c3f0 tx=0x7f5e3000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30017070 con 0x7f5e480ce4d0 2024-09-17T05:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e300053c0 con 0x7f5e480ce4d0 2024-09-17T05:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30003860 con 0x7f5e480ce4d0 2024-09-17T05:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e481f77b0 con 0x7f5e480ce4d0 2024-09-17T05:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e481f8250 con 0x7f5e480ce4d0 2024-09-17T05:01:23.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.119+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4815f310 con 0x7f5e480ce4d0 2024-09-17T05:01:23.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.123+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e3001a070 con 0x7f5e480ce4d0 2024-09-17T05:01:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.123+0000 7f5e3f400640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e24073230 0x7f5e24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.123+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5e300937d0 con 0x7f5e480ce4d0 2024-09-17T05:01:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.123+0000 7f5e46a00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e24073230 0x7f5e24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:23.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.123+0000 7f5e46a00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e24073230 0x7f5e24075740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e380af000 tx=0x7f5e3801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.131+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e480ce8e0 con 0x7f5e480ce4d0 2024-09-17T05:01:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.339+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5e481f7220 con 0x7f5e480ce4d0 2024-09-17T05:01:23.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e3f400640 1 -- 172.21.15.16:0/3124444387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f5e30001090 con 0x7f5e480ce4d0 2024-09-17T05:01:23.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e24073230 msgr2=0x7f5e24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e24073230 0x7f5e24075740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5e380af000 tx=0x7f5e3801b120 comp rx=0 tx=0).stop 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 msgr2=0x7f5e48160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48160b30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000c3f0 tx=0x7f5e3000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 shutdown_connections 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e24073230 0x7f5e24075740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e48155a40 0x7f5e48152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e480ce4d0 0x7f5e48160b30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 --2- 172.21.15.16:0/3124444387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e480afbd0 0x7f5e481f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 >> 172.21.15.16:0/3124444387 conn(0x7f5e480cd980 msgr2=0x7f5e48153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 shutdown_connections 2024-09-17T05:01:23.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:23.931+0000 7f5e4de00640 1 -- 172.21.15.16:0/3124444387 wait complete. 2024-09-17T05:01:23.954 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:01:23.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 -- 172.21.15.16:0/2172151362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c4004810 msgr2=0x7f11c4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 --2- 172.21.15.16:0/2172151362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11c4004810 0x7f11c4004c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009b30 tx=0x7f11c801d240 comp rx=0 tx=0).stop 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11dfe00640 1 -- 172.21.15.16:0/2172151362 reap_dead start 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 -- 172.21.15.16:0/2172151362 shutdown_connections 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 -- 172.21.15.16:0/2172151362 >> 172.21.15.16:0/2172151362 conn(0x7f11e00734b0 msgr2=0x7f11e00738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 -- 172.21.15.16:0/2172151362 shutdown_connections 2024-09-17T05:01:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 -- 172.21.15.16:0/2172151362 wait complete. 2024-09-17T05:01:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 Processor -- start 2024-09-17T05:01:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.115+0000 7f11e4a00640 1 -- start start 2024-09-17T05:01:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11c4004810 0x7f11e00c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0156b60 0x7f11e015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 0x7f11e015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11e00caeb0 con 0x7f11e015d470 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11e00cad30 con 0x7f11e0156b60 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f11e00cc350 con 0x7f11c4004810 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 0x7f11e015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 0x7f11e015d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45008/0 (socket says 172.21.15.16:45008) 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 -- 172.21.15.16:0/920694346 learned_addr learned my addr 172.21.15.16:0/920694346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11d6a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0156b60 0x7f11e015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11df400640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11c4004810 0x7f11e00c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 -- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11c4004810 msgr2=0x7f11e00c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11c4004810 0x7f11e00c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 -- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0156b60 msgr2=0x7f11e015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0156b60 0x7f11e015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 -- 172.21.15.16:0/920694346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e015ddc0 con 0x7f11e015d470 2024-09-17T05:01:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11df400640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11c4004810 0x7f11e00c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11dfe00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 0x7f11e015d880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f11e0073460 tx=0x7f11d8006710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800e9e0 con 0x7f11e015d470 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11d8016070 con 0x7f11e015d470 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8011460 con 0x7f11e015d470 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e00c8c00 con 0x7f11e015d470 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e00c91c0 con 0x7f11e015d470 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.119+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e00c8760 con 0x7f11e015d470 2024-09-17T05:01:24.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.123+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f11d8011600 con 0x7f11e015d470 2024-09-17T05:01:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.123+0000 7f11d7e00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11bc073230 0x7f11bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.123+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f11d8094e30 con 0x7f11e015d470 2024-09-17T05:01:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.123+0000 7f11df400640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11bc073230 0x7f11bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.123+0000 7f11df400640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11bc073230 0x7f11bc075740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009bc0 tx=0x7f11d00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.127+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11e015ddc0 con 0x7f11e015d470 2024-09-17T05:01:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.335+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e015ee60 con 0x7f11e015d470 2024-09-17T05:01:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.335+0000 7f11d7e00640 1 -- 172.21.15.16:0/920694346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f11e006e7f0 con 0x7f11e015d470 2024-09-17T05:01:24.336 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:24.336 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:24.336 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11bc073230 msgr2=0x7f11bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11bc073230 0x7f11bc075740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009bc0 tx=0x7f11d00091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 msgr2=0x7f11e015d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 0x7f11e015d880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f11e0073460 tx=0x7f11d8006710 comp rx=0 tx=0).stop 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 shutdown_connections 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11bc073230 0x7f11bc075740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11e015d470 0x7f11e015d880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11e0156b60 0x7f11e015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 --2- 172.21.15.16:0/920694346 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11c4004810 0x7f11e00c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 >> 172.21.15.16:0/920694346 conn(0x7f11e00734b0 msgr2=0x7f11e0159070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 shutdown_connections 2024-09-17T05:01:24.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.339+0000 7f11e4a00640 1 -- 172.21.15.16:0/920694346 wait complete. 2024-09-17T05:01:24.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 -- 172.21.15.16:0/2337190381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c400b2f50 msgr2=0x7f2c4015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 --2- 172.21.15.16:0/2337190381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c400b2f50 0x7f2c4015bb60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2c4006e960 tx=0x7f2c3001c830 comp rx=0 tx=0).stop 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 -- 172.21.15.16:0/2337190381 shutdown_connections 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 --2- 172.21.15.16:0/2337190381 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c4015f150 0x7f2c4015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 --2- 172.21.15.16:0/2337190381 >> v2:172.21.15.104:3300/0 conn(0x7f2c4015c0a0 0x7f2c4015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 --2- 172.21.15.16:0/2337190381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c400b2f50 0x7f2c4015bb60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 -- 172.21.15.16:0/2337190381 >> 172.21.15.16:0/2337190381 conn(0x7f2c4006f900 msgr2=0x7f2c400b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 -- 172.21.15.16:0/2337190381 shutdown_connections 2024-09-17T05:01:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 -- 172.21.15.16:0/2337190381 wait complete. 2024-09-17T05:01:24.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 Processor -- start 2024-09-17T05:01:24.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.519+0000 7f2c45e00640 1 -- start start 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c400b2f50 0x7f2c401f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c4015c0a0 0x7f2c401fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 0x7f2c40163e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4006e3d0 con 0x7f2c4015f150 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4006e250 con 0x7f2c4015c0a0 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4006e550 con 0x7f2c400b2f50 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 0x7f2c40163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 0x7f2c40163e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45026/0 (socket says 172.21.15.16:45026) 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 -- 172.21.15.16:0/285451934 learned_addr learned my addr 172.21.15.16:0/285451934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c37e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c4015c0a0 0x7f2c401fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3ea00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c400b2f50 0x7f2c401f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 -- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c400b2f50 msgr2=0x7f2c401f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c400b2f50 0x7f2c401f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 -- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c4015c0a0 msgr2=0x7f2c401fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c4015c0a0 0x7f2c401fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 -- 172.21.15.16:0/285451934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c401f9190 con 0x7f2c4015f150 2024-09-17T05:01:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c3f400640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 0x7f2c40163e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800eab0 tx=0x7f2c38009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800ce40 con 0x7f2c4015f150 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c38004c20 con 0x7f2c4015f150 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38005020 con 0x7f2c4015f150 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c401fd0b0 con 0x7f2c4015f150 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401f9460 con 0x7f2c4015f150 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.523+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c40161de0 con 0x7f2c4015f150 2024-09-17T05:01:24.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.527+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c380051e0 con 0x7f2c4015f150 2024-09-17T05:01:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.527+0000 7f2c37400640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073200 0x7f2c1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.527+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2c38095320 con 0x7f2c4015f150 2024-09-17T05:01:24.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.527+0000 7f2c3ea00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073200 0x7f2c1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.527+0000 7f2c3ea00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073200 0x7f2c1c075710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c300af000 tx=0x7f2c300a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.531+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2c3805f9b0 con 0x7f2c4015f150 2024-09-17T05:01:24.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.707+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c400ceea0 con 0x7f2c4015f150 2024-09-17T05:01:24.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c37400640 1 -- 172.21.15.16:0/285451934 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f2c380643e0 con 0x7f2c4015f150 2024-09-17T05:01:24.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:24.710 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073200 msgr2=0x7f2c1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073200 0x7f2c1c075710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2c300af000 tx=0x7f2c300a7040 comp rx=0 tx=0).stop 2024-09-17T05:01:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 msgr2=0x7f2c40163e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 0x7f2c40163e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800eab0 tx=0x7f2c38009f70 comp rx=0 tx=0).stop 2024-09-17T05:01:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 shutdown_connections 2024-09-17T05:01:24.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073200 0x7f2c1c075710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4015f150 0x7f2c40163e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c4015c0a0 0x7f2c401fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 --2- 172.21.15.16:0/285451934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c400b2f50 0x7f2c401f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 >> 172.21.15.16:0/285451934 conn(0x7f2c4006f900 msgr2=0x7f2c400cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 shutdown_connections 2024-09-17T05:01:24.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.711+0000 7f2c45e00640 1 -- 172.21.15.16:0/285451934 wait complete. 2024-09-17T05:01:24.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:01:24.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:01:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0163000640 1 -- 172.21.15.16:0/3817658855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 msgr2=0x7f0140005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0163000640 1 --2- 172.21.15.16:0/3817658855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f0140005180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f015c052060 tx=0x7f015401d590 comp rx=0 tx=0).stop 2024-09-17T05:01:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0161000640 1 -- 172.21.15.16:0/3817658855 reap_dead start 2024-09-17T05:01:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0163000640 1 -- 172.21.15.16:0/3817658855 shutdown_connections 2024-09-17T05:01:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0163000640 1 -- 172.21.15.16:0/3817658855 >> 172.21.15.16:0/3817658855 conn(0x7f015c073e00 msgr2=0x7f015c074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0163000640 1 -- 172.21.15.16:0/3817658855 shutdown_connections 2024-09-17T05:01:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.895+0000 7f0163000640 1 -- 172.21.15.16:0/3817658855 wait complete. 2024-09-17T05:01:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 Processor -- start 2024-09-17T05:01:24.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 -- start start 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f015c1f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f015c141260 0x7f015c141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f015c15ed80 0x7f015c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f015c06dfd0 con 0x7f0140004d50 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f015c06de50 con 0x7f015c141260 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f015c06e150 con 0x7f015c15ed80 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f015c1f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f015c1f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45040/0 (socket says 172.21.15.16:45040) 2024-09-17T05:01:24.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 -- 172.21.15.16:0/2353068220 learned_addr learned my addr 172.21.15.16:0/2353068220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 -- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f015c15ed80 msgr2=0x7f015c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f015c15ed80 0x7f015c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 -- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f015c141260 msgr2=0x7f015c141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0153e00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f015c141260 0x7f015c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f015c141260 0x7f015c141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 -- 172.21.15.16:0/2353068220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c0722e0 con 0x7f0140004d50 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0153e00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f015c141260 0x7f015c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f015be00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f015c1f5f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f014c001ca0 tx=0x7f014c0066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:24.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c015070 con 0x7f0140004d50 2024-09-17T05:01:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f014c011840 con 0x7f0140004d50 2024-09-17T05:01:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c011c90 con 0x7f0140004d50 2024-09-17T05:01:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c15f6d0 con 0x7f0140004d50 2024-09-17T05:01:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.899+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c0a9a10 con 0x7f0140004d50 2024-09-17T05:01:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.903+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c06e6e0 con 0x7f0140004d50 2024-09-17T05:01:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.903+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f014c018070 con 0x7f0140004d50 2024-09-17T05:01:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.903+0000 7f0158c00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0138073230 0x7f0138075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.903+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f014c095350 con 0x7f0140004d50 2024-09-17T05:01:24.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.903+0000 7f0153e00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0138073230 0x7f0138075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:24.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.907+0000 7f0153e00640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0138073230 0x7f0138075740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f015c06f0a0 tx=0x7f0144006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:24.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:24.907+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f015c15f6d0 con 0x7f0140004d50 2024-09-17T05:01:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.083+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f015c141bb0 con 0x7f0138073230 2024-09-17T05:01:25.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.091+0000 7f0158c00640 1 -- 172.21.15.16:0/2353068220 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332543 (secure 0 0 0) 0x7f015c141bb0 con 0x7f0138073230 2024-09-17T05:01:25.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:01:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0138073230 msgr2=0x7f0138075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0138073230 0x7f0138075740 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f015c06f0a0 tx=0x7f0144006b20 comp rx=0 tx=0).stop 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 msgr2=0x7f015c1f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f015c1f5f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f014c001ca0 tx=0x7f014c0066c0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 shutdown_connections 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0138073230 0x7f0138075740 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f015c15ed80 0x7f015c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f015c141260 0x7f015c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 --2- 172.21.15.16:0/2353068220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0140004d50 0x7f015c1f5f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 >> 172.21.15.16:0/2353068220 conn(0x7f015c073e00 msgr2=0x7f015c071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 shutdown_connections 2024-09-17T05:01:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.095+0000 7f0163000640 1 -- 172.21.15.16:0/2353068220 wait complete. 2024-09-17T05:01:25.117 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-17T05:01:23.464133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236200,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738520,"statfs":{"total":773094113280,"available":772852244480,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003393"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942466+0000","last_change":"2024-09-17T05:01:07.870252+0000","last_active":"2024-09-17T05:01:13.942466+0000","last_peered":"2024-09-17T05:01:13.942466+0000","last_clean":"2024-09-17T05:01:13.942466+0000","last_became_active":"2024-09-17T05:01:07.869832+0000","last_became_peered":"2024-09-17T05:01:07.869832+0000","last_unstale":"2024-09-17T05:01:13.942466+0000","last_undegraded":"2024-09-17T05:01:13.942466+0000","last_fullsized":"2024-09-17T05:01:13.942466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893796+0000","last_change":"2024-09-17T05:01:08.860472+0000","last_active":"2024-09-17T05:01:13.893796+0000","last_peered":"2024-09-17T05:01:13.893796+0000","last_clean":"2024-09-17T05:01:13.893796+0000","last_became_active":"2024-09-17T05:01:08.860121+0000","last_became_peered":"2024-09-17T05:01:08.860121+0000","last_unstale":"2024-09-17T05:01:13.893796+0000","last_undegraded":"2024-09-17T05:01:13.893796+0000","last_fullsized":"2024-09-17T05:01:13.893796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:02.947441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941059+0000","last_change":"2024-09-17T05:01:12.917747+0000","last_active":"2024-09-17T05:01:13.941059+0000","last_peered":"2024-09-17T05:01:13.941059+0000","last_clean":"2024-09-17T05:01:13.941059+0000","last_became_active":"2024-09-17T05:01:12.916952+0000","last_became_peered":"2024-09-17T05:01:12.916952+0000","last_unstale":"2024-09-17T05:01:13.941059+0000","last_undegraded":"2024-09-17T05:01:13.941059+0000","last_fullsized":"2024-09-17T05:01:13.941059+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:00.182691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893999+0000","last_change":"2024-09-17T05:01:07.858483+0000","last_active":"2024-09-17T05:01:13.893999+0000","last_peered":"2024-09-17T05:01:13.893999+0000","last_clean":"2024-09-17T05:01:13.893999+0000","last_became_active":"2024-09-17T05:01:07.858094+0000","last_became_peered":"2024-09-17T05:01:07.858094+0000","last_unstale":"2024-09-17T05:01:13.893999+0000","last_undegraded":"2024-09-17T05:01:13.893999+0000","last_fullsized":"2024-09-17T05:01:13.893999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:09.683336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894263+0000","last_change":"2024-09-17T05:01:08.886624+0000","last_active":"2024-09-17T05:01:13.894263+0000","last_peered":"2024-09-17T05:01:13.894263+0000","last_clean":"2024-09-17T05:01:13.894263+0000","last_became_active":"2024-09-17T05:01:08.886308+0000","last_became_peered":"2024-09-17T05:01:08.886308+0000","last_unstale":"2024-09-17T05:01:13.894263+0000","last_undegraded":"2024-09-17T05:01:13.894263+0000","last_fullsized":"2024-09-17T05:01:13.894263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.002578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941422+0000","last_change":"2024-09-17T05:01:12.901716+0000","last_active":"2024-09-17T05:01:13.941422+0000","last_peered":"2024-09-17T05:01:13.941422+0000","last_clean":"2024-09-17T05:01:13.941422+0000","last_became_active":"2024-09-17T05:01:12.901225+0000","last_became_peered":"2024-09-17T05:01:12.901225+0000","last_unstale":"2024-09-17T05:01:13.941422+0000","last_undegraded":"2024-09-17T05:01:13.941422+0000","last_fullsized":"2024-09-17T05:01:13.941422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:19.972140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942144+0000","last_change":"2024-09-17T05:01:07.857199+0000","last_active":"2024-09-17T05:01:13.942144+0000","last_peered":"2024-09-17T05:01:13.942144+0000","last_clean":"2024-09-17T05:01:13.942144+0000","last_became_active":"2024-09-17T05:01:07.856808+0000","last_became_peered":"2024-09-17T05:01:07.856808+0000","last_unstale":"2024-09-17T05:01:13.942144+0000","last_undegraded":"2024-09-17T05:01:13.942144+0000","last_fullsized":"2024-09-17T05:01:13.942144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943634+0000","last_change":"2024-09-17T05:01:08.880952+0000","last_active":"2024-09-17T05:01:13.943634+0000","last_peered":"2024-09-17T05:01:13.943634+0000","last_clean":"2024-09-17T05:01:13.943634+0000","last_became_active":"2024-09-17T05:01:08.880627+0000","last_became_peered":"2024-09-17T05:01:08.880627+0000","last_unstale":"2024-09-17T05:01:13.943634+0000","last_undegraded":"2024-09-17T05:01:13.943634+0000","last_fullsized":"2024-09-17T05:01:13.943634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:03.625255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943104+0000","last_change":"2024-09-17T05:01:12.903919+0000","last_active":"2024-09-17T05:01:13.943104+0000","last_peered":"2024-09-17T05:01:13.943104+0000","last_clean":"2024-09-17T05:01:13.943104+0000","last_became_active":"2024-09-17T05:01:12.901881+0000","last_became_peered":"2024-09-17T05:01:12.901881+0000","last_unstale":"2024-09-17T05:01:13.943104+0000","last_undegraded":"2024-09-17T05:01:13.943104+0000","last_fullsized":"2024-09-17T05:01:13.943104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.883298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942724+0000","last_change":"2024-09-17T05:01:07.859450+0000","last_active":"2024-09-17T05:01:13.942724+0000","last_peered":"2024-09-17T05:01:13.942724+0000","last_clean":"2024-09-17T05:01:13.942724+0000","last_became_active":"2024-09-17T05:01:07.857315+0000","last_became_peered":"2024-09-17T05:01:07.857315+0000","last_unstale":"2024-09-17T05:01:13.942724+0000","last_undegraded":"2024-09-17T05:01:13.942724+0000","last_fullsized":"2024-09-17T05:01:13.942724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892054+0000","last_change":"2024-09-17T05:01:08.893401+0000","last_active":"2024-09-17T05:01:13.892054+0000","last_peered":"2024-09-17T05:01:13.892054+0000","last_clean":"2024-09-17T05:01:13.892054+0000","last_became_active":"2024-09-17T05:01:08.892406+0000","last_became_peered":"2024-09-17T05:01:08.892406+0000","last_unstale":"2024-09-17T05:01:13.892054+0000","last_undegraded":"2024-09-17T05:01:13.892054+0000","last_fullsized":"2024-09-17T05:01:13.892054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:18.887291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942085+0000","last_change":"2024-09-17T05:01:12.918843+0000","last_active":"2024-09-17T05:01:13.942085+0000","last_peered":"2024-09-17T05:01:13.942085+0000","last_clean":"2024-09-17T05:01:13.942085+0000","last_became_active":"2024-09-17T05:01:12.916474+0000","last_became_peered":"2024-09-17T05:01:12.916474+0000","last_unstale":"2024-09-17T05:01:13.942085+0000","last_undegraded":"2024-09-17T05:01:13.942085+0000","last_fullsized":"2024-09-17T05:01:13.942085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:05.661358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891193+0000","last_change":"2024-09-17T05:01:08.858811+0000","last_active":"2024-09-17T05:01:13.891193+0000","last_peered":"2024-09-17T05:01:13.891193+0000","last_clean":"2024-09-17T05:01:13.891193+0000","last_became_active":"2024-09-17T05:01:08.858013+0000","last_became_peered":"2024-09-17T05:01:08.858013+0000","last_unstale":"2024-09-17T05:01:13.891193+0000","last_undegraded":"2024-09-17T05:01:13.891193+0000","last_fullsized":"2024-09-17T05:01:13.891193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:06.714459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891507+0000","last_change":"2024-09-17T05:01:07.873880+0000","last_active":"2024-09-17T05:01:13.891507+0000","last_peered":"2024-09-17T05:01:13.891507+0000","last_clean":"2024-09-17T05:01:13.891507+0000","last_became_active":"2024-09-17T05:01:07.870873+0000","last_became_peered":"2024-09-17T05:01:07.870873+0000","last_unstale":"2024-09-17T05:01:13.891507+0000","last_undegraded":"2024-09-17T05:01:13.891507+0000","last_fullsized":"2024-09-17T05:01:13.891507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.618135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891844+0000","last_change":"2024-09-17T05:01:12.898484+0000","last_active":"2024-09-17T05:01:13.891844+0000","last_peered":"2024-09-17T05:01:13.891844+0000","last_clean":"2024-09-17T05:01:13.891844+0000","last_became_active":"2024-09-17T05:01:12.895423+0000","last_became_peered":"2024-09-17T05:01:12.895423+0000","last_unstale":"2024-09-17T05:01:13.891844+0000","last_undegraded":"2024-09-17T05:01:13.891844+0000","last_fullsized":"2024-09-17T05:01:13.891844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:54.524295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941368+0000","last_change":"2024-09-17T05:01:08.871889+0000","last_active":"2024-09-17T05:01:13.941368+0000","last_peered":"2024-09-17T05:01:13.941368+0000","last_clean":"2024-09-17T05:01:13.941368+0000","last_became_active":"2024-09-17T05:01:08.871512+0000","last_became_peered":"2024-09-17T05:01:08.871512+0000","last_unstale":"2024-09-17T05:01:13.941368+0000","last_undegraded":"2024-09-17T05:01:13.941368+0000","last_fullsized":"2024-09-17T05:01:13.941368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892848+0000","last_change":"2024-09-17T05:01:07.865154+0000","last_active":"2024-09-17T05:01:13.892848+0000","last_peered":"2024-09-17T05:01:13.892848+0000","last_clean":"2024-09-17T05:01:13.892848+0000","last_became_active":"2024-09-17T05:01:07.864075+0000","last_became_peered":"2024-09-17T05:01:07.864075+0000","last_unstale":"2024-09-17T05:01:13.892848+0000","last_undegraded":"2024-09-17T05:01:13.892848+0000","last_fullsized":"2024-09-17T05:01:13.892848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941790+0000","last_change":"2024-09-17T05:01:12.915985+0000","last_active":"2024-09-17T05:01:13.941790+0000","last_peered":"2024-09-17T05:01:13.941790+0000","last_clean":"2024-09-17T05:01:13.941790+0000","last_became_active":"2024-09-17T05:01:12.915569+0000","last_became_peered":"2024-09-17T05:01:12.915569+0000","last_unstale":"2024-09-17T05:01:13.941790+0000","last_undegraded":"2024-09-17T05:01:13.941790+0000","last_fullsized":"2024-09-17T05:01:13.941790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:15.801136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940431+0000","last_change":"2024-09-17T05:01:08.859016+0000","last_active":"2024-09-17T05:01:13.940431+0000","last_peered":"2024-09-17T05:01:13.940431+0000","last_clean":"2024-09-17T05:01:13.940431+0000","last_became_active":"2024-09-17T05:01:08.858669+0000","last_became_peered":"2024-09-17T05:01:08.858669+0000","last_unstale":"2024-09-17T05:01:13.940431+0000","last_undegraded":"2024-09-17T05:01:13.940431+0000","last_fullsized":"2024-09-17T05:01:13.940431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:48.908013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940792+0000","last_change":"2024-09-17T05:01:07.853871+0000","last_active":"2024-09-17T05:01:13.940792+0000","last_peered":"2024-09-17T05:01:13.940792+0000","last_clean":"2024-09-17T05:01:13.940792+0000","last_became_active":"2024-09-17T05:01:07.853309+0000","last_became_peered":"2024-09-17T05:01:07.853309+0000","last_unstale":"2024-09-17T05:01:13.940792+0000","last_undegraded":"2024-09-17T05:01:13.940792+0000","last_fullsized":"2024-09-17T05:01:13.940792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941719+0000","last_change":"2024-09-17T05:01:12.917589+0000","last_active":"2024-09-17T05:01:13.941719+0000","last_peered":"2024-09-17T05:01:13.941719+0000","last_clean":"2024-09-17T05:01:13.941719+0000","last_became_active":"2024-09-17T05:01:12.917064+0000","last_became_peered":"2024-09-17T05:01:12.917064+0000","last_unstale":"2024-09-17T05:01:13.941719+0000","last_undegraded":"2024-09-17T05:01:13.941719+0000","last_fullsized":"2024-09-17T05:01:13.941719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:47.894698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890960+0000","last_change":"2024-09-17T05:01:08.865800+0000","last_active":"2024-09-17T05:01:13.890960+0000","last_peered":"2024-09-17T05:01:13.890960+0000","last_clean":"2024-09-17T05:01:13.890960+0000","last_became_active":"2024-09-17T05:01:08.865418+0000","last_became_peered":"2024-09-17T05:01:08.865418+0000","last_unstale":"2024-09-17T05:01:13.890960+0000","last_undegraded":"2024-09-17T05:01:13.890960+0000","last_fullsized":"2024-09-17T05:01:13.890960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941039+0000","last_change":"2024-09-17T05:01:07.868079+0000","last_active":"2024-09-17T05:01:13.941039+0000","last_peered":"2024-09-17T05:01:13.941039+0000","last_clean":"2024-09-17T05:01:13.941039+0000","last_became_active":"2024-09-17T05:01:07.867648+0000","last_became_peered":"2024-09-17T05:01:07.867648+0000","last_unstale":"2024-09-17T05:01:13.941039+0000","last_undegraded":"2024-09-17T05:01:13.941039+0000","last_fullsized":"2024-09-17T05:01:13.941039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943609+0000","last_change":"2024-09-17T05:01:12.913059+0000","last_active":"2024-09-17T05:01:13.943609+0000","last_peered":"2024-09-17T05:01:13.943609+0000","last_clean":"2024-09-17T05:01:13.943609+0000","last_became_active":"2024-09-17T05:01:12.912569+0000","last_became_peered":"2024-09-17T05:01:12.912569+0000","last_unstale":"2024-09-17T05:01:13.943609+0000","last_undegraded":"2024-09-17T05:01:13.943609+0000","last_fullsized":"2024-09-17T05:01:13.943609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:23.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941727+0000","last_change":"2024-09-17T05:01:08.869046+0000","last_active":"2024-09-17T05:01:13.941727+0000","last_peered":"2024-09-17T05:01:13.941727+0000","last_clean":"2024-09-17T05:01:13.941727+0000","last_became_active":"2024-09-17T05:01:08.864116+0000","last_became_peered":"2024-09-17T05:01:08.864116+0000","last_unstale":"2024-09-17T05:01:13.941727+0000","last_undegraded":"2024-09-17T05:01:13.941727+0000","last_fullsized":"2024-09-17T05:01:13.941727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:51.571131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892323+0000","last_change":"2024-09-17T05:01:07.863381+0000","last_active":"2024-09-17T05:01:13.892323+0000","last_peered":"2024-09-17T05:01:13.892323+0000","last_clean":"2024-09-17T05:01:13.892323+0000","last_became_active":"2024-09-17T05:01:07.862731+0000","last_became_peered":"2024-09-17T05:01:07.862731+0000","last_unstale":"2024-09-17T05:01:13.892323+0000","last_undegraded":"2024-09-17T05:01:13.892323+0000","last_fullsized":"2024-09-17T05:01:13.892323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:39.024164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942455+0000","last_change":"2024-09-17T05:01:12.916294+0000","last_active":"2024-09-17T05:01:13.942455+0000","last_peered":"2024-09-17T05:01:13.942455+0000","last_clean":"2024-09-17T05:01:13.942455+0000","last_became_active":"2024-09-17T05:01:12.908035+0000","last_became_peered":"2024-09-17T05:01:12.908035+0000","last_unstale":"2024-09-17T05:01:13.942455+0000","last_undegraded":"2024-09-17T05:01:13.942455+0000","last_fullsized":"2024-09-17T05:01:13.942455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:52.101015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942370+0000","last_change":"2024-09-17T05:01:08.870629+0000","last_active":"2024-09-17T05:01:13.942370+0000","last_peered":"2024-09-17T05:01:13.942370+0000","last_clean":"2024-09-17T05:01:13.942370+0000","last_became_active":"2024-09-17T05:01:08.870198+0000","last_became_peered":"2024-09-17T05:01:08.870198+0000","last_unstale":"2024-09-17T05:01:13.942370+0000","last_undegraded":"2024-09-17T05:01:13.942370+0000","last_fullsized":"2024-09-17T05:01:13.942370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:37.142368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941967+0000","last_change":"2024-09-17T05:01:07.860691+0000","last_active":"2024-09-17T05:01:13.941967+0000","last_peered":"2024-09-17T05:01:13.941967+0000","last_clean":"2024-09-17T05:01:13.941967+0000","last_became_active":"2024-09-17T05:01:07.860360+0000","last_became_peered":"2024-09-17T05:01:07.860360+0000","last_unstale":"2024-09-17T05:01:13.941967+0000","last_undegraded":"2024-09-17T05:01:13.941967+0000","last_fullsized":"2024-09-17T05:01:13.941967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940513+0000","last_change":"2024-09-17T05:01:12.900669+0000","last_active":"2024-09-17T05:01:13.940513+0000","last_peered":"2024-09-17T05:01:13.940513+0000","last_clean":"2024-09-17T05:01:13.940513+0000","last_became_active":"2024-09-17T05:01:12.900234+0000","last_became_peered":"2024-09-17T05:01:12.900234+0000","last_unstale":"2024-09-17T05:01:13.940513+0000","last_undegraded":"2024-09-17T05:01:13.940513+0000","last_fullsized":"2024-09-17T05:01:13.940513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893905+0000","last_change":"2024-09-17T05:01:08.876794+0000","last_active":"2024-09-17T05:01:13.893905+0000","last_peered":"2024-09-17T05:01:13.893905+0000","last_clean":"2024-09-17T05:01:13.893905+0000","last_became_active":"2024-09-17T05:01:08.876418+0000","last_became_peered":"2024-09-17T05:01:08.876418+0000","last_unstale":"2024-09-17T05:01:13.893905+0000","last_undegraded":"2024-09-17T05:01:13.893905+0000","last_fullsized":"2024-09-17T05:01:13.893905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:47.662092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892328+0000","last_change":"2024-09-17T05:01:07.877675+0000","last_active":"2024-09-17T05:01:13.892328+0000","last_peered":"2024-09-17T05:01:13.892328+0000","last_clean":"2024-09-17T05:01:13.892328+0000","last_became_active":"2024-09-17T05:01:07.877320+0000","last_became_peered":"2024-09-17T05:01:07.877320+0000","last_unstale":"2024-09-17T05:01:13.892328+0000","last_undegraded":"2024-09-17T05:01:13.892328+0000","last_fullsized":"2024-09-17T05:01:13.892328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:35.509947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891833+0000","last_change":"2024-09-17T05:01:12.911570+0000","last_active":"2024-09-17T05:01:13.891833+0000","last_peered":"2024-09-17T05:01:13.891833+0000","last_clean":"2024-09-17T05:01:13.891833+0000","last_became_active":"2024-09-17T05:01:12.911110+0000","last_became_peered":"2024-09-17T05:01:12.911110+0000","last_unstale":"2024-09-17T05:01:13.891833+0000","last_undegraded":"2024-09-17T05:01:13.891833+0000","last_fullsized":"2024-09-17T05:01:13.891833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:38.085327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943250+0000","last_change":"2024-09-17T05:01:08.870024+0000","last_active":"2024-09-17T05:01:13.943250+0000","last_peered":"2024-09-17T05:01:13.943250+0000","last_clean":"2024-09-17T05:01:13.943250+0000","last_became_active":"2024-09-17T05:01:08.869679+0000","last_became_peered":"2024-09-17T05:01:08.869679+0000","last_unstale":"2024-09-17T05:01:13.943250+0000","last_undegraded":"2024-09-17T05:01:13.943250+0000","last_fullsized":"2024-09-17T05:01:13.943250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:45.964067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892819+0000","last_change":"2024-09-17T05:01:07.859295+0000","last_active":"2024-09-17T05:01:13.892819+0000","last_peered":"2024-09-17T05:01:13.892819+0000","last_clean":"2024-09-17T05:01:13.892819+0000","last_became_active":"2024-09-17T05:01:07.858949+0000","last_became_peered":"2024-09-17T05:01:07.858949+0000","last_unstale":"2024-09-17T05:01:13.892819+0000","last_undegraded":"2024-09-17T05:01:13.892819+0000","last_fullsized":"2024-09-17T05:01:13.892819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:29.635679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892487+0000","last_change":"2024-09-17T05:01:12.915413+0000","last_active":"2024-09-17T05:01:13.892487+0000","last_peered":"2024-09-17T05:01:13.892487+0000","last_clean":"2024-09-17T05:01:13.892487+0000","last_became_active":"2024-09-17T05:01:12.914942+0000","last_became_peered":"2024-09-17T05:01:12.914942+0000","last_unstale":"2024-09-17T05:01:13.892487+0000","last_undegraded":"2024-09-17T05:01:13.892487+0000","last_fullsized":"2024-09-17T05:01:13.892487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:57.733227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941319+0000","last_change":"2024-09-17T05:01:08.871457+0000","last_active":"2024-09-17T05:01:13.941319+0000","last_peered":"2024-09-17T05:01:13.941319+0000","last_clean":"2024-09-17T05:01:13.941319+0000","last_became_active":"2024-09-17T05:01:08.871135+0000","last_became_peered":"2024-09-17T05:01:08.871135+0000","last_unstale":"2024-09-17T05:01:13.941319+0000","last_undegraded":"2024-09-17T05:01:13.941319+0000","last_fullsized":"2024-09-17T05:01:13.941319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:15.936190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:07.873885+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:07.869214+0000","last_became_peered":"2024-09-17T05:01:07.869214+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:16.951737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893521+0000","last_change":"2024-09-17T05:01:12.908768+0000","last_active":"2024-09-17T05:01:13.893521+0000","last_peered":"2024-09-17T05:01:13.893521+0000","last_clean":"2024-09-17T05:01:13.893521+0000","last_became_active":"2024-09-17T05:01:12.908282+0000","last_became_peered":"2024-09-17T05:01:12.908282+0000","last_unstale":"2024-09-17T05:01:13.893521+0000","last_undegraded":"2024-09-17T05:01:13.893521+0000","last_fullsized":"2024-09-17T05:01:13.893521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891855+0000","last_change":"2024-09-17T05:01:08.858109+0000","last_active":"2024-09-17T05:01:13.891855+0000","last_peered":"2024-09-17T05:01:13.891855+0000","last_clean":"2024-09-17T05:01:13.891855+0000","last_became_active":"2024-09-17T05:01:08.857293+0000","last_became_peered":"2024-09-17T05:01:08.857293+0000","last_unstale":"2024-09-17T05:01:13.891855+0000","last_undegraded":"2024-09-17T05:01:13.891855+0000","last_fullsized":"2024-09-17T05:01:13.891855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:53.180346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941193+0000","last_change":"2024-09-17T05:01:07.864654+0000","last_active":"2024-09-17T05:01:13.941193+0000","last_peered":"2024-09-17T05:01:13.941193+0000","last_clean":"2024-09-17T05:01:13.941193+0000","last_became_active":"2024-09-17T05:01:07.864166+0000","last_became_peered":"2024-09-17T05:01:07.864166+0000","last_unstale":"2024-09-17T05:01:13.941193+0000","last_undegraded":"2024-09-17T05:01:13.941193+0000","last_fullsized":"2024-09-17T05:01:13.941193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894265+0000","last_change":"2024-09-17T05:01:12.914524+0000","last_active":"2024-09-17T05:01:13.894265+0000","last_peered":"2024-09-17T05:01:13.894265+0000","last_clean":"2024-09-17T05:01:13.894265+0000","last_became_active":"2024-09-17T05:01:12.912222+0000","last_became_peered":"2024-09-17T05:01:12.912222+0000","last_unstale":"2024-09-17T05:01:13.894265+0000","last_undegraded":"2024-09-17T05:01:13.894265+0000","last_fullsized":"2024-09-17T05:01:13.894265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892306+0000","last_change":"2024-09-17T05:01:08.875660+0000","last_active":"2024-09-17T05:01:13.892306+0000","last_peered":"2024-09-17T05:01:13.892306+0000","last_clean":"2024-09-17T05:01:13.892306+0000","last_became_active":"2024-09-17T05:01:08.875265+0000","last_became_peered":"2024-09-17T05:01:08.875265+0000","last_unstale":"2024-09-17T05:01:13.892306+0000","last_undegraded":"2024-09-17T05:01:13.892306+0000","last_fullsized":"2024-09-17T05:01:13.892306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891158+0000","last_change":"2024-09-17T05:01:07.869491+0000","last_active":"2024-09-17T05:01:13.891158+0000","last_peered":"2024-09-17T05:01:13.891158+0000","last_clean":"2024-09-17T05:01:13.891158+0000","last_became_active":"2024-09-17T05:01:07.869038+0000","last_became_peered":"2024-09-17T05:01:07.869038+0000","last_unstale":"2024-09-17T05:01:13.891158+0000","last_undegraded":"2024-09-17T05:01:13.891158+0000","last_fullsized":"2024-09-17T05:01:13.891158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:00.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893711+0000","last_change":"2024-09-17T05:01:12.910830+0000","last_active":"2024-09-17T05:01:13.893711+0000","last_peered":"2024-09-17T05:01:13.893711+0000","last_clean":"2024-09-17T05:01:13.893711+0000","last_became_active":"2024-09-17T05:01:12.908497+0000","last_became_peered":"2024-09-17T05:01:12.908497+0000","last_unstale":"2024-09-17T05:01:13.893711+0000","last_undegraded":"2024-09-17T05:01:13.893711+0000","last_fullsized":"2024-09-17T05:01:13.893711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:07.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941121+0000","last_change":"2024-09-17T05:01:08.868170+0000","last_active":"2024-09-17T05:01:13.941121+0000","last_peered":"2024-09-17T05:01:13.941121+0000","last_clean":"2024-09-17T05:01:13.941121+0000","last_became_active":"2024-09-17T05:01:08.867767+0000","last_became_peered":"2024-09-17T05:01:08.867767+0000","last_unstale":"2024-09-17T05:01:13.941121+0000","last_undegraded":"2024-09-17T05:01:13.941121+0000","last_fullsized":"2024-09-17T05:01:13.941121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891555+0000","last_change":"2024-09-17T05:01:07.873908+0000","last_active":"2024-09-17T05:01:13.891555+0000","last_peered":"2024-09-17T05:01:13.891555+0000","last_clean":"2024-09-17T05:01:13.891555+0000","last_became_active":"2024-09-17T05:01:07.871028+0000","last_became_peered":"2024-09-17T05:01:07.871028+0000","last_unstale":"2024-09-17T05:01:13.891555+0000","last_undegraded":"2024-09-17T05:01:13.891555+0000","last_fullsized":"2024-09-17T05:01:13.891555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941889+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:01:13.941889+0000","last_peered":"2024-09-17T05:01:13.941889+0000","last_clean":"2024-09-17T05:01:13.941889+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:01:13.941889+0000","last_undegraded":"2024-09-17T05:01:13.941889+0000","last_fullsized":"2024-09-17T05:01:13.941889+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892300+0000","last_change":"2024-09-17T05:01:12.904503+0000","last_active":"2024-09-17T05:01:13.892300+0000","last_peered":"2024-09-17T05:01:13.892300+0000","last_clean":"2024-09-17T05:01:13.892300+0000","last_became_active":"2024-09-17T05:01:12.903910+0000","last_became_peered":"2024-09-17T05:01:12.903910+0000","last_unstale":"2024-09-17T05:01:13.892300+0000","last_undegraded":"2024-09-17T05:01:13.892300+0000","last_fullsized":"2024-09-17T05:01:13.892300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:47.490639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892152+0000","last_change":"2024-09-17T05:01:08.872909+0000","last_active":"2024-09-17T05:01:13.892152+0000","last_peered":"2024-09-17T05:01:13.892152+0000","last_clean":"2024-09-17T05:01:13.892152+0000","last_became_active":"2024-09-17T05:01:08.861070+0000","last_became_peered":"2024-09-17T05:01:08.861070+0000","last_unstale":"2024-09-17T05:01:13.892152+0000","last_undegraded":"2024-09-17T05:01:13.892152+0000","last_fullsized":"2024-09-17T05:01:13.892152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:44.525838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894016+0000","last_change":"2024-09-17T05:01:07.866417+0000","last_active":"2024-09-17T05:01:13.894016+0000","last_peered":"2024-09-17T05:01:13.894016+0000","last_clean":"2024-09-17T05:01:13.894016+0000","last_became_active":"2024-09-17T05:01:07.866088+0000","last_became_peered":"2024-09-17T05:01:07.866088+0000","last_unstale":"2024-09-17T05:01:13.894016+0000","last_undegraded":"2024-09-17T05:01:13.894016+0000","last_fullsized":"2024-09-17T05:01:13.894016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942333+0000","last_change":"2024-09-17T05:01:12.919691+0000","last_active":"2024-09-17T05:01:13.942333+0000","last_peered":"2024-09-17T05:01:13.942333+0000","last_clean":"2024-09-17T05:01:13.942333+0000","last_became_active":"2024-09-17T05:01:12.917951+0000","last_became_peered":"2024-09-17T05:01:12.917951+0000","last_unstale":"2024-09-17T05:01:13.942333+0000","last_undegraded":"2024-09-17T05:01:13.942333+0000","last_fullsized":"2024-09-17T05:01:13.942333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:11.524293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942694+0000","last_change":"2024-09-17T05:01:08.864231+0000","last_active":"2024-09-17T05:01:13.942694+0000","last_peered":"2024-09-17T05:01:13.942694+0000","last_clean":"2024-09-17T05:01:13.942694+0000","last_became_active":"2024-09-17T05:01:08.863802+0000","last_became_peered":"2024-09-17T05:01:08.863802+0000","last_unstale":"2024-09-17T05:01:13.942694+0000","last_undegraded":"2024-09-17T05:01:13.942694+0000","last_fullsized":"2024-09-17T05:01:13.942694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893511+0000","last_change":"2024-09-17T05:01:07.857650+0000","last_active":"2024-09-17T05:01:13.893511+0000","last_peered":"2024-09-17T05:01:13.893511+0000","last_clean":"2024-09-17T05:01:13.893511+0000","last_became_active":"2024-09-17T05:01:07.857311+0000","last_became_peered":"2024-09-17T05:01:07.857311+0000","last_unstale":"2024-09-17T05:01:13.893511+0000","last_undegraded":"2024-09-17T05:01:13.893511+0000","last_fullsized":"2024-09-17T05:01:13.893511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:27.902116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893168+0000","last_change":"2024-09-17T05:01:12.888243+0000","last_active":"2024-09-17T05:01:13.893168+0000","last_peered":"2024-09-17T05:01:13.893168+0000","last_clean":"2024-09-17T05:01:13.893168+0000","last_became_active":"2024-09-17T05:01:12.887793+0000","last_became_peered":"2024-09-17T05:01:12.887793+0000","last_unstale":"2024-09-17T05:01:13.893168+0000","last_undegraded":"2024-09-17T05:01:13.893168+0000","last_fullsized":"2024-09-17T05:01:13.893168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:00.157222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942457+0000","last_change":"2024-09-17T05:01:08.889497+0000","last_active":"2024-09-17T05:01:13.942457+0000","last_peered":"2024-09-17T05:01:13.942457+0000","last_clean":"2024-09-17T05:01:13.942457+0000","last_became_active":"2024-09-17T05:01:08.887064+0000","last_became_peered":"2024-09-17T05:01:08.887064+0000","last_unstale":"2024-09-17T05:01:13.942457+0000","last_undegraded":"2024-09-17T05:01:13.942457+0000","last_fullsized":"2024-09-17T05:01:13.942457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941847+0000","last_change":"2024-09-17T05:01:07.851404+0000","last_active":"2024-09-17T05:01:13.941847+0000","last_peered":"2024-09-17T05:01:13.941847+0000","last_clean":"2024-09-17T05:01:13.941847+0000","last_became_active":"2024-09-17T05:01:07.851077+0000","last_became_peered":"2024-09-17T05:01:07.851077+0000","last_unstale":"2024-09-17T05:01:13.941847+0000","last_undegraded":"2024-09-17T05:01:13.941847+0000","last_fullsized":"2024-09-17T05:01:13.941847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890772+0000","last_change":"2024-09-17T05:01:12.905280+0000","last_active":"2024-09-17T05:01:13.890772+0000","last_peered":"2024-09-17T05:01:13.890772+0000","last_clean":"2024-09-17T05:01:13.890772+0000","last_became_active":"2024-09-17T05:01:12.904787+0000","last_became_peered":"2024-09-17T05:01:12.904787+0000","last_unstale":"2024-09-17T05:01:13.890772+0000","last_undegraded":"2024-09-17T05:01:13.890772+0000","last_fullsized":"2024-09-17T05:01:13.890772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:29.666258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.884835+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.884410+0000","last_became_peered":"2024-09-17T05:01:08.884410+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:35.960729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942469+0000","last_change":"2024-09-17T05:01:07.866602+0000","last_active":"2024-09-17T05:01:13.942469+0000","last_peered":"2024-09-17T05:01:13.942469+0000","last_clean":"2024-09-17T05:01:13.942469+0000","last_became_active":"2024-09-17T05:01:07.866061+0000","last_became_peered":"2024-09-17T05:01:07.866061+0000","last_unstale":"2024-09-17T05:01:13.942469+0000","last_undegraded":"2024-09-17T05:01:13.942469+0000","last_fullsized":"2024-09-17T05:01:13.942469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943528+0000","last_change":"2024-09-17T05:01:12.904969+0000","last_active":"2024-09-17T05:01:13.943528+0000","last_peered":"2024-09-17T05:01:13.943528+0000","last_clean":"2024-09-17T05:01:13.943528+0000","last_became_active":"2024-09-17T05:01:12.904665+0000","last_became_peered":"2024-09-17T05:01:12.904665+0000","last_unstale":"2024-09-17T05:01:13.943528+0000","last_undegraded":"2024-09-17T05:01:13.943528+0000","last_fullsized":"2024-09-17T05:01:13.943528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:52.943963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890912+0000","last_change":"2024-09-17T05:01:08.872790+0000","last_active":"2024-09-17T05:01:13.890912+0000","last_peered":"2024-09-17T05:01:13.890912+0000","last_clean":"2024-09-17T05:01:13.890912+0000","last_became_active":"2024-09-17T05:01:08.872387+0000","last_became_peered":"2024-09-17T05:01:08.872387+0000","last_unstale":"2024-09-17T05:01:13.890912+0000","last_undegraded":"2024-09-17T05:01:13.890912+0000","last_fullsized":"2024-09-17T05:01:13.890912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940575+0000","last_change":"2024-09-17T05:01:07.854980+0000","last_active":"2024-09-17T05:01:13.940575+0000","last_peered":"2024-09-17T05:01:13.940575+0000","last_clean":"2024-09-17T05:01:13.940575+0000","last_became_active":"2024-09-17T05:01:07.854665+0000","last_became_peered":"2024-09-17T05:01:07.854665+0000","last_unstale":"2024-09-17T05:01:13.940575+0000","last_undegraded":"2024-09-17T05:01:13.940575+0000","last_fullsized":"2024-09-17T05:01:13.940575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893909+0000","last_change":"2024-09-17T05:01:12.908923+0000","last_active":"2024-09-17T05:01:13.893909+0000","last_peered":"2024-09-17T05:01:13.893909+0000","last_clean":"2024-09-17T05:01:13.893909+0000","last_became_active":"2024-09-17T05:01:12.908398+0000","last_became_peered":"2024-09-17T05:01:12.908398+0000","last_unstale":"2024-09-17T05:01:13.893909+0000","last_undegraded":"2024-09-17T05:01:13.893909+0000","last_fullsized":"2024-09-17T05:01:13.893909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:45.118442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941757+0000","last_change":"2024-09-17T05:01:08.856311+0000","last_active":"2024-09-17T05:01:13.941757+0000","last_peered":"2024-09-17T05:01:13.941757+0000","last_clean":"2024-09-17T05:01:13.941757+0000","last_became_active":"2024-09-17T05:01:08.855334+0000","last_became_peered":"2024-09-17T05:01:08.855334+0000","last_unstale":"2024-09-17T05:01:13.941757+0000","last_undegraded":"2024-09-17T05:01:13.941757+0000","last_fullsized":"2024-09-17T05:01:13.941757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942247+0000","last_change":"2024-09-17T05:01:07.872183+0000","last_active":"2024-09-17T05:01:13.942247+0000","last_peered":"2024-09-17T05:01:13.942247+0000","last_clean":"2024-09-17T05:01:13.942247+0000","last_became_active":"2024-09-17T05:01:07.871540+0000","last_became_peered":"2024-09-17T05:01:07.871540+0000","last_unstale":"2024-09-17T05:01:13.942247+0000","last_undegraded":"2024-09-17T05:01:13.942247+0000","last_fullsized":"2024-09-17T05:01:13.942247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894214+0000","last_change":"2024-09-17T05:01:12.906286+0000","last_active":"2024-09-17T05:01:13.894214+0000","last_peered":"2024-09-17T05:01:13.894214+0000","last_clean":"2024-09-17T05:01:13.894214+0000","last_became_active":"2024-09-17T05:01:12.905810+0000","last_became_peered":"2024-09-17T05:01:12.905810+0000","last_unstale":"2024-09-17T05:01:13.894214+0000","last_undegraded":"2024-09-17T05:01:13.894214+0000","last_fullsized":"2024-09-17T05:01:13.894214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:33.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891317+0000","last_change":"2024-09-17T05:01:08.893475+0000","last_active":"2024-09-17T05:01:13.891317+0000","last_peered":"2024-09-17T05:01:13.891317+0000","last_clean":"2024-09-17T05:01:13.891317+0000","last_became_active":"2024-09-17T05:01:08.893130+0000","last_became_peered":"2024-09-17T05:01:08.893130+0000","last_unstale":"2024-09-17T05:01:13.891317+0000","last_undegraded":"2024-09-17T05:01:13.891317+0000","last_fullsized":"2024-09-17T05:01:13.891317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891492+0000","last_change":"2024-09-17T05:01:07.869315+0000","last_active":"2024-09-17T05:01:13.891492+0000","last_peered":"2024-09-17T05:01:13.891492+0000","last_clean":"2024-09-17T05:01:13.891492+0000","last_became_active":"2024-09-17T05:01:07.864783+0000","last_became_peered":"2024-09-17T05:01:07.864783+0000","last_unstale":"2024-09-17T05:01:13.891492+0000","last_undegraded":"2024-09-17T05:01:13.891492+0000","last_fullsized":"2024-09-17T05:01:13.891492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893698+0000","last_change":"2024-09-17T05:01:12.921137+0000","last_active":"2024-09-17T05:01:13.893698+0000","last_peered":"2024-09-17T05:01:13.893698+0000","last_clean":"2024-09-17T05:01:13.893698+0000","last_became_active":"2024-09-17T05:01:12.920809+0000","last_became_peered":"2024-09-17T05:01:12.920809+0000","last_unstale":"2024-09-17T05:01:13.893698+0000","last_undegraded":"2024-09-17T05:01:13.893698+0000","last_fullsized":"2024-09-17T05:01:13.893698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892745+0000","last_change":"2024-09-17T05:01:08.892262+0000","last_active":"2024-09-17T05:01:13.892745+0000","last_peered":"2024-09-17T05:01:13.892745+0000","last_clean":"2024-09-17T05:01:13.892745+0000","last_became_active":"2024-09-17T05:01:08.891946+0000","last_became_peered":"2024-09-17T05:01:08.891946+0000","last_unstale":"2024-09-17T05:01:13.892745+0000","last_undegraded":"2024-09-17T05:01:13.892745+0000","last_fullsized":"2024-09-17T05:01:13.892745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:40.031283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941492+0000","last_change":"2024-09-17T05:01:07.872646+0000","last_active":"2024-09-17T05:01:13.941492+0000","last_peered":"2024-09-17T05:01:13.941492+0000","last_clean":"2024-09-17T05:01:13.941492+0000","last_became_active":"2024-09-17T05:01:07.872287+0000","last_became_peered":"2024-09-17T05:01:07.872287+0000","last_unstale":"2024-09-17T05:01:13.941492+0000","last_undegraded":"2024-09-17T05:01:13.941492+0000","last_fullsized":"2024-09-17T05:01:13.941492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:56.785994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891950+0000","last_change":"2024-09-17T05:01:12.900658+0000","last_active":"2024-09-17T05:01:13.891950+0000","last_peered":"2024-09-17T05:01:13.891950+0000","last_clean":"2024-09-17T05:01:13.891950+0000","last_became_active":"2024-09-17T05:01:12.900131+0000","last_became_peered":"2024-09-17T05:01:12.900131+0000","last_unstale":"2024-09-17T05:01:13.891950+0000","last_undegraded":"2024-09-17T05:01:13.891950+0000","last_fullsized":"2024-09-17T05:01:13.891950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:54.427072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941925+0000","last_change":"2024-09-17T05:01:08.869878+0000","last_active":"2024-09-17T05:01:13.941925+0000","last_peered":"2024-09-17T05:01:13.941925+0000","last_clean":"2024-09-17T05:01:13.941925+0000","last_became_active":"2024-09-17T05:01:08.869571+0000","last_became_peered":"2024-09-17T05:01:08.869571+0000","last_unstale":"2024-09-17T05:01:13.941925+0000","last_undegraded":"2024-09-17T05:01:13.941925+0000","last_fullsized":"2024-09-17T05:01:13.941925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:29.847123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:19.977855+0000","last_change":"2024-09-17T05:01:07.862966+0000","last_active":"2024-09-17T05:01:19.977855+0000","last_peered":"2024-09-17T05:01:19.977855+0000","last_clean":"2024-09-17T05:01:19.977855+0000","last_became_active":"2024-09-17T05:01:07.862605+0000","last_became_peered":"2024-09-17T05:01:07.862605+0000","last_unstale":"2024-09-17T05:01:19.977855+0000","last_undegraded":"2024-09-17T05:01:19.977855+0000","last_fullsized":"2024-09-17T05:01:19.977855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:26.467925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941104+0000","last_change":"2024-09-17T05:01:12.897023+0000","last_active":"2024-09-17T05:01:13.941104+0000","last_peered":"2024-09-17T05:01:13.941104+0000","last_clean":"2024-09-17T05:01:13.941104+0000","last_became_active":"2024-09-17T05:01:12.896528+0000","last_became_peered":"2024-09-17T05:01:12.896528+0000","last_unstale":"2024-09-17T05:01:13.941104+0000","last_undegraded":"2024-09-17T05:01:13.941104+0000","last_fullsized":"2024-09-17T05:01:13.941104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:04.392152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:08.866251+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:08.864997+0000","last_became_peered":"2024-09-17T05:01:08.864997+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:33.305210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942113+0000","last_change":"2024-09-17T05:01:07.862662+0000","last_active":"2024-09-17T05:01:13.942113+0000","last_peered":"2024-09-17T05:01:13.942113+0000","last_clean":"2024-09-17T05:01:13.942113+0000","last_became_active":"2024-09-17T05:01:07.862252+0000","last_became_peered":"2024-09-17T05:01:07.862252+0000","last_unstale":"2024-09-17T05:01:13.942113+0000","last_undegraded":"2024-09-17T05:01:13.942113+0000","last_fullsized":"2024-09-17T05:01:13.942113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:13.958450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942881+0000","last_change":"2024-09-17T05:01:12.915486+0000","last_active":"2024-09-17T05:01:13.942881+0000","last_peered":"2024-09-17T05:01:13.942881+0000","last_clean":"2024-09-17T05:01:13.942881+0000","last_became_active":"2024-09-17T05:01:12.913794+0000","last_became_peered":"2024-09-17T05:01:12.913794+0000","last_unstale":"2024-09-17T05:01:13.942881+0000","last_undegraded":"2024-09-17T05:01:13.942881+0000","last_fullsized":"2024-09-17T05:01:13.942881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:33.883073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894291+0000","last_change":"2024-09-17T05:01:08.863593+0000","last_active":"2024-09-17T05:01:13.894291+0000","last_peered":"2024-09-17T05:01:13.894291+0000","last_clean":"2024-09-17T05:01:13.894291+0000","last_became_active":"2024-09-17T05:01:08.863264+0000","last_became_peered":"2024-09-17T05:01:08.863264+0000","last_unstale":"2024-09-17T05:01:13.894291+0000","last_undegraded":"2024-09-17T05:01:13.894291+0000","last_fullsized":"2024-09-17T05:01:13.894291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:00.186616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942093+0000","last_change":"2024-09-17T05:01:07.860203+0000","last_active":"2024-09-17T05:01:13.942093+0000","last_peered":"2024-09-17T05:01:13.942093+0000","last_clean":"2024-09-17T05:01:13.942093+0000","last_became_active":"2024-09-17T05:01:07.858682+0000","last_became_peered":"2024-09-17T05:01:07.858682+0000","last_unstale":"2024-09-17T05:01:13.942093+0000","last_undegraded":"2024-09-17T05:01:13.942093+0000","last_fullsized":"2024-09-17T05:01:13.942093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940512+0000","last_change":"2024-09-17T05:01:12.912101+0000","last_active":"2024-09-17T05:01:13.940512+0000","last_peered":"2024-09-17T05:01:13.940512+0000","last_clean":"2024-09-17T05:01:13.940512+0000","last_became_active":"2024-09-17T05:01:12.911449+0000","last_became_peered":"2024-09-17T05:01:12.911449+0000","last_unstale":"2024-09-17T05:01:13.940512+0000","last_undegraded":"2024-09-17T05:01:13.940512+0000","last_fullsized":"2024-09-17T05:01:13.940512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:51.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.874496+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.874209+0000","last_became_peered":"2024-09-17T05:01:08.874209+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:06.383708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944328+0000","last_change":"2024-09-17T05:01:07.850788+0000","last_active":"2024-09-17T05:01:13.944328+0000","last_peered":"2024-09-17T05:01:13.944328+0000","last_clean":"2024-09-17T05:01:13.944328+0000","last_became_active":"2024-09-17T05:01:07.850154+0000","last_became_peered":"2024-09-17T05:01:07.850154+0000","last_unstale":"2024-09-17T05:01:13.944328+0000","last_undegraded":"2024-09-17T05:01:13.944328+0000","last_fullsized":"2024-09-17T05:01:13.944328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940756+0000","last_change":"2024-09-17T05:01:12.908537+0000","last_active":"2024-09-17T05:01:13.940756+0000","last_peered":"2024-09-17T05:01:13.940756+0000","last_clean":"2024-09-17T05:01:13.940756+0000","last_became_active":"2024-09-17T05:01:12.908060+0000","last_became_peered":"2024-09-17T05:01:12.908060+0000","last_unstale":"2024-09-17T05:01:13.940756+0000","last_undegraded":"2024-09-17T05:01:13.940756+0000","last_fullsized":"2024-09-17T05:01:13.940756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941572+0000","last_change":"2024-09-17T05:01:12.895140+0000","last_active":"2024-09-17T05:01:13.941572+0000","last_peered":"2024-09-17T05:01:13.941572+0000","last_clean":"2024-09-17T05:01:13.941572+0000","last_became_active":"2024-09-17T05:01:12.894665+0000","last_became_peered":"2024-09-17T05:01:12.894665+0000","last_unstale":"2024-09-17T05:01:13.941572+0000","last_undegraded":"2024-09-17T05:01:13.941572+0000","last_fullsized":"2024-09-17T05:01:13.941572+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:22.431196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940721+0000","last_change":"2024-09-17T05:01:08.851860+0000","last_active":"2024-09-17T05:01:13.940721+0000","last_peered":"2024-09-17T05:01:13.940721+0000","last_clean":"2024-09-17T05:01:13.940721+0000","last_became_active":"2024-09-17T05:01:08.851546+0000","last_became_peered":"2024-09-17T05:01:08.851546+0000","last_unstale":"2024-09-17T05:01:13.940721+0000","last_undegraded":"2024-09-17T05:01:13.940721+0000","last_fullsized":"2024-09-17T05:01:13.940721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:25.615656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891094+0000","last_change":"2024-09-17T05:01:07.876275+0000","last_active":"2024-09-17T05:01:13.891094+0000","last_peered":"2024-09-17T05:01:13.891094+0000","last_clean":"2024-09-17T05:01:13.891094+0000","last_became_active":"2024-09-17T05:01:07.875993+0000","last_became_peered":"2024-09-17T05:01:07.875993+0000","last_unstale":"2024-09-17T05:01:13.891094+0000","last_undegraded":"2024-09-17T05:01:13.891094+0000","last_fullsized":"2024-09-17T05:01:13.891094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:43.841239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891826+0000","last_change":"2024-09-17T05:01:12.904195+0000","last_active":"2024-09-17T05:01:13.891826+0000","last_peered":"2024-09-17T05:01:13.891826+0000","last_clean":"2024-09-17T05:01:13.891826+0000","last_became_active":"2024-09-17T05:01:12.903808+0000","last_became_peered":"2024-09-17T05:01:12.903808+0000","last_unstale":"2024-09-17T05:01:13.891826+0000","last_undegraded":"2024-09-17T05:01:13.891826+0000","last_fullsized":"2024-09-17T05:01:13.891826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:11.189684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891943+0000","last_change":"2024-09-17T05:01:08.876556+0000","last_active":"2024-09-17T05:01:13.891943+0000","last_peered":"2024-09-17T05:01:13.891943+0000","last_clean":"2024-09-17T05:01:13.891943+0000","last_became_active":"2024-09-17T05:01:08.874581+0000","last_became_peered":"2024-09-17T05:01:08.874581+0000","last_unstale":"2024-09-17T05:01:13.891943+0000","last_undegraded":"2024-09-17T05:01:13.891943+0000","last_fullsized":"2024-09-17T05:01:13.891943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:59.333120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892433+0000","last_change":"2024-09-17T05:01:07.865323+0000","last_active":"2024-09-17T05:01:13.892433+0000","last_peered":"2024-09-17T05:01:13.892433+0000","last_clean":"2024-09-17T05:01:13.892433+0000","last_became_active":"2024-09-17T05:01:07.864832+0000","last_became_peered":"2024-09-17T05:01:07.864832+0000","last_unstale":"2024-09-17T05:01:13.892433+0000","last_undegraded":"2024-09-17T05:01:13.892433+0000","last_fullsized":"2024-09-17T05:01:13.892433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891348+0000","last_change":"2024-09-17T05:01:07.870957+0000","last_active":"2024-09-17T05:01:13.891348+0000","last_peered":"2024-09-17T05:01:13.891348+0000","last_clean":"2024-09-17T05:01:13.891348+0000","last_became_active":"2024-09-17T05:01:07.869025+0000","last_became_peered":"2024-09-17T05:01:07.869025+0000","last_unstale":"2024-09-17T05:01:13.891348+0000","last_undegraded":"2024-09-17T05:01:13.891348+0000","last_fullsized":"2024-09-17T05:01:13.891348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940880+0000","last_change":"2024-09-17T05:01:08.876853+0000","last_active":"2024-09-17T05:01:13.940880+0000","last_peered":"2024-09-17T05:01:13.940880+0000","last_clean":"2024-09-17T05:01:13.940880+0000","last_became_active":"2024-09-17T05:01:08.876525+0000","last_became_peered":"2024-09-17T05:01:08.876525+0000","last_unstale":"2024-09-17T05:01:13.940880+0000","last_undegraded":"2024-09-17T05:01:13.940880+0000","last_fullsized":"2024-09-17T05:01:13.940880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:50.685520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941188+0000","last_change":"2024-09-17T05:01:12.908492+0000","last_active":"2024-09-17T05:01:13.941188+0000","last_peered":"2024-09-17T05:01:13.941188+0000","last_clean":"2024-09-17T05:01:13.941188+0000","last_became_active":"2024-09-17T05:01:12.908032+0000","last_became_peered":"2024-09-17T05:01:12.908032+0000","last_unstale":"2024-09-17T05:01:13.941188+0000","last_undegraded":"2024-09-17T05:01:13.941188+0000","last_fullsized":"2024-09-17T05:01:13.941188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893141+0000","last_change":"2024-09-17T05:01:07.859924+0000","last_active":"2024-09-17T05:01:13.893141+0000","last_peered":"2024-09-17T05:01:13.893141+0000","last_clean":"2024-09-17T05:01:13.893141+0000","last_became_active":"2024-09-17T05:01:07.859692+0000","last_became_peered":"2024-09-17T05:01:07.859692+0000","last_unstale":"2024-09-17T05:01:13.893141+0000","last_undegraded":"2024-09-17T05:01:13.893141+0000","last_fullsized":"2024-09-17T05:01:13.893141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.935910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943063+0000","last_change":"2024-09-17T05:01:08.886919+0000","last_active":"2024-09-17T05:01:13.943063+0000","last_peered":"2024-09-17T05:01:13.943063+0000","last_clean":"2024-09-17T05:01:13.943063+0000","last_became_active":"2024-09-17T05:01:08.886565+0000","last_became_peered":"2024-09-17T05:01:08.886565+0000","last_unstale":"2024-09-17T05:01:13.943063+0000","last_undegraded":"2024-09-17T05:01:13.943063+0000","last_fullsized":"2024-09-17T05:01:13.943063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941737+0000","last_change":"2024-09-17T05:01:12.919497+0000","last_active":"2024-09-17T05:01:13.941737+0000","last_peered":"2024-09-17T05:01:13.941737+0000","last_clean":"2024-09-17T05:01:13.941737+0000","last_became_active":"2024-09-17T05:01:12.919097+0000","last_became_peered":"2024-09-17T05:01:12.919097+0000","last_unstale":"2024-09-17T05:01:13.941737+0000","last_undegraded":"2024-09-17T05:01:13.941737+0000","last_fullsized":"2024-09-17T05:01:13.941737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:33.403463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941489+0000","last_change":"2024-09-17T05:01:08.868818+0000","last_active":"2024-09-17T05:01:13.941489+0000","last_peered":"2024-09-17T05:01:13.941489+0000","last_clean":"2024-09-17T05:01:13.941489+0000","last_became_active":"2024-09-17T05:01:08.868496+0000","last_became_peered":"2024-09-17T05:01:08.868496+0000","last_unstale":"2024-09-17T05:01:13.941489+0000","last_undegraded":"2024-09-17T05:01:13.941489+0000","last_fullsized":"2024-09-17T05:01:13.941489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:17.846926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941062+0000","last_change":"2024-09-17T05:01:08.881690+0000","last_active":"2024-09-17T05:01:13.941062+0000","last_peered":"2024-09-17T05:01:13.941062+0000","last_clean":"2024-09-17T05:01:13.941062+0000","last_became_active":"2024-09-17T05:01:08.880670+0000","last_became_peered":"2024-09-17T05:01:08.880670+0000","last_unstale":"2024-09-17T05:01:13.941062+0000","last_undegraded":"2024-09-17T05:01:13.941062+0000","last_fullsized":"2024-09-17T05:01:13.941062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891163+0000","last_change":"2024-09-17T05:01:08.867225+0000","last_active":"2024-09-17T05:01:13.891163+0000","last_peered":"2024-09-17T05:01:13.891163+0000","last_clean":"2024-09-17T05:01:13.891163+0000","last_became_active":"2024-09-17T05:01:08.866861+0000","last_became_peered":"2024-09-17T05:01:08.866861+0000","last_unstale":"2024-09-17T05:01:13.891163+0000","last_undegraded":"2024-09-17T05:01:13.891163+0000","last_fullsized":"2024-09-17T05:01:13.891163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:40.190786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892992+0000","last_change":"2024-09-17T05:01:08.862318+0000","last_active":"2024-09-17T05:01:13.892992+0000","last_peered":"2024-09-17T05:01:13.892992+0000","last_clean":"2024-09-17T05:01:13.892992+0000","last_became_active":"2024-09-17T05:01:08.861957+0000","last_became_peered":"2024-09-17T05:01:08.861957+0000","last_unstale":"2024-09-17T05:01:13.892992+0000","last_undegraded":"2024-09-17T05:01:13.892992+0000","last_fullsized":"2024-09-17T05:01:13.892992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:26.208177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893817+0000","last_change":"2024-09-17T05:01:08.883574+0000","last_active":"2024-09-17T05:01:13.893817+0000","last_peered":"2024-09-17T05:01:13.893817+0000","last_clean":"2024-09-17T05:01:13.893817+0000","last_became_active":"2024-09-17T05:01:08.883327+0000","last_became_peered":"2024-09-17T05:01:08.883327+0000","last_unstale":"2024-09-17T05:01:13.893817+0000","last_undegraded":"2024-09-17T05:01:13.893817+0000","last_fullsized":"2024-09-17T05:01:13.893817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893848+0000","last_change":"2024-09-17T05:01:08.877996+0000","last_active":"2024-09-17T05:01:13.893848+0000","last_peered":"2024-09-17T05:01:13.893848+0000","last_clean":"2024-09-17T05:01:13.893848+0000","last_became_active":"2024-09-17T05:01:08.877545+0000","last_became_peered":"2024-09-17T05:01:08.877545+0000","last_unstale":"2024-09-17T05:01:13.893848+0000","last_undegraded":"2024-09-17T05:01:13.893848+0000","last_fullsized":"2024-09-17T05:01:13.893848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:43.858748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943694+0000","last_change":"2024-09-17T05:01:08.874485+0000","last_active":"2024-09-17T05:01:13.943694+0000","last_peered":"2024-09-17T05:01:13.943694+0000","last_clean":"2024-09-17T05:01:13.943694+0000","last_became_active":"2024-09-17T05:01:08.874134+0000","last_became_peered":"2024-09-17T05:01:08.874134+0000","last_unstale":"2024-09-17T05:01:13.943694+0000","last_undegraded":"2024-09-17T05:01:13.943694+0000","last_fullsized":"2024-09-17T05:01:13.943694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:10.348602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893707+0000","last_change":"2024-09-17T05:01:08.886688+0000","last_active":"2024-09-17T05:01:13.893707+0000","last_peered":"2024-09-17T05:01:13.893707+0000","last_clean":"2024-09-17T05:01:13.893707+0000","last_became_active":"2024-09-17T05:01:08.886385+0000","last_became_peered":"2024-09-17T05:01:08.886385+0000","last_unstale":"2024-09-17T05:01:13.893707+0000","last_undegraded":"2024-09-17T05:01:13.893707+0000","last_fullsized":"2024-09-17T05:01:13.893707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:29.234911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891025+0000","last_change":"2024-09-17T05:01:08.866999+0000","last_active":"2024-09-17T05:01:13.891025+0000","last_peered":"2024-09-17T05:01:13.891025+0000","last_clean":"2024-09-17T05:01:13.891025+0000","last_became_active":"2024-09-17T05:01:08.866661+0000","last_became_peered":"2024-09-17T05:01:08.866661+0000","last_unstale":"2024-09-17T05:01:13.891025+0000","last_undegraded":"2024-09-17T05:01:13.891025+0000","last_fullsized":"2024-09-17T05:01:13.891025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:40.685724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892397+0000","last_change":"2024-09-17T05:01:08.871375+0000","last_active":"2024-09-17T05:01:13.892397+0000","last_peered":"2024-09-17T05:01:13.892397+0000","last_clean":"2024-09-17T05:01:13.892397+0000","last_became_active":"2024-09-17T05:01:08.870439+0000","last_became_peered":"2024-09-17T05:01:08.870439+0000","last_unstale":"2024-09-17T05:01:13.892397+0000","last_undegraded":"2024-09-17T05:01:13.892397+0000","last_fullsized":"2024-09-17T05:01:13.892397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:49.417178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942955+0000","last_change":"2024-09-17T05:01:08.866055+0000","last_active":"2024-09-17T05:01:13.942955+0000","last_peered":"2024-09-17T05:01:13.942955+0000","last_clean":"2024-09-17T05:01:13.942955+0000","last_became_active":"2024-09-17T05:01:08.865155+0000","last_became_peered":"2024-09-17T05:01:08.865155+0000","last_unstale":"2024-09-17T05:01:13.942955+0000","last_undegraded":"2024-09-17T05:01:13.942955+0000","last_fullsized":"2024-09-17T05:01:13.942955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940728+0000","last_change":"2024-09-17T05:01:08.868372+0000","last_active":"2024-09-17T05:01:13.940728+0000","last_peered":"2024-09-17T05:01:13.940728+0000","last_clean":"2024-09-17T05:01:13.940728+0000","last_became_active":"2024-09-17T05:01:08.867742+0000","last_became_peered":"2024-09-17T05:01:08.867742+0000","last_unstale":"2024-09-17T05:01:13.940728+0000","last_undegraded":"2024-09-17T05:01:13.940728+0000","last_fullsized":"2024-09-17T05:01:13.940728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:39.401927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942609+0000","last_change":"2024-09-17T05:01:08.884233+0000","last_active":"2024-09-17T05:01:13.942609+0000","last_peered":"2024-09-17T05:01:13.942609+0000","last_clean":"2024-09-17T05:01:13.942609+0000","last_became_active":"2024-09-17T05:01:08.883484+0000","last_became_peered":"2024-09-17T05:01:08.883484+0000","last_unstale":"2024-09-17T05:01:13.942609+0000","last_undegraded":"2024-09-17T05:01:13.942609+0000","last_fullsized":"2024-09-17T05:01:13.942609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892259+0000","last_change":"2024-09-17T05:01:08.862441+0000","last_active":"2024-09-17T05:01:13.892259+0000","last_peered":"2024-09-17T05:01:13.892259+0000","last_clean":"2024-09-17T05:01:13.892259+0000","last_became_active":"2024-09-17T05:01:08.860705+0000","last_became_peered":"2024-09-17T05:01:08.860705+0000","last_unstale":"2024-09-17T05:01:13.892259+0000","last_undegraded":"2024-09-17T05:01:13.892259+0000","last_fullsized":"2024-09-17T05:01:13.892259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:47.926845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942230+0000","last_change":"2024-09-17T05:01:08.869890+0000","last_active":"2024-09-17T05:01:13.942230+0000","last_peered":"2024-09-17T05:01:13.942230+0000","last_clean":"2024-09-17T05:01:13.942230+0000","last_became_active":"2024-09-17T05:01:08.869367+0000","last_became_peered":"2024-09-17T05:01:08.869367+0000","last_unstale":"2024-09-17T05:01:13.942230+0000","last_undegraded":"2024-09-17T05:01:13.942230+0000","last_fullsized":"2024-09-17T05:01:13.942230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892235+0000","last_change":"2024-09-17T05:01:08.891532+0000","last_active":"2024-09-17T05:01:13.892235+0000","last_peered":"2024-09-17T05:01:13.892235+0000","last_clean":"2024-09-17T05:01:13.892235+0000","last_became_active":"2024-09-17T05:01:08.891207+0000","last_became_peered":"2024-09-17T05:01:08.891207+0000","last_unstale":"2024-09-17T05:01:13.892235+0000","last_undegraded":"2024-09-17T05:01:13.892235+0000","last_fullsized":"2024-09-17T05:01:13.892235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:45.826779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890942+0000","last_change":"2024-09-17T05:01:08.864037+0000","last_active":"2024-09-17T05:01:13.890942+0000","last_peered":"2024-09-17T05:01:13.890942+0000","last_clean":"2024-09-17T05:01:13.890942+0000","last_became_active":"2024-09-17T05:01:08.863691+0000","last_became_peered":"2024-09-17T05:01:08.863691+0000","last_unstale":"2024-09-17T05:01:13.890942+0000","last_undegraded":"2024-09-17T05:01:13.890942+0000","last_fullsized":"2024-09-17T05:01:13.890942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892250+0000","last_change":"2024-09-17T05:01:08.860804+0000","last_active":"2024-09-17T05:01:13.892250+0000","last_peered":"2024-09-17T05:01:13.892250+0000","last_clean":"2024-09-17T05:01:13.892250+0000","last_became_active":"2024-09-17T05:01:08.860278+0000","last_became_peered":"2024-09-17T05:01:08.860278+0000","last_unstale":"2024-09-17T05:01:13.892250+0000","last_undegraded":"2024-09-17T05:01:13.892250+0000","last_fullsized":"2024-09-17T05:01:13.892250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:41.450008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892724+0000","last_change":"2024-09-17T05:01:08.876606+0000","last_active":"2024-09-17T05:01:13.892724+0000","last_peered":"2024-09-17T05:01:13.892724+0000","last_clean":"2024-09-17T05:01:13.892724+0000","last_became_active":"2024-09-17T05:01:08.875868+0000","last_became_peered":"2024-09-17T05:01:08.875868+0000","last_unstale":"2024-09-17T05:01:13.892724+0000","last_undegraded":"2024-09-17T05:01:13.892724+0000","last_fullsized":"2024-09-17T05:01:13.892724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:36.517067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893030+0000","last_change":"2024-09-17T05:01:08.882586+0000","last_active":"2024-09-17T05:01:13.893030+0000","last_peered":"2024-09-17T05:01:13.893030+0000","last_clean":"2024-09-17T05:01:13.893030+0000","last_became_active":"2024-09-17T05:01:08.882293+0000","last_became_peered":"2024-09-17T05:01:08.882293+0000","last_unstale":"2024-09-17T05:01:13.893030+0000","last_undegraded":"2024-09-17T05:01:13.893030+0000","last_fullsized":"2024-09-17T05:01:13.893030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:55.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942578+0000","last_change":"2024-09-17T05:01:08.882456+0000","last_active":"2024-09-17T05:01:13.942578+0000","last_peered":"2024-09-17T05:01:13.942578+0000","last_clean":"2024-09-17T05:01:13.942578+0000","last_became_active":"2024-09-17T05:01:08.881935+0000","last_became_peered":"2024-09-17T05:01:08.881935+0000","last_unstale":"2024-09-17T05:01:13.942578+0000","last_undegraded":"2024-09-17T05:01:13.942578+0000","last_fullsized":"2024-09-17T05:01:13.942578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:01.625408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941814+0000","last_change":"2024-09-17T05:01:08.862457+0000","last_active":"2024-09-17T05:01:13.941814+0000","last_peered":"2024-09-17T05:01:13.941814+0000","last_clean":"2024-09-17T05:01:13.941814+0000","last_became_active":"2024-09-17T05:01:08.862099+0000","last_became_peered":"2024-09-17T05:01:08.862099+0000","last_unstale":"2024-09-17T05:01:13.941814+0000","last_undegraded":"2024-09-17T05:01:13.941814+0000","last_fullsized":"2024-09-17T05:01:13.941814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:47.337576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941761+0000","last_change":"2024-09-17T05:01:08.876965+0000","last_active":"2024-09-17T05:01:13.941761+0000","last_peered":"2024-09-17T05:01:13.941761+0000","last_clean":"2024-09-17T05:01:13.941761+0000","last_became_active":"2024-09-17T05:01:08.876594+0000","last_became_peered":"2024-09-17T05:01:08.876594+0000","last_unstale":"2024-09-17T05:01:13.941761+0000","last_undegraded":"2024-09-17T05:01:13.941761+0000","last_fullsized":"2024-09-17T05:01:13.941761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:07.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944208+0000","last_change":"2024-09-17T05:01:08.880532+0000","last_active":"2024-09-17T05:01:13.944208+0000","last_peered":"2024-09-17T05:01:13.944208+0000","last_clean":"2024-09-17T05:01:13.944208+0000","last_became_active":"2024-09-17T05:01:08.880215+0000","last_became_peered":"2024-09-17T05:01:08.880215+0000","last_unstale":"2024-09-17T05:01:13.944208+0000","last_undegraded":"2024-09-17T05:01:13.944208+0000","last_fullsized":"2024-09-17T05:01:13.944208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891730+0000","last_change":"2024-09-17T05:01:08.857708+0000","last_active":"2024-09-17T05:01:13.891730+0000","last_peered":"2024-09-17T05:01:13.891730+0000","last_clean":"2024-09-17T05:01:13.891730+0000","last_became_active":"2024-09-17T05:01:08.856723+0000","last_became_peered":"2024-09-17T05:01:08.856723+0000","last_unstale":"2024-09-17T05:01:13.891730+0000","last_undegraded":"2024-09-17T05:01:13.891730+0000","last_fullsized":"2024-09-17T05:01:13.891730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:19.705413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942123+0000","last_change":"2024-09-17T05:01:08.876806+0000","last_active":"2024-09-17T05:01:13.942123+0000","last_peered":"2024-09-17T05:01:13.942123+0000","last_clean":"2024-09-17T05:01:13.942123+0000","last_became_active":"2024-09-17T05:01:08.876453+0000","last_became_peered":"2024-09-17T05:01:08.876453+0000","last_unstale":"2024-09-17T05:01:13.942123+0000","last_undegraded":"2024-09-17T05:01:13.942123+0000","last_fullsized":"2024-09-17T05:01:13.942123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:02.271107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942651+0000","last_change":"2024-09-17T05:01:08.863011+0000","last_active":"2024-09-17T05:01:13.942651+0000","last_peered":"2024-09-17T05:01:13.942651+0000","last_clean":"2024-09-17T05:01:13.942651+0000","last_became_active":"2024-09-17T05:01:08.862561+0000","last_became_peered":"2024-09-17T05:01:08.862561+0000","last_unstale":"2024-09-17T05:01:13.942651+0000","last_undegraded":"2024-09-17T05:01:13.942651+0000","last_fullsized":"2024-09-17T05:01:13.942651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:40.251476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941484+0000","last_change":"2024-09-17T05:01:08.877286+0000","last_active":"2024-09-17T05:01:13.941484+0000","last_peered":"2024-09-17T05:01:13.941484+0000","last_clean":"2024-09-17T05:01:13.941484+0000","last_became_active":"2024-09-17T05:01:08.876336+0000","last_became_peered":"2024-09-17T05:01:08.876336+0000","last_unstale":"2024-09-17T05:01:13.941484+0000","last_undegraded":"2024-09-17T05:01:13.941484+0000","last_fullsized":"2024-09-17T05:01:13.941484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:48.065065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943117+0000","last_change":"2024-09-17T05:01:08.863862+0000","last_active":"2024-09-17T05:01:13.943117+0000","last_peered":"2024-09-17T05:01:13.943117+0000","last_clean":"2024-09-17T05:01:13.943117+0000","last_became_active":"2024-09-17T05:01:08.863514+0000","last_became_peered":"2024-09-17T05:01:08.863514+0000","last_unstale":"2024-09-17T05:01:13.943117+0000","last_undegraded":"2024-09-17T05:01:13.943117+0000","last_fullsized":"2024-09-17T05:01:13.943117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:54.036443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893288+0000","last_change":"2024-09-17T05:01:08.860436+0000","last_active":"2024-09-17T05:01:13.893288+0000","last_peered":"2024-09-17T05:01:13.893288+0000","last_clean":"2024-09-17T05:01:13.893288+0000","last_became_active":"2024-09-17T05:01:08.860010+0000","last_became_peered":"2024-09-17T05:01:08.860010+0000","last_unstale":"2024-09-17T05:01:13.893288+0000","last_undegraded":"2024-09-17T05:01:13.893288+0000","last_fullsized":"2024-09-17T05:01:13.893288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:00.968662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893613+0000","last_change":"2024-09-17T05:01:08.878717+0000","last_active":"2024-09-17T05:01:13.893613+0000","last_peered":"2024-09-17T05:01:13.893613+0000","last_clean":"2024-09-17T05:01:13.893613+0000","last_became_active":"2024-09-17T05:01:08.878392+0000","last_became_peered":"2024-09-17T05:01:08.878392+0000","last_unstale":"2024-09-17T05:01:13.893613+0000","last_undegraded":"2024-09-17T05:01:13.893613+0000","last_fullsized":"2024-09-17T05:01:13.893613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:50.747553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942379+0000","last_change":"2024-09-17T05:01:08.889143+0000","last_active":"2024-09-17T05:01:13.942379+0000","last_peered":"2024-09-17T05:01:13.942379+0000","last_clean":"2024-09-17T05:01:13.942379+0000","last_became_active":"2024-09-17T05:01:08.888428+0000","last_became_peered":"2024-09-17T05:01:08.888428+0000","last_unstale":"2024-09-17T05:01:13.942379+0000","last_undegraded":"2024-09-17T05:01:13.942379+0000","last_fullsized":"2024-09-17T05:01:13.942379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:01:25.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:01:25.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fc000640 1 -- 172.21.15.16:0/4083590285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 msgr2=0x7f85dc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fc000640 1 --2- 172.21.15.16:0/4083590285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85dc005160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009eb0 tx=0x7f85ec01d590 comp rx=0 tx=0).stop 2024-09-17T05:01:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fa000640 1 -- 172.21.15.16:0/4083590285 reap_dead start 2024-09-17T05:01:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fc000640 1 -- 172.21.15.16:0/4083590285 shutdown_connections 2024-09-17T05:01:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fc000640 1 -- 172.21.15.16:0/4083590285 >> 172.21.15.16:0/4083590285 conn(0x7f85f406f900 msgr2=0x7f85f40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fc000640 1 -- 172.21.15.16:0/4083590285 shutdown_connections 2024-09-17T05:01:25.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.315+0000 7f85fc000640 1 -- 172.21.15.16:0/4083590285 wait complete. 2024-09-17T05:01:25.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 Processor -- start 2024-09-17T05:01:25.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 -- start start 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85f4145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f85f4146620 0x7f85f4159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f41f5650 0x7f85f41f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4073c50 con 0x7f85dc004d50 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f85f4073ad0 con 0x7f85f41f5650 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f85f4073dd0 con 0x7f85f4146620 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85f4145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85f4145f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45062/0 (socket says 172.21.15.16:45062) 2024-09-17T05:01:25.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 -- 172.21.15.16:0/2835452997 learned_addr learned my addr 172.21.15.16:0/2835452997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f8c00640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f85f4146620 0x7f85f4159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fa000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f41f5650 0x7f85f41f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 -- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f85f4146620 msgr2=0x7f85f4159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f85f4146620 0x7f85f4159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 -- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f41f5650 msgr2=0x7f85f41f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f41f5650 0x7f85f41f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 -- 172.21.15.16:0/2835452997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f4146a30 con 0x7f85dc004d50 2024-09-17T05:01:25.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f9600640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85f4145f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f85e8000fc0 tx=0x7f85e800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8015070 con 0x7f85dc004d50 2024-09-17T05:01:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85e8010980 con 0x7f85dc004d50 2024-09-17T05:01:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8010d90 con 0x7f85dc004d50 2024-09-17T05:01:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f41f8c10 con 0x7f85dc004d50 2024-09-17T05:01:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.319+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f41f91d0 con 0x7f85dc004d50 2024-09-17T05:01:25.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.323+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f41f9410 con 0x7f85dc004d50 2024-09-17T05:01:25.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.323+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f85e8018070 con 0x7f85dc004d50 2024-09-17T05:01:25.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.323+0000 7f85f1600640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85d4073230 0x7f85d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.323+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f85e8094a20 con 0x7f85dc004d50 2024-09-17T05:01:25.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.323+0000 7f85f8c00640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85d4073230 0x7f85d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.327+0000 7f85f8c00640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85d4073230 0x7f85d4075740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85f414c120 tx=0x7f85e00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:25.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.327+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f85f41f9410 con 0x7f85dc004d50 2024-09-17T05:01:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.499+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85f40b3690 con 0x7f85d4073230 2024-09-17T05:01:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.507+0000 7f85f1600640 1 -- 172.21.15.16:0/2835452997 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332543 (secure 0 0 0) 0x7f85f40b3690 con 0x7f85d4073230 2024-09-17T05:01:25.511 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:01:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85d4073230 msgr2=0x7f85d4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85d4073230 0x7f85d4075740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f85f414c120 tx=0x7f85e00091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 msgr2=0x7f85f4145f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85f4145f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f85e8000fc0 tx=0x7f85e800ea90 comp rx=0 tx=0).stop 2024-09-17T05:01:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 shutdown_connections 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85d4073230 0x7f85d4075740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f85f41f5650 0x7f85f41f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f85f4146620 0x7f85f4159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 --2- 172.21.15.16:0/2835452997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85dc004d50 0x7f85f4145f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 >> 172.21.15.16:0/2835452997 conn(0x7f85f406f900 msgr2=0x7f85f415c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 shutdown_connections 2024-09-17T05:01:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.511+0000 7f85fc000640 1 -- 172.21.15.16:0/2835452997 wait complete. 2024-09-17T05:01:25.532 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-17T05:01:25.464815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236200,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738520,"statfs":{"total":773094113280,"available":772852244480,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.576204"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942466+0000","last_change":"2024-09-17T05:01:07.870252+0000","last_active":"2024-09-17T05:01:13.942466+0000","last_peered":"2024-09-17T05:01:13.942466+0000","last_clean":"2024-09-17T05:01:13.942466+0000","last_became_active":"2024-09-17T05:01:07.869832+0000","last_became_peered":"2024-09-17T05:01:07.869832+0000","last_unstale":"2024-09-17T05:01:13.942466+0000","last_undegraded":"2024-09-17T05:01:13.942466+0000","last_fullsized":"2024-09-17T05:01:13.942466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893796+0000","last_change":"2024-09-17T05:01:08.860472+0000","last_active":"2024-09-17T05:01:13.893796+0000","last_peered":"2024-09-17T05:01:13.893796+0000","last_clean":"2024-09-17T05:01:13.893796+0000","last_became_active":"2024-09-17T05:01:08.860121+0000","last_became_peered":"2024-09-17T05:01:08.860121+0000","last_unstale":"2024-09-17T05:01:13.893796+0000","last_undegraded":"2024-09-17T05:01:13.893796+0000","last_fullsized":"2024-09-17T05:01:13.893796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:02.947441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941059+0000","last_change":"2024-09-17T05:01:12.917747+0000","last_active":"2024-09-17T05:01:13.941059+0000","last_peered":"2024-09-17T05:01:13.941059+0000","last_clean":"2024-09-17T05:01:13.941059+0000","last_became_active":"2024-09-17T05:01:12.916952+0000","last_became_peered":"2024-09-17T05:01:12.916952+0000","last_unstale":"2024-09-17T05:01:13.941059+0000","last_undegraded":"2024-09-17T05:01:13.941059+0000","last_fullsized":"2024-09-17T05:01:13.941059+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:00.182691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893999+0000","last_change":"2024-09-17T05:01:07.858483+0000","last_active":"2024-09-17T05:01:13.893999+0000","last_peered":"2024-09-17T05:01:13.893999+0000","last_clean":"2024-09-17T05:01:13.893999+0000","last_became_active":"2024-09-17T05:01:07.858094+0000","last_became_peered":"2024-09-17T05:01:07.858094+0000","last_unstale":"2024-09-17T05:01:13.893999+0000","last_undegraded":"2024-09-17T05:01:13.893999+0000","last_fullsized":"2024-09-17T05:01:13.893999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:09.683336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894263+0000","last_change":"2024-09-17T05:01:08.886624+0000","last_active":"2024-09-17T05:01:13.894263+0000","last_peered":"2024-09-17T05:01:13.894263+0000","last_clean":"2024-09-17T05:01:13.894263+0000","last_became_active":"2024-09-17T05:01:08.886308+0000","last_became_peered":"2024-09-17T05:01:08.886308+0000","last_unstale":"2024-09-17T05:01:13.894263+0000","last_undegraded":"2024-09-17T05:01:13.894263+0000","last_fullsized":"2024-09-17T05:01:13.894263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.002578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941422+0000","last_change":"2024-09-17T05:01:12.901716+0000","last_active":"2024-09-17T05:01:13.941422+0000","last_peered":"2024-09-17T05:01:13.941422+0000","last_clean":"2024-09-17T05:01:13.941422+0000","last_became_active":"2024-09-17T05:01:12.901225+0000","last_became_peered":"2024-09-17T05:01:12.901225+0000","last_unstale":"2024-09-17T05:01:13.941422+0000","last_undegraded":"2024-09-17T05:01:13.941422+0000","last_fullsized":"2024-09-17T05:01:13.941422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:19.972140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942144+0000","last_change":"2024-09-17T05:01:07.857199+0000","last_active":"2024-09-17T05:01:13.942144+0000","last_peered":"2024-09-17T05:01:13.942144+0000","last_clean":"2024-09-17T05:01:13.942144+0000","last_became_active":"2024-09-17T05:01:07.856808+0000","last_became_peered":"2024-09-17T05:01:07.856808+0000","last_unstale":"2024-09-17T05:01:13.942144+0000","last_undegraded":"2024-09-17T05:01:13.942144+0000","last_fullsized":"2024-09-17T05:01:13.942144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943634+0000","last_change":"2024-09-17T05:01:08.880952+0000","last_active":"2024-09-17T05:01:13.943634+0000","last_peered":"2024-09-17T05:01:13.943634+0000","last_clean":"2024-09-17T05:01:13.943634+0000","last_became_active":"2024-09-17T05:01:08.880627+0000","last_became_peered":"2024-09-17T05:01:08.880627+0000","last_unstale":"2024-09-17T05:01:13.943634+0000","last_undegraded":"2024-09-17T05:01:13.943634+0000","last_fullsized":"2024-09-17T05:01:13.943634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:03.625255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943104+0000","last_change":"2024-09-17T05:01:12.903919+0000","last_active":"2024-09-17T05:01:13.943104+0000","last_peered":"2024-09-17T05:01:13.943104+0000","last_clean":"2024-09-17T05:01:13.943104+0000","last_became_active":"2024-09-17T05:01:12.901881+0000","last_became_peered":"2024-09-17T05:01:12.901881+0000","last_unstale":"2024-09-17T05:01:13.943104+0000","last_undegraded":"2024-09-17T05:01:13.943104+0000","last_fullsized":"2024-09-17T05:01:13.943104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.883298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942724+0000","last_change":"2024-09-17T05:01:07.859450+0000","last_active":"2024-09-17T05:01:13.942724+0000","last_peered":"2024-09-17T05:01:13.942724+0000","last_clean":"2024-09-17T05:01:13.942724+0000","last_became_active":"2024-09-17T05:01:07.857315+0000","last_became_peered":"2024-09-17T05:01:07.857315+0000","last_unstale":"2024-09-17T05:01:13.942724+0000","last_undegraded":"2024-09-17T05:01:13.942724+0000","last_fullsized":"2024-09-17T05:01:13.942724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892054+0000","last_change":"2024-09-17T05:01:08.893401+0000","last_active":"2024-09-17T05:01:13.892054+0000","last_peered":"2024-09-17T05:01:13.892054+0000","last_clean":"2024-09-17T05:01:13.892054+0000","last_became_active":"2024-09-17T05:01:08.892406+0000","last_became_peered":"2024-09-17T05:01:08.892406+0000","last_unstale":"2024-09-17T05:01:13.892054+0000","last_undegraded":"2024-09-17T05:01:13.892054+0000","last_fullsized":"2024-09-17T05:01:13.892054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:18.887291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942085+0000","last_change":"2024-09-17T05:01:12.918843+0000","last_active":"2024-09-17T05:01:13.942085+0000","last_peered":"2024-09-17T05:01:13.942085+0000","last_clean":"2024-09-17T05:01:13.942085+0000","last_became_active":"2024-09-17T05:01:12.916474+0000","last_became_peered":"2024-09-17T05:01:12.916474+0000","last_unstale":"2024-09-17T05:01:13.942085+0000","last_undegraded":"2024-09-17T05:01:13.942085+0000","last_fullsized":"2024-09-17T05:01:13.942085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:05.661358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891193+0000","last_change":"2024-09-17T05:01:08.858811+0000","last_active":"2024-09-17T05:01:13.891193+0000","last_peered":"2024-09-17T05:01:13.891193+0000","last_clean":"2024-09-17T05:01:13.891193+0000","last_became_active":"2024-09-17T05:01:08.858013+0000","last_became_peered":"2024-09-17T05:01:08.858013+0000","last_unstale":"2024-09-17T05:01:13.891193+0000","last_undegraded":"2024-09-17T05:01:13.891193+0000","last_fullsized":"2024-09-17T05:01:13.891193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:06.714459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891507+0000","last_change":"2024-09-17T05:01:07.873880+0000","last_active":"2024-09-17T05:01:13.891507+0000","last_peered":"2024-09-17T05:01:13.891507+0000","last_clean":"2024-09-17T05:01:13.891507+0000","last_became_active":"2024-09-17T05:01:07.870873+0000","last_became_peered":"2024-09-17T05:01:07.870873+0000","last_unstale":"2024-09-17T05:01:13.891507+0000","last_undegraded":"2024-09-17T05:01:13.891507+0000","last_fullsized":"2024-09-17T05:01:13.891507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.618135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891844+0000","last_change":"2024-09-17T05:01:12.898484+0000","last_active":"2024-09-17T05:01:13.891844+0000","last_peered":"2024-09-17T05:01:13.891844+0000","last_clean":"2024-09-17T05:01:13.891844+0000","last_became_active":"2024-09-17T05:01:12.895423+0000","last_became_peered":"2024-09-17T05:01:12.895423+0000","last_unstale":"2024-09-17T05:01:13.891844+0000","last_undegraded":"2024-09-17T05:01:13.891844+0000","last_fullsized":"2024-09-17T05:01:13.891844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:54.524295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941368+0000","last_change":"2024-09-17T05:01:08.871889+0000","last_active":"2024-09-17T05:01:13.941368+0000","last_peered":"2024-09-17T05:01:13.941368+0000","last_clean":"2024-09-17T05:01:13.941368+0000","last_became_active":"2024-09-17T05:01:08.871512+0000","last_became_peered":"2024-09-17T05:01:08.871512+0000","last_unstale":"2024-09-17T05:01:13.941368+0000","last_undegraded":"2024-09-17T05:01:13.941368+0000","last_fullsized":"2024-09-17T05:01:13.941368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892848+0000","last_change":"2024-09-17T05:01:07.865154+0000","last_active":"2024-09-17T05:01:13.892848+0000","last_peered":"2024-09-17T05:01:13.892848+0000","last_clean":"2024-09-17T05:01:13.892848+0000","last_became_active":"2024-09-17T05:01:07.864075+0000","last_became_peered":"2024-09-17T05:01:07.864075+0000","last_unstale":"2024-09-17T05:01:13.892848+0000","last_undegraded":"2024-09-17T05:01:13.892848+0000","last_fullsized":"2024-09-17T05:01:13.892848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941790+0000","last_change":"2024-09-17T05:01:12.915985+0000","last_active":"2024-09-17T05:01:13.941790+0000","last_peered":"2024-09-17T05:01:13.941790+0000","last_clean":"2024-09-17T05:01:13.941790+0000","last_became_active":"2024-09-17T05:01:12.915569+0000","last_became_peered":"2024-09-17T05:01:12.915569+0000","last_unstale":"2024-09-17T05:01:13.941790+0000","last_undegraded":"2024-09-17T05:01:13.941790+0000","last_fullsized":"2024-09-17T05:01:13.941790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:15.801136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940431+0000","last_change":"2024-09-17T05:01:08.859016+0000","last_active":"2024-09-17T05:01:13.940431+0000","last_peered":"2024-09-17T05:01:13.940431+0000","last_clean":"2024-09-17T05:01:13.940431+0000","last_became_active":"2024-09-17T05:01:08.858669+0000","last_became_peered":"2024-09-17T05:01:08.858669+0000","last_unstale":"2024-09-17T05:01:13.940431+0000","last_undegraded":"2024-09-17T05:01:13.940431+0000","last_fullsized":"2024-09-17T05:01:13.940431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:48.908013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940792+0000","last_change":"2024-09-17T05:01:07.853871+0000","last_active":"2024-09-17T05:01:13.940792+0000","last_peered":"2024-09-17T05:01:13.940792+0000","last_clean":"2024-09-17T05:01:13.940792+0000","last_became_active":"2024-09-17T05:01:07.853309+0000","last_became_peered":"2024-09-17T05:01:07.853309+0000","last_unstale":"2024-09-17T05:01:13.940792+0000","last_undegraded":"2024-09-17T05:01:13.940792+0000","last_fullsized":"2024-09-17T05:01:13.940792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941719+0000","last_change":"2024-09-17T05:01:12.917589+0000","last_active":"2024-09-17T05:01:13.941719+0000","last_peered":"2024-09-17T05:01:13.941719+0000","last_clean":"2024-09-17T05:01:13.941719+0000","last_became_active":"2024-09-17T05:01:12.917064+0000","last_became_peered":"2024-09-17T05:01:12.917064+0000","last_unstale":"2024-09-17T05:01:13.941719+0000","last_undegraded":"2024-09-17T05:01:13.941719+0000","last_fullsized":"2024-09-17T05:01:13.941719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:47.894698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890960+0000","last_change":"2024-09-17T05:01:08.865800+0000","last_active":"2024-09-17T05:01:13.890960+0000","last_peered":"2024-09-17T05:01:13.890960+0000","last_clean":"2024-09-17T05:01:13.890960+0000","last_became_active":"2024-09-17T05:01:08.865418+0000","last_became_peered":"2024-09-17T05:01:08.865418+0000","last_unstale":"2024-09-17T05:01:13.890960+0000","last_undegraded":"2024-09-17T05:01:13.890960+0000","last_fullsized":"2024-09-17T05:01:13.890960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941039+0000","last_change":"2024-09-17T05:01:07.868079+0000","last_active":"2024-09-17T05:01:13.941039+0000","last_peered":"2024-09-17T05:01:13.941039+0000","last_clean":"2024-09-17T05:01:13.941039+0000","last_became_active":"2024-09-17T05:01:07.867648+0000","last_became_peered":"2024-09-17T05:01:07.867648+0000","last_unstale":"2024-09-17T05:01:13.941039+0000","last_undegraded":"2024-09-17T05:01:13.941039+0000","last_fullsized":"2024-09-17T05:01:13.941039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943609+0000","last_change":"2024-09-17T05:01:12.913059+0000","last_active":"2024-09-17T05:01:13.943609+0000","last_peered":"2024-09-17T05:01:13.943609+0000","last_clean":"2024-09-17T05:01:13.943609+0000","last_became_active":"2024-09-17T05:01:12.912569+0000","last_became_peered":"2024-09-17T05:01:12.912569+0000","last_unstale":"2024-09-17T05:01:13.943609+0000","last_undegraded":"2024-09-17T05:01:13.943609+0000","last_fullsized":"2024-09-17T05:01:13.943609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:23.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941727+0000","last_change":"2024-09-17T05:01:08.869046+0000","last_active":"2024-09-17T05:01:13.941727+0000","last_peered":"2024-09-17T05:01:13.941727+0000","last_clean":"2024-09-17T05:01:13.941727+0000","last_became_active":"2024-09-17T05:01:08.864116+0000","last_became_peered":"2024-09-17T05:01:08.864116+0000","last_unstale":"2024-09-17T05:01:13.941727+0000","last_undegraded":"2024-09-17T05:01:13.941727+0000","last_fullsized":"2024-09-17T05:01:13.941727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:51.571131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892323+0000","last_change":"2024-09-17T05:01:07.863381+0000","last_active":"2024-09-17T05:01:13.892323+0000","last_peered":"2024-09-17T05:01:13.892323+0000","last_clean":"2024-09-17T05:01:13.892323+0000","last_became_active":"2024-09-17T05:01:07.862731+0000","last_became_peered":"2024-09-17T05:01:07.862731+0000","last_unstale":"2024-09-17T05:01:13.892323+0000","last_undegraded":"2024-09-17T05:01:13.892323+0000","last_fullsized":"2024-09-17T05:01:13.892323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:39.024164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942455+0000","last_change":"2024-09-17T05:01:12.916294+0000","last_active":"2024-09-17T05:01:13.942455+0000","last_peered":"2024-09-17T05:01:13.942455+0000","last_clean":"2024-09-17T05:01:13.942455+0000","last_became_active":"2024-09-17T05:01:12.908035+0000","last_became_peered":"2024-09-17T05:01:12.908035+0000","last_unstale":"2024-09-17T05:01:13.942455+0000","last_undegraded":"2024-09-17T05:01:13.942455+0000","last_fullsized":"2024-09-17T05:01:13.942455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:52.101015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942370+0000","last_change":"2024-09-17T05:01:08.870629+0000","last_active":"2024-09-17T05:01:13.942370+0000","last_peered":"2024-09-17T05:01:13.942370+0000","last_clean":"2024-09-17T05:01:13.942370+0000","last_became_active":"2024-09-17T05:01:08.870198+0000","last_became_peered":"2024-09-17T05:01:08.870198+0000","last_unstale":"2024-09-17T05:01:13.942370+0000","last_undegraded":"2024-09-17T05:01:13.942370+0000","last_fullsized":"2024-09-17T05:01:13.942370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:37.142368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941967+0000","last_change":"2024-09-17T05:01:07.860691+0000","last_active":"2024-09-17T05:01:13.941967+0000","last_peered":"2024-09-17T05:01:13.941967+0000","last_clean":"2024-09-17T05:01:13.941967+0000","last_became_active":"2024-09-17T05:01:07.860360+0000","last_became_peered":"2024-09-17T05:01:07.860360+0000","last_unstale":"2024-09-17T05:01:13.941967+0000","last_undegraded":"2024-09-17T05:01:13.941967+0000","last_fullsized":"2024-09-17T05:01:13.941967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940513+0000","last_change":"2024-09-17T05:01:12.900669+0000","last_active":"2024-09-17T05:01:13.940513+0000","last_peered":"2024-09-17T05:01:13.940513+0000","last_clean":"2024-09-17T05:01:13.940513+0000","last_became_active":"2024-09-17T05:01:12.900234+0000","last_became_peered":"2024-09-17T05:01:12.900234+0000","last_unstale":"2024-09-17T05:01:13.940513+0000","last_undegraded":"2024-09-17T05:01:13.940513+0000","last_fullsized":"2024-09-17T05:01:13.940513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893905+0000","last_change":"2024-09-17T05:01:08.876794+0000","last_active":"2024-09-17T05:01:13.893905+0000","last_peered":"2024-09-17T05:01:13.893905+0000","last_clean":"2024-09-17T05:01:13.893905+0000","last_became_active":"2024-09-17T05:01:08.876418+0000","last_became_peered":"2024-09-17T05:01:08.876418+0000","last_unstale":"2024-09-17T05:01:13.893905+0000","last_undegraded":"2024-09-17T05:01:13.893905+0000","last_fullsized":"2024-09-17T05:01:13.893905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:47.662092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892328+0000","last_change":"2024-09-17T05:01:07.877675+0000","last_active":"2024-09-17T05:01:13.892328+0000","last_peered":"2024-09-17T05:01:13.892328+0000","last_clean":"2024-09-17T05:01:13.892328+0000","last_became_active":"2024-09-17T05:01:07.877320+0000","last_became_peered":"2024-09-17T05:01:07.877320+0000","last_unstale":"2024-09-17T05:01:13.892328+0000","last_undegraded":"2024-09-17T05:01:13.892328+0000","last_fullsized":"2024-09-17T05:01:13.892328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:35.509947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891833+0000","last_change":"2024-09-17T05:01:12.911570+0000","last_active":"2024-09-17T05:01:13.891833+0000","last_peered":"2024-09-17T05:01:13.891833+0000","last_clean":"2024-09-17T05:01:13.891833+0000","last_became_active":"2024-09-17T05:01:12.911110+0000","last_became_peered":"2024-09-17T05:01:12.911110+0000","last_unstale":"2024-09-17T05:01:13.891833+0000","last_undegraded":"2024-09-17T05:01:13.891833+0000","last_fullsized":"2024-09-17T05:01:13.891833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:38.085327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943250+0000","last_change":"2024-09-17T05:01:08.870024+0000","last_active":"2024-09-17T05:01:13.943250+0000","last_peered":"2024-09-17T05:01:13.943250+0000","last_clean":"2024-09-17T05:01:13.943250+0000","last_became_active":"2024-09-17T05:01:08.869679+0000","last_became_peered":"2024-09-17T05:01:08.869679+0000","last_unstale":"2024-09-17T05:01:13.943250+0000","last_undegraded":"2024-09-17T05:01:13.943250+0000","last_fullsized":"2024-09-17T05:01:13.943250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:45.964067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892819+0000","last_change":"2024-09-17T05:01:07.859295+0000","last_active":"2024-09-17T05:01:13.892819+0000","last_peered":"2024-09-17T05:01:13.892819+0000","last_clean":"2024-09-17T05:01:13.892819+0000","last_became_active":"2024-09-17T05:01:07.858949+0000","last_became_peered":"2024-09-17T05:01:07.858949+0000","last_unstale":"2024-09-17T05:01:13.892819+0000","last_undegraded":"2024-09-17T05:01:13.892819+0000","last_fullsized":"2024-09-17T05:01:13.892819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:29.635679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892487+0000","last_change":"2024-09-17T05:01:12.915413+0000","last_active":"2024-09-17T05:01:13.892487+0000","last_peered":"2024-09-17T05:01:13.892487+0000","last_clean":"2024-09-17T05:01:13.892487+0000","last_became_active":"2024-09-17T05:01:12.914942+0000","last_became_peered":"2024-09-17T05:01:12.914942+0000","last_unstale":"2024-09-17T05:01:13.892487+0000","last_undegraded":"2024-09-17T05:01:13.892487+0000","last_fullsized":"2024-09-17T05:01:13.892487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:57.733227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941319+0000","last_change":"2024-09-17T05:01:08.871457+0000","last_active":"2024-09-17T05:01:13.941319+0000","last_peered":"2024-09-17T05:01:13.941319+0000","last_clean":"2024-09-17T05:01:13.941319+0000","last_became_active":"2024-09-17T05:01:08.871135+0000","last_became_peered":"2024-09-17T05:01:08.871135+0000","last_unstale":"2024-09-17T05:01:13.941319+0000","last_undegraded":"2024-09-17T05:01:13.941319+0000","last_fullsized":"2024-09-17T05:01:13.941319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:15.936190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:07.873885+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:07.869214+0000","last_became_peered":"2024-09-17T05:01:07.869214+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:16.951737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893521+0000","last_change":"2024-09-17T05:01:12.908768+0000","last_active":"2024-09-17T05:01:13.893521+0000","last_peered":"2024-09-17T05:01:13.893521+0000","last_clean":"2024-09-17T05:01:13.893521+0000","last_became_active":"2024-09-17T05:01:12.908282+0000","last_became_peered":"2024-09-17T05:01:12.908282+0000","last_unstale":"2024-09-17T05:01:13.893521+0000","last_undegraded":"2024-09-17T05:01:13.893521+0000","last_fullsized":"2024-09-17T05:01:13.893521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891855+0000","last_change":"2024-09-17T05:01:08.858109+0000","last_active":"2024-09-17T05:01:13.891855+0000","last_peered":"2024-09-17T05:01:13.891855+0000","last_clean":"2024-09-17T05:01:13.891855+0000","last_became_active":"2024-09-17T05:01:08.857293+0000","last_became_peered":"2024-09-17T05:01:08.857293+0000","last_unstale":"2024-09-17T05:01:13.891855+0000","last_undegraded":"2024-09-17T05:01:13.891855+0000","last_fullsized":"2024-09-17T05:01:13.891855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:53.180346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941193+0000","last_change":"2024-09-17T05:01:07.864654+0000","last_active":"2024-09-17T05:01:13.941193+0000","last_peered":"2024-09-17T05:01:13.941193+0000","last_clean":"2024-09-17T05:01:13.941193+0000","last_became_active":"2024-09-17T05:01:07.864166+0000","last_became_peered":"2024-09-17T05:01:07.864166+0000","last_unstale":"2024-09-17T05:01:13.941193+0000","last_undegraded":"2024-09-17T05:01:13.941193+0000","last_fullsized":"2024-09-17T05:01:13.941193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894265+0000","last_change":"2024-09-17T05:01:12.914524+0000","last_active":"2024-09-17T05:01:13.894265+0000","last_peered":"2024-09-17T05:01:13.894265+0000","last_clean":"2024-09-17T05:01:13.894265+0000","last_became_active":"2024-09-17T05:01:12.912222+0000","last_became_peered":"2024-09-17T05:01:12.912222+0000","last_unstale":"2024-09-17T05:01:13.894265+0000","last_undegraded":"2024-09-17T05:01:13.894265+0000","last_fullsized":"2024-09-17T05:01:13.894265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892306+0000","last_change":"2024-09-17T05:01:08.875660+0000","last_active":"2024-09-17T05:01:13.892306+0000","last_peered":"2024-09-17T05:01:13.892306+0000","last_clean":"2024-09-17T05:01:13.892306+0000","last_became_active":"2024-09-17T05:01:08.875265+0000","last_became_peered":"2024-09-17T05:01:08.875265+0000","last_unstale":"2024-09-17T05:01:13.892306+0000","last_undegraded":"2024-09-17T05:01:13.892306+0000","last_fullsized":"2024-09-17T05:01:13.892306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891158+0000","last_change":"2024-09-17T05:01:07.869491+0000","last_active":"2024-09-17T05:01:13.891158+0000","last_peered":"2024-09-17T05:01:13.891158+0000","last_clean":"2024-09-17T05:01:13.891158+0000","last_became_active":"2024-09-17T05:01:07.869038+0000","last_became_peered":"2024-09-17T05:01:07.869038+0000","last_unstale":"2024-09-17T05:01:13.891158+0000","last_undegraded":"2024-09-17T05:01:13.891158+0000","last_fullsized":"2024-09-17T05:01:13.891158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:00.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893711+0000","last_change":"2024-09-17T05:01:12.910830+0000","last_active":"2024-09-17T05:01:13.893711+0000","last_peered":"2024-09-17T05:01:13.893711+0000","last_clean":"2024-09-17T05:01:13.893711+0000","last_became_active":"2024-09-17T05:01:12.908497+0000","last_became_peered":"2024-09-17T05:01:12.908497+0000","last_unstale":"2024-09-17T05:01:13.893711+0000","last_undegraded":"2024-09-17T05:01:13.893711+0000","last_fullsized":"2024-09-17T05:01:13.893711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:07.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941121+0000","last_change":"2024-09-17T05:01:08.868170+0000","last_active":"2024-09-17T05:01:13.941121+0000","last_peered":"2024-09-17T05:01:13.941121+0000","last_clean":"2024-09-17T05:01:13.941121+0000","last_became_active":"2024-09-17T05:01:08.867767+0000","last_became_peered":"2024-09-17T05:01:08.867767+0000","last_unstale":"2024-09-17T05:01:13.941121+0000","last_undegraded":"2024-09-17T05:01:13.941121+0000","last_fullsized":"2024-09-17T05:01:13.941121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891555+0000","last_change":"2024-09-17T05:01:07.873908+0000","last_active":"2024-09-17T05:01:13.891555+0000","last_peered":"2024-09-17T05:01:13.891555+0000","last_clean":"2024-09-17T05:01:13.891555+0000","last_became_active":"2024-09-17T05:01:07.871028+0000","last_became_peered":"2024-09-17T05:01:07.871028+0000","last_unstale":"2024-09-17T05:01:13.891555+0000","last_undegraded":"2024-09-17T05:01:13.891555+0000","last_fullsized":"2024-09-17T05:01:13.891555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941889+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:01:13.941889+0000","last_peered":"2024-09-17T05:01:13.941889+0000","last_clean":"2024-09-17T05:01:13.941889+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:01:13.941889+0000","last_undegraded":"2024-09-17T05:01:13.941889+0000","last_fullsized":"2024-09-17T05:01:13.941889+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892300+0000","last_change":"2024-09-17T05:01:12.904503+0000","last_active":"2024-09-17T05:01:13.892300+0000","last_peered":"2024-09-17T05:01:13.892300+0000","last_clean":"2024-09-17T05:01:13.892300+0000","last_became_active":"2024-09-17T05:01:12.903910+0000","last_became_peered":"2024-09-17T05:01:12.903910+0000","last_unstale":"2024-09-17T05:01:13.892300+0000","last_undegraded":"2024-09-17T05:01:13.892300+0000","last_fullsized":"2024-09-17T05:01:13.892300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:47.490639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892152+0000","last_change":"2024-09-17T05:01:08.872909+0000","last_active":"2024-09-17T05:01:13.892152+0000","last_peered":"2024-09-17T05:01:13.892152+0000","last_clean":"2024-09-17T05:01:13.892152+0000","last_became_active":"2024-09-17T05:01:08.861070+0000","last_became_peered":"2024-09-17T05:01:08.861070+0000","last_unstale":"2024-09-17T05:01:13.892152+0000","last_undegraded":"2024-09-17T05:01:13.892152+0000","last_fullsized":"2024-09-17T05:01:13.892152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:44.525838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894016+0000","last_change":"2024-09-17T05:01:07.866417+0000","last_active":"2024-09-17T05:01:13.894016+0000","last_peered":"2024-09-17T05:01:13.894016+0000","last_clean":"2024-09-17T05:01:13.894016+0000","last_became_active":"2024-09-17T05:01:07.866088+0000","last_became_peered":"2024-09-17T05:01:07.866088+0000","last_unstale":"2024-09-17T05:01:13.894016+0000","last_undegraded":"2024-09-17T05:01:13.894016+0000","last_fullsized":"2024-09-17T05:01:13.894016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942333+0000","last_change":"2024-09-17T05:01:12.919691+0000","last_active":"2024-09-17T05:01:13.942333+0000","last_peered":"2024-09-17T05:01:13.942333+0000","last_clean":"2024-09-17T05:01:13.942333+0000","last_became_active":"2024-09-17T05:01:12.917951+0000","last_became_peered":"2024-09-17T05:01:12.917951+0000","last_unstale":"2024-09-17T05:01:13.942333+0000","last_undegraded":"2024-09-17T05:01:13.942333+0000","last_fullsized":"2024-09-17T05:01:13.942333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:11.524293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942694+0000","last_change":"2024-09-17T05:01:08.864231+0000","last_active":"2024-09-17T05:01:13.942694+0000","last_peered":"2024-09-17T05:01:13.942694+0000","last_clean":"2024-09-17T05:01:13.942694+0000","last_became_active":"2024-09-17T05:01:08.863802+0000","last_became_peered":"2024-09-17T05:01:08.863802+0000","last_unstale":"2024-09-17T05:01:13.942694+0000","last_undegraded":"2024-09-17T05:01:13.942694+0000","last_fullsized":"2024-09-17T05:01:13.942694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893511+0000","last_change":"2024-09-17T05:01:07.857650+0000","last_active":"2024-09-17T05:01:13.893511+0000","last_peered":"2024-09-17T05:01:13.893511+0000","last_clean":"2024-09-17T05:01:13.893511+0000","last_became_active":"2024-09-17T05:01:07.857311+0000","last_became_peered":"2024-09-17T05:01:07.857311+0000","last_unstale":"2024-09-17T05:01:13.893511+0000","last_undegraded":"2024-09-17T05:01:13.893511+0000","last_fullsized":"2024-09-17T05:01:13.893511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:27.902116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893168+0000","last_change":"2024-09-17T05:01:12.888243+0000","last_active":"2024-09-17T05:01:13.893168+0000","last_peered":"2024-09-17T05:01:13.893168+0000","last_clean":"2024-09-17T05:01:13.893168+0000","last_became_active":"2024-09-17T05:01:12.887793+0000","last_became_peered":"2024-09-17T05:01:12.887793+0000","last_unstale":"2024-09-17T05:01:13.893168+0000","last_undegraded":"2024-09-17T05:01:13.893168+0000","last_fullsized":"2024-09-17T05:01:13.893168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:00.157222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942457+0000","last_change":"2024-09-17T05:01:08.889497+0000","last_active":"2024-09-17T05:01:13.942457+0000","last_peered":"2024-09-17T05:01:13.942457+0000","last_clean":"2024-09-17T05:01:13.942457+0000","last_became_active":"2024-09-17T05:01:08.887064+0000","last_became_peered":"2024-09-17T05:01:08.887064+0000","last_unstale":"2024-09-17T05:01:13.942457+0000","last_undegraded":"2024-09-17T05:01:13.942457+0000","last_fullsized":"2024-09-17T05:01:13.942457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941847+0000","last_change":"2024-09-17T05:01:07.851404+0000","last_active":"2024-09-17T05:01:13.941847+0000","last_peered":"2024-09-17T05:01:13.941847+0000","last_clean":"2024-09-17T05:01:13.941847+0000","last_became_active":"2024-09-17T05:01:07.851077+0000","last_became_peered":"2024-09-17T05:01:07.851077+0000","last_unstale":"2024-09-17T05:01:13.941847+0000","last_undegraded":"2024-09-17T05:01:13.941847+0000","last_fullsized":"2024-09-17T05:01:13.941847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890772+0000","last_change":"2024-09-17T05:01:12.905280+0000","last_active":"2024-09-17T05:01:13.890772+0000","last_peered":"2024-09-17T05:01:13.890772+0000","last_clean":"2024-09-17T05:01:13.890772+0000","last_became_active":"2024-09-17T05:01:12.904787+0000","last_became_peered":"2024-09-17T05:01:12.904787+0000","last_unstale":"2024-09-17T05:01:13.890772+0000","last_undegraded":"2024-09-17T05:01:13.890772+0000","last_fullsized":"2024-09-17T05:01:13.890772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:29.666258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.884835+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.884410+0000","last_became_peered":"2024-09-17T05:01:08.884410+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:35.960729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942469+0000","last_change":"2024-09-17T05:01:07.866602+0000","last_active":"2024-09-17T05:01:13.942469+0000","last_peered":"2024-09-17T05:01:13.942469+0000","last_clean":"2024-09-17T05:01:13.942469+0000","last_became_active":"2024-09-17T05:01:07.866061+0000","last_became_peered":"2024-09-17T05:01:07.866061+0000","last_unstale":"2024-09-17T05:01:13.942469+0000","last_undegraded":"2024-09-17T05:01:13.942469+0000","last_fullsized":"2024-09-17T05:01:13.942469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943528+0000","last_change":"2024-09-17T05:01:12.904969+0000","last_active":"2024-09-17T05:01:13.943528+0000","last_peered":"2024-09-17T05:01:13.943528+0000","last_clean":"2024-09-17T05:01:13.943528+0000","last_became_active":"2024-09-17T05:01:12.904665+0000","last_became_peered":"2024-09-17T05:01:12.904665+0000","last_unstale":"2024-09-17T05:01:13.943528+0000","last_undegraded":"2024-09-17T05:01:13.943528+0000","last_fullsized":"2024-09-17T05:01:13.943528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:52.943963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890912+0000","last_change":"2024-09-17T05:01:08.872790+0000","last_active":"2024-09-17T05:01:13.890912+0000","last_peered":"2024-09-17T05:01:13.890912+0000","last_clean":"2024-09-17T05:01:13.890912+0000","last_became_active":"2024-09-17T05:01:08.872387+0000","last_became_peered":"2024-09-17T05:01:08.872387+0000","last_unstale":"2024-09-17T05:01:13.890912+0000","last_undegraded":"2024-09-17T05:01:13.890912+0000","last_fullsized":"2024-09-17T05:01:13.890912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940575+0000","last_change":"2024-09-17T05:01:07.854980+0000","last_active":"2024-09-17T05:01:13.940575+0000","last_peered":"2024-09-17T05:01:13.940575+0000","last_clean":"2024-09-17T05:01:13.940575+0000","last_became_active":"2024-09-17T05:01:07.854665+0000","last_became_peered":"2024-09-17T05:01:07.854665+0000","last_unstale":"2024-09-17T05:01:13.940575+0000","last_undegraded":"2024-09-17T05:01:13.940575+0000","last_fullsized":"2024-09-17T05:01:13.940575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893909+0000","last_change":"2024-09-17T05:01:12.908923+0000","last_active":"2024-09-17T05:01:13.893909+0000","last_peered":"2024-09-17T05:01:13.893909+0000","last_clean":"2024-09-17T05:01:13.893909+0000","last_became_active":"2024-09-17T05:01:12.908398+0000","last_became_peered":"2024-09-17T05:01:12.908398+0000","last_unstale":"2024-09-17T05:01:13.893909+0000","last_undegraded":"2024-09-17T05:01:13.893909+0000","last_fullsized":"2024-09-17T05:01:13.893909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:45.118442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941757+0000","last_change":"2024-09-17T05:01:08.856311+0000","last_active":"2024-09-17T05:01:13.941757+0000","last_peered":"2024-09-17T05:01:13.941757+0000","last_clean":"2024-09-17T05:01:13.941757+0000","last_became_active":"2024-09-17T05:01:08.855334+0000","last_became_peered":"2024-09-17T05:01:08.855334+0000","last_unstale":"2024-09-17T05:01:13.941757+0000","last_undegraded":"2024-09-17T05:01:13.941757+0000","last_fullsized":"2024-09-17T05:01:13.941757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942247+0000","last_change":"2024-09-17T05:01:07.872183+0000","last_active":"2024-09-17T05:01:13.942247+0000","last_peered":"2024-09-17T05:01:13.942247+0000","last_clean":"2024-09-17T05:01:13.942247+0000","last_became_active":"2024-09-17T05:01:07.871540+0000","last_became_peered":"2024-09-17T05:01:07.871540+0000","last_unstale":"2024-09-17T05:01:13.942247+0000","last_undegraded":"2024-09-17T05:01:13.942247+0000","last_fullsized":"2024-09-17T05:01:13.942247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894214+0000","last_change":"2024-09-17T05:01:12.906286+0000","last_active":"2024-09-17T05:01:13.894214+0000","last_peered":"2024-09-17T05:01:13.894214+0000","last_clean":"2024-09-17T05:01:13.894214+0000","last_became_active":"2024-09-17T05:01:12.905810+0000","last_became_peered":"2024-09-17T05:01:12.905810+0000","last_unstale":"2024-09-17T05:01:13.894214+0000","last_undegraded":"2024-09-17T05:01:13.894214+0000","last_fullsized":"2024-09-17T05:01:13.894214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:33.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891317+0000","last_change":"2024-09-17T05:01:08.893475+0000","last_active":"2024-09-17T05:01:13.891317+0000","last_peered":"2024-09-17T05:01:13.891317+0000","last_clean":"2024-09-17T05:01:13.891317+0000","last_became_active":"2024-09-17T05:01:08.893130+0000","last_became_peered":"2024-09-17T05:01:08.893130+0000","last_unstale":"2024-09-17T05:01:13.891317+0000","last_undegraded":"2024-09-17T05:01:13.891317+0000","last_fullsized":"2024-09-17T05:01:13.891317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891492+0000","last_change":"2024-09-17T05:01:07.869315+0000","last_active":"2024-09-17T05:01:13.891492+0000","last_peered":"2024-09-17T05:01:13.891492+0000","last_clean":"2024-09-17T05:01:13.891492+0000","last_became_active":"2024-09-17T05:01:07.864783+0000","last_became_peered":"2024-09-17T05:01:07.864783+0000","last_unstale":"2024-09-17T05:01:13.891492+0000","last_undegraded":"2024-09-17T05:01:13.891492+0000","last_fullsized":"2024-09-17T05:01:13.891492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893698+0000","last_change":"2024-09-17T05:01:12.921137+0000","last_active":"2024-09-17T05:01:13.893698+0000","last_peered":"2024-09-17T05:01:13.893698+0000","last_clean":"2024-09-17T05:01:13.893698+0000","last_became_active":"2024-09-17T05:01:12.920809+0000","last_became_peered":"2024-09-17T05:01:12.920809+0000","last_unstale":"2024-09-17T05:01:13.893698+0000","last_undegraded":"2024-09-17T05:01:13.893698+0000","last_fullsized":"2024-09-17T05:01:13.893698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892745+0000","last_change":"2024-09-17T05:01:08.892262+0000","last_active":"2024-09-17T05:01:13.892745+0000","last_peered":"2024-09-17T05:01:13.892745+0000","last_clean":"2024-09-17T05:01:13.892745+0000","last_became_active":"2024-09-17T05:01:08.891946+0000","last_became_peered":"2024-09-17T05:01:08.891946+0000","last_unstale":"2024-09-17T05:01:13.892745+0000","last_undegraded":"2024-09-17T05:01:13.892745+0000","last_fullsized":"2024-09-17T05:01:13.892745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:40.031283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941492+0000","last_change":"2024-09-17T05:01:07.872646+0000","last_active":"2024-09-17T05:01:13.941492+0000","last_peered":"2024-09-17T05:01:13.941492+0000","last_clean":"2024-09-17T05:01:13.941492+0000","last_became_active":"2024-09-17T05:01:07.872287+0000","last_became_peered":"2024-09-17T05:01:07.872287+0000","last_unstale":"2024-09-17T05:01:13.941492+0000","last_undegraded":"2024-09-17T05:01:13.941492+0000","last_fullsized":"2024-09-17T05:01:13.941492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:56.785994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891950+0000","last_change":"2024-09-17T05:01:12.900658+0000","last_active":"2024-09-17T05:01:13.891950+0000","last_peered":"2024-09-17T05:01:13.891950+0000","last_clean":"2024-09-17T05:01:13.891950+0000","last_became_active":"2024-09-17T05:01:12.900131+0000","last_became_peered":"2024-09-17T05:01:12.900131+0000","last_unstale":"2024-09-17T05:01:13.891950+0000","last_undegraded":"2024-09-17T05:01:13.891950+0000","last_fullsized":"2024-09-17T05:01:13.891950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:54.427072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941925+0000","last_change":"2024-09-17T05:01:08.869878+0000","last_active":"2024-09-17T05:01:13.941925+0000","last_peered":"2024-09-17T05:01:13.941925+0000","last_clean":"2024-09-17T05:01:13.941925+0000","last_became_active":"2024-09-17T05:01:08.869571+0000","last_became_peered":"2024-09-17T05:01:08.869571+0000","last_unstale":"2024-09-17T05:01:13.941925+0000","last_undegraded":"2024-09-17T05:01:13.941925+0000","last_fullsized":"2024-09-17T05:01:13.941925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:29.847123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:19.977855+0000","last_change":"2024-09-17T05:01:07.862966+0000","last_active":"2024-09-17T05:01:19.977855+0000","last_peered":"2024-09-17T05:01:19.977855+0000","last_clean":"2024-09-17T05:01:19.977855+0000","last_became_active":"2024-09-17T05:01:07.862605+0000","last_became_peered":"2024-09-17T05:01:07.862605+0000","last_unstale":"2024-09-17T05:01:19.977855+0000","last_undegraded":"2024-09-17T05:01:19.977855+0000","last_fullsized":"2024-09-17T05:01:19.977855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:26.467925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941104+0000","last_change":"2024-09-17T05:01:12.897023+0000","last_active":"2024-09-17T05:01:13.941104+0000","last_peered":"2024-09-17T05:01:13.941104+0000","last_clean":"2024-09-17T05:01:13.941104+0000","last_became_active":"2024-09-17T05:01:12.896528+0000","last_became_peered":"2024-09-17T05:01:12.896528+0000","last_unstale":"2024-09-17T05:01:13.941104+0000","last_undegraded":"2024-09-17T05:01:13.941104+0000","last_fullsized":"2024-09-17T05:01:13.941104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:04.392152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:08.866251+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:08.864997+0000","last_became_peered":"2024-09-17T05:01:08.864997+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:33.305210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942113+0000","last_change":"2024-09-17T05:01:07.862662+0000","last_active":"2024-09-17T05:01:13.942113+0000","last_peered":"2024-09-17T05:01:13.942113+0000","last_clean":"2024-09-17T05:01:13.942113+0000","last_became_active":"2024-09-17T05:01:07.862252+0000","last_became_peered":"2024-09-17T05:01:07.862252+0000","last_unstale":"2024-09-17T05:01:13.942113+0000","last_undegraded":"2024-09-17T05:01:13.942113+0000","last_fullsized":"2024-09-17T05:01:13.942113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:13.958450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942881+0000","last_change":"2024-09-17T05:01:12.915486+0000","last_active":"2024-09-17T05:01:13.942881+0000","last_peered":"2024-09-17T05:01:13.942881+0000","last_clean":"2024-09-17T05:01:13.942881+0000","last_became_active":"2024-09-17T05:01:12.913794+0000","last_became_peered":"2024-09-17T05:01:12.913794+0000","last_unstale":"2024-09-17T05:01:13.942881+0000","last_undegraded":"2024-09-17T05:01:13.942881+0000","last_fullsized":"2024-09-17T05:01:13.942881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:33.883073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894291+0000","last_change":"2024-09-17T05:01:08.863593+0000","last_active":"2024-09-17T05:01:13.894291+0000","last_peered":"2024-09-17T05:01:13.894291+0000","last_clean":"2024-09-17T05:01:13.894291+0000","last_became_active":"2024-09-17T05:01:08.863264+0000","last_became_peered":"2024-09-17T05:01:08.863264+0000","last_unstale":"2024-09-17T05:01:13.894291+0000","last_undegraded":"2024-09-17T05:01:13.894291+0000","last_fullsized":"2024-09-17T05:01:13.894291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:00.186616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942093+0000","last_change":"2024-09-17T05:01:07.860203+0000","last_active":"2024-09-17T05:01:13.942093+0000","last_peered":"2024-09-17T05:01:13.942093+0000","last_clean":"2024-09-17T05:01:13.942093+0000","last_became_active":"2024-09-17T05:01:07.858682+0000","last_became_peered":"2024-09-17T05:01:07.858682+0000","last_unstale":"2024-09-17T05:01:13.942093+0000","last_undegraded":"2024-09-17T05:01:13.942093+0000","last_fullsized":"2024-09-17T05:01:13.942093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940512+0000","last_change":"2024-09-17T05:01:12.912101+0000","last_active":"2024-09-17T05:01:13.940512+0000","last_peered":"2024-09-17T05:01:13.940512+0000","last_clean":"2024-09-17T05:01:13.940512+0000","last_became_active":"2024-09-17T05:01:12.911449+0000","last_became_peered":"2024-09-17T05:01:12.911449+0000","last_unstale":"2024-09-17T05:01:13.940512+0000","last_undegraded":"2024-09-17T05:01:13.940512+0000","last_fullsized":"2024-09-17T05:01:13.940512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:51.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.874496+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.874209+0000","last_became_peered":"2024-09-17T05:01:08.874209+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:06.383708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944328+0000","last_change":"2024-09-17T05:01:07.850788+0000","last_active":"2024-09-17T05:01:13.944328+0000","last_peered":"2024-09-17T05:01:13.944328+0000","last_clean":"2024-09-17T05:01:13.944328+0000","last_became_active":"2024-09-17T05:01:07.850154+0000","last_became_peered":"2024-09-17T05:01:07.850154+0000","last_unstale":"2024-09-17T05:01:13.944328+0000","last_undegraded":"2024-09-17T05:01:13.944328+0000","last_fullsized":"2024-09-17T05:01:13.944328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940756+0000","last_change":"2024-09-17T05:01:12.908537+0000","last_active":"2024-09-17T05:01:13.940756+0000","last_peered":"2024-09-17T05:01:13.940756+0000","last_clean":"2024-09-17T05:01:13.940756+0000","last_became_active":"2024-09-17T05:01:12.908060+0000","last_became_peered":"2024-09-17T05:01:12.908060+0000","last_unstale":"2024-09-17T05:01:13.940756+0000","last_undegraded":"2024-09-17T05:01:13.940756+0000","last_fullsized":"2024-09-17T05:01:13.940756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941572+0000","last_change":"2024-09-17T05:01:12.895140+0000","last_active":"2024-09-17T05:01:13.941572+0000","last_peered":"2024-09-17T05:01:13.941572+0000","last_clean":"2024-09-17T05:01:13.941572+0000","last_became_active":"2024-09-17T05:01:12.894665+0000","last_became_peered":"2024-09-17T05:01:12.894665+0000","last_unstale":"2024-09-17T05:01:13.941572+0000","last_undegraded":"2024-09-17T05:01:13.941572+0000","last_fullsized":"2024-09-17T05:01:13.941572+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:22.431196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940721+0000","last_change":"2024-09-17T05:01:08.851860+0000","last_active":"2024-09-17T05:01:13.940721+0000","last_peered":"2024-09-17T05:01:13.940721+0000","last_clean":"2024-09-17T05:01:13.940721+0000","last_became_active":"2024-09-17T05:01:08.851546+0000","last_became_peered":"2024-09-17T05:01:08.851546+0000","last_unstale":"2024-09-17T05:01:13.940721+0000","last_undegraded":"2024-09-17T05:01:13.940721+0000","last_fullsized":"2024-09-17T05:01:13.940721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:25.615656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891094+0000","last_change":"2024-09-17T05:01:07.876275+0000","last_active":"2024-09-17T05:01:13.891094+0000","last_peered":"2024-09-17T05:01:13.891094+0000","last_clean":"2024-09-17T05:01:13.891094+0000","last_became_active":"2024-09-17T05:01:07.875993+0000","last_became_peered":"2024-09-17T05:01:07.875993+0000","last_unstale":"2024-09-17T05:01:13.891094+0000","last_undegraded":"2024-09-17T05:01:13.891094+0000","last_fullsized":"2024-09-17T05:01:13.891094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:43.841239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891826+0000","last_change":"2024-09-17T05:01:12.904195+0000","last_active":"2024-09-17T05:01:13.891826+0000","last_peered":"2024-09-17T05:01:13.891826+0000","last_clean":"2024-09-17T05:01:13.891826+0000","last_became_active":"2024-09-17T05:01:12.903808+0000","last_became_peered":"2024-09-17T05:01:12.903808+0000","last_unstale":"2024-09-17T05:01:13.891826+0000","last_undegraded":"2024-09-17T05:01:13.891826+0000","last_fullsized":"2024-09-17T05:01:13.891826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:11.189684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891943+0000","last_change":"2024-09-17T05:01:08.876556+0000","last_active":"2024-09-17T05:01:13.891943+0000","last_peered":"2024-09-17T05:01:13.891943+0000","last_clean":"2024-09-17T05:01:13.891943+0000","last_became_active":"2024-09-17T05:01:08.874581+0000","last_became_peered":"2024-09-17T05:01:08.874581+0000","last_unstale":"2024-09-17T05:01:13.891943+0000","last_undegraded":"2024-09-17T05:01:13.891943+0000","last_fullsized":"2024-09-17T05:01:13.891943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:59.333120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892433+0000","last_change":"2024-09-17T05:01:07.865323+0000","last_active":"2024-09-17T05:01:13.892433+0000","last_peered":"2024-09-17T05:01:13.892433+0000","last_clean":"2024-09-17T05:01:13.892433+0000","last_became_active":"2024-09-17T05:01:07.864832+0000","last_became_peered":"2024-09-17T05:01:07.864832+0000","last_unstale":"2024-09-17T05:01:13.892433+0000","last_undegraded":"2024-09-17T05:01:13.892433+0000","last_fullsized":"2024-09-17T05:01:13.892433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891348+0000","last_change":"2024-09-17T05:01:07.870957+0000","last_active":"2024-09-17T05:01:13.891348+0000","last_peered":"2024-09-17T05:01:13.891348+0000","last_clean":"2024-09-17T05:01:13.891348+0000","last_became_active":"2024-09-17T05:01:07.869025+0000","last_became_peered":"2024-09-17T05:01:07.869025+0000","last_unstale":"2024-09-17T05:01:13.891348+0000","last_undegraded":"2024-09-17T05:01:13.891348+0000","last_fullsized":"2024-09-17T05:01:13.891348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940880+0000","last_change":"2024-09-17T05:01:08.876853+0000","last_active":"2024-09-17T05:01:13.940880+0000","last_peered":"2024-09-17T05:01:13.940880+0000","last_clean":"2024-09-17T05:01:13.940880+0000","last_became_active":"2024-09-17T05:01:08.876525+0000","last_became_peered":"2024-09-17T05:01:08.876525+0000","last_unstale":"2024-09-17T05:01:13.940880+0000","last_undegraded":"2024-09-17T05:01:13.940880+0000","last_fullsized":"2024-09-17T05:01:13.940880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:50.685520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941188+0000","last_change":"2024-09-17T05:01:12.908492+0000","last_active":"2024-09-17T05:01:13.941188+0000","last_peered":"2024-09-17T05:01:13.941188+0000","last_clean":"2024-09-17T05:01:13.941188+0000","last_became_active":"2024-09-17T05:01:12.908032+0000","last_became_peered":"2024-09-17T05:01:12.908032+0000","last_unstale":"2024-09-17T05:01:13.941188+0000","last_undegraded":"2024-09-17T05:01:13.941188+0000","last_fullsized":"2024-09-17T05:01:13.941188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893141+0000","last_change":"2024-09-17T05:01:07.859924+0000","last_active":"2024-09-17T05:01:13.893141+0000","last_peered":"2024-09-17T05:01:13.893141+0000","last_clean":"2024-09-17T05:01:13.893141+0000","last_became_active":"2024-09-17T05:01:07.859692+0000","last_became_peered":"2024-09-17T05:01:07.859692+0000","last_unstale":"2024-09-17T05:01:13.893141+0000","last_undegraded":"2024-09-17T05:01:13.893141+0000","last_fullsized":"2024-09-17T05:01:13.893141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.935910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943063+0000","last_change":"2024-09-17T05:01:08.886919+0000","last_active":"2024-09-17T05:01:13.943063+0000","last_peered":"2024-09-17T05:01:13.943063+0000","last_clean":"2024-09-17T05:01:13.943063+0000","last_became_active":"2024-09-17T05:01:08.886565+0000","last_became_peered":"2024-09-17T05:01:08.886565+0000","last_unstale":"2024-09-17T05:01:13.943063+0000","last_undegraded":"2024-09-17T05:01:13.943063+0000","last_fullsized":"2024-09-17T05:01:13.943063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941737+0000","last_change":"2024-09-17T05:01:12.919497+0000","last_active":"2024-09-17T05:01:13.941737+0000","last_peered":"2024-09-17T05:01:13.941737+0000","last_clean":"2024-09-17T05:01:13.941737+0000","last_became_active":"2024-09-17T05:01:12.919097+0000","last_became_peered":"2024-09-17T05:01:12.919097+0000","last_unstale":"2024-09-17T05:01:13.941737+0000","last_undegraded":"2024-09-17T05:01:13.941737+0000","last_fullsized":"2024-09-17T05:01:13.941737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:33.403463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941489+0000","last_change":"2024-09-17T05:01:08.868818+0000","last_active":"2024-09-17T05:01:13.941489+0000","last_peered":"2024-09-17T05:01:13.941489+0000","last_clean":"2024-09-17T05:01:13.941489+0000","last_became_active":"2024-09-17T05:01:08.868496+0000","last_became_peered":"2024-09-17T05:01:08.868496+0000","last_unstale":"2024-09-17T05:01:13.941489+0000","last_undegraded":"2024-09-17T05:01:13.941489+0000","last_fullsized":"2024-09-17T05:01:13.941489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:17.846926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941062+0000","last_change":"2024-09-17T05:01:08.881690+0000","last_active":"2024-09-17T05:01:13.941062+0000","last_peered":"2024-09-17T05:01:13.941062+0000","last_clean":"2024-09-17T05:01:13.941062+0000","last_became_active":"2024-09-17T05:01:08.880670+0000","last_became_peered":"2024-09-17T05:01:08.880670+0000","last_unstale":"2024-09-17T05:01:13.941062+0000","last_undegraded":"2024-09-17T05:01:13.941062+0000","last_fullsized":"2024-09-17T05:01:13.941062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891163+0000","last_change":"2024-09-17T05:01:08.867225+0000","last_active":"2024-09-17T05:01:13.891163+0000","last_peered":"2024-09-17T05:01:13.891163+0000","last_clean":"2024-09-17T05:01:13.891163+0000","last_became_active":"2024-09-17T05:01:08.866861+0000","last_became_peered":"2024-09-17T05:01:08.866861+0000","last_unstale":"2024-09-17T05:01:13.891163+0000","last_undegraded":"2024-09-17T05:01:13.891163+0000","last_fullsized":"2024-09-17T05:01:13.891163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:40.190786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892992+0000","last_change":"2024-09-17T05:01:08.862318+0000","last_active":"2024-09-17T05:01:13.892992+0000","last_peered":"2024-09-17T05:01:13.892992+0000","last_clean":"2024-09-17T05:01:13.892992+0000","last_became_active":"2024-09-17T05:01:08.861957+0000","last_became_peered":"2024-09-17T05:01:08.861957+0000","last_unstale":"2024-09-17T05:01:13.892992+0000","last_undegraded":"2024-09-17T05:01:13.892992+0000","last_fullsized":"2024-09-17T05:01:13.892992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:26.208177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893817+0000","last_change":"2024-09-17T05:01:08.883574+0000","last_active":"2024-09-17T05:01:13.893817+0000","last_peered":"2024-09-17T05:01:13.893817+0000","last_clean":"2024-09-17T05:01:13.893817+0000","last_became_active":"2024-09-17T05:01:08.883327+0000","last_became_peered":"2024-09-17T05:01:08.883327+0000","last_unstale":"2024-09-17T05:01:13.893817+0000","last_undegraded":"2024-09-17T05:01:13.893817+0000","last_fullsized":"2024-09-17T05:01:13.893817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893848+0000","last_change":"2024-09-17T05:01:08.877996+0000","last_active":"2024-09-17T05:01:13.893848+0000","last_peered":"2024-09-17T05:01:13.893848+0000","last_clean":"2024-09-17T05:01:13.893848+0000","last_became_active":"2024-09-17T05:01:08.877545+0000","last_became_peered":"2024-09-17T05:01:08.877545+0000","last_unstale":"2024-09-17T05:01:13.893848+0000","last_undegraded":"2024-09-17T05:01:13.893848+0000","last_fullsized":"2024-09-17T05:01:13.893848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:43.858748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943694+0000","last_change":"2024-09-17T05:01:08.874485+0000","last_active":"2024-09-17T05:01:13.943694+0000","last_peered":"2024-09-17T05:01:13.943694+0000","last_clean":"2024-09-17T05:01:13.943694+0000","last_became_active":"2024-09-17T05:01:08.874134+0000","last_became_peered":"2024-09-17T05:01:08.874134+0000","last_unstale":"2024-09-17T05:01:13.943694+0000","last_undegraded":"2024-09-17T05:01:13.943694+0000","last_fullsized":"2024-09-17T05:01:13.943694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:10.348602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893707+0000","last_change":"2024-09-17T05:01:08.886688+0000","last_active":"2024-09-17T05:01:13.893707+0000","last_peered":"2024-09-17T05:01:13.893707+0000","last_clean":"2024-09-17T05:01:13.893707+0000","last_became_active":"2024-09-17T05:01:08.886385+0000","last_became_peered":"2024-09-17T05:01:08.886385+0000","last_unstale":"2024-09-17T05:01:13.893707+0000","last_undegraded":"2024-09-17T05:01:13.893707+0000","last_fullsized":"2024-09-17T05:01:13.893707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:29.234911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891025+0000","last_change":"2024-09-17T05:01:08.866999+0000","last_active":"2024-09-17T05:01:13.891025+0000","last_peered":"2024-09-17T05:01:13.891025+0000","last_clean":"2024-09-17T05:01:13.891025+0000","last_became_active":"2024-09-17T05:01:08.866661+0000","last_became_peered":"2024-09-17T05:01:08.866661+0000","last_unstale":"2024-09-17T05:01:13.891025+0000","last_undegraded":"2024-09-17T05:01:13.891025+0000","last_fullsized":"2024-09-17T05:01:13.891025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:40.685724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892397+0000","last_change":"2024-09-17T05:01:08.871375+0000","last_active":"2024-09-17T05:01:13.892397+0000","last_peered":"2024-09-17T05:01:13.892397+0000","last_clean":"2024-09-17T05:01:13.892397+0000","last_became_active":"2024-09-17T05:01:08.870439+0000","last_became_peered":"2024-09-17T05:01:08.870439+0000","last_unstale":"2024-09-17T05:01:13.892397+0000","last_undegraded":"2024-09-17T05:01:13.892397+0000","last_fullsized":"2024-09-17T05:01:13.892397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:49.417178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942955+0000","last_change":"2024-09-17T05:01:08.866055+0000","last_active":"2024-09-17T05:01:13.942955+0000","last_peered":"2024-09-17T05:01:13.942955+0000","last_clean":"2024-09-17T05:01:13.942955+0000","last_became_active":"2024-09-17T05:01:08.865155+0000","last_became_peered":"2024-09-17T05:01:08.865155+0000","last_unstale":"2024-09-17T05:01:13.942955+0000","last_undegraded":"2024-09-17T05:01:13.942955+0000","last_fullsized":"2024-09-17T05:01:13.942955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940728+0000","last_change":"2024-09-17T05:01:08.868372+0000","last_active":"2024-09-17T05:01:13.940728+0000","last_peered":"2024-09-17T05:01:13.940728+0000","last_clean":"2024-09-17T05:01:13.940728+0000","last_became_active":"2024-09-17T05:01:08.867742+0000","last_became_peered":"2024-09-17T05:01:08.867742+0000","last_unstale":"2024-09-17T05:01:13.940728+0000","last_undegraded":"2024-09-17T05:01:13.940728+0000","last_fullsized":"2024-09-17T05:01:13.940728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:39.401927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942609+0000","last_change":"2024-09-17T05:01:08.884233+0000","last_active":"2024-09-17T05:01:13.942609+0000","last_peered":"2024-09-17T05:01:13.942609+0000","last_clean":"2024-09-17T05:01:13.942609+0000","last_became_active":"2024-09-17T05:01:08.883484+0000","last_became_peered":"2024-09-17T05:01:08.883484+0000","last_unstale":"2024-09-17T05:01:13.942609+0000","last_undegraded":"2024-09-17T05:01:13.942609+0000","last_fullsized":"2024-09-17T05:01:13.942609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892259+0000","last_change":"2024-09-17T05:01:08.862441+0000","last_active":"2024-09-17T05:01:13.892259+0000","last_peered":"2024-09-17T05:01:13.892259+0000","last_clean":"2024-09-17T05:01:13.892259+0000","last_became_active":"2024-09-17T05:01:08.860705+0000","last_became_peered":"2024-09-17T05:01:08.860705+0000","last_unstale":"2024-09-17T05:01:13.892259+0000","last_undegraded":"2024-09-17T05:01:13.892259+0000","last_fullsized":"2024-09-17T05:01:13.892259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:47.926845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942230+0000","last_change":"2024-09-17T05:01:08.869890+0000","last_active":"2024-09-17T05:01:13.942230+0000","last_peered":"2024-09-17T05:01:13.942230+0000","last_clean":"2024-09-17T05:01:13.942230+0000","last_became_active":"2024-09-17T05:01:08.869367+0000","last_became_peered":"2024-09-17T05:01:08.869367+0000","last_unstale":"2024-09-17T05:01:13.942230+0000","last_undegraded":"2024-09-17T05:01:13.942230+0000","last_fullsized":"2024-09-17T05:01:13.942230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892235+0000","last_change":"2024-09-17T05:01:08.891532+0000","last_active":"2024-09-17T05:01:13.892235+0000","last_peered":"2024-09-17T05:01:13.892235+0000","last_clean":"2024-09-17T05:01:13.892235+0000","last_became_active":"2024-09-17T05:01:08.891207+0000","last_became_peered":"2024-09-17T05:01:08.891207+0000","last_unstale":"2024-09-17T05:01:13.892235+0000","last_undegraded":"2024-09-17T05:01:13.892235+0000","last_fullsized":"2024-09-17T05:01:13.892235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:45.826779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890942+0000","last_change":"2024-09-17T05:01:08.864037+0000","last_active":"2024-09-17T05:01:13.890942+0000","last_peered":"2024-09-17T05:01:13.890942+0000","last_clean":"2024-09-17T05:01:13.890942+0000","last_became_active":"2024-09-17T05:01:08.863691+0000","last_became_peered":"2024-09-17T05:01:08.863691+0000","last_unstale":"2024-09-17T05:01:13.890942+0000","last_undegraded":"2024-09-17T05:01:13.890942+0000","last_fullsized":"2024-09-17T05:01:13.890942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892250+0000","last_change":"2024-09-17T05:01:08.860804+0000","last_active":"2024-09-17T05:01:13.892250+0000","last_peered":"2024-09-17T05:01:13.892250+0000","last_clean":"2024-09-17T05:01:13.892250+0000","last_became_active":"2024-09-17T05:01:08.860278+0000","last_became_peered":"2024-09-17T05:01:08.860278+0000","last_unstale":"2024-09-17T05:01:13.892250+0000","last_undegraded":"2024-09-17T05:01:13.892250+0000","last_fullsized":"2024-09-17T05:01:13.892250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:41.450008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892724+0000","last_change":"2024-09-17T05:01:08.876606+0000","last_active":"2024-09-17T05:01:13.892724+0000","last_peered":"2024-09-17T05:01:13.892724+0000","last_clean":"2024-09-17T05:01:13.892724+0000","last_became_active":"2024-09-17T05:01:08.875868+0000","last_became_peered":"2024-09-17T05:01:08.875868+0000","last_unstale":"2024-09-17T05:01:13.892724+0000","last_undegraded":"2024-09-17T05:01:13.892724+0000","last_fullsized":"2024-09-17T05:01:13.892724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:36.517067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893030+0000","last_change":"2024-09-17T05:01:08.882586+0000","last_active":"2024-09-17T05:01:13.893030+0000","last_peered":"2024-09-17T05:01:13.893030+0000","last_clean":"2024-09-17T05:01:13.893030+0000","last_became_active":"2024-09-17T05:01:08.882293+0000","last_became_peered":"2024-09-17T05:01:08.882293+0000","last_unstale":"2024-09-17T05:01:13.893030+0000","last_undegraded":"2024-09-17T05:01:13.893030+0000","last_fullsized":"2024-09-17T05:01:13.893030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:55.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942578+0000","last_change":"2024-09-17T05:01:08.882456+0000","last_active":"2024-09-17T05:01:13.942578+0000","last_peered":"2024-09-17T05:01:13.942578+0000","last_clean":"2024-09-17T05:01:13.942578+0000","last_became_active":"2024-09-17T05:01:08.881935+0000","last_became_peered":"2024-09-17T05:01:08.881935+0000","last_unstale":"2024-09-17T05:01:13.942578+0000","last_undegraded":"2024-09-17T05:01:13.942578+0000","last_fullsized":"2024-09-17T05:01:13.942578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:01.625408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941814+0000","last_change":"2024-09-17T05:01:08.862457+0000","last_active":"2024-09-17T05:01:13.941814+0000","last_peered":"2024-09-17T05:01:13.941814+0000","last_clean":"2024-09-17T05:01:13.941814+0000","last_became_active":"2024-09-17T05:01:08.862099+0000","last_became_peered":"2024-09-17T05:01:08.862099+0000","last_unstale":"2024-09-17T05:01:13.941814+0000","last_undegraded":"2024-09-17T05:01:13.941814+0000","last_fullsized":"2024-09-17T05:01:13.941814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:47.337576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941761+0000","last_change":"2024-09-17T05:01:08.876965+0000","last_active":"2024-09-17T05:01:13.941761+0000","last_peered":"2024-09-17T05:01:13.941761+0000","last_clean":"2024-09-17T05:01:13.941761+0000","last_became_active":"2024-09-17T05:01:08.876594+0000","last_became_peered":"2024-09-17T05:01:08.876594+0000","last_unstale":"2024-09-17T05:01:13.941761+0000","last_undegraded":"2024-09-17T05:01:13.941761+0000","last_fullsized":"2024-09-17T05:01:13.941761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:07.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944208+0000","last_change":"2024-09-17T05:01:08.880532+0000","last_active":"2024-09-17T05:01:13.944208+0000","last_peered":"2024-09-17T05:01:13.944208+0000","last_clean":"2024-09-17T05:01:13.944208+0000","last_became_active":"2024-09-17T05:01:08.880215+0000","last_became_peered":"2024-09-17T05:01:08.880215+0000","last_unstale":"2024-09-17T05:01:13.944208+0000","last_undegraded":"2024-09-17T05:01:13.944208+0000","last_fullsized":"2024-09-17T05:01:13.944208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891730+0000","last_change":"2024-09-17T05:01:08.857708+0000","last_active":"2024-09-17T05:01:13.891730+0000","last_peered":"2024-09-17T05:01:13.891730+0000","last_clean":"2024-09-17T05:01:13.891730+0000","last_became_active":"2024-09-17T05:01:08.856723+0000","last_became_peered":"2024-09-17T05:01:08.856723+0000","last_unstale":"2024-09-17T05:01:13.891730+0000","last_undegraded":"2024-09-17T05:01:13.891730+0000","last_fullsized":"2024-09-17T05:01:13.891730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:19.705413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942123+0000","last_change":"2024-09-17T05:01:08.876806+0000","last_active":"2024-09-17T05:01:13.942123+0000","last_peered":"2024-09-17T05:01:13.942123+0000","last_clean":"2024-09-17T05:01:13.942123+0000","last_became_active":"2024-09-17T05:01:08.876453+0000","last_became_peered":"2024-09-17T05:01:08.876453+0000","last_unstale":"2024-09-17T05:01:13.942123+0000","last_undegraded":"2024-09-17T05:01:13.942123+0000","last_fullsized":"2024-09-17T05:01:13.942123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:02.271107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942651+0000","last_change":"2024-09-17T05:01:08.863011+0000","last_active":"2024-09-17T05:01:13.942651+0000","last_peered":"2024-09-17T05:01:13.942651+0000","last_clean":"2024-09-17T05:01:13.942651+0000","last_became_active":"2024-09-17T05:01:08.862561+0000","last_became_peered":"2024-09-17T05:01:08.862561+0000","last_unstale":"2024-09-17T05:01:13.942651+0000","last_undegraded":"2024-09-17T05:01:13.942651+0000","last_fullsized":"2024-09-17T05:01:13.942651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:40.251476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941484+0000","last_change":"2024-09-17T05:01:08.877286+0000","last_active":"2024-09-17T05:01:13.941484+0000","last_peered":"2024-09-17T05:01:13.941484+0000","last_clean":"2024-09-17T05:01:13.941484+0000","last_became_active":"2024-09-17T05:01:08.876336+0000","last_became_peered":"2024-09-17T05:01:08.876336+0000","last_unstale":"2024-09-17T05:01:13.941484+0000","last_undegraded":"2024-09-17T05:01:13.941484+0000","last_fullsized":"2024-09-17T05:01:13.941484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:48.065065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943117+0000","last_change":"2024-09-17T05:01:08.863862+0000","last_active":"2024-09-17T05:01:13.943117+0000","last_peered":"2024-09-17T05:01:13.943117+0000","last_clean":"2024-09-17T05:01:13.943117+0000","last_became_active":"2024-09-17T05:01:08.863514+0000","last_became_peered":"2024-09-17T05:01:08.863514+0000","last_unstale":"2024-09-17T05:01:13.943117+0000","last_undegraded":"2024-09-17T05:01:13.943117+0000","last_fullsized":"2024-09-17T05:01:13.943117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:54.036443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893288+0000","last_change":"2024-09-17T05:01:08.860436+0000","last_active":"2024-09-17T05:01:13.893288+0000","last_peered":"2024-09-17T05:01:13.893288+0000","last_clean":"2024-09-17T05:01:13.893288+0000","last_became_active":"2024-09-17T05:01:08.860010+0000","last_became_peered":"2024-09-17T05:01:08.860010+0000","last_unstale":"2024-09-17T05:01:13.893288+0000","last_undegraded":"2024-09-17T05:01:13.893288+0000","last_fullsized":"2024-09-17T05:01:13.893288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:00.968662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893613+0000","last_change":"2024-09-17T05:01:08.878717+0000","last_active":"2024-09-17T05:01:13.893613+0000","last_peered":"2024-09-17T05:01:13.893613+0000","last_clean":"2024-09-17T05:01:13.893613+0000","last_became_active":"2024-09-17T05:01:08.878392+0000","last_became_peered":"2024-09-17T05:01:08.878392+0000","last_unstale":"2024-09-17T05:01:13.893613+0000","last_undegraded":"2024-09-17T05:01:13.893613+0000","last_fullsized":"2024-09-17T05:01:13.893613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:50.747553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942379+0000","last_change":"2024-09-17T05:01:08.889143+0000","last_active":"2024-09-17T05:01:13.942379+0000","last_peered":"2024-09-17T05:01:13.942379+0000","last_clean":"2024-09-17T05:01:13.942379+0000","last_became_active":"2024-09-17T05:01:08.888428+0000","last_became_peered":"2024-09-17T05:01:08.888428+0000","last_unstale":"2024-09-17T05:01:13.942379+0000","last_undegraded":"2024-09-17T05:01:13.942379+0000","last_fullsized":"2024-09-17T05:01:13.942379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:01:25.537 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:01:25.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T05:01:25.538 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 -- 172.21.15.16:0/2378042927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 msgr2=0x7fad70004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 --2- 172.21.15.16:0/2378042927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad70004c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad740016f0 tx=0x7fad7401c750 comp rx=0 tx=0).stop 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 -- 172.21.15.16:0/2378042927 shutdown_connections 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 --2- 172.21.15.16:0/2378042927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad70004c20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 -- 172.21.15.16:0/2378042927 >> 172.21.15.16:0/2378042927 conn(0x7fad8c0b42d0 msgr2=0x7fad8c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 -- 172.21.15.16:0/2378042927 shutdown_connections 2024-09-17T05:01:25.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 -- 172.21.15.16:0/2378042927 wait complete. 2024-09-17T05:01:25.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 Processor -- start 2024-09-17T05:01:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.735+0000 7fad91400640 1 -- start start 2024-09-17T05:01:25.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad8c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad8c071a00 0x7fad8c1faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad8c157860 0x7fad8c1fb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0b25d0 con 0x7fad70004810 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad8c0b2450 con 0x7fad8c157860 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c0b2750 con 0x7fad8c071a00 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad8c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad8c0714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45076/0 (socket says 172.21.15.16:45076) 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 -- 172.21.15.16:0/1632554931 learned_addr learned my addr 172.21.15.16:0/1632554931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 -- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad8c071a00 msgr2=0x7fad8c1faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad81e00640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad8c071a00 0x7fad8c1faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8b200640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad8c157860 0x7fad8c1fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad8c071a00 0x7fad8c1faec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 -- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad8c157860 msgr2=0x7fad8c1fb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad8c157860 0x7fad8c1fb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 -- 172.21.15.16:0/1632554931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c1fca60 con 0x7fad70004810 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad81e00640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad8c071a00 0x7fad8c1faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8b200640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad8c157860 0x7fad8c1fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad8a800640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad8c0714c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006600 tx=0x7fad7c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c015070 con 0x7fad70004810 2024-09-17T05:01:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad7c011920 con 0x7fad70004810 2024-09-17T05:01:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8c1fc780 con 0x7fad70004810 2024-09-17T05:01:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011d70 con 0x7fad70004810 2024-09-17T05:01:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8c1fa7b0 con 0x7fad70004810 2024-09-17T05:01:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.739+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8c161c00 con 0x7fad70004810 2024-09-17T05:01:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.743+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad7c018070 con 0x7fad70004810 2024-09-17T05:01:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.743+0000 7fad83400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad68073230 0x7fad68075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.743+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fad7c095210 con 0x7fad70004810 2024-09-17T05:01:25.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.743+0000 7fad81e00640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad68073230 0x7fad68075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.743+0000 7fad81e00640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad68073230 0x7fad68075740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad740016c0 tx=0x7fad74019540 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:25.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.747+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad7c05fee0 con 0x7fad70004810 2024-09-17T05:01:25.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fad8c0ccb40 con 0x7fad70004810 2024-09-17T05:01:25.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad83400640 1 -- 172.21.15.16:0/1632554931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fad7c05f7e0 con 0x7fad70004810 2024-09-17T05:01:25.923 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:25.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad68073230 msgr2=0x7fad68075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad68073230 0x7fad68075740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad740016c0 tx=0x7fad74019540 comp rx=0 tx=0).stop 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 msgr2=0x7fad8c0714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad8c0714c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fad7c006600 tx=0x7fad7c0039d0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 shutdown_connections 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad68073230 0x7fad68075740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad8c157860 0x7fad8c1fb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad8c071a00 0x7fad8c1faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 --2- 172.21.15.16:0/1632554931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad70004810 0x7fad8c0714c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.923+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 >> 172.21.15.16:0/1632554931 conn(0x7fad8c0b42d0 msgr2=0x7fad8c0b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.927+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 shutdown_connections 2024-09-17T05:01:25.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:25.927+0000 7fad91400640 1 -- 172.21.15.16:0/1632554931 wait complete. 2024-09-17T05:01:25.946 INFO:teuthology.orchestra.run.smithi016.stdout:[{"version":31,"timestamp":"2024-09-17T05:00:56.478242+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"2933.842623"}]}] 2024-09-17T05:01:25.946 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T05:01:25.947 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T05:01:25.947 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T05:01:25.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T05:01:26.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 -- 172.21.15.16:0/3107784185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 msgr2=0x7fd8a8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 --2- 172.21.15.16:0/3107784185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8a8003ed0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40016c0 tx=0x7fd8b401c730 comp rx=0 tx=0).stop 2024-09-17T05:01:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 -- 172.21.15.16:0/3107784185 shutdown_connections 2024-09-17T05:01:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 --2- 172.21.15.16:0/3107784185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8a8003ed0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 -- 172.21.15.16:0/3107784185 >> 172.21.15.16:0/3107784185 conn(0x7fd8c00d2270 msgr2=0x7fd8c00d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 -- 172.21.15.16:0/3107784185 shutdown_connections 2024-09-17T05:01:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 -- 172.21.15.16:0/3107784185 wait complete. 2024-09-17T05:01:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 Processor -- start 2024-09-17T05:01:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.107+0000 7fd8c8400640 1 -- start start 2024-09-17T05:01:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8c015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8c015f780 0x7fd8c00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8c00b6650 0x7fd8c01fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00d06d0 con 0x7fd8a8003a60 2024-09-17T05:01:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00d0550 con 0x7fd8c00b6650 2024-09-17T05:01:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c00d0850 con 0x7fd8c015f780 2024-09-17T05:01:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8c015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8c015f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45086/0 (socket says 172.21.15.16:45086) 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 -- 172.21.15.16:0/1854353271 learned_addr learned my addr 172.21.15.16:0/1854353271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5000640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8c015f780 0x7fd8c00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c6400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8c00b6650 0x7fd8c01fbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 -- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8c015f780 msgr2=0x7fd8c00b6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8c015f780 0x7fd8c00b6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 -- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8c00b6650 msgr2=0x7fd8c01fbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8c00b6650 0x7fd8c01fbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 -- 172.21.15.16:0/1854353271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c015fb90 con 0x7fd8a8003a60 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c5a00640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8c015f240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8b401ae30 tx=0x7fd8b4003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40b9070 con 0x7fd8a8003a60 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b40ae040 con 0x7fd8a8003a60 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c015e630 con 0x7fd8a8003a60 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40ad070 con 0x7fd8a8003a60 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c015d870 con 0x7fd8a8003a60 2024-09-17T05:01:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.111+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c00d0db0 con 0x7fd8a8003a60 2024-09-17T05:01:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.115+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd8b4006350 con 0x7fd8a8003a60 2024-09-17T05:01:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.115+0000 7fd8bd600640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd8a0073200 0x7fd8a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.115+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd8b412d930 con 0x7fd8a8003a60 2024-09-17T05:01:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.115+0000 7fd8c5000640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd8a0073200 0x7fd8a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.115+0000 7fd8c5000640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd8a0073200 0x7fd8a0075710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac005ec0 tx=0x7fd8ac0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.119+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd8c015fb90 con 0x7fd8a8003a60 2024-09-17T05:01:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.339+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd8c00ca400 con 0x7fd8a8003a60 2024-09-17T05:01:26.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8bd600640 1 -- 172.21.15.16:0/1854353271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186463 (secure 0 0 0) 0x7fd8b4016490 con 0x7fd8a8003a60 2024-09-17T05:01:26.344 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd8a0073200 msgr2=0x7fd8a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd8a0073200 0x7fd8a0075710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac005ec0 tx=0x7fd8ac0091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 msgr2=0x7fd8c015f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8c015f240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8b401ae30 tx=0x7fd8b4003be0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 shutdown_connections 2024-09-17T05:01:26.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd8a0073200 0x7fd8a0075710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8c00b6650 0x7fd8c01fbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd8c015f780 0x7fd8c00b6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 --2- 172.21.15.16:0/1854353271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a8003a60 0x7fd8c015f240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 >> 172.21.15.16:0/1854353271 conn(0x7fd8c00d2270 msgr2=0x7fd8c00ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.343+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 shutdown_connections 2024-09-17T05:01:26.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.347+0000 7fd8c8400640 1 -- 172.21.15.16:0/1854353271 wait complete. 2024-09-17T05:01:26.365 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4132,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2105036570},{"type":"v1","addr":"172.21.15.104:6801","nonce":2105036570}]},"active_addr":"172.21.15.104:6801/2105036570","active_change":"2024-09-17T05:00:47.447685+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4130,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":1393301502}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":2938793450}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":140890318}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.104:0","nonce":3049755837}]}]} 2024-09-17T05:01:26.370 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T05:01:26.370 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T05:01:26.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 -- 172.21.15.16:0/707509952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899015f750 msgr2=0x7f899015fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 --2- 172.21.15.16:0/707509952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899015f750 0x7f899015fb60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8990072540 tx=0x7f898801cb80 comp rx=0 tx=0).stop 2024-09-17T05:01:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 -- 172.21.15.16:0/707509952 shutdown_connections 2024-09-17T05:01:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 --2- 172.21.15.16:0/707509952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899015f750 0x7f899015fb60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 --2- 172.21.15.16:0/707509952 >> v2:172.21.15.104:3300/0 conn(0x7f899006ce90 0x7f89900cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 -- 172.21.15.16:0/707509952 >> 172.21.15.16:0/707509952 conn(0x7f89900d0860 msgr2=0x7f89900d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 -- 172.21.15.16:0/707509952 shutdown_connections 2024-09-17T05:01:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.527+0000 7f8996800640 1 -- 172.21.15.16:0/707509952 wait complete. 2024-09-17T05:01:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 Processor -- start 2024-09-17T05:01:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 -- start start 2024-09-17T05:01:26.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 0x7f8990157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89900cf860 0x7f8990156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f899015f750 0x7f89901463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89901689e0 con 0x7f899006ce90 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89900b5a80 con 0x7f89900cf860 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8990168b60 con 0x7f899015f750 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 0x7f8990157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 0x7f8990157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45104/0 (socket says 172.21.15.16:45104) 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 -- 172.21.15.16:0/3181598325 learned_addr learned my addr 172.21.15.16:0/3181598325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8987e00640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89900cf860 0x7f8990156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 -- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f899015f750 msgr2=0x7f89901463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898fe00640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f899015f750 0x7f89901463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f899015f750 0x7f89901463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 -- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89900cf860 msgr2=0x7f8990156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89900cf860 0x7f8990156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 -- 172.21.15.16:0/3181598325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901f6740 con 0x7f899006ce90 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8987e00640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89900cf860 0x7f8990156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898fe00640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f899015f750 0x7f89901463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f898f400640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 0x7f8990157570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f898000e7c0 tx=0x7f898000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898000ccc0 con 0x7f899006ce90 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f898000ce80 con 0x7f899006ce90 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980005280 con 0x7f899006ce90 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901f6a20 con 0x7f899006ce90 2024-09-17T05:01:26.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.531+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901f6fe0 con 0x7f899006ce90 2024-09-17T05:01:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.535+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89900cf320 con 0x7f899006ce90 2024-09-17T05:01:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.535+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8980005420 con 0x7f899006ce90 2024-09-17T05:01:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.535+0000 7f8987400640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f896c073200 0x7f896c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.535+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8980094f50 con 0x7f899006ce90 2024-09-17T05:01:26.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.535+0000 7f8987e00640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f896c073200 0x7f896c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.535+0000 7f8987e00640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f896c073200 0x7f896c075710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8978000f70 tx=0x7f8978009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:26.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.539+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f898005fcb0 con 0x7f899006ce90 2024-09-17T05:01:26.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.715+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89900ccbb0 con 0x7f899006ce90 2024-09-17T05:01:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8987400640 1 -- 172.21.15.16:0/3181598325 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f898005f5b0 con 0x7f899006ce90 2024-09-17T05:01:26.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:26.720 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f896c073200 msgr2=0x7f896c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f896c073200 0x7f896c075710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8978000f70 tx=0x7f8978009040 comp rx=0 tx=0).stop 2024-09-17T05:01:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 msgr2=0x7f8990157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 0x7f8990157570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f898000e7c0 tx=0x7f898000ec80 comp rx=0 tx=0).stop 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 shutdown_connections 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f896c073200 0x7f896c075710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f899015f750 0x7f89901463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89900cf860 0x7f8990156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 --2- 172.21.15.16:0/3181598325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f899006ce90 0x7f8990157570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 >> 172.21.15.16:0/3181598325 conn(0x7f89900d0860 msgr2=0x7f8990158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 shutdown_connections 2024-09-17T05:01:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.719+0000 7f8996800640 1 -- 172.21.15.16:0/3181598325 wait complete. 2024-09-17T05:01:26.742 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T05:01:26.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 -- 172.21.15.16:0/1821526818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 msgr2=0x7f76e815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 --2- 172.21.15.16:0/1821526818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 0x7f76e815bb60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009870 tx=0x7f76d801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 -- 172.21.15.16:0/1821526818 shutdown_connections 2024-09-17T05:01:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 --2- 172.21.15.16:0/1821526818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 0x7f76e815bb60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 -- 172.21.15.16:0/1821526818 >> 172.21.15.16:0/1821526818 conn(0x7f76e806f900 msgr2=0x7f76e80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 -- 172.21.15.16:0/1821526818 shutdown_connections 2024-09-17T05:01:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.899+0000 7f76ede00640 1 -- 172.21.15.16:0/1821526818 wait complete. 2024-09-17T05:01:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 Processor -- start 2024-09-17T05:01:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 -- start start 2024-09-17T05:01:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e8073240 0x7f76e81fa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 0x7f76e814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 0x7f76e806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8169db0 con 0x7f76e815c0a0 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76e8169c30 con 0x7f76e8073240 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f76e8169f30 con 0x7f76e80b2f50 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 0x7f76e806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 0x7f76e806d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45116/0 (socket says 172.21.15.16:45116) 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 -- 172.21.15.16:0/1434844891 learned_addr learned my addr 172.21.15.16:0/1434844891 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e6000640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 0x7f76e814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e6a00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e8073240 0x7f76e81fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 -- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 msgr2=0x7f76e814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 0x7f76e814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 -- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e8073240 msgr2=0x7f76e81fa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e8073240 0x7f76e81fa230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 -- 172.21.15.16:0/1434844891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81f7360 con 0x7f76e815c0a0 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76e7400640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 0x7f76e806d210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c730 tx=0x7f76e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0003b80 con 0x7f76e815c0a0 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76e0003d20 con 0x7f76e815c0a0 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e81f6710 con 0x7f76e815c0a0 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e000aa50 con 0x7f76e815c0a0 2024-09-17T05:01:26.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.903+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e81f6cd0 con 0x7f76e815c0a0 2024-09-17T05:01:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.907+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e80b1010 con 0x7f76e815c0a0 2024-09-17T05:01:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.907+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76e000abf0 con 0x7f76e815c0a0 2024-09-17T05:01:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.907+0000 7f76df400640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76c8073200 0x7f76c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.907+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f76e0094390 con 0x7f76e815c0a0 2024-09-17T05:01:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.907+0000 7f76e6a00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76c8073200 0x7f76c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.911+0000 7f76e6a00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76c8073200 0x7f76c8075710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76d80af000 tx=0x7f76d801b010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:26.911+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76e005f060 con 0x7f76e815c0a0 2024-09-17T05:01:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.087+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76e81f7530 con 0x7f76e815c0a0 2024-09-17T05:01:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.087+0000 7f76df400640 1 -- 172.21.15.16:0/1434844891 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f76e0078e20 con 0x7f76e815c0a0 2024-09-17T05:01:27.089 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:27.090 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76c8073200 msgr2=0x7f76c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76c8073200 0x7f76c8075710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76d80af000 tx=0x7f76d801b010 comp rx=0 tx=0).stop 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 msgr2=0x7f76e806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 0x7f76e806d210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76e000c730 tx=0x7f76e000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 shutdown_connections 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76c8073200 0x7f76c8075710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76e815c0a0 0x7f76e806d210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76e80b2f50 0x7f76e814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 --2- 172.21.15.16:0/1434844891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76e8073240 0x7f76e81fa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 >> 172.21.15.16:0/1434844891 conn(0x7f76e806f900 msgr2=0x7f76e80b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 shutdown_connections 2024-09-17T05:01:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.091+0000 7f76ede00640 1 -- 172.21.15.16:0/1434844891 wait complete. 2024-09-17T05:01:27.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T05:01:27.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T05:01:27.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T05:01:27.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T05:01:27.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T05:01:27.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T05:01:27.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T05:01:27.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T05:01:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 -- 172.21.15.16:0/248623271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 msgr2=0x7f9dcc0741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 --2- 172.21.15.16:0/248623271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 0x7f9dcc0741d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009bc0 tx=0x7f9dc801c9a0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 -- 172.21.15.16:0/248623271 shutdown_connections 2024-09-17T05:01:27.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 --2- 172.21.15.16:0/248623271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 0x7f9dcc0741d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 -- 172.21.15.16:0/248623271 >> 172.21.15.16:0/248623271 conn(0x7f9dcc0d0370 msgr2=0x7f9dcc0d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 -- 172.21.15.16:0/248623271 shutdown_connections 2024-09-17T05:01:27.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7f9dd4800640 1 -- 172.21.15.16:0/248623271 wait complete. 2024-09-17T05:01:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 -- 172.21.15.16:0/1354887403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa4070810 msgr2=0x7fcfa41622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 --2- 172.21.15.16:0/1354887403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa4070810 0x7fcfa41622f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40cdbc0 tx=0x7fcf9401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 -- 172.21.15.16:0/1354887403 shutdown_connections 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 --2- 172.21.15.16:0/1354887403 >> v2:172.21.15.16:3300/0 conn(0x7fcfa40cc4e0 0x7fcfa4162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 --2- 172.21.15.16:0/1354887403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa4070810 0x7fcfa41622f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 --2- 172.21.15.16:0/1354887403 >> v2:172.21.15.104:3300/0 conn(0x7fcfa40cc0d0 0x7fcfa40702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 -- 172.21.15.16:0/1354887403 >> 172.21.15.16:0/1354887403 conn(0x7fcfa40cfed0 msgr2=0x7fcfa40d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 -- 172.21.15.16:0/1354887403 shutdown_connections 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.403+0000 7fcfa8c00640 1 -- 172.21.15.16:0/1354887403 wait complete. 2024-09-17T05:01:27.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 Processor -- start 2024-09-17T05:01:27.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 -- start start 2024-09-17T05:01:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 0x7f9dcc13f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc0b3610 0x7f9dcc0ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9dcc162300 0x7f9dcc2000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc16d000 con 0x7f9dcc0b3610 2024-09-17T05:01:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9dcc16ce80 con 0x7f9dcc06ff40 2024-09-17T05:01:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd4800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9dcc16d180 con 0x7f9dcc162300 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd2800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9dcc162300 0x7f9dcc2000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd2800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9dcc162300 0x7f9dcc2000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49414/0 (socket says 172.21.15.16:49414) 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.407+0000 7f9dd2800640 1 -- 172.21.15.16:0/500545894 learned_addr learned my addr 172.21.15.16:0/500545894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 0x7f9dcc13f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 -- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9dcc162300 msgr2=0x7f9dcc2000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9dcc162300 0x7f9dcc2000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 -- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc0b3610 msgr2=0x7f9dcc0ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dcc0b3610 0x7f9dcc0ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 -- 172.21.15.16:0/500545894 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dcc13f6c0 con 0x7f9dcc06ff40 2024-09-17T05:01:27.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 Processor -- start 2024-09-17T05:01:27.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd1e00640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 0x7f9dcc13f990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c3c0 tx=0x7f9dc000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 -- start start 2024-09-17T05:01:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 -- 172.21.15.16:0/257764884 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 msgr2=0x7fdc3c0d46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 --2- 172.21.15.16:0/257764884 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c0d46d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009840 tx=0x7fdc2c01c680 comp rx=0 tx=0).stop 2024-09-17T05:01:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 -- 172.21.15.16:0/257764884 shutdown_connections 2024-09-17T05:01:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 --2- 172.21.15.16:0/257764884 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c0d46d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 --2- 172.21.15.16:0/257764884 >> v2:172.21.15.16:3300/0 conn(0x7fdc3c0d32f0 0x7fdc3c0d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 -- 172.21.15.16:0/257764884 >> 172.21.15.16:0/257764884 conn(0x7fdc3c09d800 msgr2=0x7fdc3c0b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 -- 172.21.15.16:0/257764884 shutdown_connections 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fdc42000640 1 -- 172.21.15.16:0/257764884 wait complete. 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0017070 con 0x7f9dcc06ff40 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dc0003600 con 0x7f9dcc06ff40 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0003a90 con 0x7f9dcc06ff40 2024-09-17T05:01:27.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dcc0cd870 con 0x7f9dcc06ff40 2024-09-17T05:01:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dcc0cde30 con 0x7f9dcc06ff40 2024-09-17T05:01:27.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9dcc0741d0 con 0x7f9dcc06ff40 2024-09-17T05:01:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 -- 172.21.15.16:0/3737892361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c0ccb90 msgr2=0x7f600c141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 --2- 172.21.15.16:0/3737892361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c0ccb90 0x7f600c141ce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f600c161030 tx=0x7f5ff801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40cc0d0 0x7fcfa414a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa40cc4e0 0x7fcfa416cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa414fe00 0x7fcfa416d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa40ce160 con 0x7fcfa414fe00 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa40cdfe0 con 0x7fcfa40cc0d0 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.411+0000 7fcfa8c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa40ce2e0 con 0x7fcfa40cc4e0 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa414fe00 0x7fcfa416d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa414fe00 0x7fcfa416d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43306/0 (socket says 172.21.15.16:43306) 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 -- 172.21.15.16:0/891186100 learned_addr learned my addr 172.21.15.16:0/891186100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 -- 172.21.15.16:0/3737892361 shutdown_connections 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 --2- 172.21.15.16:0/3737892361 >> v2:172.21.15.104:3300/0 conn(0x7f600c142220 0x7f600c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 --2- 172.21.15.16:0/3737892361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c0ccb90 0x7f600c141ce0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f600c161030 tx=0x7f5ff801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 --2- 172.21.15.16:0/3737892361 >> v2:172.21.15.16:3300/0 conn(0x7f600c0701b0 0x7f600c0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 -- 172.21.15.16:0/3737892361 >> 172.21.15.16:0/3737892361 conn(0x7f600c0d2000 msgr2=0x7f600c0d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 -- 172.21.15.16:0/3553980326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 msgr2=0x7f99800e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 --2- 172.21.15.16:0/3553980326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 0x7f99800e3cd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f99740098f0 tx=0x7f997401c6f0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fdc42000640 1 Processor -- start 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 -- 172.21.15.16:0/3553980326 shutdown_connections 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 -- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa40cc4e0 msgr2=0x7fcfa416cf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcfa40cc4e0 0x7fcfa416cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 -- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40cc0d0 msgr2=0x7fcfa414a0d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfa40cc0d0 0x7fcfa414a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 -- 172.21.15.16:0/891186100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa40afc90 con 0x7fcfa414fe00 2024-09-17T05:01:27.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fdc42000640 1 -- start start 2024-09-17T05:01:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 --2- 172.21.15.16:0/3553980326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 0x7f99800e2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 --2- 172.21.15.16:0/3553980326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 0x7f99800e3cd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 -- 172.21.15.16:0/3553980326 >> 172.21.15.16:0/3553980326 conn(0x7f998001ead0 msgr2=0x7f998003a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 -- 172.21.15.16:0/3737892361 shutdown_connections 2024-09-17T05:01:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f6012e00640 1 -- 172.21.15.16:0/3737892361 wait complete. 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcfa3e00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa414fe00 0x7fcfa416d460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e940 tx=0x7fcf9c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9dc001a070 con 0x7f9dcc06ff40 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dc6000640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9db0073250 0x7f9db0075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9dc005a4f0 con 0x7f9dcc06ff40 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dd1400640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9db0073250 0x7f9db0075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dc6000640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7f9db007d730 0x7f9db007fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 --> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9db00802f0 con 0x7f9db007d730 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c004d30 con 0x7fcfa414fe00 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf9c00cb20 con 0x7fcfa414fe00 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c004d30 con 0x7fcfa414fe00 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fcfa8c00640 1 -- 172.21.15.16:0/891186100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa40aff70 con 0x7fcfa414fe00 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fcfa8c00640 1 -- 172.21.15.16:0/891186100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa40b0530 con 0x7fcfa414fe00 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc42000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c0d2b30 0x7fdc3c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc42000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc3c0d32f0 0x7fdc3c206890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc42000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 -- 172.21.15.16:0/3553980326 shutdown_connections 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.415+0000 7f998de00640 1 -- 172.21.15.16:0/3553980326 wait complete. 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 -- 172.21.15.16:0/1487042616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b48035440 msgr2=0x7f6b480e0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 --2- 172.21.15.16:0/1487042616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b48035440 0x7f6b480e0890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b4800ec40 tx=0x7f6b3c01c750 comp rx=0 tx=0).stop 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 -- 172.21.15.16:0/1487042616 shutdown_connections 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 --2- 172.21.15.16:0/1487042616 >> v2:172.21.15.16:3300/0 conn(0x7f6b480e1a80 0x7f6b480e0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 --2- 172.21.15.16:0/1487042616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b48035440 0x7f6b480e0890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 -- 172.21.15.16:0/1487042616 >> 172.21.15.16:0/1487042616 conn(0x7f6b4801eaf0 msgr2=0x7f6b4803a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 -- 172.21.15.16:0/1487042616 shutdown_connections 2024-09-17T05:01:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc42000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c16c0e0 con 0x7fdc3c0d2b30 2024-09-17T05:01:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc42000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c16bf60 con 0x7fdc3c0d32f0 2024-09-17T05:01:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc42000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3c16c260 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6012e00640 1 Processor -- start 2024-09-17T05:01:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6b4f200640 1 -- 172.21.15.16:0/1487042616 wait complete. 2024-09-17T05:01:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcfa40ce790 con 0x7fcfa414fe00 2024-09-17T05:01:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f998de00640 1 Processor -- start 2024-09-17T05:01:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f6012e00640 1 -- start start 2024-09-17T05:01:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49436/0 (socket says 172.21.15.16:49436) 2024-09-17T05:01:27.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f58ad000640 1 -- 172.21.15.16:0/2996863566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a8166710 msgr2=0x7f58a8166b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9dc005aa70 con 0x7f9dcc06ff40 2024-09-17T05:01:27.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f58ad000640 1 --2- 172.21.15.16:0/2996863566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a8166710 0x7f58a8166b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58a808d110 tx=0x7f58a001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 -- 172.21.15.16:0/960695623 learned_addr learned my addr 172.21.15.16:0/960695623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f998de00640 1 -- start start 2024-09-17T05:01:27.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7f9dd1400640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9db0073250 0x7f9db0075760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc001230 tx=0x7f9dbc001100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3b400640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c0d2b30 0x7fdc3c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3aa00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc3c0d32f0 0x7fdc3c206890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 -- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc3c0d32f0 msgr2=0x7fdc3c206890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc3c0d32f0 0x7fdc3c206890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 -- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c0d2b30 msgr2=0x7fdc3c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c0d2b30 0x7fdc3c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 -- 172.21.15.16:0/960695623 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3c206dd0 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.419+0000 7fdc3be00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c160330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009790 tx=0x7fdc2c005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 -- 172.21.15.16:0/2996863566 shutdown_connections 2024-09-17T05:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 --2- 172.21.15.16:0/2996863566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a8166710 0x7f58a8166b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 --2- 172.21.15.16:0/2996863566 >> v2:172.21.15.16:3300/0 conn(0x7f58a80cde00 0x7f58a80b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 --2- 172.21.15.16:0/2996863566 >> v2:172.21.15.104:3300/0 conn(0x7f58a8162cb0 0x7f58a80b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 -- 172.21.15.16:0/2996863566 >> 172.21.15.16:0/2996863566 conn(0x7f58a80b3c50 msgr2=0x7f58a80b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f9dd2800640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7f9db007d730 0x7f9db007fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0ac030 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc2c004a90 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c004ec0 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 -- 172.21.15.16:0/4179222301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fb80d2b30 msgr2=0x7f2fb815e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 --2- 172.21.15.16:0/4179222301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fb80d2b30 0x7f2fb815e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2fb80d1660 tx=0x7f2fa801c800 comp rx=0 tx=0).stop 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 -- 172.21.15.16:0/4179222301 shutdown_connections 2024-09-17T05:01:27.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 --2- 172.21.15.16:0/4179222301 >> v2:172.21.15.16:3300/0 conn(0x7f2fb80d3a00 0x7f2fb80d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 --2- 172.21.15.16:0/4179222301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fb80d2b30 0x7f2fb815e770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 -- 172.21.15.16:0/4179222301 >> 172.21.15.16:0/4179222301 conn(0x7f2fb809d800 msgr2=0x7f2fb80b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7fdc42000640 1 -- 172.21.15.16:0/960695623 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3c15fb70 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 -- 172.21.15.16:0/4179222301 shutdown_connections 2024-09-17T05:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f2fbf200640 1 -- 172.21.15.16:0/4179222301 wait complete. 2024-09-17T05:01:27.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7fdc42000640 1 -- 172.21.15.16:0/960695623 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3c206000 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6b4f200640 1 Processor -- start 2024-09-17T05:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 -- 172.21.15.16:0/2996863566 shutdown_connections 2024-09-17T05:01:27.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99800e16d0 0x7f99800d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c0701b0 0x7f600c1f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c142220 0x7f600c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c158130 0x7f600c161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0d1380 con 0x7f600c0701b0 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0d1200 con 0x7f600c142220 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f600c0d1500 con 0x7f600c158130 2024-09-17T05:01:27.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f9dd2800640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7f9db007d730 0x7f9db007fc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc10001040 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998de00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 0x7f99800d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f58ad000640 1 -- 172.21.15.16:0/2996863566 wait complete. 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 0x7f998017de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99800e1080 con 0x7f99800e16d0 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998de00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99800e0f00 con 0x7f99800e38c0 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998de00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f99800068b0 con 0x7f99800e21f0 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f9987e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 0x7f99800d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f9987e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 0x7f99800d91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49454/0 (socket says 172.21.15.16:49454) 2024-09-17T05:01:27.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f9987e00640 1 -- 172.21.15.16:0/552325602 learned_addr learned my addr 172.21.15.16:0/552325602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f998d400640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 0x7f998017de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcf9c024480 con 0x7fcfa414fe00 2024-09-17T05:01:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== osd.0 v2:172.21.15.16:6824/3154417598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9db00802f0 con 0x7f9db007d730 2024-09-17T05:01:27.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f2fbf200640 1 Processor -- start 2024-09-17T05:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f9987e00640 1 -- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 msgr2=0x7f998017de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f9987e00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99800e38c0 0x7f998017de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f9987e00640 1 -- 172.21.15.16:0/552325602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99800e16d0 msgr2=0x7f99800d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f9987e00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99800e16d0 0x7f99800d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f9987e00640 1 -- 172.21.15.16:0/552325602 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99800d9bb0 con 0x7f99800e21f0 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c142220 0x7f600c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c142220 0x7f600c0cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:43840/0 (socket says 172.21.15.16:43840) 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6012400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c158130 0x7f600c161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 -- 172.21.15.16:0/3595045040 learned_addr learned my addr 172.21.15.16:0/3595045040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 -- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c158130 msgr2=0x7f600c161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c158130 0x7f600c161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 -- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c0701b0 msgr2=0x7f600c1f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f600c0701b0 0x7f600c1f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.423+0000 7f6011000640 1 -- 172.21.15.16:0/3595045040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c0cc5f0 con 0x7f600c142220 2024-09-17T05:01:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6011000640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c142220 0x7f600c0cc0b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ff801cda0 tx=0x7f5ff8005a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80b2040 con 0x7f600c142220 2024-09-17T05:01:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ff8004040 con 0x7f600c142220 2024-09-17T05:01:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80043f0 con 0x7f600c142220 2024-09-17T05:01:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f2fbf200640 1 -- start start 2024-09-17T05:01:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c161040 con 0x7f600c142220 2024-09-17T05:01:27.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c1615c0 con 0x7f600c142220 2024-09-17T05:01:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc2c0af070 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f58ad000640 1 Processor -- start 2024-09-17T05:01:27.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f58ad000640 1 -- start start 2024-09-17T05:01:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6b4f200640 1 -- start start 2024-09-17T05:01:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9be00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf80073400 0x7fcf80075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcf9c095250 con 0x7fcfa414fe00 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fb80d2b30 0x7f2fb8161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fb815ecb0 0x7f2fb82009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb815ddb0 0x7f2fb815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb816c050 con 0x7f2fb815ddb0 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb816bed0 con 0x7f2fb815ecb0 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb816c1d0 con 0x7f2fb80d2b30 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fdc33400640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc1c073300 0x7fdc1c075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9b400640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf80073400 0x7fcf80075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9be00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7fcf8007cbc0 0x7fcf8007f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 --> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf9c01de50 con 0x7fcf8007cbc0 2024-09-17T05:01:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf9c095640 con 0x7fcfa414fe00 2024-09-17T05:01:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b480e1670 0x7f6b480caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b480e1a80 0x7f6b48051330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f9987e00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 0x7f99800d91d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f997c00b410 tx=0x7f997c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb815ddb0 0x7f2fb815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcfa3400640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7fcf8007cbc0 0x7fcf8007f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcfa3400640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7fcf8007cbc0 0x7fcf8007f050 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9b400640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf80073400 0x7fcf80075910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b890 tx=0x7fcf8c005c80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.427+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f600c1f9230 con 0x7f600c142220 2024-09-17T05:01:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ff80bd030 con 0x7f600c142220 2024-09-17T05:01:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c01d070 con 0x7f99800e21f0 2024-09-17T05:01:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f997c010ba0 con 0x7f99800e21f0 2024-09-17T05:01:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c018440 con 0x7f99800e21f0 2024-09-17T05:01:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b48051870 0x7f6b48051c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b480ec7c0 con 0x7f6b48051870 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6b480ec640 con 0x7f6b480e1670 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6b480ec940 con 0x7f6b480e1a80 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb815ddb0 0x7f2fb815d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43334/0 (socket says 172.21.15.16:43334) 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 -- 172.21.15.16:0/3263954758 learned_addr learned my addr 172.21.15.16:0/3263954758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 -- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fb80d2b30 msgr2=0x7f2fb8161e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2fb80d2b30 0x7f2fb8161e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 -- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fb815ecb0 msgr2=0x7f2fb82009a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fb7400640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fb815ecb0 0x7f2fb82009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2fb815ecb0 0x7f2fb82009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 -- 172.21.15.16:0/3263954758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb815daf0 con 0x7f2fb815ddb0 2024-09-17T05:01:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbd200640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb815ddb0 0x7f2fb815d5b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000ba10 tx=0x7f2fb000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0019070 con 0x7f2fb815ddb0 2024-09-17T05:01:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 -- 172.21.15.16:0/3263954758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb80a1d30 con 0x7f2fb815ddb0 2024-09-17T05:01:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fbf200640 1 -- 172.21.15.16:0/3263954758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb8161390 con 0x7f2fb815ddb0 2024-09-17T05:01:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== osd.1 v2:172.21.15.16:6809/40873108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf9c01de50 con 0x7fcf8007cbc0 2024-09-17T05:01:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6009600640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fec073200 0x7f5fec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b480e1670 0x7f6b480caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b480e1670 0x7f6b480caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:43872/0 (socket says 172.21.15.16:43872) 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4de00640 1 -- 172.21.15.16:0/964169978 learned_addr learned my addr 172.21.15.16:0/964169978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f6b4d400640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b480e1a80 0x7f6b48051330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58ad000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a80cde00 0x7f58a81fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58ad000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a8162cb0 0x7f58a816c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58ad000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a8166710 0x7f58a814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58ad000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58a80cf8a0 con 0x7f58a8166710 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58ad000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58a80cf720 con 0x7f58a8162cb0 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58ad000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f58a80cfa20 con 0x7f58a80cde00 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58a7400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a80cde00 0x7f58a81fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58a7400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a80cde00 0x7f58a81fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49466/0 (socket says 172.21.15.16:49466) 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f58a7400640 1 -- 172.21.15.16:0/3081051200 learned_addr learned my addr 172.21.15.16:0/3081051200 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f589ea00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a8162cb0 0x7f58a816c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fb000cc30 con 0x7f2fb815ddb0 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0023060 con 0x7f2fb815ddb0 2024-09-17T05:01:27.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f2fbf200640 1 -- 172.21.15.16:0/3263954758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f90001040 con 0x7f2fb815ddb0 2024-09-17T05:01:27.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdc2c12e4e0 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc33400640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fdc1c07cac0 0x7fdc1c07ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5ff812dfc0 con 0x7f600c142220 2024-09-17T05:01:27.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6011a00640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fec073200 0x7f5fec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99800de8f0 con 0x7f99800e21f0 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.431+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99800deeb0 con 0x7f99800e21f0 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99800d9e90 con 0x7f99800e21f0 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4e800640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b48051870 0x7f6b48051c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58a7400640 1 -- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a8162cb0 msgr2=0x7f58a816c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58a7400640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a8162cb0 0x7f58a816c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58a7400640 1 -- 172.21.15.16:0/3081051200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a8166710 msgr2=0x7f58a814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58a7400640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a8166710 0x7f58a814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58a7400640 1 -- 172.21.15.16:0/3081051200 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a8165740 con 0x7f58a80cde00 2024-09-17T05:01:27.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6011a00640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fec073200 0x7f5fec075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f600000b9a0 tx=0x7f6000005dc0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6009600640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7f5fec07c9c0 0x7f5fec07ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 --> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ff80a9ec0 con 0x7f5fec07c9c0 2024-09-17T05:01:27.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ff81305c0 con 0x7f600c142220 2024-09-17T05:01:27.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4de00640 1 -- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b480e1a80 msgr2=0x7f6b48051330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4de00640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b480e1a80 0x7f6b48051330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4de00640 1 -- 172.21.15.16:0/964169978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b48051870 msgr2=0x7f6b48051c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4de00640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b48051870 0x7f6b48051c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4de00640 1 -- 172.21.15.16:0/964169978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b48055dc0 con 0x7f6b480e1670 2024-09-17T05:01:27.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58a7400640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a80cde00 0x7f58a81fc8f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f589800ccb0 tx=0x7f589800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 --> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc1c07f620 con 0x7fdc1c07cac0 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc2c12e8d0 con 0x7fdc3c0d42c0 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc3aa00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fdc1c07cac0 0x7fdc1c07ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc3b400640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc1c073300 0x7fdc1c075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4de00640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b480e1670 0x7f6b480caa20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b440 tx=0x7f6b4000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40015020 con 0x7f6b480e1670 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b40011980 con 0x7f6b480e1670 2024-09-17T05:01:27.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40011dd0 con 0x7f6b480e1670 2024-09-17T05:01:27.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc3b400640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc1c073300 0x7fdc1c075810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc34003ae0 tx=0x7fdc34003eb0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7fdc3aa00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fdc1c07cac0 0x7fdc1c07ef50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b480560a0 con 0x7f6b480e1670 2024-09-17T05:01:27.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898004b30 con 0x7f58a80cde00 2024-09-17T05:01:27.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5898004dc0 con 0x7f58a80cde00 2024-09-17T05:01:27.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898005280 con 0x7f58a80cde00 2024-09-17T05:01:27.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b480de3c0 con 0x7f6b480e1670 2024-09-17T05:01:27.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b480e0890 con 0x7f6b480e1670 2024-09-17T05:01:27.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f997c020070 con 0x7f99800e21f0 2024-09-17T05:01:27.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a8165a20 con 0x7f58a80cde00 2024-09-17T05:01:27.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.435+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a814e250 con 0x7f58a80cde00 2024-09-17T05:01:27.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5880001040 con 0x7f58a80cde00 2024-09-17T05:01:27.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f6012400640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7f5fec07c9c0 0x7f5fec07ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== osd.2 v2:172.21.15.16:6808/3210692914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc1c07f620 con 0x7fdc1c07cac0 2024-09-17T05:01:27.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f9984c00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99680732d0 0x7f99680757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f997c0986f0 con 0x7f99800e21f0 2024-09-17T05:01:27.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f9984c00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f996807caa0 0x7f996807ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 --> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f996807f600 con 0x7f996807caa0 2024-09-17T05:01:27.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2fb001c070 con 0x7f2fb815ddb0 2024-09-17T05:01:27.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f6012400640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7f5fec07c9c0 0x7f5fec07ee50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f997c098ae0 con 0x7f99800e21f0 2024-09-17T05:01:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b40018020 con 0x7f6b480e1670 2024-09-17T05:01:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.439+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5898005420 con 0x7f58a80cde00 2024-09-17T05:01:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fafe00640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2f98073200 0x7f2f98075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2fb0094ca0 con 0x7f2fb815ddb0 2024-09-17T05:01:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fafe00640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f2f9807c9c0 0x7f2f9807ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 --> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fb0014e50 con 0x7f2f9807c9c0 2024-09-17T05:01:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2fb0095090 con 0x7f2fb815ddb0 2024-09-17T05:01:27.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== osd.3 v2:172.21.15.16:6800/3613434049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ff80a9ec0 con 0x7f5fec07c9c0 2024-09-17T05:01:27.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b46000640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b30073230 0x7f6b30075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6b40094d00 con 0x7f6b480e1670 2024-09-17T05:01:27.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fb7e00640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2f98073200 0x7f2f98075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b4d400640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b30073230 0x7f6b30075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f998ca00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99680732d0 0x7f99680757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f998d400640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f996807caa0 0x7f996807ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589fe00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073230 0x7f5884075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f998d400640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f996807caa0 0x7f996807ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fb7400640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f2f9807c9c0 0x7f2f9807ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b46000640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f6b3007c980 0x7f6b3007ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 --> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b40013ec0 con 0x7f6b3007c980 2024-09-17T05:01:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b40095190 con 0x7f6b480e1670 2024-09-17T05:01:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5898094aa0 con 0x7f58a80cde00 2024-09-17T05:01:27.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589ea00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073230 0x7f5884075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fb7e00640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2f98073200 0x7f2f98075710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80af000 tx=0x7f2fa801b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b4d400640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b30073230 0x7f6b30075740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0af000 tx=0x7f6b3c01af90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f6b4e800640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f6b3007c980 0x7f6b3007ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f998ca00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99680732d0 0x7f99680757e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99740ae000 tx=0x7f997400b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== osd.4 v2:172.21.15.104:6810/2719508844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f996807f600 con 0x7f996807caa0 2024-09-17T05:01:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589fe00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f588407c970 0x7f588407ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589ea00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073230 0x7f5884075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f58a80511b0 tx=0x7f5890006f90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 --> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5898004820 con 0x7f588407c970 2024-09-17T05:01:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58980992a0 con 0x7f58a80cde00 2024-09-17T05:01:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.443+0000 7f2fb7400640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f2f9807c9c0 0x7f2f9807ee50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.447+0000 7f58a7e00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f588407c970 0x7f588407ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.447+0000 7f6b4e800640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f6b3007c980 0x7f6b3007ee10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.447+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== osd.7 v2:172.21.15.104:6818/2403343602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fb0014e50 con 0x7f2f9807c9c0 2024-09-17T05:01:27.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.447+0000 7f58a7e00640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f588407c970 0x7f588407ee00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.447+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== osd.5 v2:172.21.15.104:6802/464547405 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b40013ec0 con 0x7f6b3007c980 2024-09-17T05:01:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.447+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== osd.6 v2:172.21.15.104:6826/3199393218 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5898004820 con 0x7f588407c970 2024-09-17T05:01:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.459+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 --> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9dcc0ae280 con 0x7f9db007d730 2024-09-17T05:01:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.459+0000 7f9dc6000640 1 -- 172.21.15.16:0/500545894 <== osd.0 v2:172.21.15.16:6824/3154417598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9dcc0ae280 con 0x7f9db007d730 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7f9db007d730 msgr2=0x7f9db007fc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7f9db007d730 0x7f9db007fc20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9db0073250 msgr2=0x7f9db0075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9db0073250 0x7f9db0075760 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc001230 tx=0x7f9dbc001100 comp rx=0 tx=0).stop 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 msgr2=0x7f9dcc13f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 --2- 172.21.15.16:0/500545894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dcc06ff40 0x7f9dcc13f990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c3c0 tx=0x7f9dc000c880 comp rx=0 tx=0).stop 2024-09-17T05:01:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd2800640 1 -- 172.21.15.16:0/500545894 reap_dead start 2024-09-17T05:01:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 shutdown_connections 2024-09-17T05:01:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.463+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 >> 172.21.15.16:0/500545894 conn(0x7f9dcc0d0370 msgr2=0x7f9dcc06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.467+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 shutdown_connections 2024-09-17T05:01:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.467+0000 7f9dd4800640 1 -- 172.21.15.16:0/500545894 wait complete. 2024-09-17T05:01:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.467+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 --> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcfa40ce790 con 0x7fcf8007cbc0 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.467+0000 7fcf9be00640 1 -- 172.21.15.16:0/891186100 <== osd.1 v2:172.21.15.16:6809/40873108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcfa40ce790 con 0x7fcf8007cbc0 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7fcf8007cbc0 msgr2=0x7fcf8007f050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7fcf8007cbc0 0x7fcf8007f050 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf80073400 msgr2=0x7fcf80075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf80073400 0x7fcf80075910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00b890 tx=0x7fcf8c005c80 comp rx=0 tx=0).stop 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa414fe00 msgr2=0x7fcfa416d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 --2- 172.21.15.16:0/891186100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfa414fe00 0x7fcfa416d460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e940 tx=0x7fcf9c00ee00 comp rx=0 tx=0).stop 2024-09-17T05:01:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 --> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc10002de0 con 0x7fdc1c07cac0 2024-09-17T05:01:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcfa3e00640 1 -- 172.21.15.16:0/891186100 reap_dead start 2024-09-17T05:01:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 shutdown_connections 2024-09-17T05:01:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 >> 172.21.15.16:0/891186100 conn(0x7fcfa40cfed0 msgr2=0x7fcfa4161d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 --> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f600c0cafe0 con 0x7f5fec07c9c0 2024-09-17T05:01:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 shutdown_connections 2024-09-17T05:01:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7fcf98c00640 1 -- 172.21.15.16:0/891186100 wait complete. 2024-09-17T05:01:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6009600640 1 -- 172.21.15.16:0/3595045040 <== osd.3 v2:172.21.15.16:6800/3613434049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f600c0cafe0 con 0x7f5fec07c9c0 2024-09-17T05:01:27.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 --> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99800e3d70 con 0x7f996807caa0 2024-09-17T05:01:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7f5fec07c9c0 msgr2=0x7f5fec07ee50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7f5fec07c9c0 0x7f5fec07ee50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fec073200 msgr2=0x7f5fec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fec073200 0x7f5fec075710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f600000b9a0 tx=0x7f6000005dc0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c142220 msgr2=0x7f600c0cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012e00640 1 --2- 172.21.15.16:0/3595045040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c142220 0x7f600c0cc0b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ff801cda0 tx=0x7f5ff8005a30 comp rx=0 tx=0).stop 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f6012400640 1 -- 172.21.15.16:0/3595045040 reap_dead start 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.471+0000 7f9984c00640 1 -- 172.21.15.16:0/552325602 <== osd.4 v2:172.21.15.104:6810/2719508844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f99800e3d70 con 0x7f996807caa0 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f996807caa0 msgr2=0x7f996807ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f996807caa0 0x7f996807ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99680732d0 msgr2=0x7f99680757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99680732d0 0x7f99680757e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99740ae000 tx=0x7f997400b040 comp rx=0 tx=0).stop 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 msgr2=0x7f99800d91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 --2- 172.21.15.16:0/552325602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99800e21f0 0x7f99800d91d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f997c00b410 tx=0x7f997c00b8d0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998d400640 1 -- 172.21.15.16:0/552325602 reap_dead start 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc33400640 1 -- 172.21.15.16:0/960695623 <== osd.2 v2:172.21.15.16:6808/3210692914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdc10002de0 con 0x7fdc1c07cac0 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 shutdown_connections 2024-09-17T05:01:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 >> 172.21.15.16:0/552325602 conn(0x7f998001ead0 msgr2=0x7f998000c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 shutdown_connections 2024-09-17T05:01:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f998de00640 1 -- 172.21.15.16:0/552325602 wait complete. 2024-09-17T05:01:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fbf200640 1 -- 172.21.15.16:0/3263954758 --> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f90002de0 con 0x7f2f9807c9c0 2024-09-17T05:01:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 --> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b48056270 con 0x7f6b3007c980 2024-09-17T05:01:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 shutdown_connections 2024-09-17T05:01:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 >> 172.21.15.16:0/3595045040 conn(0x7f600c0d2000 msgr2=0x7f600c0b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 shutdown_connections 2024-09-17T05:01:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6012e00640 1 -- 172.21.15.16:0/3595045040 wait complete. 2024-09-17T05:01:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b46000640 1 -- 172.21.15.16:0/964169978 <== osd.5 v2:172.21.15.104:6802/464547405 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6b48056270 con 0x7f6b3007c980 2024-09-17T05:01:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fafe00640 1 -- 172.21.15.16:0/3263954758 <== osd.7 v2:172.21.15.104:6818/2403343602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f90002de0 con 0x7f2f9807c9c0 2024-09-17T05:01:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fdc1c07cac0 msgr2=0x7fdc1c07ef50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc30c00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fdc1c07cac0 0x7fdc1c07ef50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f6b3007c980 msgr2=0x7f6b3007ee10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f6b3007c980 0x7f6b3007ee10 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b30073230 msgr2=0x7f6b30075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b30073230 0x7f6b30075740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0af000 tx=0x7f6b3c01af90 comp rx=0 tx=0).stop 2024-09-17T05:01:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b480e1670 msgr2=0x7f6b480caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4f200640 1 --2- 172.21.15.16:0/964169978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b480e1670 0x7f6b480caa20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000b440 tx=0x7f6b4000b900 comp rx=0 tx=0).stop 2024-09-17T05:01:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f2f9807c9c0 msgr2=0x7f2f9807ee50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fad600640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f2f9807c9c0 0x7f2f9807ee50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc1c073300 msgr2=0x7fdc1c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc30c00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc1c073300 0x7fdc1c075810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc34003ae0 tx=0x7fdc34003eb0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 msgr2=0x7fdc3c160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc30c00640 1 --2- 172.21.15.16:0/960695623 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc3c0d42c0 0x7fdc3c160330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009790 tx=0x7fdc2c005730 comp rx=0 tx=0).stop 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7fdc3be00640 1 -- 172.21.15.16:0/960695623 reap_dead start 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f6b4e800640 1 -- 172.21.15.16:0/964169978 reap_dead start 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2f98073200 msgr2=0x7f2f98075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fad600640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2f98073200 0x7f2f98075710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80af000 tx=0x7f2fa801b040 comp rx=0 tx=0).stop 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb815ddb0 msgr2=0x7f2fb815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fad600640 1 --2- 172.21.15.16:0/3263954758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fb815ddb0 0x7f2fb815d5b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000ba10 tx=0x7f2fb000bed0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.475+0000 7f2fbd200640 1 -- 172.21.15.16:0/3263954758 reap_dead start 2024-09-17T05:01:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 shutdown_connections 2024-09-17T05:01:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 >> 172.21.15.16:0/964169978 conn(0x7f6b4801eaf0 msgr2=0x7f6b480db780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 shutdown_connections 2024-09-17T05:01:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 >> 172.21.15.16:0/3263954758 conn(0x7f2fb809d800 msgr2=0x7f2fb80d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 shutdown_connections 2024-09-17T05:01:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 >> 172.21.15.16:0/960695623 conn(0x7fdc3c09d800 msgr2=0x7fdc3c06f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 shutdown_connections 2024-09-17T05:01:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f2fad600640 1 -- 172.21.15.16:0/3263954758 wait complete. 2024-09-17T05:01:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 shutdown_connections 2024-09-17T05:01:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f6b4f200640 1 -- 172.21.15.16:0/964169978 wait complete. 2024-09-17T05:01:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 --> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5880002de0 con 0x7f588407c970 2024-09-17T05:01:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 shutdown_connections 2024-09-17T05:01:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.479+0000 7f589fe00640 1 -- 172.21.15.16:0/3081051200 <== osd.6 v2:172.21.15.104:6826/3199393218 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5880002de0 con 0x7f588407c970 2024-09-17T05:01:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7fdc30c00640 1 -- 172.21.15.16:0/960695623 wait complete. 2024-09-17T05:01:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f588407c970 msgr2=0x7f588407ee00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f588407c970 0x7f588407ee00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073230 msgr2=0x7f5884075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073230 0x7f5884075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f58a80511b0 tx=0x7f5890006f90 comp rx=0 tx=0).stop 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a80cde00 msgr2=0x7f58a81fc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 --2- 172.21.15.16:0/3081051200 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58a80cde00 0x7f58a81fc8f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f589800ccb0 tx=0x7f589800a620 comp rx=0 tx=0).stop 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58a7e00640 1 -- 172.21.15.16:0/3081051200 reap_dead start 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 shutdown_connections 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 >> 172.21.15.16:0/3081051200 conn(0x7f58a80b3c50 msgr2=0x7f58a80b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 shutdown_connections 2024-09-17T05:01:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.483+0000 7f58ad000640 1 -- 172.21.15.16:0/3081051200 wait complete. 2024-09-17T05:01:27.509 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:27.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T05:01:27.511 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-17T05:01:27.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:01:27.515 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-17T05:01:27.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:01:27.525 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-17T05:01:27.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:01:27.528 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-17T05:01:27.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:01:27.530 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-17T05:01:27.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:01:27.531 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:27.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:01:27.534 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2024-09-17T05:01:27.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T05:01:27.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 -- 172.21.15.16:0/4102706314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060033bc0 msgr2=0x7fa0600511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 --2- 172.21.15.16:0/4102706314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060033bc0 0x7fa0600511b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098f0 tx=0x7fa05c01c720 comp rx=0 tx=0).stop 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 -- 172.21.15.16:0/4102706314 shutdown_connections 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 --2- 172.21.15.16:0/4102706314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 0x7fa0600edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 --2- 172.21.15.16:0/4102706314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa060033bc0 0x7fa0600511b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 -- 172.21.15.16:0/4102706314 >> 172.21.15.16:0/4102706314 conn(0x7fa06001d9b0 msgr2=0x7fa060039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 -- 172.21.15.16:0/4102706314 shutdown_connections 2024-09-17T05:01:27.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.787+0000 7fa06e200640 1 -- 172.21.15.16:0/4102706314 wait complete. 2024-09-17T05:01:27.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.791+0000 7fa06e200640 1 Processor -- start 2024-09-17T05:01:27.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.791+0000 7fa06e200640 1 -- start start 2024-09-17T05:01:27.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.795+0000 7fa06e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 0x7fa0600ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.795+0000 7fa06e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 0x7fa0600e3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.795+0000 7fa06ce00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 0x7fa0600ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.795+0000 7fa06ce00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 0x7fa0600ddb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49492/0 (socket says 172.21.15.16:49492) 2024-09-17T05:01:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.795+0000 7fa067e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 0x7fa0600e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 0x7fa0600e3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:43912/0 (socket says 172.21.15.16:43912) 2024-09-17T05:01:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06ce00640 1 -- 172.21.15.16:0/1786137577 learned_addr learned my addr 172.21.15.16:0/1786137577 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0600e3bd0 0x7fa0600de340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0600f0ab0 con 0x7fa0600e3bd0 2024-09-17T05:01:27.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0600f0930 con 0x7fa0600516f0 2024-09-17T05:01:27.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa0600f0c30 con 0x7fa060006320 2024-09-17T05:01:27.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 -- 172.21.15.16:0/1536834225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 msgr2=0x7f71ac050990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b4a00640 1 -- 172.21.15.16:0/1536834225 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a40a66e0 con 0x7f71ac033bc0 2024-09-17T05:01:27.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 --2- 172.21.15.16:0/1536834225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac050990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098f0 tx=0x7f71a401c870 comp rx=0 tx=0).stop 2024-09-17T05:01:27.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 -- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 msgr2=0x7fa0600ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 0x7fa0600ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 -- 172.21.15.16:0/1786137577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0600e3bd0 msgr2=0x7fa0600de340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0600e3bd0 0x7fa0600de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa060052eb0 con 0x7fa0600516f0 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 -- 172.21.15.16:0/1536834225 shutdown_connections 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 --2- 172.21.15.16:0/1536834225 >> v2:172.21.15.16:3300/0 conn(0x7f71ac04edf0 0x7f71ac04f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 --2- 172.21.15.16:0/1536834225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac050990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 -- 172.21.15.16:0/1536834225 >> 172.21.15.16:0/1536834225 conn(0x7f71ac01d9b0 msgr2=0x7f71ac039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa067e00640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 0x7fa0600e3690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa05400b3e0 tx=0x7fa05400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 -- 172.21.15.16:0/1536834225 shutdown_connections 2024-09-17T05:01:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7f71b7200640 1 -- 172.21.15.16:0/1536834225 wait complete. 2024-09-17T05:01:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054015020 con 0x7fa0600516f0 2024-09-17T05:01:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa060053190 con 0x7fa0600516f0 2024-09-17T05:01:27.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.799+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0600e4400 con 0x7fa0600516f0 2024-09-17T05:01:27.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.803+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa054011920 con 0x7fa0600516f0 2024-09-17T05:01:27.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.803+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054015020 con 0x7fa0600516f0 2024-09-17T05:01:27.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.803+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0440052a0 con 0x7fa0600516f0 2024-09-17T05:01:27.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.803+0000 7f71b7200640 1 Processor -- start 2024-09-17T05:01:27.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa054018020 con 0x7fa0600516f0 2024-09-17T05:01:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7fa064c00640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa048073430 0x7fa048075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.803+0000 7f71b7200640 1 -- start start 2024-09-17T05:01:27.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa054093ec0 con 0x7fa0600516f0 2024-09-17T05:01:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b7200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac0df680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 -- 172.21.15.16:0/1114457647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b80051d0 msgr2=0x7f61b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 --2- 172.21.15.16:0/1114457647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b80051d0 0x7f61b800b9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009c00 tx=0x7f61c401d2b0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ac04edf0 0x7f71ac0d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 -- 172.21.15.16:0/1114457647 shutdown_connections 2024-09-17T05:01:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 --2- 172.21.15.16:0/1114457647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61b80051d0 0x7f61b800b9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac0511b0 0x7f71ac0f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7fa06ce00640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa048073430 0x7fa048075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 -- 172.21.15.16:0/1114457647 >> 172.21.15.16:0/1114457647 conn(0x7f61d40ccae0 msgr2=0x7f61d40ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ac04edf0 0x7f71ac0d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 -- 172.21.15.16:0/1114457647 shutdown_connections 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f61db000640 1 -- 172.21.15.16:0/1114457647 wait complete. 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b5e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac0df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b5e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac0df680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49520/0 (socket says 172.21.15.16:49520) 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b5e00640 1 -- 172.21.15.16:0/1972699963 learned_addr learned my addr 172.21.15.16:0/1972699963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0f4ce0 con 0x7f71ac04edf0 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0f4b60 con 0x7f71ac0511b0 2024-09-17T05:01:27.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.807+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f71ac0f4e60 con 0x7f71ac033bc0 2024-09-17T05:01:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7fa06ce00640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa048073430 0x7fa048075940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098f0 tx=0x7fa05c0b3040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b5e00640 1 -- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac0511b0 msgr2=0x7f71ac0f1840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:01:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b5e00640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac0511b0 0x7f71ac0f1840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b5e00640 1 -- 172.21.15.16:0/1972699963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ac04edf0 msgr2=0x7f71ac0d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b5e00640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ac04edf0 0x7f71ac0d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b5e00640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac0d31e0 con 0x7f71ac033bc0 2024-09-17T05:01:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b5e00640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac0df680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71a40b0040 tx=0x7f71a40a6560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9ebe00640 1 -- 172.21.15.16:0/1266523779 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9d0004810 msgr2=0x7ff9d0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/1266523779 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9d0004810 0x7ff9d0004c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009cf0 tx=0x7ff9e401d030 comp rx=0 tx=0).stop 2024-09-17T05:01:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4004250 con 0x7f71ac033bc0 2024-09-17T05:01:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a40044e0 con 0x7f71ac033bc0 2024-09-17T05:01:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac184f80 con 0x7f71ac033bc0 2024-09-17T05:01:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003840 con 0x7f71ac033bc0 2024-09-17T05:01:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac185540 con 0x7f71ac033bc0 2024-09-17T05:01:27.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9eb400640 1 -- 172.21.15.16:0/1266523779 reap_dead start 2024-09-17T05:01:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa05401f020 con 0x7fa0600516f0 2024-09-17T05:01:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9ebe00640 1 -- 172.21.15.16:0/1266523779 shutdown_connections 2024-09-17T05:01:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9ebe00640 1 -- 172.21.15.16:0/1266523779 >> 172.21.15.16:0/1266523779 conn(0x7ff9ec06fbe0 msgr2=0x7ff9ec06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9ebe00640 1 -- 172.21.15.16:0/1266523779 shutdown_connections 2024-09-17T05:01:27.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f61db000640 1 Processor -- start 2024-09-17T05:01:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f61db000640 1 -- start start 2024-09-17T05:01:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c0052a0 con 0x7f71ac033bc0 2024-09-17T05:01:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.811+0000 7ff9ebe00640 1 -- 172.21.15.16:0/1266523779 wait complete. 2024-09-17T05:01:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71a40c0030 con 0x7f71ac033bc0 2024-09-17T05:01:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f6d4c400640 1 -- 172.21.15.16:0/4126740729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d440cfcd0 msgr2=0x7f6d440d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f61db000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b80051d0 0x7f61d40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f6d4c400640 1 --2- 172.21.15.16:0/4126740729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d440cfcd0 0x7f6d440d00e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6d440b0ce0 tx=0x7f6d3801caf0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.815+0000 7f61db000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 0x7f61d41fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 Processor -- start 2024-09-17T05:01:27.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f71aa000640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7194073250 0x7f7194075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61db000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415b220 0x7f61d415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 -- start start 2024-09-17T05:01:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 0x7f61d41fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 0x7f61d41fd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49534/0 (socket says 172.21.15.16:49534) 2024-09-17T05:01:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f6d4c400640 1 -- 172.21.15.16:0/4126740729 shutdown_connections 2024-09-17T05:01:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f6d4c400640 1 --2- 172.21.15.16:0/4126740729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d440cfcd0 0x7f6d440d00e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f6d4c400640 1 --2- 172.21.15.16:0/4126740729 >> v2:172.21.15.16:3300/0 conn(0x7f6d44155400 0x7f6d44155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 -- 172.21.15.16:0/263056143 learned_addr learned my addr 172.21.15.16:0/263056143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f71a412e460 con 0x7f71ac033bc0 2024-09-17T05:01:27.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f71b5400640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7194073250 0x7f7194075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 -- 172.21.15.16:0/1081496094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc058300 msgr2=0x7f86dc0f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 --2- 172.21.15.16:0/1081496094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc058300 0x7f86dc0f2360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86e407ad80 tx=0x7f86e4098650 comp rx=0 tx=0).stop 2024-09-17T05:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61d415da10 con 0x7f61d415b220 2024-09-17T05:01:27.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61d40cc6b0 con 0x7f61b80051d0 2024-09-17T05:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f6d4c400640 1 -- 172.21.15.16:0/4126740729 >> 172.21.15.16:0/4126740729 conn(0x7f6d440b2dd0 msgr2=0x7f6d440b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 -- 172.21.15.16:0/1081496094 shutdown_connections 2024-09-17T05:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 --2- 172.21.15.16:0/1081496094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc058300 0x7f86dc0f2360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 -- 172.21.15.16:0/1081496094 >> 172.21.15.16:0/1081496094 conn(0x7f86dc00e210 msgr2=0x7f86dc00e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f61d40b5930 con 0x7f61d40cf890 2024-09-17T05:01:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f71b5400640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7194073250 0x7f7194075760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0dd4e0 tx=0x7f71a0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 -- 172.21.15.16:0/1081496094 shutdown_connections 2024-09-17T05:01:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f86e9e00640 1 -- 172.21.15.16:0/1081496094 wait complete. 2024-09-17T05:01:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f6d4c400640 1 -- 172.21.15.16:0/4126740729 shutdown_connections 2024-09-17T05:01:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 -- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b80051d0 msgr2=0x7f61d40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3e00640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b80051d0 0x7f61d40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 0x7ff9ec161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b80051d0 0x7f61d40b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d9000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415b220 0x7f61d415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9ec15e380 0x7ff9ec0ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 -- 172.21.15.16:0/263056143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415b220 msgr2=0x7f61d415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9ec0ac760 0x7ff9ec2058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9eaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 0x7ff9ec161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec0d0c10 con 0x7ff9ec15e380 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ec0d0a90 con 0x7ff9d0004810 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ebe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ec0d0d90 con 0x7ff9ec0ac760 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9eaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 0x7ff9ec161e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:43956/0 (socket says 172.21.15.16:43956) 2024-09-17T05:01:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9eaa00640 1 -- 172.21.15.16:0/691239572 learned_addr learned my addr 172.21.15.16:0/691239572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415b220 0x7f61d415b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3400640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d41f9910 con 0x7f61d40cf890 2024-09-17T05:01:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9ea000640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9ec15e380 0x7ff9ec0ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7ff9eb400640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9ec0ac760 0x7ff9ec2058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f61d3e00640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b80051d0 0x7f61d40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9eaa00640 1 -- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9ec0ac760 msgr2=0x7ff9ec2058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9eaa00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9ec0ac760 0x7ff9ec2058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9eaa00640 1 -- 172.21.15.16:0/691239572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9ec15e380 msgr2=0x7ff9ec0ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9eaa00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9ec15e380 0x7ff9ec0ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61d3400640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 0x7f61d41fd0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00c730 tx=0x7f61cc00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9eaa00640 1 -- 172.21.15.16:0/691239572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ec0acb70 con 0x7ff9d0004810 2024-09-17T05:01:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.819+0000 7f6d4c400640 1 -- 172.21.15.16:0/4126740729 wait complete. 2024-09-17T05:01:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 Processor -- start 2024-09-17T05:01:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9eaa00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 0x7ff9ec161e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8003f30 tx=0x7ff9d8001700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 -- start start 2024-09-17T05:01:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f6d4c400640 1 Processor -- start 2024-09-17T05:01:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8011640 con 0x7ff9d0004810 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9d8016020 con 0x7ff9d0004810 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8011a40 con 0x7ff9d0004810 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ec1627b0 con 0x7ff9d0004810 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ec162d70 con 0x7ff9d0004810 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc003b80 con 0x7f61d40cf890 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d415bb70 con 0x7f61d40cf890 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61cc003d20 con 0x7f61d40cf890 2024-09-17T05:01:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d41f9b90 con 0x7f61d40cf890 2024-09-17T05:01:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f6d4c400640 1 -- start start 2024-09-17T05:01:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00aab0 con 0x7f61d40cf890 2024-09-17T05:01:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc006f30 0x7f86dc0d9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d40af8f0 con 0x7f61d40cf890 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 0x7f86dc0e6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc058300 0x7f86dc1850d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc0f6080 con 0x7f86dc053370 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc0f5f00 con 0x7f86dc006f30 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e9e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc0f6200 con 0x7f86dc058300 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 0x7f86dc0e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 0x7f86dc0e6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43464/0 (socket says 172.21.15.16:43464) 2024-09-17T05:01:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.823+0000 7f86e2000640 1 -- 172.21.15.16:0/3873920102 learned_addr learned my addr 172.21.15.16:0/3873920102 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f71a40164f0 con 0x7f71ac033bc0 2024-09-17T05:01:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e3400640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc058300 0x7f86dc1850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e2000640 1 -- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc058300 msgr2=0x7f86dc1850d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e2000640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc058300 0x7f86dc1850d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e2000640 1 -- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc006f30 msgr2=0x7f86dc0d9bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:01:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e2000640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc006f30 0x7f86dc0d9bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e2000640 1 -- 172.21.15.16:0/3873920102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc0da920 con 0x7f86dc053370 2024-09-17T05:01:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7fd5c5400640 1 -- 172.21.15.16:0/3080203972 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0009510 con 0x7fd5ac004810 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d44073dc0 0x7f6d4414b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d440cfcd0 0x7f6d4415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 0x7f6d4415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440cc7b0 con 0x7f6d440cfcd0 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d440cc630 con 0x7f6d44155400 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6d440cc930 con 0x7f6d44073dc0 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 0x7f6d4415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 0x7f6d4415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:43966/0 (socket says 172.21.15.16:43966) 2024-09-17T05:01:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 -- 172.21.15.16:0/1803355313 learned_addr learned my addr 172.21.15.16:0/1803355313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ec162fb0 con 0x7ff9d0004810 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d49000640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d440cfcd0 0x7f6d4415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d49a00640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d44073dc0 0x7f6d4414b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 -- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d44073dc0 msgr2=0x7f6d4414b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d44073dc0 0x7f6d4414b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 -- 172.21.15.16:0/1803355313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d440cfcd0 msgr2=0x7f6d4415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86e2000640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 0x7f86dc0e6670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c390 tx=0x7f86d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d440cfcd0 0x7f6d4415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 -- 172.21.15.16:0/1803355313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d44152ca0 con 0x7f6d44155400 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d49000640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d440cfcd0 0x7f6d4415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4a400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 0x7f6d4415b0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009b10 tx=0x7f6d380a57e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0017070 con 0x7f86dc053370 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86d00053e0 con 0x7f86dc053370 2024-09-17T05:01:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0003860 con 0x7f86dc053370 2024-09-17T05:01:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380bc020 con 0x7f6d44155400 2024-09-17T05:01:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d380053b0 con 0x7f6d44155400 2024-09-17T05:01:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380bc020 con 0x7f6d44155400 2024-09-17T05:01:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4414e4c0 con 0x7f6d44155400 2024-09-17T05:01:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4415b9c0 con 0x7f6d44155400 2024-09-17T05:01:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc0dac00 con 0x7f86dc053370 2024-09-17T05:01:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 -- 172.21.15.16:0/3080203972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 msgr2=0x7fd5ac004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 --2- 172.21.15.16:0/3080203972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 0x7fd5ac004c20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009940 tx=0x7fd5b001c740 comp rx=0 tx=0).stop 2024-09-17T05:01:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc0e4ac0 con 0x7f86dc053370 2024-09-17T05:01:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 -- 172.21.15.16:0/3080203972 shutdown_connections 2024-09-17T05:01:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 --2- 172.21.15.16:0/3080203972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 0x7fd5ac004c20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 -- 172.21.15.16:0/3080203972 >> 172.21.15.16:0/3080203972 conn(0x7fd5c80b2250 msgr2=0x7fd5c80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc0e7330 con 0x7f86dc053370 2024-09-17T05:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff9d8011be0 con 0x7ff9d0004810 2024-09-17T05:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7ff9e2a00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9cc0732d0 0x7ff9cc0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.827+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f61cc00ac50 con 0x7f61d40cf890 2024-09-17T05:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f61cbe00640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f61b4073200 0x7f61b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f61cc094500 con 0x7f61d40cf890 2024-09-17T05:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff9d80952a0 con 0x7ff9d0004810 2024-09-17T05:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d4415bc20 con 0x7f6d44155400 2024-09-17T05:01:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 -- 172.21.15.16:0/3080203972 shutdown_connections 2024-09-17T05:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7f61d3e00640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f61b4073200 0x7f61b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7ff9ea000640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9cc0732d0 0x7ff9cc0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7ff9ea000640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9cc0732d0 0x7ff9cc0757e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40ae000 tx=0x7ff9e4019dd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f61d415bb70 con 0x7f61d40cf890 2024-09-17T05:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7f61d3e00640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f61b4073200 0x7f61b4075710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000f70 tx=0x7f61c000c040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.831+0000 7fd5cd200640 1 -- 172.21.15.16:0/3080203972 wait complete. 2024-09-17T05:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f86d001a070 con 0x7f86dc053370 2024-09-17T05:01:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff9ec162fb0 con 0x7ff9d0004810 2024-09-17T05:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7f86db400640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86c40732d0 0x7f86c40757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f86d0093840 con 0x7f86dc053370 2024-09-17T05:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.835+0000 7f86e2a00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86c40732d0 0x7f86c40757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.839+0000 7fd5cd200640 1 Processor -- start 2024-09-17T05:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.839+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d380bf070 con 0x7f6d44155400 2024-09-17T05:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.839+0000 7f86e2a00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86c40732d0 0x7f86c40757e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86d4001510 tx=0x7f86d4007a20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.839+0000 7fd5cd200640 1 -- start start 2024-09-17T05:01:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7f6d41600640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d24073400 0x7f6d24075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6d38002020 con 0x7f6d44155400 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7f6d49a00640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d24073400 0x7f6d24075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5cd200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 0x7fd5c80c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5cd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 0x7fd5c81f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5cd200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd5c81f4e00 0x7fd5c81586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5cd200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c815a9a0 con 0x7fd5c814ee70 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5cd200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c815a820 con 0x7fd5ac004810 2024-09-17T05:01:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5cd200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c815ab20 con 0x7fd5c81f4e00 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5bd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 0x7fd5c81f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5bd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 0x7fd5c81f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43474/0 (socket says 172.21.15.16:43474) 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5bd400640 1 -- 172.21.15.16:0/2530316553 learned_addr learned my addr 172.21.15.16:0/2530316553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5c6800640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd5c81f4e00 0x7fd5c81586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7fd5c5e00640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 0x7fd5c80c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.843+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d38016770 con 0x7f6d44155400 2024-09-17T05:01:27.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.847+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f86d005e590 con 0x7f86dc053370 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bd400640 1 -- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd5c81f4e00 msgr2=0x7fd5c81586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bd400640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd5c81f4e00 0x7fd5c81586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bd400640 1 -- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 msgr2=0x7fd5c80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bd400640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 0x7fd5c80c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bd400640 1 -- 172.21.15.16:0/2530316553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80b3070 con 0x7fd5c814ee70 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bd400640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 0x7fd5c81f48c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5b001cfe0 tx=0x7fd5b00a7ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00b2040 con 0x7fd5c814ee70 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c81f5640 con 0x7fd5c814ee70 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5b00bd070 con 0x7fd5c814ee70 2024-09-17T05:01:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.851+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00b2040 con 0x7fd5c814ee70 2024-09-17T05:01:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.847+0000 7f6d49a00640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d24073400 0x7f6d24075910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009ad0 tx=0x7f6d3c009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.855+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c80ca3f0 con 0x7fd5c814ee70 2024-09-17T05:01:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.855+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd5b00a59c0 con 0x7fd5c814ee70 2024-09-17T05:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.859+0000 7fd5bea00640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5a4073400 0x7fd5a4075910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.859+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c815af10 con 0x7fd5c814ee70 2024-09-17T05:01:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.859+0000 7fd5c5e00640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5a4073400 0x7fd5a4075910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.859+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd5b012bf10 con 0x7fd5c814ee70 2024-09-17T05:01:27.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.859+0000 7fd5c5e00640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5a4073400 0x7fd5a4075910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5c806bbd0 tx=0x7fd5b8005040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd5b0016450 con 0x7fd5c814ee70 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3817659997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c4074ae0 msgr2=0x7fc7c4074050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3817659997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c4074ae0 0x7fc7c4074050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0013f30 tx=0x7fc7b0024e40 comp rx=0 tx=0).stop 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3817659997 shutdown_connections 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3817659997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073270 0x7fc7c415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3817659997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c4074ae0 0x7fc7c4074050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3817659997 >> 172.21.15.16:0/3817659997 conn(0x7fc7c40ccae0 msgr2=0x7fc7c40ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3817659997 shutdown_connections 2024-09-17T05:01:27.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.871+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3817659997 wait complete. 2024-09-17T05:01:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.879+0000 7fc7c9c00640 1 Processor -- start 2024-09-17T05:01:27.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.879+0000 7fc7c9c00640 1 -- start start 2024-09-17T05:01:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.879+0000 7fc7c9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 0x7fc7c40c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.879+0000 7fc7c9c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073680 0x7fc7c41f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 0x7fc7c40c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 0x7fc7c40c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43478/0 (socket says 172.21.15.16:43478) 2024-09-17T05:01:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c9c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c41f83a0 0x7fc7c4149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 -- 172.21.15.16:0/3715564941 learned_addr learned my addr 172.21.15.16:0/3715564941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2000640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073680 0x7fc7c41f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4162200 con 0x7fc7c4073270 2024-09-17T05:01:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c4162080 con 0x7fc7c41f83a0 2024-09-17T05:01:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c4162380 con 0x7fc7c4073680 2024-09-17T05:01:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c3400640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c41f83a0 0x7fc7c4149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 -- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073680 msgr2=0x7fc7c41f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073680 0x7fc7c41f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 -- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c41f83a0 msgr2=0x7fc7c4149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c41f83a0 0x7fc7c4149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c40c5430 con 0x7fc7c4073270 2024-09-17T05:01:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2000640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073680 0x7fc7c41f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c3400640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c41f83a0 0x7fc7c4149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:27.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.883+0000 7fc7c2a00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 0x7fc7c40c4be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009230 tx=0x7fc7b0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.887+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00c6070 con 0x7fc7c4073270 2024-09-17T05:01:27.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.887+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b0005320 con 0x7fc7c4073270 2024-09-17T05:01:27.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.887+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00ba040 con 0x7fc7c4073270 2024-09-17T05:01:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.887+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c40c5710 con 0x7fc7c4073270 2024-09-17T05:01:27.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.887+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c40c5cd0 con 0x7fc7c4073270 2024-09-17T05:01:27.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.891+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7b00046d0 con 0x7fc7c4073270 2024-09-17T05:01:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.891+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c4074ae0 con 0x7fc7c4073270 2024-09-17T05:01:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.895+0000 7fc7baa00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc7a4073230 0x7fc7a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.899+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc7b0136b60 con 0x7fc7c4073270 2024-09-17T05:01:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.899+0000 7fc7c2000640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc7a4073230 0x7fc7a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:27.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.899+0000 7fc7c2000640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc7a4073230 0x7fc7a4075740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40ca3d0 tx=0x7fc7bc009340 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:27.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:27.903+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc7b001ea60 con 0x7fc7c4073270 2024-09-17T05:01:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.191+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa044005590 con 0x7fa0600516f0 2024-09-17T05:01:28.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.191+0000 7fa064c00640 1 -- 172.21.15.16:0/1786137577 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa05405ec20 con 0x7fa0600516f0 2024-09-17T05:01:28.193 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.191+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f718c005860 con 0x7f71ac033bc0 2024-09-17T05:01:28.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.191+0000 7f71aa000640 1 -- 172.21.15.16:0/1972699963 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71a40f92b0 con 0x7f71ac033bc0 2024-09-17T05:01:28.194 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.195+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa048073430 msgr2=0x7fa048075940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.195+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa048073430 0x7fa048075940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098f0 tx=0x7fa05c0b3040 comp rx=0 tx=0).stop 2024-09-17T05:01:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.195+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7194073250 msgr2=0x7f7194075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7194073250 0x7f7194075760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0dd4e0 tx=0x7f71a0009040 comp rx=0 tx=0).stop 2024-09-17T05:01:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 msgr2=0x7f71ac0df680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac0df680 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f71a40b0040 tx=0x7f71a40a6560 comp rx=0 tx=0).stop 2024-09-17T05:01:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 msgr2=0x7fa0600e3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 0x7fa0600e3690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa05400b3e0 tx=0x7fa05400b8a0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 shutdown_connections 2024-09-17T05:01:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7194073250 0x7f7194075760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71ac0511b0 0x7f71ac0f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71ac04edf0 0x7f71ac0d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 --2- 172.21.15.16:0/1972699963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71ac033bc0 0x7f71ac0df680 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 >> 172.21.15.16:0/1972699963 conn(0x7f71ac01d9b0 msgr2=0x7f71ac0e0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 shutdown_connections 2024-09-17T05:01:28.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa048073430 0x7fa048075940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 shutdown_connections 2024-09-17T05:01:28.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f71b7200640 1 -- 172.21.15.16:0/1972699963 wait complete. 2024-09-17T05:01:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0600e3bd0 0x7fa0600de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0600516f0 0x7fa0600e3690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 --2- 172.21.15.16:0/1786137577 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa060006320 0x7fa0600ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 >> 172.21.15.16:0/1786137577 conn(0x7fa06001d9b0 msgr2=0x7fa0600e17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 shutdown_connections 2024-09-17T05:01:28.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7fa06e200640 1 -- 172.21.15.16:0/1786137577 wait complete. 2024-09-17T05:01:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.199+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f86dc00eee0 con 0x7f86dc053370 2024-09-17T05:01:28.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.203+0000 7f86db400640 1 -- 172.21.15.16:0/3873920102 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86d005de90 con 0x7f86dc053370 2024-09-17T05:01:28.203 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86c40732d0 msgr2=0x7f86c40757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86c40732d0 0x7f86c40757e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f86d4001510 tx=0x7f86d4007a20 comp rx=0 tx=0).stop 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 msgr2=0x7f86dc0e6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 0x7f86dc0e6670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c390 tx=0x7f86d000c850 comp rx=0 tx=0).stop 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 shutdown_connections 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86c40732d0 0x7f86c40757e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc058300 0x7f86dc1850d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc053370 0x7f86dc0e6670 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 --2- 172.21.15.16:0/3873920102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc006f30 0x7f86dc0d9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 >> 172.21.15.16:0/3873920102 conn(0x7f86dc00e210 msgr2=0x7f86dc0f1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 shutdown_connections 2024-09-17T05:01:28.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f86e9e00640 1 -- 172.21.15.16:0/3873920102 wait complete. 2024-09-17T05:01:28.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.207+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f61d40cd8e0 con 0x7f61d40cf890 2024-09-17T05:01:28.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.211+0000 7f61cbe00640 1 -- 172.21.15.16:0/263056143 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61d41f9910 con 0x7f61d40cf890 2024-09-17T05:01:28.211 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:28.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.211+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff9ec0d0380 con 0x7ff9d0004810 2024-09-17T05:01:28.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.211+0000 7ff9e2a00640 1 -- 172.21.15.16:0/691239572 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff9d8020020 con 0x7ff9d0004810 2024-09-17T05:01:28.213 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-17T05:01:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f61b4073200 msgr2=0x7f61b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f61b4073200 0x7f61b4075710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61c0000f70 tx=0x7f61c000c040 comp rx=0 tx=0).stop 2024-09-17T05:01:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 msgr2=0x7f61d41fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 0x7f61d41fd0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00c730 tx=0x7f61cc00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 shutdown_connections 2024-09-17T05:01:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f61b4073200 0x7f61b4075710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61d415b220 0x7f61d415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f61d40cf890 0x7f61d41fd0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 --2- 172.21.15.16:0/263056143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b80051d0 0x7f61d40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 >> 172.21.15.16:0/263056143 conn(0x7f61d40ccae0 msgr2=0x7f61d40ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9cc0732d0 msgr2=0x7ff9cc0757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 shutdown_connections 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9cc0732d0 0x7ff9cc0757e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff9e40ae000 tx=0x7ff9e4019dd0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 msgr2=0x7ff9ec161e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 0x7ff9ec161e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8003f30 tx=0x7ff9d8001700 comp rx=0 tx=0).stop 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 shutdown_connections 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9cc0732d0 0x7ff9cc0757e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9ec0ac760 0x7ff9ec2058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9ec15e380 0x7ff9ec0ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 --2- 172.21.15.16:0/691239572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9d0004810 0x7ff9ec161e40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 >> 172.21.15.16:0/691239572 conn(0x7ff9ec06fbe0 msgr2=0x7ff9ec0af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.215+0000 7f61db000640 1 -- 172.21.15.16:0/263056143 wait complete. 2024-09-17T05:01:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.219+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 shutdown_connections 2024-09-17T05:01:28.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.219+0000 7ff9ebe00640 1 -- 172.21.15.16:0/691239572 wait complete. 2024-09-17T05:01:28.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.223+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6d4414e9e0 con 0x7f6d44155400 2024-09-17T05:01:28.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.223+0000 7f6d41600640 1 -- 172.21.15.16:0/1803355313 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d380f9430 con 0x7f6d44155400 2024-09-17T05:01:28.225 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d24073400 msgr2=0x7f6d24075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d24073400 0x7f6d24075910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009ad0 tx=0x7f6d3c009260 comp rx=0 tx=0).stop 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 msgr2=0x7f6d4415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 0x7f6d4415b0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009b10 tx=0x7f6d380a57e0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 shutdown_connections 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d24073400 0x7f6d24075910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.227+0000 7f6d4c400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d44155400 0x7f6d4415b0b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.231+0000 7f6d4c400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d440cfcd0 0x7f6d4415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.231+0000 7f6d4c400640 1 --2- 172.21.15.16:0/1803355313 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d44073dc0 0x7f6d4414b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.231+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 >> 172.21.15.16:0/1803355313 conn(0x7f6d440b2dd0 msgr2=0x7f6d44068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.231+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 shutdown_connections 2024-09-17T05:01:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.231+0000 7f6d4c400640 1 -- 172.21.15.16:0/1803355313 wait complete. 2024-09-17T05:01:28.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-17T05:01:28.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.0 2024-09-17T05:01:28.244 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.247+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd5c80cb210 con 0x7fd5c814ee70 2024-09-17T05:01:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.247+0000 7fd5bea00640 1 -- 172.21.15.16:0/2530316553 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5b0016450 con 0x7fd5c814ee70 2024-09-17T05:01:28.248 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-09-17T05:01:28.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-09-17T05:01:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5a4073400 msgr2=0x7fd5a4075910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5a4073400 0x7fd5a4075910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5c806bbd0 tx=0x7fd5b8005040 comp rx=0 tx=0).stop 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 msgr2=0x7fd5c81f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 0x7fd5c81f48c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd5b001cfe0 tx=0x7fd5b00a7ed0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 shutdown_connections 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5a4073400 0x7fd5a4075910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd5c81f4e00 0x7fd5c81586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5c814ee70 0x7fd5c81f48c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 --2- 172.21.15.16:0/2530316553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5ac004810 0x7fd5c80c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 >> 172.21.15.16:0/2530316553 conn(0x7fd5c80b2250 msgr2=0x7fd5c80b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 shutdown_connections 2024-09-17T05:01:28.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.251+0000 7fd5cd200640 1 -- 172.21.15.16:0/2530316553 wait complete. 2024-09-17T05:01:28.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-17T05:01:28.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2024-09-17T05:01:28.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2024-09-17T05:01:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc7c415a750 con 0x7fc7c4073270 2024-09-17T05:01:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7baa00640 1 -- 172.21.15.16:0/3715564941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7b0101720 con 0x7fc7c4073270 2024-09-17T05:01:28.271 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2024-09-17T05:01:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc7a4073230 msgr2=0x7fc7a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc7a4073230 0x7fc7a4075740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40ca3d0 tx=0x7fc7bc009340 comp rx=0 tx=0).stop 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 msgr2=0x7fc7c40c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 0x7fc7c40c4be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009230 tx=0x7fc7b0005d60 comp rx=0 tx=0).stop 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 shutdown_connections 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc7a4073230 0x7fc7a4075740 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c41f83a0 0x7fc7c4149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc7c4073680 0x7fc7c41f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 --2- 172.21.15.16:0/3715564941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7c4073270 0x7fc7c40c4be0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 >> 172.21.15.16:0/3715564941 conn(0x7fc7c40ccae0 msgr2=0x7fc7c415c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 shutdown_connections 2024-09-17T05:01:28.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:28.271+0000 7fc7c9c00640 1 -- 172.21.15.16:0/3715564941 wait complete. 2024-09-17T05:01:28.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-17T05:01:28.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2024-09-17T05:01:29.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:01:29.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:01:29.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:01:29.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:01:29.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:01:29.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:01:29.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T05:01:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 -- 172.21.15.16:0/535735410 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c0cff90 msgr2=0x7f960c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 --2- 172.21.15.16:0/535735410 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c0cff90 0x7f960c0d03a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9604009bc0 tx=0x7f960401cbc0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 -- 172.21.15.16:0/535735410 shutdown_connections 2024-09-17T05:01:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 --2- 172.21.15.16:0/535735410 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c0cff90 0x7f960c0d03a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 --2- 172.21.15.16:0/535735410 >> v2:172.21.15.16:3300/0 conn(0x7f960c144e50 0x7f960c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 -- 172.21.15.16:0/535735410 >> 172.21.15.16:0/535735410 conn(0x7f960c073ca0 msgr2=0x7f960c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 -- 172.21.15.16:0/535735410 shutdown_connections 2024-09-17T05:01:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.479+0000 7f9613000640 1 -- 172.21.15.16:0/535735410 wait complete. 2024-09-17T05:01:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.483+0000 7f9613000640 1 Processor -- start 2024-09-17T05:01:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.483+0000 7f9613000640 1 -- start start 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.483+0000 7f9613000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c06efb0 0x7f960c1fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.483+0000 7f9613000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 0x7f960c140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9613000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f960c144e50 0x7f960c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9613000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f960c169a80 con 0x7f960c0cff90 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9613000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f960c160010 con 0x7f960c144e50 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9613000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f960c169c00 con 0x7f960c06efb0 2024-09-17T05:01:29.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 0x7f960c140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 0x7f960c140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43490/0 (socket says 172.21.15.16:43490) 2024-09-17T05:01:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 -- 172.21.15.16:0/3854949986 learned_addr learned my addr 172.21.15.16:0/3854949986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9612600640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f960c144e50 0x7f960c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 -- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c06efb0 msgr2=0x7f960c1fded0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T05:01:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c06efb0 0x7f960c1fded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 -- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f960c144e50 msgr2=0x7f960c159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f960c144e50 0x7f960c159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 -- 172.21.15.16:0/3854949986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c15d3e0 con 0x7f960c0cff90 2024-09-17T05:01:29.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9611200640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 0x7f960c140ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c400 tx=0x7f95f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8011790 con 0x7f960c0cff90 2024-09-17T05:01:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f8017070 con 0x7f960c0cff90 2024-09-17T05:01:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8011ca0 con 0x7f960c0cff90 2024-09-17T05:01:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c0b25e0 con 0x7f960c0cff90 2024-09-17T05:01:29.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.487+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c15d960 con 0x7f960c0cff90 2024-09-17T05:01:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.491+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c0d03a0 con 0x7f960c0cff90 2024-09-17T05:01:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.499+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f95f801c460 con 0x7f960c0cff90 2024-09-17T05:01:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.499+0000 7f9609600640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f95ec073200 0x7f95ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.499+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f95f8094e50 con 0x7f960c0cff90 2024-09-17T05:01:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.503+0000 7f9611c00640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f95ec073200 0x7f95ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.503+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f95f805fb10 con 0x7f960c0cff90 2024-09-17T05:01:29.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f9611c00640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f95ec073200 0x7f95ec075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96000099d0 tx=0x7f96000091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 -- 172.21.15.16:0/1167342259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 msgr2=0x7f20ec0d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 --2- 172.21.15.16:0/1167342259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec0d1b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0ce030 tx=0x7f20d801c6b0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 -- 172.21.15.16:0/1167342259 shutdown_connections 2024-09-17T05:01:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 --2- 172.21.15.16:0/1167342259 >> v2:172.21.15.104:3300/0 conn(0x7f20ec0cf850 0x7f20ec0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 --2- 172.21.15.16:0/1167342259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec0d1b80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 -- 172.21.15.16:0/1167342259 >> 172.21.15.16:0/1167342259 conn(0x7f20ec0b42d0 msgr2=0x7f20ec0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.507+0000 7f20f2600640 1 -- 172.21.15.16:0/1167342259 shutdown_connections 2024-09-17T05:01:29.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.511+0000 7f20f2600640 1 -- 172.21.15.16:0/1167342259 wait complete. 2024-09-17T05:01:29.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.511+0000 7f20f2600640 1 Processor -- start 2024-09-17T05:01:29.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.511+0000 7f20f2600640 1 -- start start 2024-09-17T05:01:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20ec0d02d0 0x7f20ec1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec1fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec14c320 0x7f20ec0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec169e20 con 0x7f20ec159f70 2024-09-17T05:01:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec169ca0 con 0x7f20ec14c320 2024-09-17T05:01:29.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec169fa0 con 0x7f20ec0d02d0 2024-09-17T05:01:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec1fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec1fcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43508/0 (socket says 172.21.15.16:43508) 2024-09-17T05:01:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 -- 172.21.15.16:0/980878116 learned_addr learned my addr 172.21.15.16:0/980878116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eb400640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20ec0d02d0 0x7f20ec1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20ebe00640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec14c320 0x7f20ec0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 -- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20ec0d02d0 msgr2=0x7f20ec1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20ec0d02d0 0x7f20ec1f8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 -- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec14c320 msgr2=0x7f20ec0d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec14c320 0x7f20ec0d2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 -- 172.21.15.16:0/980878116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec150f30 con 0x7f20ec159f70 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20eaa00640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec1fcac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b9c0 tx=0x7f20e400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4003b80 con 0x7f20ec159f70 2024-09-17T05:01:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20e4003e10 con 0x7f20ec159f70 2024-09-17T05:01:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e400cb40 con 0x7f20ec159f70 2024-09-17T05:01:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec14c730 con 0x7f20ec159f70 2024-09-17T05:01:29.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.515+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec1f9540 con 0x7f20ec159f70 2024-09-17T05:01:29.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.519+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec1f8fd0 con 0x7f20ec159f70 2024-09-17T05:01:29.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.519+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20e4006d10 con 0x7f20ec159f70 2024-09-17T05:01:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.523+0000 7f20e3400640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20cc073230 0x7f20cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.523+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f20e4094160 con 0x7f20ec159f70 2024-09-17T05:01:29.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.527+0000 7f20eb400640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20cc073230 0x7f20cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.527+0000 7f20eb400640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20cc073230 0x7f20cc075740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20d80af000 tx=0x7f20d801aef0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.531+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f20e405eda0 con 0x7f20ec159f70 2024-09-17T05:01:29.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e64a00640 1 -- 172.21.15.16:0/1547384865 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58009730 con 0x7f2e6806dfa0 2024-09-17T05:01:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e6ce00640 1 -- 172.21.15.16:0/1547384865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 msgr2=0x7f2e4c006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/1547384865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 0x7f2e4c006a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009b90 tx=0x7f2e5801d2b0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e66800640 1 -- 172.21.15.16:0/1547384865 reap_dead start 2024-09-17T05:01:29.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e6ce00640 1 -- 172.21.15.16:0/1547384865 shutdown_connections 2024-09-17T05:01:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e6ce00640 1 -- 172.21.15.16:0/1547384865 >> 172.21.15.16:0/1547384865 conn(0x7f2e680b2dd0 msgr2=0x7f2e680b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e6ce00640 1 -- 172.21.15.16:0/1547384865 shutdown_connections 2024-09-17T05:01:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.551+0000 7f2e6ce00640 1 -- 172.21.15.16:0/1547384865 wait complete. 2024-09-17T05:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 Processor -- start 2024-09-17T05:01:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 -- start start 2024-09-17T05:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 0x7f2e68144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e68150bd0 0x7f2e681e3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 0x7f2e681e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6806e740 con 0x7f2e68150bd0 2024-09-17T05:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6806e5c0 con 0x7f2e681e3680 2024-09-17T05:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.555+0000 7f2e6ce00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6806e8c0 con 0x7f2e6806dfa0 2024-09-17T05:01:29.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 0x7f2e681e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 0x7f2e681e1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:44034/0 (socket says 172.21.15.16:44034) 2024-09-17T05:01:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e65e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 0x7f2e68144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 -- 172.21.15.16:0/2601090242 learned_addr learned my addr 172.21.15.16:0/2601090242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e65400640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e68150bd0 0x7f2e681e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 -- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 msgr2=0x7f2e68144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 0x7f2e68144a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 -- 172.21.15.16:0/2601090242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e68150bd0 msgr2=0x7f2e681e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e68150bd0 0x7f2e681e3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 -- 172.21.15.16:0/2601090242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e681e3a90 con 0x7f2e681e3680 2024-09-17T05:01:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e65400640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e68150bd0 0x7f2e681e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:01:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e66800640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 0x7f2e681e1de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2e68074b30 tx=0x7f2e6000fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000dda0 con 0x7f2e681e3680 2024-09-17T05:01:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e60004b30 con 0x7f2e681e3680 2024-09-17T05:01:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e60005020 con 0x7f2e681e3680 2024-09-17T05:01:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e681f1340 con 0x7f2e681e3680 2024-09-17T05:01:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.559+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e681f1900 con 0x7f2e681e3680 2024-09-17T05:01:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.563+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e6806ee00 con 0x7f2e681e3680 2024-09-17T05:01:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2e600051c0 con 0x7f2e681e3680 2024-09-17T05:01:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a40a00640 1 -- 172.21.15.16:0/164490831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0b2070 con 0x7f8a440b4c70 2024-09-17T05:01:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 -- 172.21.15.16:0/164490831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a440b4c70 msgr2=0x7f8a44071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 --2- 172.21.15.16:0/164490831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a440b4c70 0x7f8a44071210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009bc0 tx=0x7f8a3c01c8a0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 -- 172.21.15.16:0/164490831 shutdown_connections 2024-09-17T05:01:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 --2- 172.21.15.16:0/164490831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a440b4c70 0x7f8a44071210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 -- 172.21.15.16:0/164490831 >> 172.21.15.16:0/164490831 conn(0x7f8a440d10f0 msgr2=0x7f8a440d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 -- 172.21.15.16:0/164490831 shutdown_connections 2024-09-17T05:01:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f8a43200640 1 -- 172.21.15.16:0/164490831 wait complete. 2024-09-17T05:01:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f2e5e000640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e48073200 0x7f2e48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e60094dc0 con 0x7f2e681e3680 2024-09-17T05:01:29.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.567+0000 7f2e65e00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e48073200 0x7f2e48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f2e65e00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e48073200 0x7f2e48075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2e54001250 tx=0x7f2e54001590 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f8a43200640 1 Processor -- start 2024-09-17T05:01:29.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2e60022020 con 0x7f2e681e3680 2024-09-17T05:01:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f8a43200640 1 -- start start 2024-09-17T05:01:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 -- 172.21.15.16:0/2899575450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b43f0 msgr2=0x7f0f840cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/2899575450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b43f0 0x7f0f840cd080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00afd0 tx=0x7f0f7c020000 comp rx=0 tx=0).stop 2024-09-17T05:01:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 -- 172.21.15.16:0/2295424502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2480cf7f0 msgr2=0x7fd2481621e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 --2- 172.21.15.16:0/2295424502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2480cf7f0 0x7fd2481621e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd24806e720 tx=0x7fd23801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 -- 172.21.15.16:0/2295424502 shutdown_connections 2024-09-17T05:01:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 -- 172.21.15.16:0/2899575450 shutdown_connections 2024-09-17T05:01:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 --2- 172.21.15.16:0/2295424502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2480cf7f0 0x7fd2481621e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 -- 172.21.15.16:0/2295424502 >> 172.21.15.16:0/2295424502 conn(0x7fd2480d1940 msgr2=0x7fd2480d42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 -- 172.21.15.16:0/2295424502 shutdown_connections 2024-09-17T05:01:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7fd24d400640 1 -- 172.21.15.16:0/2295424502 wait complete. 2024-09-17T05:01:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/2899575450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b43f0 0x7f0f840cd080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 -- 172.21.15.16:0/2899575450 >> 172.21.15.16:0/2899575450 conn(0x7f0f8406ce90 msgr2=0x7f0f8406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 -- 172.21.15.16:0/2899575450 shutdown_connections 2024-09-17T05:01:29.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f0f8aa00640 1 -- 172.21.15.16:0/2899575450 wait complete. 2024-09-17T05:01:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7fd24d400640 1 Processor -- start 2024-09-17T05:01:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7fd24d400640 1 -- start start 2024-09-17T05:01:29.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f0f8aa00640 1 Processor -- start 2024-09-17T05:01:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.571+0000 7f8a43200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 0x7f8a4413d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a43200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a440b4c70 0x7f8a44152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a43200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a4415c020 0x7f8a44152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a43200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4416a110 con 0x7f8a44071750 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a43200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44169f90 con 0x7f8a440b4c70 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a43200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4416a290 con 0x7f8a4415c020 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 0x7f8a4413d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 0x7f8a4413d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43544/0 (socket says 172.21.15.16:43544) 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 -- 172.21.15.16:0/3292123737 learned_addr learned my addr 172.21.15.16:0/3292123737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 -- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a4415c020 msgr2=0x7f8a44152f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a4415c020 0x7f8a44152f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 -- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a440b4c70 msgr2=0x7f8a44152a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a440b4c70 0x7f8a44152a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 -- 172.21.15.16:0/3292123737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a44140fa0 con 0x7f8a44071750 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a41e00640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 0x7f8a4413d710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c01af50 tx=0x7f8a3c0a55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0cc070 con 0x7f8a44071750 2024-09-17T05:01:29.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a3c004040 con 0x7f8a44071750 2024-09-17T05:01:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0cc070 con 0x7f8a44071750 2024-09-17T05:01:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a441537f0 con 0x7f8a44071750 2024-09-17T05:01:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a441409e0 con 0x7f8a44071750 2024-09-17T05:01:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.575+0000 7f0f8aa00640 1 -- start start 2024-09-17T05:01:29.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a44071210 con 0x7f8a44071750 2024-09-17T05:01:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2480cf7f0 0x7fd248205f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2480cfc00 0x7fd2480c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd248162720 0x7fd2480cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd247e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2480cf7f0 0x7fd248205f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd248155230 con 0x7fd2480cf7f0 2024-09-17T05:01:29.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481550b0 con 0x7fd248162720 2024-09-17T05:01:29.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f0f8aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8408b4a0 0x7f0f84207eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f0f8aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 0x7f0f8408a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd2481553b0 con 0x7fd2480cfc00 2024-09-17T05:01:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f0f8aa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f840b43f0 0x7f0f8408aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f0f8aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406ff50 con 0x7f0f8408b4a0 2024-09-17T05:01:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f0f8aa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406fdd0 con 0x7f0f840b3fe0 2024-09-17T05:01:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7f0f8aa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0f84176e30 con 0x7f0f840b43f0 2024-09-17T05:01:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd247e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2480cf7f0 0x7fd248205f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43548/0 (socket says 172.21.15.16:43548) 2024-09-17T05:01:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd247e00640 1 -- 172.21.15.16:0/2450853793 learned_addr learned my addr 172.21.15.16:0/2450853793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd24ca00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd248162720 0x7fd2480cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.579+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2480cfc00 0x7fd2480c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f8a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f840b43f0 0x7f0f8408aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a3c004230 con 0x7f8a44071750 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 0x7f0f8408a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 0x7f0f8408a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:44060/0 (socket says 172.21.15.16:44060) 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 -- 172.21.15.16:0/1845209204 learned_addr learned my addr 172.21.15.16:0/1845209204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f8a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f840b43f0 0x7f0f8408aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49654/0 (socket says 172.21.15.16:49654) 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f89600640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8408b4a0 0x7f0f84207eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 -- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f840b43f0 msgr2=0x7f0f8408aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f840b43f0 0x7f0f8408aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 -- 172.21.15.16:0/1845209204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8408b4a0 msgr2=0x7f0f84207eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8408b4a0 0x7f0f84207eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 -- 172.21.15.16:0/1845209204 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f840ca220 con 0x7f0f840b3fe0 2024-09-17T05:01:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f8a3a000640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a24073430 0x7f8a24075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f88c00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 0x7f0f8408a540 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c420 tx=0x7f0f7800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd23fe00640 1 -- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd248162720 msgr2=0x7fd2480cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd248162720 0x7fd2480cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd23fe00640 1 -- 172.21.15.16:0/2450853793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2480cf7f0 msgr2=0x7fd248205f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2480cf7f0 0x7fd248205f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd23fe00640 1 -- 172.21.15.16:0/2450853793 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd248158060 con 0x7fd2480cfc00 2024-09-17T05:01:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78017070 con 0x7f0f840b3fe0 2024-09-17T05:01:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f780053c0 con 0x7f0f840b3fe0 2024-09-17T05:01:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8a3c13d0e0 con 0x7f8a44071750 2024-09-17T05:01:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78003860 con 0x7f0f840b3fe0 2024-09-17T05:01:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2480cfc00 0x7fd2480c6e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c3c0 tx=0x7fd23000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f8a41400640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a24073430 0x7f8a24075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230017070 con 0x7fd2480cfc00 2024-09-17T05:01:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f840ca500 con 0x7f0f840b3fe0 2024-09-17T05:01:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2300053c0 con 0x7fd2480cfc00 2024-09-17T05:01:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230003860 con 0x7fd2480cfc00 2024-09-17T05:01:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f840caac0 con 0x7f0f840b3fe0 2024-09-17T05:01:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2480c73c0 con 0x7fd2480cfc00 2024-09-17T05:01:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.583+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd248158600 con 0x7fd2480cfc00 2024-09-17T05:01:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.587+0000 7f8a41400640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a24073430 0x7f8a24075940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006fd0 tx=0x7f8a30006b40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.587+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84168160 con 0x7f0f840b3fe0 2024-09-17T05:01:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.587+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0f7801a070 con 0x7f0f840b3fe0 2024-09-17T05:01:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.587+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd23001a070 con 0x7fd2480cfc00 2024-09-17T05:01:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7f0f81600640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f64077640 0x7f0f64079b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2480b9080 con 0x7fd2480cfc00 2024-09-17T05:01:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0f780938b0 con 0x7f0f840b3fe0 2024-09-17T05:01:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7f0f89600640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f64077640 0x7f0f64079b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7f0f89600640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f64077640 0x7f0f64079b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f70001630 tx=0x7f0f70009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7fd244c00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd224073250 0x7fd224075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd2300937c0 con 0x7fd2480cfc00 2024-09-17T05:01:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.591+0000 7fd247e00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd224073250 0x7fd224075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.595+0000 7fd247e00640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd224073250 0x7fd224075760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd238009230 tx=0x7fd23801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.595+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8a44071b60 con 0x7f8a44071750 2024-09-17T05:01:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.595+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0f7805dea0 con 0x7f0f840b3fe0 2024-09-17T05:01:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.595+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd248162b30 con 0x7fd2480cfc00 2024-09-17T05:01:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 -- 172.21.15.16:0/2493333852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 msgr2=0x7f082415b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 --2- 172.21.15.16:0/2493333852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 0x7f082415b4a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0824071ce0 tx=0x7f081401c770 comp rx=0 tx=0).stop 2024-09-17T05:01:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 -- 172.21.15.16:0/2493333852 shutdown_connections 2024-09-17T05:01:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 --2- 172.21.15.16:0/2493333852 >> v2:172.21.15.16:3300/0 conn(0x7f08240cbbf0 0x7f08240cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 --2- 172.21.15.16:0/2493333852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 0x7f082415b4a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 -- 172.21.15.16:0/2493333852 >> 172.21.15.16:0/2493333852 conn(0x7f08240ccae0 msgr2=0x7f08240ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 -- 172.21.15.16:0/2493333852 shutdown_connections 2024-09-17T05:01:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 -- 172.21.15.16:0/2493333852 wait complete. 2024-09-17T05:01:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 Processor -- start 2024-09-17T05:01:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.603+0000 7f082a800640 1 -- start start 2024-09-17T05:01:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f082a800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 0x7f0824154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f082a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08240cbbf0 0x7f0824157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f082a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 0x7f0824158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f082a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08240cecf0 con 0x7f08240cbbf0 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f082a800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08240ceb70 con 0x7f082415fb20 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f082a800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f08240cee70 con 0x7f08240ca6b0 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 0x7f0824154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 0x7f0824154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:49660/0 (socket says 172.21.15.16:49660) 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 -- 172.21.15.16:0/3798441 learned_addr learned my addr 172.21.15.16:0/3798441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823e00640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 0x7f0824158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0822a00640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08240cbbf0 0x7f0824157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 -- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 msgr2=0x7f0824158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 0x7f0824158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 -- 172.21.15.16:0/3798441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08240cbbf0 msgr2=0x7f0824157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08240cbbf0 0x7f0824157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 -- 172.21.15.16:0/3798441 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08240b5ee0 con 0x7f08240ca6b0 2024-09-17T05:01:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f0823400640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 0x7f0824154c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f081401cc50 tx=0x7f08140a5630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.607+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140b2040 con 0x7f08240ca6b0 2024-09-17T05:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.611+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08140ba030 con 0x7f08240ca6b0 2024-09-17T05:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.611+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0824164ab0 con 0x7f08240ca6b0 2024-09-17T05:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.611+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140b6850 con 0x7f08240ca6b0 2024-09-17T05:01:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.611+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0824165070 con 0x7f08240ca6b0 2024-09-17T05:01:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.611+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08240cc000 con 0x7f08240ca6b0 2024-09-17T05:01:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.611+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08140b69f0 con 0x7f08240ca6b0 2024-09-17T05:01:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.615+0000 7f081b400640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0804073300 0x7f0804075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.615+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f08141328f0 con 0x7f08240ca6b0 2024-09-17T05:01:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.615+0000 7f0822a00640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0804073300 0x7f0804075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.615+0000 7f0822a00640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0804073300 0x7f0804075810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f08241f99f0 tx=0x7f08100091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:29.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.615+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08140c4030 con 0x7f08240ca6b0 2024-09-17T05:01:29.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f960c0cd1b0 con 0x7f960c0cff90 2024-09-17T05:01:29.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9609600640 1 -- 172.21.15.16:0/3854949986 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95f805f410 con 0x7f960c0cff90 2024-09-17T05:01:29.743 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f95ec073200 msgr2=0x7f95ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f95ec073200 0x7f95ec075710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96000099d0 tx=0x7f96000091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 msgr2=0x7f960c140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 0x7f960c140ae0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c400 tx=0x7f95f800c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 shutdown_connections 2024-09-17T05:01:29.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f95ec073200 0x7f95ec075710 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96000099d0 tx=0x7f96000091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f960c144e50 0x7f960c159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f960c0cff90 0x7f960c140ae0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 --2- 172.21.15.16:0/3854949986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f960c06efb0 0x7f960c1fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.743+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 >> 172.21.15.16:0/3854949986 conn(0x7f960c073ca0 msgr2=0x7f960c15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.747+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 shutdown_connections 2024-09-17T05:01:29.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.747+0000 7f9613000640 1 -- 172.21.15.16:0/3854949986 wait complete. 2024-09-17T05:01:29.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-17T05:01:29.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:01:29.772 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:01:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.883+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8a440b14f0 con 0x7f8a44071750 2024-09-17T05:01:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.883+0000 7f8a3a000640 1 -- 172.21.15.16:0/3292123737 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a3c109be0 con 0x7f8a44071750 2024-09-17T05:01:29.885 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-17T05:01:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a24073430 msgr2=0x7f8a24075940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a24073430 0x7f8a24075940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8a30006fd0 tx=0x7f8a30006b40 comp rx=0 tx=0).stop 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 msgr2=0x7f8a4413d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 0x7f8a4413d710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c01af50 tx=0x7f8a3c0a55e0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 shutdown_connections 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a24073430 0x7f8a24075940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a4415c020 0x7f8a44152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a440b4c70 0x7f8a44152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 --2- 172.21.15.16:0/3292123737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a44071750 0x7f8a4413d710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 >> 172.21.15.16:0/3292123737 conn(0x7f8a440d10f0 msgr2=0x7f8a4415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 shutdown_connections 2024-09-17T05:01:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.887+0000 7f8a43200640 1 -- 172.21.15.16:0/3292123737 wait complete. 2024-09-17T05:01:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.891+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0f840b4c20 con 0x7f0f840b3fe0 2024-09-17T05:01:29.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.891+0000 7f0f81600640 1 -- 172.21.15.16:0/1845209204 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0f78091e20 con 0x7f0f840b3fe0 2024-09-17T05:01:29.893 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-17T05:01:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f64077640 msgr2=0x7f0f64079b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f64077640 0x7f0f64079b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f70001630 tx=0x7f0f70009040 comp rx=0 tx=0).stop 2024-09-17T05:01:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 msgr2=0x7f0f8408a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 0x7f0f8408a540 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c420 tx=0x7f0f7800c8e0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 shutdown_connections 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f64077640 0x7f0f64079b50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f840b43f0 0x7f0f8408aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f840b3fe0 0x7f0f8408a540 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 --2- 172.21.15.16:0/1845209204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f8408b4a0 0x7f0f84207eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 >> 172.21.15.16:0/1845209204 conn(0x7f0f8406ce90 msgr2=0x7f0f8406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 shutdown_connections 2024-09-17T05:01:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.895+0000 7f0f8aa00640 1 -- 172.21.15.16:0/1845209204 wait complete. 2024-09-17T05:01:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.899+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f20ec0b2100 con 0x7f20ec159f70 2024-09-17T05:01:29.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.899+0000 7f20e3400640 1 -- 172.21.15.16:0/980878116 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f20e405e6a0 con 0x7f20ec159f70 2024-09-17T05:01:29.900 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:29.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20cc073230 msgr2=0x7f20cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20cc073230 0x7f20cc075740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20d80af000 tx=0x7f20d801aef0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 msgr2=0x7f20ec1fcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec1fcac0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b9c0 tx=0x7f20e400be80 comp rx=0 tx=0).stop 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 shutdown_connections 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20cc073230 0x7f20cc075740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec14c320 0x7f20ec0d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20ec159f70 0x7f20ec1fcac0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 --2- 172.21.15.16:0/980878116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f20ec0d02d0 0x7f20ec1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 >> 172.21.15.16:0/980878116 conn(0x7f20ec0b42d0 msgr2=0x7f20ec0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 shutdown_connections 2024-09-17T05:01:29.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.903+0000 7f20f2600640 1 -- 172.21.15.16:0/980878116 wait complete. 2024-09-17T05:01:29.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-17T05:01:29.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2024-09-17T05:01:29.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2024-09-17T05:01:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2e681521b0 con 0x7f2e681e3680 2024-09-17T05:01:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e5e000640 1 -- 172.21.15.16:0/2601090242 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e681e3a90 con 0x7f2e681e3680 2024-09-17T05:01:29.942 INFO:teuthology.orchestra.run.smithi016.stdout:64424509447 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e48073200 msgr2=0x7f2e48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e48073200 0x7f2e48075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2e54001250 tx=0x7f2e54001590 comp rx=0 tx=0).stop 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 msgr2=0x7f2e681e1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 0x7f2e681e1de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2e68074b30 tx=0x7f2e6000fed0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 shutdown_connections 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e48073200 0x7f2e48075710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e681e3680 0x7f2e681e1de0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e68150bd0 0x7f2e681e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 --2- 172.21.15.16:0/2601090242 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e6806dfa0 0x7f2e68144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 >> 172.21.15.16:0/2601090242 conn(0x7f2e680b2dd0 msgr2=0x7f2e680b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 shutdown_connections 2024-09-17T05:01:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.943+0000 7f2e6ce00640 1 -- 172.21.15.16:0/2601090242 wait complete. 2024-09-17T05:01:29.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-09-17T05:01:29.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.967+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd2480b8da0 con 0x7fd2480cfc00 2024-09-17T05:01:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.967+0000 7fd244c00640 1 -- 172.21.15.16:0/2450853793 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd23005e010 con 0x7fd2480cfc00 2024-09-17T05:01:29.968 INFO:teuthology.orchestra.run.smithi016.stdout:64424509446 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd224073250 msgr2=0x7fd224075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd224073250 0x7fd224075760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd238009230 tx=0x7fd23801b120 comp rx=0 tx=0).stop 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2480cfc00 msgr2=0x7fd2480c6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2480cfc00 0x7fd2480c6e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd23000c3c0 tx=0x7fd23000c880 comp rx=0 tx=0).stop 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 shutdown_connections 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd224073250 0x7fd224075760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd248162720 0x7fd2480cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2480cfc00 0x7fd2480c6e80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 --2- 172.21.15.16:0/2450853793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2480cf7f0 0x7fd248205f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 >> 172.21.15.16:0/2450853793 conn(0x7fd2480d1940 msgr2=0x7fd2480c8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 shutdown_connections 2024-09-17T05:01:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.971+0000 7fd24d400640 1 -- 172.21.15.16:0/2450853793 wait complete. 2024-09-17T05:01:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.975+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f082406d060 con 0x7f08240ca6b0 2024-09-17T05:01:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f081b400640 1 -- 172.21.15.16:0/3798441 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08140164b0 con 0x7f08240ca6b0 2024-09-17T05:01:29.978 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2024-09-17T05:01:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0804073300 msgr2=0x7f0804075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0804073300 0x7f0804075810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f08241f99f0 tx=0x7f08100091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 msgr2=0x7f0824154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 0x7f0824154c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f081401cc50 tx=0x7f08140a5630 comp rx=0 tx=0).stop 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 shutdown_connections 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0804073300 0x7f0804075810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f082415fb20 0x7f0824158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08240cbbf0 0x7f0824157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 --2- 172.21.15.16:0/3798441 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08240ca6b0 0x7f0824154c10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 >> 172.21.15.16:0/3798441 conn(0x7f08240ccae0 msgr2=0x7f082415a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 shutdown_connections 2024-09-17T05:01:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:29.979+0000 7f082a800640 1 -- 172.21.15.16:0/3798441 wait complete. 2024-09-17T05:01:29.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-17T05:01:30.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-09-17T05:01:30.002 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:30.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T05:01:30.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T05:01:30.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T05:01:30.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 -- 172.21.15.16:0/4035362341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 msgr2=0x7fcd080b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 --2- 172.21.15.16:0/4035362341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd080b2100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009c80 tx=0x7fcd0001cc10 comp rx=0 tx=0).stop 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 -- 172.21.15.16:0/4035362341 shutdown_connections 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 --2- 172.21.15.16:0/4035362341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd080b2100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 --2- 172.21.15.16:0/4035362341 >> v2:172.21.15.104:3300/0 conn(0x7fcd080d02d0 0x7fcd080cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 -- 172.21.15.16:0/4035362341 >> 172.21.15.16:0/4035362341 conn(0x7fcd080b42d0 msgr2=0x7fcd080b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 -- 172.21.15.16:0/4035362341 shutdown_connections 2024-09-17T05:01:30.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 -- 172.21.15.16:0/4035362341 wait complete. 2024-09-17T05:01:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.943+0000 7fcd0f400640 1 Processor -- start 2024-09-17T05:01:30.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 -- start start 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd0806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd080d02d0 0x7fcd081f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd08159f70 0x7fcd080cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08169e20 con 0x7fcd080cf680 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08169ca0 con 0x7fcd08159f70 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcd08169fa0 con 0x7fcd080d02d0 2024-09-17T05:01:30.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd0806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd0806dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43606/0 (socket says 172.21.15.16:43606) 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 -- 172.21.15.16:0/3292844049 learned_addr learned my addr 172.21.15.16:0/3292844049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0d400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd08159f70 0x7fcd080cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fccffe00640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd080d02d0 0x7fcd081f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 -- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd080d02d0 msgr2=0x7fcd081f8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd080d02d0 0x7fcd081f8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 -- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd08159f70 msgr2=0x7fcd080cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd08159f70 0x7fcd080cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 -- 172.21.15.16:0/3292844049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd081f90c0 con 0x7fcd080cf680 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0ca00640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd0806dfa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c3c0 tx=0x7fccf800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8017070 con 0x7fcd080cf680 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccf80053c0 con 0x7fcd080cf680 2024-09-17T05:01:30.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd081f93a0 con 0x7fcd080cf680 2024-09-17T05:01:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.947+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd081fc0e0 con 0x7fcd080cf680 2024-09-17T05:01:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.951+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8003860 con 0x7fcd080cf680 2024-09-17T05:01:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.951+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce00052a0 con 0x7fcd080cf680 2024-09-17T05:01:30.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.951+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fccf801a070 con 0x7fcd080cf680 2024-09-17T05:01:30.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.951+0000 7fcd05600640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcce4073200 0x7fcce4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:30.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.951+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fccf8093790 con 0x7fcd080cf680 2024-09-17T05:01:30.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.955+0000 7fccffe00640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcce4073200 0x7fcce4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:30.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.955+0000 7fccffe00640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcce4073200 0x7fcce4075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fccf00058d0 tx=0x7fccf0005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:30.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:30.959+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd080cfa90 con 0x7fcd080cf680 2024-09-17T05:01:30.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T05:01:30.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T05:01:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.099+0000 7f2e95400640 1 -- 172.21.15.16:0/2980363643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 msgr2=0x7f2e900b5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.099+0000 7f2e95400640 1 --2- 172.21.15.16:0/2980363643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900b5b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2e88009c80 tx=0x7f2e8801cd10 comp rx=0 tx=0).stop 2024-09-17T05:01:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 -- 172.21.15.16:0/2980363643 shutdown_connections 2024-09-17T05:01:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 --2- 172.21.15.16:0/2980363643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900b5b40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 --2- 172.21.15.16:0/2980363643 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 0x7f2e90155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 -- 172.21.15.16:0/2980363643 >> 172.21.15.16:0/2980363643 conn(0x7f2e90073ca0 msgr2=0x7f2e900740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 -- 172.21.15.16:0/2980363643 shutdown_connections 2024-09-17T05:01:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 -- 172.21.15.16:0/2980363643 wait complete. 2024-09-17T05:01:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 Processor -- start 2024-09-17T05:01:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.103+0000 7f2e95400640 1 -- start start 2024-09-17T05:01:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e9006efb0 0x7f2e90158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 0x7f2e900aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2e900712e0 con 0x7f2e900b5730 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90071160 con 0x7f2e9006efb0 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9016c300 con 0x7f2e90162b90 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e8f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 0x7f2e900aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e8f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43632/0 (socket says 172.21.15.16:43632) 2024-09-17T05:01:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e8f400640 1 -- 172.21.15.16:0/1513882987 learned_addr learned my addr 172.21.15.16:0/1513882987 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e8fe00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e9006efb0 0x7f2e90158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 -- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e9006efb0 msgr2=0x7f2e90158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e9006efb0 0x7f2e90158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 -- 172.21.15.16:0/1513882987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 msgr2=0x7f2e900ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 -- 172.21.15.16:0/1513882987 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e90151ed0 con 0x7f2e90162b90 2024-09-17T05:01:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e94a00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 0x7f2e900aeef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e880ac000 tx=0x7f2e88005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e880b2040 con 0x7f2e90162b90 2024-09-17T05:01:31.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e88004130 con 0x7f2e90162b90 2024-09-17T05:01:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88004540 con 0x7f2e90162b90 2024-09-17T05:01:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e901594e0 con 0x7f2e90162b90 2024-09-17T05:01:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.107+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e90152450 con 0x7f2e90162b90 2024-09-17T05:01:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.111+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e900b5b40 con 0x7f2e90162b90 2024-09-17T05:01:31.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.111+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2e880bd060 con 0x7f2e90162b90 2024-09-17T05:01:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.115+0000 7f2e87e00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e70073200 0x7f2e70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.115+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e8812e190 con 0x7f2e90162b90 2024-09-17T05:01:31.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.115+0000 7f2e8fe00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e70073200 0x7f2e70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.115+0000 7f2e8fe00640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e70073200 0x7f2e70075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009600 tx=0x7f2e800091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.119+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2e88016990 con 0x7f2e90162b90 2024-09-17T05:01:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 -- 172.21.15.16:0/2723554783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6380cc680 msgr2=0x7fc6380cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 --2- 172.21.15.16:0/2723554783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6380cc680 0x7fc6380cca90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc63815f0c0 tx=0x7fc63001caa0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 -- 172.21.15.16:0/2723554783 shutdown_connections 2024-09-17T05:01:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 --2- 172.21.15.16:0/2723554783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6380cc680 0x7fc6380cca90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 --2- 172.21.15.16:0/2723554783 >> v2:172.21.15.104:3300/0 conn(0x7fc63815c460 0x7fc6381443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 -- 172.21.15.16:0/2723554783 >> 172.21.15.16:0/2723554783 conn(0x7fc6380cdb10 msgr2=0x7fc6380cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 -- 172.21.15.16:0/2723554783 shutdown_connections 2024-09-17T05:01:31.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 -- 172.21.15.16:0/2723554783 wait complete. 2024-09-17T05:01:31.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 Processor -- start 2024-09-17T05:01:31.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.139+0000 7fc637200640 1 -- start start 2024-09-17T05:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc638071210 0x7fc63815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6380cc680 0x7fc63813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 0x7fc638164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc63806d580 con 0x7fc63815c460 2024-09-17T05:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc63806d400 con 0x7fc638071210 2024-09-17T05:01:31.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc63806d700 con 0x7fc6380cc680 2024-09-17T05:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 0x7fc638164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 0x7fc638164540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43662/0 (socket says 172.21.15.16:43662) 2024-09-17T05:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 -- 172.21.15.16:0/474311933 learned_addr learned my addr 172.21.15.16:0/474311933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc635e00640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc638071210 0x7fc63815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc635400640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6380cc680 0x7fc63813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 -- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6380cc680 msgr2=0x7fc63813da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6380cc680 0x7fc63813da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 -- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc638071210 msgr2=0x7fc63815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc638071210 0x7fc63815a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 -- 172.21.15.16:0/474311933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63813df50 con 0x7fc63815c460 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc636800640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 0x7fc638164540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc63001c510 tx=0x7fc630003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6300b2040 con 0x7fc63815c460 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6300bb070 con 0x7fc63815c460 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630004330 con 0x7fc63815c460 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc638141950 con 0x7fc63815c460 2024-09-17T05:01:31.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.143+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc638141f10 con 0x7fc63815c460 2024-09-17T05:01:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.147+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6380cca90 con 0x7fc63815c460 2024-09-17T05:01:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.147+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc6380cd7e0 con 0x7fc63815c460 2024-09-17T05:01:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.147+0000 7fc62e000640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc618073280 0x7fc618075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.147+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc63012f6b0 con 0x7fc63815c460 2024-09-17T05:01:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.147+0000 7fc635e00640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc618073280 0x7fc618075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.151+0000 7fc635e00640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc618073280 0x7fc618075790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc628009a10 tx=0x7fc6280091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.155+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc630016750 con 0x7fc63815c460 2024-09-17T05:01:31.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1416886698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d00cc6e0 msgr2=0x7fa1d00ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1416886698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d00cc6e0 0x7fa1d00ccaf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4005fd0 tx=0x7fa1c401c760 comp rx=0 tx=0).stop 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1416886698 shutdown_connections 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1416886698 >> v2:172.21.15.104:3300/0 conn(0x7fa1d0164020 0x7fa1d0166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1416886698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d00cc6e0 0x7fa1d00ccaf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1416886698 >> 172.21.15.16:0/1416886698 conn(0x7fa1d00cef10 msgr2=0x7fa1d00cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1416886698 shutdown_connections 2024-09-17T05:01:31.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.199+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1416886698 wait complete. 2024-09-17T05:01:31.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.203+0000 7fa1d5e00640 1 Processor -- start 2024-09-17T05:01:31.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 -- start start 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 0x7fa1d0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d0160000 0x7fa1d0160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0164020 0x7fa1d01614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0169560 con 0x7fa1d0164020 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01693e0 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d01696e0 con 0x7fa1d0160000 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 0x7fa1d0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 0x7fa1d0155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:44144/0 (socket says 172.21.15.16:44144) 2024-09-17T05:01:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d4a00640 1 -- 172.21.15.16:0/1384027873 learned_addr learned my addr 172.21.15.16:0/1384027873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1cfe00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d0160000 0x7fa1d0160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d5400640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0164020 0x7fa1d01614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d4a00640 1 -- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d0160000 msgr2=0x7fa1d0160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d4a00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d0160000 0x7fa1d0160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.207+0000 7fa1d4a00640 1 -- 172.21.15.16:0/1384027873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0164020 msgr2=0x7fa1d01614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1d4a00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0164020 0x7fa1d01614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1d4a00640 1 -- 172.21.15.16:0/1384027873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d0162fb0 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1d5400640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0164020 0x7fa1d01614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:31.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1d4a00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 0x7fa1d0155e50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009790 tx=0x7fa1c4005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c40bb020 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1c40047c0 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4004bd0 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d0166290 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.211+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01627b0 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.215+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d00cf570 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.215+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1c40be020 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa1ccc00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1b0073230 0x7fa1b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa1c412ef60 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa1cfe00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1b0073230 0x7fa1b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 -- 172.21.15.16:0/2801376214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460155840 msgr2=0x7fa4600b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 --2- 172.21.15.16:0/2801376214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460155840 0x7fa4600b58b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa458009bc0 tx=0x7fa45801cb30 comp rx=0 tx=0).stop 2024-09-17T05:01:31.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 -- 172.21.15.16:0/2801376214 shutdown_connections 2024-09-17T05:01:31.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 --2- 172.21.15.16:0/2801376214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460155840 0x7fa4600b58b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 -- 172.21.15.16:0/2801376214 >> 172.21.15.16:0/2801376214 conn(0x7fa460073ca0 msgr2=0x7fa4600740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 -- 172.21.15.16:0/2801376214 shutdown_connections 2024-09-17T05:01:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 -- 172.21.15.16:0/2801376214 wait complete. 2024-09-17T05:01:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa1cfe00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1b0073230 0x7fa1b0075740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00b8030 tx=0x7fa1bc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.219+0000 7fa45fe00640 1 Processor -- start 2024-09-17T05:01:31.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa1c40c5030 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 -- start start 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 0x7fa46016c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa460155840 0x7fa46014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460162b70 0x7fa46014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4601716b0 con 0x7fa460162b70 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa460171530 con 0x7fa4600b54a0 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa460171830 con 0x7fa460155840 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 0x7fa46016c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 0x7fa46016c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:44158/0 (socket says 172.21.15.16:44158) 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 -- 172.21.15.16:0/1078372628 learned_addr learned my addr 172.21.15.16:0/1078372628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45f400640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460162b70 0x7fa46014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45e000640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa460155840 0x7fa46014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 -- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa460155840 msgr2=0x7fa46014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa460155840 0x7fa46014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 -- 172.21.15.16:0/1078372628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460162b70 msgr2=0x7fa46014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460162b70 0x7fa46014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 -- 172.21.15.16:0/1078372628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa460075040 con 0x7fa4600b54a0 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45f400640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460162b70 0x7fa46014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45ea00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 0x7fa46016c830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa45000c410 tx=0x7fa45000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450011790 con 0x7fa4600b54a0 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa450017070 con 0x7fa4600b54a0 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450011c60 con 0x7fa4600b54a0 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.223+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa460162060 con 0x7fa4600b54a0 2024-09-17T05:01:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.227+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa460154f60 con 0x7fa4600b54a0 2024-09-17T05:01:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.227+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4600b58b0 con 0x7fa4600b54a0 2024-09-17T05:01:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.227+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa45001c460 con 0x7fa4600b54a0 2024-09-17T05:01:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.227+0000 7fa457400640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa440073200 0x7fa440075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.231+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa450094690 con 0x7fa4600b54a0 2024-09-17T05:01:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.231+0000 7fa45e000640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa440073200 0x7fa440075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.231+0000 7fa45e000640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa440073200 0x7fa440075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa460160890 tx=0x7fa44c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.231+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa45005f3e0 con 0x7fa4600b54a0 2024-09-17T05:01:31.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.259+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcce0005590 con 0x7fcd080cf680 2024-09-17T05:01:31.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.259+0000 7fcd05600640 1 -- 172.21.15.16:0/3292844049 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fccf805e620 con 0x7fcd080cf680 2024-09-17T05:01:31.261 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2024-09-17T05:01:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.259+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcce4073200 msgr2=0x7fcce4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.259+0000 7fcd0f400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcce4073200 0x7fcce4075710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fccf00058d0 tx=0x7fccf0005860 comp rx=0 tx=0).stop 2024-09-17T05:01:31.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.259+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 msgr2=0x7fcd0806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.259+0000 7fcd0f400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd0806dfa0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c3c0 tx=0x7fccf800c880 comp rx=0 tx=0).stop 2024-09-17T05:01:31.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 shutdown_connections 2024-09-17T05:01:31.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcce4073200 0x7fcce4075710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd08159f70 0x7fcd080cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd080d02d0 0x7fcd081f8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 --2- 172.21.15.16:0/3292844049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd080cf680 0x7fcd0806dfa0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 >> 172.21.15.16:0/3292844049 conn(0x7fcd080b42d0 msgr2=0x7fcd08071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 shutdown_connections 2024-09-17T05:01:31.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.263+0000 7fcd0f400640 1 -- 172.21.15.16:0/3292844049 wait complete. 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 -- 172.21.15.16:0/117847899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c145050 msgr2=0x7f2b1c145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 --2- 172.21.15.16:0/117847899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c145050 0x7f2b1c145460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c06dea0 tx=0x7f2b1401cba0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 -- 172.21.15.16:0/117847899 shutdown_connections 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 --2- 172.21.15.16:0/117847899 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c145050 0x7f2b1c145460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 --2- 172.21.15.16:0/117847899 >> v2:172.21.15.16:3300/0 conn(0x7f2b1c160060 0x7f2b1c160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 --2- 172.21.15.16:0/117847899 >> v2:172.21.15.104:3300/0 conn(0x7f2b1c06efb0 0x7f2b1c06f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 -- 172.21.15.16:0/117847899 >> 172.21.15.16:0/117847899 conn(0x7f2b1c073ca0 msgr2=0x7f2b1c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 -- 172.21.15.16:0/117847899 shutdown_connections 2024-09-17T05:01:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.283+0000 7f2b1be00640 1 -- 172.21.15.16:0/117847899 wait complete. 2024-09-17T05:01:31.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.7 2024-09-17T05:01:31.286 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:31.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 Processor -- start 2024-09-17T05:01:31.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 -- start start 2024-09-17T05:01:31.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c06efb0 0x7f2b1c142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c145050 0x7f2b1c0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 0x7f2b1c1f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c0cfb10 con 0x7f2b1c160060 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c06dcf0 con 0x7f2b1c145050 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1be00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1c0cfc90 con 0x7f2b1c06efb0 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 0x7f2b1c1f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 0x7f2b1c1f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43706/0 (socket says 172.21.15.16:43706) 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 -- 172.21.15.16:0/3562856311 learned_addr learned my addr 172.21.15.16:0/3562856311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1aa00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c06efb0 0x7f2b1c142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1a000640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c145050 0x7f2b1c0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 -- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c06efb0 msgr2=0x7f2b1c142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c06efb0 0x7f2b1c142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 -- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c145050 msgr2=0x7f2b1c0b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c145050 0x7f2b1c0b5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.287+0000 7f2b1b400640 1 -- 172.21.15.16:0/3562856311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c157050 con 0x7f2b1c160060 2024-09-17T05:01:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b1b400640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 0x7f2b1c1f8270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009b10 tx=0x7f2b14009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140bc070 con 0x7f2b1c160060 2024-09-17T05:01:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b140042f0 con 0x7f2b1c160060 2024-09-17T05:01:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c157330 con 0x7f2b1c160060 2024-09-17T05:01:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140b2040 con 0x7f2b1c160060 2024-09-17T05:01:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c0b4c70 con 0x7f2b1c160060 2024-09-17T05:01:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.291+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1c145460 con 0x7f2b1c160060 2024-09-17T05:01:31.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.295+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b140bf070 con 0x7f2b1c160060 2024-09-17T05:01:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.295+0000 7f2b12a00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2afc073200 0x7f2afc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.295+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2b1412e330 con 0x7f2b1c160060 2024-09-17T05:01:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.295+0000 7f2b1aa00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2afc073200 0x7f2afc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.295+0000 7f2b1aa00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2afc073200 0x7f2afc075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00bb50 tx=0x7f2b0c00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.299+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b14016790 con 0x7f2b1c160060 2024-09-17T05:01:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.427+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2e90152f30 con 0x7f2e90162b90 2024-09-17T05:01:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.427+0000 7f2e87e00640 1 -- 172.21.15.16:0/1513882987 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e880f8810 con 0x7f2e90162b90 2024-09-17T05:01:31.428 INFO:teuthology.orchestra.run.smithi016.stdout:64424509450 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e70073200 msgr2=0x7f2e70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e70073200 0x7f2e70075710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009600 tx=0x7f2e800091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 msgr2=0x7f2e900aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 0x7f2e900aeef0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2e880ac000 tx=0x7f2e88005c20 comp rx=0 tx=0).stop 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 shutdown_connections 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2e70073200 0x7f2e70075710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2e90162b90 0x7f2e900aeef0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e900b5730 0x7f2e900ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 --2- 172.21.15.16:0/1513882987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e9006efb0 0x7f2e90158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 >> 172.21.15.16:0/1513882987 conn(0x7f2e90073ca0 msgr2=0x7f2e900740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 shutdown_connections 2024-09-17T05:01:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.431+0000 7f2e95400640 1 -- 172.21.15.16:0/1513882987 wait complete. 2024-09-17T05:01:31.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509450 for osd.6 2024-09-17T05:01:31.462 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2b1c074150 con 0x7f2b1c160060 2024-09-17T05:01:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b12a00640 1 -- 172.21.15.16:0/3562856311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2b140f8ac0 con 0x7f2b1c160060 2024-09-17T05:01:31.479 INFO:teuthology.orchestra.run.smithi016.stdout:64424509448 2024-09-17T05:01:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2afc073200 msgr2=0x7f2afc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2afc073200 0x7f2afc075710 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00bb50 tx=0x7f2b0c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 msgr2=0x7f2b1c1f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 0x7f2b1c1f8270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009b10 tx=0x7f2b14009260 comp rx=0 tx=0).stop 2024-09-17T05:01:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 shutdown_connections 2024-09-17T05:01:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2afc073200 0x7f2afc075710 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1c160060 0x7f2b1c1f8270 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c145050 0x7f2b1c0b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 --2- 172.21.15.16:0/3562856311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b1c06efb0 0x7f2b1c142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 >> 172.21.15.16:0/3562856311 conn(0x7f2b1c073ca0 msgr2=0x7f2b1c15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 shutdown_connections 2024-09-17T05:01:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.479+0000 7f2b1be00640 1 -- 172.21.15.16:0/3562856311 wait complete. 2024-09-17T05:01:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.491+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc63815dec0 con 0x7fc63815c460 2024-09-17T05:01:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc62e000640 1 -- 172.21.15.16:0/474311933 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc6300fa370 con 0x7fc63815c460 2024-09-17T05:01:31.494 INFO:teuthology.orchestra.run.smithi016.stdout:64424509451 2024-09-17T05:01:31.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc618073280 msgr2=0x7fc618075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc618073280 0x7fc618075790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc628009a10 tx=0x7fc6280091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 msgr2=0x7fc638164540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 0x7fc638164540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc63001c510 tx=0x7fc630003f40 comp rx=0 tx=0).stop 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 shutdown_connections 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc618073280 0x7fc618075790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc63815c460 0x7fc638164540 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6380cc680 0x7fc63813da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 --2- 172.21.15.16:0/474311933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc638071210 0x7fc63815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.495+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 >> 172.21.15.16:0/474311933 conn(0x7fc6380cdb10 msgr2=0x7fc6380cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.499+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 shutdown_connections 2024-09-17T05:01:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.499+0000 7fc637200640 1 -- 172.21.15.16:0/474311933 wait complete. 2024-09-17T05:01:31.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.2 2024-09-17T05:01:31.502 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:31.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.1 2024-09-17T05:01:31.521 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:31.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa46016dbf0 con 0x7fa4600b54a0 2024-09-17T05:01:31.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa457400640 1 -- 172.21.15.16:0/1078372628 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa45005ece0 con 0x7fa4600b54a0 2024-09-17T05:01:31.527 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa440073200 msgr2=0x7fa440075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa440073200 0x7fa440075710 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa460160890 tx=0x7fa44c009040 comp rx=0 tx=0).stop 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 msgr2=0x7fa46016c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 0x7fa46016c830 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa45000c410 tx=0x7fa45000c8d0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 shutdown_connections 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa440073200 0x7fa440075710 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa460162b70 0x7fa46014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa460155840 0x7fa46014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 --2- 172.21.15.16:0/1078372628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4600b54a0 0x7fa46016c830 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 >> 172.21.15.16:0/1078372628 conn(0x7fa460073ca0 msgr2=0x7fa46016e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 shutdown_connections 2024-09-17T05:01:31.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.527+0000 7fa45fe00640 1 -- 172.21.15.16:0/1078372628 wait complete. 2024-09-17T05:01:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa1d0071e90 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1ccc00640 1 -- 172.21.15.16:0/1384027873 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1c4016410 con 0x7fa1d00cc6e0 2024-09-17T05:01:31.547 INFO:teuthology.orchestra.run.smithi016.stdout:64424509449 2024-09-17T05:01:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1b0073230 msgr2=0x7fa1b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1b0073230 0x7fa1b0075740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00b8030 tx=0x7fa1bc009040 comp rx=0 tx=0).stop 2024-09-17T05:01:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 msgr2=0x7fa1d0155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 0x7fa1d0155e50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009790 tx=0x7fa1c4005790 comp rx=0 tx=0).stop 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 shutdown_connections 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1b0073230 0x7fa1b0075740 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d0164020 0x7fa1d01614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1d0160000 0x7fa1d0160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 --2- 172.21.15.16:0/1384027873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1d00cc6e0 0x7fa1d0155e50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 >> 172.21.15.16:0/1384027873 conn(0x7fa1d00cef10 msgr2=0x7fa1d006e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509449 for osd.3 2024-09-17T05:01:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 shutdown_connections 2024-09-17T05:01:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.547+0000 7fa1d5e00640 1 -- 172.21.15.16:0/1384027873 wait complete. 2024-09-17T05:01:31.551 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:31.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509449 for osd.4 2024-09-17T05:01:31.569 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:31.570 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:01:31.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:01:31.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 -- 172.21.15.16:0/824264757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c160880 msgr2=0x7f1e8c167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 --2- 172.21.15.16:0/824264757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c160880 0x7f1e8c167500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0b3050 tx=0x7f1e8401cd40 comp rx=0 tx=0).stop 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 -- 172.21.15.16:0/824264757 shutdown_connections 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 --2- 172.21.15.16:0/824264757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c160880 0x7f1e8c167500 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 --2- 172.21.15.16:0/824264757 >> v2:172.21.15.104:3300/0 conn(0x7f1e8c15d600 0x7f1e8c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 -- 172.21.15.16:0/824264757 >> 172.21.15.16:0/824264757 conn(0x7f1e8c06f900 msgr2=0x7f1e8c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 -- 172.21.15.16:0/824264757 shutdown_connections 2024-09-17T05:01:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.783+0000 7f1e90a00640 1 -- 172.21.15.16:0/824264757 wait complete. 2024-09-17T05:01:31.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 Processor -- start 2024-09-17T05:01:31.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 -- start start 2024-09-17T05:01:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e8c0cf390 0x7f1e8c13ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 0x7f1e8c0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8c160880 0x7f1e8c0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c0b4b40 con 0x7f1e8c15d600 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c0b5280 con 0x7f1e8c160880 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8c16a260 con 0x7f1e8c0cf390 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 0x7f1e8c0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 0x7f1e8c0b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43718/0 (socket says 172.21.15.16:43718) 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 -- 172.21.15.16:0/2481268317 learned_addr learned my addr 172.21.15.16:0/2481268317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e89e00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e8c0cf390 0x7f1e8c13ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e8a800640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8c160880 0x7f1e8c0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 -- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e8c0cf390 msgr2=0x7f1e8c13ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e8c0cf390 0x7f1e8c13ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 -- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8c160880 msgr2=0x7f1e8c0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8c160880 0x7f1e8c0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 -- 172.21.15.16:0/2481268317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c1fc370 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e83e00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 0x7f1e8c0b0ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400b440 tx=0x7f1e7400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74015020 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e74011980 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c1fc650 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74011dd0 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.787+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1fcc10 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.791+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c0cee50 con 0x7f1e8c15d600 2024-09-17T05:01:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.791+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e74018020 con 0x7f1e8c15d600 2024-09-17T05:01:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.791+0000 7f1e82000640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e68073230 0x7f1e68075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.791+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1e740942b0 con 0x7f1e8c15d600 2024-09-17T05:01:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.791+0000 7f1e89e00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e68073230 0x7f1e68075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.791+0000 7f1e89e00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e68073230 0x7f1e68075740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0afb90 tx=0x7f1e7c00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.795+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e7401f030 con 0x7f1e8c15d600 2024-09-17T05:01:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.971+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e8c074cf0 con 0x7f1e68073230 2024-09-17T05:01:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.975+0000 7f1e82000640 1 -- 172.21.15.16:0/2481268317 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f1e8c074cf0 con 0x7f1e68073230 2024-09-17T05:01:31.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e68073230 msgr2=0x7f1e68075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e68073230 0x7f1e68075740 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0afb90 tx=0x7f1e7c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 msgr2=0x7f1e8c0b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 0x7f1e8c0b0ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400b440 tx=0x7f1e7400b900 comp rx=0 tx=0).stop 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 shutdown_connections 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e68073230 0x7f1e68075740 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0afb90 tx=0x7f1e7c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8c160880 0x7f1e8c0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8c15d600 0x7f1e8c0b0ef0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 --2- 172.21.15.16:0/2481268317 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e8c0cf390 0x7f1e8c13ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 >> 172.21.15.16:0/2481268317 conn(0x7f1e8c06f900 msgr2=0x7f1e8c145560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 shutdown_connections 2024-09-17T05:01:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:31.979+0000 7f1e90a00640 1 -- 172.21.15.16:0/2481268317 wait complete. 2024-09-17T05:01:32.002 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-17T05:01:31.466554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236200,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738520,"statfs":{"total":773094113280,"available":772852244480,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003869"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942466+0000","last_change":"2024-09-17T05:01:07.870252+0000","last_active":"2024-09-17T05:01:13.942466+0000","last_peered":"2024-09-17T05:01:13.942466+0000","last_clean":"2024-09-17T05:01:13.942466+0000","last_became_active":"2024-09-17T05:01:07.869832+0000","last_became_peered":"2024-09-17T05:01:07.869832+0000","last_unstale":"2024-09-17T05:01:13.942466+0000","last_undegraded":"2024-09-17T05:01:13.942466+0000","last_fullsized":"2024-09-17T05:01:13.942466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893796+0000","last_change":"2024-09-17T05:01:08.860472+0000","last_active":"2024-09-17T05:01:13.893796+0000","last_peered":"2024-09-17T05:01:13.893796+0000","last_clean":"2024-09-17T05:01:13.893796+0000","last_became_active":"2024-09-17T05:01:08.860121+0000","last_became_peered":"2024-09-17T05:01:08.860121+0000","last_unstale":"2024-09-17T05:01:13.893796+0000","last_undegraded":"2024-09-17T05:01:13.893796+0000","last_fullsized":"2024-09-17T05:01:13.893796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:02.947441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941059+0000","last_change":"2024-09-17T05:01:12.917747+0000","last_active":"2024-09-17T05:01:13.941059+0000","last_peered":"2024-09-17T05:01:13.941059+0000","last_clean":"2024-09-17T05:01:13.941059+0000","last_became_active":"2024-09-17T05:01:12.916952+0000","last_became_peered":"2024-09-17T05:01:12.916952+0000","last_unstale":"2024-09-17T05:01:13.941059+0000","last_undegraded":"2024-09-17T05:01:13.941059+0000","last_fullsized":"2024-09-17T05:01:13.941059+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:00.182691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893999+0000","last_change":"2024-09-17T05:01:07.858483+0000","last_active":"2024-09-17T05:01:13.893999+0000","last_peered":"2024-09-17T05:01:13.893999+0000","last_clean":"2024-09-17T05:01:13.893999+0000","last_became_active":"2024-09-17T05:01:07.858094+0000","last_became_peered":"2024-09-17T05:01:07.858094+0000","last_unstale":"2024-09-17T05:01:13.893999+0000","last_undegraded":"2024-09-17T05:01:13.893999+0000","last_fullsized":"2024-09-17T05:01:13.893999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:09.683336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894263+0000","last_change":"2024-09-17T05:01:08.886624+0000","last_active":"2024-09-17T05:01:13.894263+0000","last_peered":"2024-09-17T05:01:13.894263+0000","last_clean":"2024-09-17T05:01:13.894263+0000","last_became_active":"2024-09-17T05:01:08.886308+0000","last_became_peered":"2024-09-17T05:01:08.886308+0000","last_unstale":"2024-09-17T05:01:13.894263+0000","last_undegraded":"2024-09-17T05:01:13.894263+0000","last_fullsized":"2024-09-17T05:01:13.894263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.002578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941422+0000","last_change":"2024-09-17T05:01:12.901716+0000","last_active":"2024-09-17T05:01:13.941422+0000","last_peered":"2024-09-17T05:01:13.941422+0000","last_clean":"2024-09-17T05:01:13.941422+0000","last_became_active":"2024-09-17T05:01:12.901225+0000","last_became_peered":"2024-09-17T05:01:12.901225+0000","last_unstale":"2024-09-17T05:01:13.941422+0000","last_undegraded":"2024-09-17T05:01:13.941422+0000","last_fullsized":"2024-09-17T05:01:13.941422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:19.972140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942144+0000","last_change":"2024-09-17T05:01:07.857199+0000","last_active":"2024-09-17T05:01:13.942144+0000","last_peered":"2024-09-17T05:01:13.942144+0000","last_clean":"2024-09-17T05:01:13.942144+0000","last_became_active":"2024-09-17T05:01:07.856808+0000","last_became_peered":"2024-09-17T05:01:07.856808+0000","last_unstale":"2024-09-17T05:01:13.942144+0000","last_undegraded":"2024-09-17T05:01:13.942144+0000","last_fullsized":"2024-09-17T05:01:13.942144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943634+0000","last_change":"2024-09-17T05:01:08.880952+0000","last_active":"2024-09-17T05:01:13.943634+0000","last_peered":"2024-09-17T05:01:13.943634+0000","last_clean":"2024-09-17T05:01:13.943634+0000","last_became_active":"2024-09-17T05:01:08.880627+0000","last_became_peered":"2024-09-17T05:01:08.880627+0000","last_unstale":"2024-09-17T05:01:13.943634+0000","last_undegraded":"2024-09-17T05:01:13.943634+0000","last_fullsized":"2024-09-17T05:01:13.943634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:03.625255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943104+0000","last_change":"2024-09-17T05:01:12.903919+0000","last_active":"2024-09-17T05:01:13.943104+0000","last_peered":"2024-09-17T05:01:13.943104+0000","last_clean":"2024-09-17T05:01:13.943104+0000","last_became_active":"2024-09-17T05:01:12.901881+0000","last_became_peered":"2024-09-17T05:01:12.901881+0000","last_unstale":"2024-09-17T05:01:13.943104+0000","last_undegraded":"2024-09-17T05:01:13.943104+0000","last_fullsized":"2024-09-17T05:01:13.943104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.883298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942724+0000","last_change":"2024-09-17T05:01:07.859450+0000","last_active":"2024-09-17T05:01:13.942724+0000","last_peered":"2024-09-17T05:01:13.942724+0000","last_clean":"2024-09-17T05:01:13.942724+0000","last_became_active":"2024-09-17T05:01:07.857315+0000","last_became_peered":"2024-09-17T05:01:07.857315+0000","last_unstale":"2024-09-17T05:01:13.942724+0000","last_undegraded":"2024-09-17T05:01:13.942724+0000","last_fullsized":"2024-09-17T05:01:13.942724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892054+0000","last_change":"2024-09-17T05:01:08.893401+0000","last_active":"2024-09-17T05:01:13.892054+0000","last_peered":"2024-09-17T05:01:13.892054+0000","last_clean":"2024-09-17T05:01:13.892054+0000","last_became_active":"2024-09-17T05:01:08.892406+0000","last_became_peered":"2024-09-17T05:01:08.892406+0000","last_unstale":"2024-09-17T05:01:13.892054+0000","last_undegraded":"2024-09-17T05:01:13.892054+0000","last_fullsized":"2024-09-17T05:01:13.892054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:18.887291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942085+0000","last_change":"2024-09-17T05:01:12.918843+0000","last_active":"2024-09-17T05:01:13.942085+0000","last_peered":"2024-09-17T05:01:13.942085+0000","last_clean":"2024-09-17T05:01:13.942085+0000","last_became_active":"2024-09-17T05:01:12.916474+0000","last_became_peered":"2024-09-17T05:01:12.916474+0000","last_unstale":"2024-09-17T05:01:13.942085+0000","last_undegraded":"2024-09-17T05:01:13.942085+0000","last_fullsized":"2024-09-17T05:01:13.942085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:05.661358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891193+0000","last_change":"2024-09-17T05:01:08.858811+0000","last_active":"2024-09-17T05:01:13.891193+0000","last_peered":"2024-09-17T05:01:13.891193+0000","last_clean":"2024-09-17T05:01:13.891193+0000","last_became_active":"2024-09-17T05:01:08.858013+0000","last_became_peered":"2024-09-17T05:01:08.858013+0000","last_unstale":"2024-09-17T05:01:13.891193+0000","last_undegraded":"2024-09-17T05:01:13.891193+0000","last_fullsized":"2024-09-17T05:01:13.891193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:06.714459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891507+0000","last_change":"2024-09-17T05:01:07.873880+0000","last_active":"2024-09-17T05:01:13.891507+0000","last_peered":"2024-09-17T05:01:13.891507+0000","last_clean":"2024-09-17T05:01:13.891507+0000","last_became_active":"2024-09-17T05:01:07.870873+0000","last_became_peered":"2024-09-17T05:01:07.870873+0000","last_unstale":"2024-09-17T05:01:13.891507+0000","last_undegraded":"2024-09-17T05:01:13.891507+0000","last_fullsized":"2024-09-17T05:01:13.891507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.618135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891844+0000","last_change":"2024-09-17T05:01:12.898484+0000","last_active":"2024-09-17T05:01:13.891844+0000","last_peered":"2024-09-17T05:01:13.891844+0000","last_clean":"2024-09-17T05:01:13.891844+0000","last_became_active":"2024-09-17T05:01:12.895423+0000","last_became_peered":"2024-09-17T05:01:12.895423+0000","last_unstale":"2024-09-17T05:01:13.891844+0000","last_undegraded":"2024-09-17T05:01:13.891844+0000","last_fullsized":"2024-09-17T05:01:13.891844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:54.524295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941368+0000","last_change":"2024-09-17T05:01:08.871889+0000","last_active":"2024-09-17T05:01:13.941368+0000","last_peered":"2024-09-17T05:01:13.941368+0000","last_clean":"2024-09-17T05:01:13.941368+0000","last_became_active":"2024-09-17T05:01:08.871512+0000","last_became_peered":"2024-09-17T05:01:08.871512+0000","last_unstale":"2024-09-17T05:01:13.941368+0000","last_undegraded":"2024-09-17T05:01:13.941368+0000","last_fullsized":"2024-09-17T05:01:13.941368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892848+0000","last_change":"2024-09-17T05:01:07.865154+0000","last_active":"2024-09-17T05:01:13.892848+0000","last_peered":"2024-09-17T05:01:13.892848+0000","last_clean":"2024-09-17T05:01:13.892848+0000","last_became_active":"2024-09-17T05:01:07.864075+0000","last_became_peered":"2024-09-17T05:01:07.864075+0000","last_unstale":"2024-09-17T05:01:13.892848+0000","last_undegraded":"2024-09-17T05:01:13.892848+0000","last_fullsized":"2024-09-17T05:01:13.892848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941790+0000","last_change":"2024-09-17T05:01:12.915985+0000","last_active":"2024-09-17T05:01:13.941790+0000","last_peered":"2024-09-17T05:01:13.941790+0000","last_clean":"2024-09-17T05:01:13.941790+0000","last_became_active":"2024-09-17T05:01:12.915569+0000","last_became_peered":"2024-09-17T05:01:12.915569+0000","last_unstale":"2024-09-17T05:01:13.941790+0000","last_undegraded":"2024-09-17T05:01:13.941790+0000","last_fullsized":"2024-09-17T05:01:13.941790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:15.801136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940431+0000","last_change":"2024-09-17T05:01:08.859016+0000","last_active":"2024-09-17T05:01:13.940431+0000","last_peered":"2024-09-17T05:01:13.940431+0000","last_clean":"2024-09-17T05:01:13.940431+0000","last_became_active":"2024-09-17T05:01:08.858669+0000","last_became_peered":"2024-09-17T05:01:08.858669+0000","last_unstale":"2024-09-17T05:01:13.940431+0000","last_undegraded":"2024-09-17T05:01:13.940431+0000","last_fullsized":"2024-09-17T05:01:13.940431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:48.908013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940792+0000","last_change":"2024-09-17T05:01:07.853871+0000","last_active":"2024-09-17T05:01:13.940792+0000","last_peered":"2024-09-17T05:01:13.940792+0000","last_clean":"2024-09-17T05:01:13.940792+0000","last_became_active":"2024-09-17T05:01:07.853309+0000","last_became_peered":"2024-09-17T05:01:07.853309+0000","last_unstale":"2024-09-17T05:01:13.940792+0000","last_undegraded":"2024-09-17T05:01:13.940792+0000","last_fullsized":"2024-09-17T05:01:13.940792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941719+0000","last_change":"2024-09-17T05:01:12.917589+0000","last_active":"2024-09-17T05:01:13.941719+0000","last_peered":"2024-09-17T05:01:13.941719+0000","last_clean":"2024-09-17T05:01:13.941719+0000","last_became_active":"2024-09-17T05:01:12.917064+0000","last_became_peered":"2024-09-17T05:01:12.917064+0000","last_unstale":"2024-09-17T05:01:13.941719+0000","last_undegraded":"2024-09-17T05:01:13.941719+0000","last_fullsized":"2024-09-17T05:01:13.941719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:47.894698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890960+0000","last_change":"2024-09-17T05:01:08.865800+0000","last_active":"2024-09-17T05:01:13.890960+0000","last_peered":"2024-09-17T05:01:13.890960+0000","last_clean":"2024-09-17T05:01:13.890960+0000","last_became_active":"2024-09-17T05:01:08.865418+0000","last_became_peered":"2024-09-17T05:01:08.865418+0000","last_unstale":"2024-09-17T05:01:13.890960+0000","last_undegraded":"2024-09-17T05:01:13.890960+0000","last_fullsized":"2024-09-17T05:01:13.890960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941039+0000","last_change":"2024-09-17T05:01:07.868079+0000","last_active":"2024-09-17T05:01:13.941039+0000","last_peered":"2024-09-17T05:01:13.941039+0000","last_clean":"2024-09-17T05:01:13.941039+0000","last_became_active":"2024-09-17T05:01:07.867648+0000","last_became_peered":"2024-09-17T05:01:07.867648+0000","last_unstale":"2024-09-17T05:01:13.941039+0000","last_undegraded":"2024-09-17T05:01:13.941039+0000","last_fullsized":"2024-09-17T05:01:13.941039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943609+0000","last_change":"2024-09-17T05:01:12.913059+0000","last_active":"2024-09-17T05:01:13.943609+0000","last_peered":"2024-09-17T05:01:13.943609+0000","last_clean":"2024-09-17T05:01:13.943609+0000","last_became_active":"2024-09-17T05:01:12.912569+0000","last_became_peered":"2024-09-17T05:01:12.912569+0000","last_unstale":"2024-09-17T05:01:13.943609+0000","last_undegraded":"2024-09-17T05:01:13.943609+0000","last_fullsized":"2024-09-17T05:01:13.943609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:23.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941727+0000","last_change":"2024-09-17T05:01:08.869046+0000","last_active":"2024-09-17T05:01:13.941727+0000","last_peered":"2024-09-17T05:01:13.941727+0000","last_clean":"2024-09-17T05:01:13.941727+0000","last_became_active":"2024-09-17T05:01:08.864116+0000","last_became_peered":"2024-09-17T05:01:08.864116+0000","last_unstale":"2024-09-17T05:01:13.941727+0000","last_undegraded":"2024-09-17T05:01:13.941727+0000","last_fullsized":"2024-09-17T05:01:13.941727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:51.571131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892323+0000","last_change":"2024-09-17T05:01:07.863381+0000","last_active":"2024-09-17T05:01:13.892323+0000","last_peered":"2024-09-17T05:01:13.892323+0000","last_clean":"2024-09-17T05:01:13.892323+0000","last_became_active":"2024-09-17T05:01:07.862731+0000","last_became_peered":"2024-09-17T05:01:07.862731+0000","last_unstale":"2024-09-17T05:01:13.892323+0000","last_undegraded":"2024-09-17T05:01:13.892323+0000","last_fullsized":"2024-09-17T05:01:13.892323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:39.024164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942455+0000","last_change":"2024-09-17T05:01:12.916294+0000","last_active":"2024-09-17T05:01:13.942455+0000","last_peered":"2024-09-17T05:01:13.942455+0000","last_clean":"2024-09-17T05:01:13.942455+0000","last_became_active":"2024-09-17T05:01:12.908035+0000","last_became_peered":"2024-09-17T05:01:12.908035+0000","last_unstale":"2024-09-17T05:01:13.942455+0000","last_undegraded":"2024-09-17T05:01:13.942455+0000","last_fullsized":"2024-09-17T05:01:13.942455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:52.101015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942370+0000","last_change":"2024-09-17T05:01:08.870629+0000","last_active":"2024-09-17T05:01:13.942370+0000","last_peered":"2024-09-17T05:01:13.942370+0000","last_clean":"2024-09-17T05:01:13.942370+0000","last_became_active":"2024-09-17T05:01:08.870198+0000","last_became_peered":"2024-09-17T05:01:08.870198+0000","last_unstale":"2024-09-17T05:01:13.942370+0000","last_undegraded":"2024-09-17T05:01:13.942370+0000","last_fullsized":"2024-09-17T05:01:13.942370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:37.142368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941967+0000","last_change":"2024-09-17T05:01:07.860691+0000","last_active":"2024-09-17T05:01:13.941967+0000","last_peered":"2024-09-17T05:01:13.941967+0000","last_clean":"2024-09-17T05:01:13.941967+0000","last_became_active":"2024-09-17T05:01:07.860360+0000","last_became_peered":"2024-09-17T05:01:07.860360+0000","last_unstale":"2024-09-17T05:01:13.941967+0000","last_undegraded":"2024-09-17T05:01:13.941967+0000","last_fullsized":"2024-09-17T05:01:13.941967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940513+0000","last_change":"2024-09-17T05:01:12.900669+0000","last_active":"2024-09-17T05:01:13.940513+0000","last_peered":"2024-09-17T05:01:13.940513+0000","last_clean":"2024-09-17T05:01:13.940513+0000","last_became_active":"2024-09-17T05:01:12.900234+0000","last_became_peered":"2024-09-17T05:01:12.900234+0000","last_unstale":"2024-09-17T05:01:13.940513+0000","last_undegraded":"2024-09-17T05:01:13.940513+0000","last_fullsized":"2024-09-17T05:01:13.940513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893905+0000","last_change":"2024-09-17T05:01:08.876794+0000","last_active":"2024-09-17T05:01:13.893905+0000","last_peered":"2024-09-17T05:01:13.893905+0000","last_clean":"2024-09-17T05:01:13.893905+0000","last_became_active":"2024-09-17T05:01:08.876418+0000","last_became_peered":"2024-09-17T05:01:08.876418+0000","last_unstale":"2024-09-17T05:01:13.893905+0000","last_undegraded":"2024-09-17T05:01:13.893905+0000","last_fullsized":"2024-09-17T05:01:13.893905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:47.662092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892328+0000","last_change":"2024-09-17T05:01:07.877675+0000","last_active":"2024-09-17T05:01:13.892328+0000","last_peered":"2024-09-17T05:01:13.892328+0000","last_clean":"2024-09-17T05:01:13.892328+0000","last_became_active":"2024-09-17T05:01:07.877320+0000","last_became_peered":"2024-09-17T05:01:07.877320+0000","last_unstale":"2024-09-17T05:01:13.892328+0000","last_undegraded":"2024-09-17T05:01:13.892328+0000","last_fullsized":"2024-09-17T05:01:13.892328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:35.509947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891833+0000","last_change":"2024-09-17T05:01:12.911570+0000","last_active":"2024-09-17T05:01:13.891833+0000","last_peered":"2024-09-17T05:01:13.891833+0000","last_clean":"2024-09-17T05:01:13.891833+0000","last_became_active":"2024-09-17T05:01:12.911110+0000","last_became_peered":"2024-09-17T05:01:12.911110+0000","last_unstale":"2024-09-17T05:01:13.891833+0000","last_undegraded":"2024-09-17T05:01:13.891833+0000","last_fullsized":"2024-09-17T05:01:13.891833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:38.085327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943250+0000","last_change":"2024-09-17T05:01:08.870024+0000","last_active":"2024-09-17T05:01:13.943250+0000","last_peered":"2024-09-17T05:01:13.943250+0000","last_clean":"2024-09-17T05:01:13.943250+0000","last_became_active":"2024-09-17T05:01:08.869679+0000","last_became_peered":"2024-09-17T05:01:08.869679+0000","last_unstale":"2024-09-17T05:01:13.943250+0000","last_undegraded":"2024-09-17T05:01:13.943250+0000","last_fullsized":"2024-09-17T05:01:13.943250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:45.964067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892819+0000","last_change":"2024-09-17T05:01:07.859295+0000","last_active":"2024-09-17T05:01:13.892819+0000","last_peered":"2024-09-17T05:01:13.892819+0000","last_clean":"2024-09-17T05:01:13.892819+0000","last_became_active":"2024-09-17T05:01:07.858949+0000","last_became_peered":"2024-09-17T05:01:07.858949+0000","last_unstale":"2024-09-17T05:01:13.892819+0000","last_undegraded":"2024-09-17T05:01:13.892819+0000","last_fullsized":"2024-09-17T05:01:13.892819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:29.635679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892487+0000","last_change":"2024-09-17T05:01:12.915413+0000","last_active":"2024-09-17T05:01:13.892487+0000","last_peered":"2024-09-17T05:01:13.892487+0000","last_clean":"2024-09-17T05:01:13.892487+0000","last_became_active":"2024-09-17T05:01:12.914942+0000","last_became_peered":"2024-09-17T05:01:12.914942+0000","last_unstale":"2024-09-17T05:01:13.892487+0000","last_undegraded":"2024-09-17T05:01:13.892487+0000","last_fullsized":"2024-09-17T05:01:13.892487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:57.733227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941319+0000","last_change":"2024-09-17T05:01:08.871457+0000","last_active":"2024-09-17T05:01:13.941319+0000","last_peered":"2024-09-17T05:01:13.941319+0000","last_clean":"2024-09-17T05:01:13.941319+0000","last_became_active":"2024-09-17T05:01:08.871135+0000","last_became_peered":"2024-09-17T05:01:08.871135+0000","last_unstale":"2024-09-17T05:01:13.941319+0000","last_undegraded":"2024-09-17T05:01:13.941319+0000","last_fullsized":"2024-09-17T05:01:13.941319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:15.936190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:07.873885+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:07.869214+0000","last_became_peered":"2024-09-17T05:01:07.869214+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:16.951737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893521+0000","last_change":"2024-09-17T05:01:12.908768+0000","last_active":"2024-09-17T05:01:13.893521+0000","last_peered":"2024-09-17T05:01:13.893521+0000","last_clean":"2024-09-17T05:01:13.893521+0000","last_became_active":"2024-09-17T05:01:12.908282+0000","last_became_peered":"2024-09-17T05:01:12.908282+0000","last_unstale":"2024-09-17T05:01:13.893521+0000","last_undegraded":"2024-09-17T05:01:13.893521+0000","last_fullsized":"2024-09-17T05:01:13.893521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891855+0000","last_change":"2024-09-17T05:01:08.858109+0000","last_active":"2024-09-17T05:01:13.891855+0000","last_peered":"2024-09-17T05:01:13.891855+0000","last_clean":"2024-09-17T05:01:13.891855+0000","last_became_active":"2024-09-17T05:01:08.857293+0000","last_became_peered":"2024-09-17T05:01:08.857293+0000","last_unstale":"2024-09-17T05:01:13.891855+0000","last_undegraded":"2024-09-17T05:01:13.891855+0000","last_fullsized":"2024-09-17T05:01:13.891855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:53.180346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941193+0000","last_change":"2024-09-17T05:01:07.864654+0000","last_active":"2024-09-17T05:01:13.941193+0000","last_peered":"2024-09-17T05:01:13.941193+0000","last_clean":"2024-09-17T05:01:13.941193+0000","last_became_active":"2024-09-17T05:01:07.864166+0000","last_became_peered":"2024-09-17T05:01:07.864166+0000","last_unstale":"2024-09-17T05:01:13.941193+0000","last_undegraded":"2024-09-17T05:01:13.941193+0000","last_fullsized":"2024-09-17T05:01:13.941193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894265+0000","last_change":"2024-09-17T05:01:12.914524+0000","last_active":"2024-09-17T05:01:13.894265+0000","last_peered":"2024-09-17T05:01:13.894265+0000","last_clean":"2024-09-17T05:01:13.894265+0000","last_became_active":"2024-09-17T05:01:12.912222+0000","last_became_peered":"2024-09-17T05:01:12.912222+0000","last_unstale":"2024-09-17T05:01:13.894265+0000","last_undegraded":"2024-09-17T05:01:13.894265+0000","last_fullsized":"2024-09-17T05:01:13.894265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892306+0000","last_change":"2024-09-17T05:01:08.875660+0000","last_active":"2024-09-17T05:01:13.892306+0000","last_peered":"2024-09-17T05:01:13.892306+0000","last_clean":"2024-09-17T05:01:13.892306+0000","last_became_active":"2024-09-17T05:01:08.875265+0000","last_became_peered":"2024-09-17T05:01:08.875265+0000","last_unstale":"2024-09-17T05:01:13.892306+0000","last_undegraded":"2024-09-17T05:01:13.892306+0000","last_fullsized":"2024-09-17T05:01:13.892306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891158+0000","last_change":"2024-09-17T05:01:07.869491+0000","last_active":"2024-09-17T05:01:13.891158+0000","last_peered":"2024-09-17T05:01:13.891158+0000","last_clean":"2024-09-17T05:01:13.891158+0000","last_became_active":"2024-09-17T05:01:07.869038+0000","last_became_peered":"2024-09-17T05:01:07.869038+0000","last_unstale":"2024-09-17T05:01:13.891158+0000","last_undegraded":"2024-09-17T05:01:13.891158+0000","last_fullsized":"2024-09-17T05:01:13.891158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:00.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893711+0000","last_change":"2024-09-17T05:01:12.910830+0000","last_active":"2024-09-17T05:01:13.893711+0000","last_peered":"2024-09-17T05:01:13.893711+0000","last_clean":"2024-09-17T05:01:13.893711+0000","last_became_active":"2024-09-17T05:01:12.908497+0000","last_became_peered":"2024-09-17T05:01:12.908497+0000","last_unstale":"2024-09-17T05:01:13.893711+0000","last_undegraded":"2024-09-17T05:01:13.893711+0000","last_fullsized":"2024-09-17T05:01:13.893711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:07.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941121+0000","last_change":"2024-09-17T05:01:08.868170+0000","last_active":"2024-09-17T05:01:13.941121+0000","last_peered":"2024-09-17T05:01:13.941121+0000","last_clean":"2024-09-17T05:01:13.941121+0000","last_became_active":"2024-09-17T05:01:08.867767+0000","last_became_peered":"2024-09-17T05:01:08.867767+0000","last_unstale":"2024-09-17T05:01:13.941121+0000","last_undegraded":"2024-09-17T05:01:13.941121+0000","last_fullsized":"2024-09-17T05:01:13.941121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891555+0000","last_change":"2024-09-17T05:01:07.873908+0000","last_active":"2024-09-17T05:01:13.891555+0000","last_peered":"2024-09-17T05:01:13.891555+0000","last_clean":"2024-09-17T05:01:13.891555+0000","last_became_active":"2024-09-17T05:01:07.871028+0000","last_became_peered":"2024-09-17T05:01:07.871028+0000","last_unstale":"2024-09-17T05:01:13.891555+0000","last_undegraded":"2024-09-17T05:01:13.891555+0000","last_fullsized":"2024-09-17T05:01:13.891555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941889+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:01:13.941889+0000","last_peered":"2024-09-17T05:01:13.941889+0000","last_clean":"2024-09-17T05:01:13.941889+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:01:13.941889+0000","last_undegraded":"2024-09-17T05:01:13.941889+0000","last_fullsized":"2024-09-17T05:01:13.941889+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892300+0000","last_change":"2024-09-17T05:01:12.904503+0000","last_active":"2024-09-17T05:01:13.892300+0000","last_peered":"2024-09-17T05:01:13.892300+0000","last_clean":"2024-09-17T05:01:13.892300+0000","last_became_active":"2024-09-17T05:01:12.903910+0000","last_became_peered":"2024-09-17T05:01:12.903910+0000","last_unstale":"2024-09-17T05:01:13.892300+0000","last_undegraded":"2024-09-17T05:01:13.892300+0000","last_fullsized":"2024-09-17T05:01:13.892300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:47.490639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892152+0000","last_change":"2024-09-17T05:01:08.872909+0000","last_active":"2024-09-17T05:01:13.892152+0000","last_peered":"2024-09-17T05:01:13.892152+0000","last_clean":"2024-09-17T05:01:13.892152+0000","last_became_active":"2024-09-17T05:01:08.861070+0000","last_became_peered":"2024-09-17T05:01:08.861070+0000","last_unstale":"2024-09-17T05:01:13.892152+0000","last_undegraded":"2024-09-17T05:01:13.892152+0000","last_fullsized":"2024-09-17T05:01:13.892152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:44.525838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894016+0000","last_change":"2024-09-17T05:01:07.866417+0000","last_active":"2024-09-17T05:01:13.894016+0000","last_peered":"2024-09-17T05:01:13.894016+0000","last_clean":"2024-09-17T05:01:13.894016+0000","last_became_active":"2024-09-17T05:01:07.866088+0000","last_became_peered":"2024-09-17T05:01:07.866088+0000","last_unstale":"2024-09-17T05:01:13.894016+0000","last_undegraded":"2024-09-17T05:01:13.894016+0000","last_fullsized":"2024-09-17T05:01:13.894016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942333+0000","last_change":"2024-09-17T05:01:12.919691+0000","last_active":"2024-09-17T05:01:13.942333+0000","last_peered":"2024-09-17T05:01:13.942333+0000","last_clean":"2024-09-17T05:01:13.942333+0000","last_became_active":"2024-09-17T05:01:12.917951+0000","last_became_peered":"2024-09-17T05:01:12.917951+0000","last_unstale":"2024-09-17T05:01:13.942333+0000","last_undegraded":"2024-09-17T05:01:13.942333+0000","last_fullsized":"2024-09-17T05:01:13.942333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:11.524293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942694+0000","last_change":"2024-09-17T05:01:08.864231+0000","last_active":"2024-09-17T05:01:13.942694+0000","last_peered":"2024-09-17T05:01:13.942694+0000","last_clean":"2024-09-17T05:01:13.942694+0000","last_became_active":"2024-09-17T05:01:08.863802+0000","last_became_peered":"2024-09-17T05:01:08.863802+0000","last_unstale":"2024-09-17T05:01:13.942694+0000","last_undegraded":"2024-09-17T05:01:13.942694+0000","last_fullsized":"2024-09-17T05:01:13.942694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893511+0000","last_change":"2024-09-17T05:01:07.857650+0000","last_active":"2024-09-17T05:01:13.893511+0000","last_peered":"2024-09-17T05:01:13.893511+0000","last_clean":"2024-09-17T05:01:13.893511+0000","last_became_active":"2024-09-17T05:01:07.857311+0000","last_became_peered":"2024-09-17T05:01:07.857311+0000","last_unstale":"2024-09-17T05:01:13.893511+0000","last_undegraded":"2024-09-17T05:01:13.893511+0000","last_fullsized":"2024-09-17T05:01:13.893511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:27.902116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893168+0000","last_change":"2024-09-17T05:01:12.888243+0000","last_active":"2024-09-17T05:01:13.893168+0000","last_peered":"2024-09-17T05:01:13.893168+0000","last_clean":"2024-09-17T05:01:13.893168+0000","last_became_active":"2024-09-17T05:01:12.887793+0000","last_became_peered":"2024-09-17T05:01:12.887793+0000","last_unstale":"2024-09-17T05:01:13.893168+0000","last_undegraded":"2024-09-17T05:01:13.893168+0000","last_fullsized":"2024-09-17T05:01:13.893168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:00.157222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942457+0000","last_change":"2024-09-17T05:01:08.889497+0000","last_active":"2024-09-17T05:01:13.942457+0000","last_peered":"2024-09-17T05:01:13.942457+0000","last_clean":"2024-09-17T05:01:13.942457+0000","last_became_active":"2024-09-17T05:01:08.887064+0000","last_became_peered":"2024-09-17T05:01:08.887064+0000","last_unstale":"2024-09-17T05:01:13.942457+0000","last_undegraded":"2024-09-17T05:01:13.942457+0000","last_fullsized":"2024-09-17T05:01:13.942457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941847+0000","last_change":"2024-09-17T05:01:07.851404+0000","last_active":"2024-09-17T05:01:13.941847+0000","last_peered":"2024-09-17T05:01:13.941847+0000","last_clean":"2024-09-17T05:01:13.941847+0000","last_became_active":"2024-09-17T05:01:07.851077+0000","last_became_peered":"2024-09-17T05:01:07.851077+0000","last_unstale":"2024-09-17T05:01:13.941847+0000","last_undegraded":"2024-09-17T05:01:13.941847+0000","last_fullsized":"2024-09-17T05:01:13.941847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890772+0000","last_change":"2024-09-17T05:01:12.905280+0000","last_active":"2024-09-17T05:01:13.890772+0000","last_peered":"2024-09-17T05:01:13.890772+0000","last_clean":"2024-09-17T05:01:13.890772+0000","last_became_active":"2024-09-17T05:01:12.904787+0000","last_became_peered":"2024-09-17T05:01:12.904787+0000","last_unstale":"2024-09-17T05:01:13.890772+0000","last_undegraded":"2024-09-17T05:01:13.890772+0000","last_fullsized":"2024-09-17T05:01:13.890772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:29.666258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.884835+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.884410+0000","last_became_peered":"2024-09-17T05:01:08.884410+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:35.960729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942469+0000","last_change":"2024-09-17T05:01:07.866602+0000","last_active":"2024-09-17T05:01:13.942469+0000","last_peered":"2024-09-17T05:01:13.942469+0000","last_clean":"2024-09-17T05:01:13.942469+0000","last_became_active":"2024-09-17T05:01:07.866061+0000","last_became_peered":"2024-09-17T05:01:07.866061+0000","last_unstale":"2024-09-17T05:01:13.942469+0000","last_undegraded":"2024-09-17T05:01:13.942469+0000","last_fullsized":"2024-09-17T05:01:13.942469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943528+0000","last_change":"2024-09-17T05:01:12.904969+0000","last_active":"2024-09-17T05:01:13.943528+0000","last_peered":"2024-09-17T05:01:13.943528+0000","last_clean":"2024-09-17T05:01:13.943528+0000","last_became_active":"2024-09-17T05:01:12.904665+0000","last_became_peered":"2024-09-17T05:01:12.904665+0000","last_unstale":"2024-09-17T05:01:13.943528+0000","last_undegraded":"2024-09-17T05:01:13.943528+0000","last_fullsized":"2024-09-17T05:01:13.943528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:52.943963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890912+0000","last_change":"2024-09-17T05:01:08.872790+0000","last_active":"2024-09-17T05:01:13.890912+0000","last_peered":"2024-09-17T05:01:13.890912+0000","last_clean":"2024-09-17T05:01:13.890912+0000","last_became_active":"2024-09-17T05:01:08.872387+0000","last_became_peered":"2024-09-17T05:01:08.872387+0000","last_unstale":"2024-09-17T05:01:13.890912+0000","last_undegraded":"2024-09-17T05:01:13.890912+0000","last_fullsized":"2024-09-17T05:01:13.890912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940575+0000","last_change":"2024-09-17T05:01:07.854980+0000","last_active":"2024-09-17T05:01:13.940575+0000","last_peered":"2024-09-17T05:01:13.940575+0000","last_clean":"2024-09-17T05:01:13.940575+0000","last_became_active":"2024-09-17T05:01:07.854665+0000","last_became_peered":"2024-09-17T05:01:07.854665+0000","last_unstale":"2024-09-17T05:01:13.940575+0000","last_undegraded":"2024-09-17T05:01:13.940575+0000","last_fullsized":"2024-09-17T05:01:13.940575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893909+0000","last_change":"2024-09-17T05:01:12.908923+0000","last_active":"2024-09-17T05:01:13.893909+0000","last_peered":"2024-09-17T05:01:13.893909+0000","last_clean":"2024-09-17T05:01:13.893909+0000","last_became_active":"2024-09-17T05:01:12.908398+0000","last_became_peered":"2024-09-17T05:01:12.908398+0000","last_unstale":"2024-09-17T05:01:13.893909+0000","last_undegraded":"2024-09-17T05:01:13.893909+0000","last_fullsized":"2024-09-17T05:01:13.893909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:45.118442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941757+0000","last_change":"2024-09-17T05:01:08.856311+0000","last_active":"2024-09-17T05:01:13.941757+0000","last_peered":"2024-09-17T05:01:13.941757+0000","last_clean":"2024-09-17T05:01:13.941757+0000","last_became_active":"2024-09-17T05:01:08.855334+0000","last_became_peered":"2024-09-17T05:01:08.855334+0000","last_unstale":"2024-09-17T05:01:13.941757+0000","last_undegraded":"2024-09-17T05:01:13.941757+0000","last_fullsized":"2024-09-17T05:01:13.941757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942247+0000","last_change":"2024-09-17T05:01:07.872183+0000","last_active":"2024-09-17T05:01:13.942247+0000","last_peered":"2024-09-17T05:01:13.942247+0000","last_clean":"2024-09-17T05:01:13.942247+0000","last_became_active":"2024-09-17T05:01:07.871540+0000","last_became_peered":"2024-09-17T05:01:07.871540+0000","last_unstale":"2024-09-17T05:01:13.942247+0000","last_undegraded":"2024-09-17T05:01:13.942247+0000","last_fullsized":"2024-09-17T05:01:13.942247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894214+0000","last_change":"2024-09-17T05:01:12.906286+0000","last_active":"2024-09-17T05:01:13.894214+0000","last_peered":"2024-09-17T05:01:13.894214+0000","last_clean":"2024-09-17T05:01:13.894214+0000","last_became_active":"2024-09-17T05:01:12.905810+0000","last_became_peered":"2024-09-17T05:01:12.905810+0000","last_unstale":"2024-09-17T05:01:13.894214+0000","last_undegraded":"2024-09-17T05:01:13.894214+0000","last_fullsized":"2024-09-17T05:01:13.894214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:33.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891317+0000","last_change":"2024-09-17T05:01:08.893475+0000","last_active":"2024-09-17T05:01:13.891317+0000","last_peered":"2024-09-17T05:01:13.891317+0000","last_clean":"2024-09-17T05:01:13.891317+0000","last_became_active":"2024-09-17T05:01:08.893130+0000","last_became_peered":"2024-09-17T05:01:08.893130+0000","last_unstale":"2024-09-17T05:01:13.891317+0000","last_undegraded":"2024-09-17T05:01:13.891317+0000","last_fullsized":"2024-09-17T05:01:13.891317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891492+0000","last_change":"2024-09-17T05:01:07.869315+0000","last_active":"2024-09-17T05:01:13.891492+0000","last_peered":"2024-09-17T05:01:13.891492+0000","last_clean":"2024-09-17T05:01:13.891492+0000","last_became_active":"2024-09-17T05:01:07.864783+0000","last_became_peered":"2024-09-17T05:01:07.864783+0000","last_unstale":"2024-09-17T05:01:13.891492+0000","last_undegraded":"2024-09-17T05:01:13.891492+0000","last_fullsized":"2024-09-17T05:01:13.891492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893698+0000","last_change":"2024-09-17T05:01:12.921137+0000","last_active":"2024-09-17T05:01:13.893698+0000","last_peered":"2024-09-17T05:01:13.893698+0000","last_clean":"2024-09-17T05:01:13.893698+0000","last_became_active":"2024-09-17T05:01:12.920809+0000","last_became_peered":"2024-09-17T05:01:12.920809+0000","last_unstale":"2024-09-17T05:01:13.893698+0000","last_undegraded":"2024-09-17T05:01:13.893698+0000","last_fullsized":"2024-09-17T05:01:13.893698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892745+0000","last_change":"2024-09-17T05:01:08.892262+0000","last_active":"2024-09-17T05:01:13.892745+0000","last_peered":"2024-09-17T05:01:13.892745+0000","last_clean":"2024-09-17T05:01:13.892745+0000","last_became_active":"2024-09-17T05:01:08.891946+0000","last_became_peered":"2024-09-17T05:01:08.891946+0000","last_unstale":"2024-09-17T05:01:13.892745+0000","last_undegraded":"2024-09-17T05:01:13.892745+0000","last_fullsized":"2024-09-17T05:01:13.892745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:40.031283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941492+0000","last_change":"2024-09-17T05:01:07.872646+0000","last_active":"2024-09-17T05:01:13.941492+0000","last_peered":"2024-09-17T05:01:13.941492+0000","last_clean":"2024-09-17T05:01:13.941492+0000","last_became_active":"2024-09-17T05:01:07.872287+0000","last_became_peered":"2024-09-17T05:01:07.872287+0000","last_unstale":"2024-09-17T05:01:13.941492+0000","last_undegraded":"2024-09-17T05:01:13.941492+0000","last_fullsized":"2024-09-17T05:01:13.941492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:56.785994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891950+0000","last_change":"2024-09-17T05:01:12.900658+0000","last_active":"2024-09-17T05:01:13.891950+0000","last_peered":"2024-09-17T05:01:13.891950+0000","last_clean":"2024-09-17T05:01:13.891950+0000","last_became_active":"2024-09-17T05:01:12.900131+0000","last_became_peered":"2024-09-17T05:01:12.900131+0000","last_unstale":"2024-09-17T05:01:13.891950+0000","last_undegraded":"2024-09-17T05:01:13.891950+0000","last_fullsized":"2024-09-17T05:01:13.891950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:54.427072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941925+0000","last_change":"2024-09-17T05:01:08.869878+0000","last_active":"2024-09-17T05:01:13.941925+0000","last_peered":"2024-09-17T05:01:13.941925+0000","last_clean":"2024-09-17T05:01:13.941925+0000","last_became_active":"2024-09-17T05:01:08.869571+0000","last_became_peered":"2024-09-17T05:01:08.869571+0000","last_unstale":"2024-09-17T05:01:13.941925+0000","last_undegraded":"2024-09-17T05:01:13.941925+0000","last_fullsized":"2024-09-17T05:01:13.941925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:29.847123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:19.977855+0000","last_change":"2024-09-17T05:01:07.862966+0000","last_active":"2024-09-17T05:01:19.977855+0000","last_peered":"2024-09-17T05:01:19.977855+0000","last_clean":"2024-09-17T05:01:19.977855+0000","last_became_active":"2024-09-17T05:01:07.862605+0000","last_became_peered":"2024-09-17T05:01:07.862605+0000","last_unstale":"2024-09-17T05:01:19.977855+0000","last_undegraded":"2024-09-17T05:01:19.977855+0000","last_fullsized":"2024-09-17T05:01:19.977855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:26.467925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941104+0000","last_change":"2024-09-17T05:01:12.897023+0000","last_active":"2024-09-17T05:01:13.941104+0000","last_peered":"2024-09-17T05:01:13.941104+0000","last_clean":"2024-09-17T05:01:13.941104+0000","last_became_active":"2024-09-17T05:01:12.896528+0000","last_became_peered":"2024-09-17T05:01:12.896528+0000","last_unstale":"2024-09-17T05:01:13.941104+0000","last_undegraded":"2024-09-17T05:01:13.941104+0000","last_fullsized":"2024-09-17T05:01:13.941104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:04.392152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:08.866251+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:08.864997+0000","last_became_peered":"2024-09-17T05:01:08.864997+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:33.305210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942113+0000","last_change":"2024-09-17T05:01:07.862662+0000","last_active":"2024-09-17T05:01:13.942113+0000","last_peered":"2024-09-17T05:01:13.942113+0000","last_clean":"2024-09-17T05:01:13.942113+0000","last_became_active":"2024-09-17T05:01:07.862252+0000","last_became_peered":"2024-09-17T05:01:07.862252+0000","last_unstale":"2024-09-17T05:01:13.942113+0000","last_undegraded":"2024-09-17T05:01:13.942113+0000","last_fullsized":"2024-09-17T05:01:13.942113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:13.958450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942881+0000","last_change":"2024-09-17T05:01:12.915486+0000","last_active":"2024-09-17T05:01:13.942881+0000","last_peered":"2024-09-17T05:01:13.942881+0000","last_clean":"2024-09-17T05:01:13.942881+0000","last_became_active":"2024-09-17T05:01:12.913794+0000","last_became_peered":"2024-09-17T05:01:12.913794+0000","last_unstale":"2024-09-17T05:01:13.942881+0000","last_undegraded":"2024-09-17T05:01:13.942881+0000","last_fullsized":"2024-09-17T05:01:13.942881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:33.883073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894291+0000","last_change":"2024-09-17T05:01:08.863593+0000","last_active":"2024-09-17T05:01:13.894291+0000","last_peered":"2024-09-17T05:01:13.894291+0000","last_clean":"2024-09-17T05:01:13.894291+0000","last_became_active":"2024-09-17T05:01:08.863264+0000","last_became_peered":"2024-09-17T05:01:08.863264+0000","last_unstale":"2024-09-17T05:01:13.894291+0000","last_undegraded":"2024-09-17T05:01:13.894291+0000","last_fullsized":"2024-09-17T05:01:13.894291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:00.186616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942093+0000","last_change":"2024-09-17T05:01:07.860203+0000","last_active":"2024-09-17T05:01:13.942093+0000","last_peered":"2024-09-17T05:01:13.942093+0000","last_clean":"2024-09-17T05:01:13.942093+0000","last_became_active":"2024-09-17T05:01:07.858682+0000","last_became_peered":"2024-09-17T05:01:07.858682+0000","last_unstale":"2024-09-17T05:01:13.942093+0000","last_undegraded":"2024-09-17T05:01:13.942093+0000","last_fullsized":"2024-09-17T05:01:13.942093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940512+0000","last_change":"2024-09-17T05:01:12.912101+0000","last_active":"2024-09-17T05:01:13.940512+0000","last_peered":"2024-09-17T05:01:13.940512+0000","last_clean":"2024-09-17T05:01:13.940512+0000","last_became_active":"2024-09-17T05:01:12.911449+0000","last_became_peered":"2024-09-17T05:01:12.911449+0000","last_unstale":"2024-09-17T05:01:13.940512+0000","last_undegraded":"2024-09-17T05:01:13.940512+0000","last_fullsized":"2024-09-17T05:01:13.940512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:51.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.874496+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.874209+0000","last_became_peered":"2024-09-17T05:01:08.874209+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:06.383708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944328+0000","last_change":"2024-09-17T05:01:07.850788+0000","last_active":"2024-09-17T05:01:13.944328+0000","last_peered":"2024-09-17T05:01:13.944328+0000","last_clean":"2024-09-17T05:01:13.944328+0000","last_became_active":"2024-09-17T05:01:07.850154+0000","last_became_peered":"2024-09-17T05:01:07.850154+0000","last_unstale":"2024-09-17T05:01:13.944328+0000","last_undegraded":"2024-09-17T05:01:13.944328+0000","last_fullsized":"2024-09-17T05:01:13.944328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940756+0000","last_change":"2024-09-17T05:01:12.908537+0000","last_active":"2024-09-17T05:01:13.940756+0000","last_peered":"2024-09-17T05:01:13.940756+0000","last_clean":"2024-09-17T05:01:13.940756+0000","last_became_active":"2024-09-17T05:01:12.908060+0000","last_became_peered":"2024-09-17T05:01:12.908060+0000","last_unstale":"2024-09-17T05:01:13.940756+0000","last_undegraded":"2024-09-17T05:01:13.940756+0000","last_fullsized":"2024-09-17T05:01:13.940756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941572+0000","last_change":"2024-09-17T05:01:12.895140+0000","last_active":"2024-09-17T05:01:13.941572+0000","last_peered":"2024-09-17T05:01:13.941572+0000","last_clean":"2024-09-17T05:01:13.941572+0000","last_became_active":"2024-09-17T05:01:12.894665+0000","last_became_peered":"2024-09-17T05:01:12.894665+0000","last_unstale":"2024-09-17T05:01:13.941572+0000","last_undegraded":"2024-09-17T05:01:13.941572+0000","last_fullsized":"2024-09-17T05:01:13.941572+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:22.431196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940721+0000","last_change":"2024-09-17T05:01:08.851860+0000","last_active":"2024-09-17T05:01:13.940721+0000","last_peered":"2024-09-17T05:01:13.940721+0000","last_clean":"2024-09-17T05:01:13.940721+0000","last_became_active":"2024-09-17T05:01:08.851546+0000","last_became_peered":"2024-09-17T05:01:08.851546+0000","last_unstale":"2024-09-17T05:01:13.940721+0000","last_undegraded":"2024-09-17T05:01:13.940721+0000","last_fullsized":"2024-09-17T05:01:13.940721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:25.615656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891094+0000","last_change":"2024-09-17T05:01:07.876275+0000","last_active":"2024-09-17T05:01:13.891094+0000","last_peered":"2024-09-17T05:01:13.891094+0000","last_clean":"2024-09-17T05:01:13.891094+0000","last_became_active":"2024-09-17T05:01:07.875993+0000","last_became_peered":"2024-09-17T05:01:07.875993+0000","last_unstale":"2024-09-17T05:01:13.891094+0000","last_undegraded":"2024-09-17T05:01:13.891094+0000","last_fullsized":"2024-09-17T05:01:13.891094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:43.841239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891826+0000","last_change":"2024-09-17T05:01:12.904195+0000","last_active":"2024-09-17T05:01:13.891826+0000","last_peered":"2024-09-17T05:01:13.891826+0000","last_clean":"2024-09-17T05:01:13.891826+0000","last_became_active":"2024-09-17T05:01:12.903808+0000","last_became_peered":"2024-09-17T05:01:12.903808+0000","last_unstale":"2024-09-17T05:01:13.891826+0000","last_undegraded":"2024-09-17T05:01:13.891826+0000","last_fullsized":"2024-09-17T05:01:13.891826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:11.189684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891943+0000","last_change":"2024-09-17T05:01:08.876556+0000","last_active":"2024-09-17T05:01:13.891943+0000","last_peered":"2024-09-17T05:01:13.891943+0000","last_clean":"2024-09-17T05:01:13.891943+0000","last_became_active":"2024-09-17T05:01:08.874581+0000","last_became_peered":"2024-09-17T05:01:08.874581+0000","last_unstale":"2024-09-17T05:01:13.891943+0000","last_undegraded":"2024-09-17T05:01:13.891943+0000","last_fullsized":"2024-09-17T05:01:13.891943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:59.333120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892433+0000","last_change":"2024-09-17T05:01:07.865323+0000","last_active":"2024-09-17T05:01:13.892433+0000","last_peered":"2024-09-17T05:01:13.892433+0000","last_clean":"2024-09-17T05:01:13.892433+0000","last_became_active":"2024-09-17T05:01:07.864832+0000","last_became_peered":"2024-09-17T05:01:07.864832+0000","last_unstale":"2024-09-17T05:01:13.892433+0000","last_undegraded":"2024-09-17T05:01:13.892433+0000","last_fullsized":"2024-09-17T05:01:13.892433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891348+0000","last_change":"2024-09-17T05:01:07.870957+0000","last_active":"2024-09-17T05:01:13.891348+0000","last_peered":"2024-09-17T05:01:13.891348+0000","last_clean":"2024-09-17T05:01:13.891348+0000","last_became_active":"2024-09-17T05:01:07.869025+0000","last_became_peered":"2024-09-17T05:01:07.869025+0000","last_unstale":"2024-09-17T05:01:13.891348+0000","last_undegraded":"2024-09-17T05:01:13.891348+0000","last_fullsized":"2024-09-17T05:01:13.891348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940880+0000","last_change":"2024-09-17T05:01:08.876853+0000","last_active":"2024-09-17T05:01:13.940880+0000","last_peered":"2024-09-17T05:01:13.940880+0000","last_clean":"2024-09-17T05:01:13.940880+0000","last_became_active":"2024-09-17T05:01:08.876525+0000","last_became_peered":"2024-09-17T05:01:08.876525+0000","last_unstale":"2024-09-17T05:01:13.940880+0000","last_undegraded":"2024-09-17T05:01:13.940880+0000","last_fullsized":"2024-09-17T05:01:13.940880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:50.685520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941188+0000","last_change":"2024-09-17T05:01:12.908492+0000","last_active":"2024-09-17T05:01:13.941188+0000","last_peered":"2024-09-17T05:01:13.941188+0000","last_clean":"2024-09-17T05:01:13.941188+0000","last_became_active":"2024-09-17T05:01:12.908032+0000","last_became_peered":"2024-09-17T05:01:12.908032+0000","last_unstale":"2024-09-17T05:01:13.941188+0000","last_undegraded":"2024-09-17T05:01:13.941188+0000","last_fullsized":"2024-09-17T05:01:13.941188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893141+0000","last_change":"2024-09-17T05:01:07.859924+0000","last_active":"2024-09-17T05:01:13.893141+0000","last_peered":"2024-09-17T05:01:13.893141+0000","last_clean":"2024-09-17T05:01:13.893141+0000","last_became_active":"2024-09-17T05:01:07.859692+0000","last_became_peered":"2024-09-17T05:01:07.859692+0000","last_unstale":"2024-09-17T05:01:13.893141+0000","last_undegraded":"2024-09-17T05:01:13.893141+0000","last_fullsized":"2024-09-17T05:01:13.893141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.935910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943063+0000","last_change":"2024-09-17T05:01:08.886919+0000","last_active":"2024-09-17T05:01:13.943063+0000","last_peered":"2024-09-17T05:01:13.943063+0000","last_clean":"2024-09-17T05:01:13.943063+0000","last_became_active":"2024-09-17T05:01:08.886565+0000","last_became_peered":"2024-09-17T05:01:08.886565+0000","last_unstale":"2024-09-17T05:01:13.943063+0000","last_undegraded":"2024-09-17T05:01:13.943063+0000","last_fullsized":"2024-09-17T05:01:13.943063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941737+0000","last_change":"2024-09-17T05:01:12.919497+0000","last_active":"2024-09-17T05:01:13.941737+0000","last_peered":"2024-09-17T05:01:13.941737+0000","last_clean":"2024-09-17T05:01:13.941737+0000","last_became_active":"2024-09-17T05:01:12.919097+0000","last_became_peered":"2024-09-17T05:01:12.919097+0000","last_unstale":"2024-09-17T05:01:13.941737+0000","last_undegraded":"2024-09-17T05:01:13.941737+0000","last_fullsized":"2024-09-17T05:01:13.941737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:33.403463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941489+0000","last_change":"2024-09-17T05:01:08.868818+0000","last_active":"2024-09-17T05:01:13.941489+0000","last_peered":"2024-09-17T05:01:13.941489+0000","last_clean":"2024-09-17T05:01:13.941489+0000","last_became_active":"2024-09-17T05:01:08.868496+0000","last_became_peered":"2024-09-17T05:01:08.868496+0000","last_unstale":"2024-09-17T05:01:13.941489+0000","last_undegraded":"2024-09-17T05:01:13.941489+0000","last_fullsized":"2024-09-17T05:01:13.941489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:17.846926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941062+0000","last_change":"2024-09-17T05:01:08.881690+0000","last_active":"2024-09-17T05:01:13.941062+0000","last_peered":"2024-09-17T05:01:13.941062+0000","last_clean":"2024-09-17T05:01:13.941062+0000","last_became_active":"2024-09-17T05:01:08.880670+0000","last_became_peered":"2024-09-17T05:01:08.880670+0000","last_unstale":"2024-09-17T05:01:13.941062+0000","last_undegraded":"2024-09-17T05:01:13.941062+0000","last_fullsized":"2024-09-17T05:01:13.941062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891163+0000","last_change":"2024-09-17T05:01:08.867225+0000","last_active":"2024-09-17T05:01:13.891163+0000","last_peered":"2024-09-17T05:01:13.891163+0000","last_clean":"2024-09-17T05:01:13.891163+0000","last_became_active":"2024-09-17T05:01:08.866861+0000","last_became_peered":"2024-09-17T05:01:08.866861+0000","last_unstale":"2024-09-17T05:01:13.891163+0000","last_undegraded":"2024-09-17T05:01:13.891163+0000","last_fullsized":"2024-09-17T05:01:13.891163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:40.190786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892992+0000","last_change":"2024-09-17T05:01:08.862318+0000","last_active":"2024-09-17T05:01:13.892992+0000","last_peered":"2024-09-17T05:01:13.892992+0000","last_clean":"2024-09-17T05:01:13.892992+0000","last_became_active":"2024-09-17T05:01:08.861957+0000","last_became_peered":"2024-09-17T05:01:08.861957+0000","last_unstale":"2024-09-17T05:01:13.892992+0000","last_undegraded":"2024-09-17T05:01:13.892992+0000","last_fullsized":"2024-09-17T05:01:13.892992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:26.208177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893817+0000","last_change":"2024-09-17T05:01:08.883574+0000","last_active":"2024-09-17T05:01:13.893817+0000","last_peered":"2024-09-17T05:01:13.893817+0000","last_clean":"2024-09-17T05:01:13.893817+0000","last_became_active":"2024-09-17T05:01:08.883327+0000","last_became_peered":"2024-09-17T05:01:08.883327+0000","last_unstale":"2024-09-17T05:01:13.893817+0000","last_undegraded":"2024-09-17T05:01:13.893817+0000","last_fullsized":"2024-09-17T05:01:13.893817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893848+0000","last_change":"2024-09-17T05:01:08.877996+0000","last_active":"2024-09-17T05:01:13.893848+0000","last_peered":"2024-09-17T05:01:13.893848+0000","last_clean":"2024-09-17T05:01:13.893848+0000","last_became_active":"2024-09-17T05:01:08.877545+0000","last_became_peered":"2024-09-17T05:01:08.877545+0000","last_unstale":"2024-09-17T05:01:13.893848+0000","last_undegraded":"2024-09-17T05:01:13.893848+0000","last_fullsized":"2024-09-17T05:01:13.893848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:43.858748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943694+0000","last_change":"2024-09-17T05:01:08.874485+0000","last_active":"2024-09-17T05:01:13.943694+0000","last_peered":"2024-09-17T05:01:13.943694+0000","last_clean":"2024-09-17T05:01:13.943694+0000","last_became_active":"2024-09-17T05:01:08.874134+0000","last_became_peered":"2024-09-17T05:01:08.874134+0000","last_unstale":"2024-09-17T05:01:13.943694+0000","last_undegraded":"2024-09-17T05:01:13.943694+0000","last_fullsized":"2024-09-17T05:01:13.943694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:10.348602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893707+0000","last_change":"2024-09-17T05:01:08.886688+0000","last_active":"2024-09-17T05:01:13.893707+0000","last_peered":"2024-09-17T05:01:13.893707+0000","last_clean":"2024-09-17T05:01:13.893707+0000","last_became_active":"2024-09-17T05:01:08.886385+0000","last_became_peered":"2024-09-17T05:01:08.886385+0000","last_unstale":"2024-09-17T05:01:13.893707+0000","last_undegraded":"2024-09-17T05:01:13.893707+0000","last_fullsized":"2024-09-17T05:01:13.893707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:29.234911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891025+0000","last_change":"2024-09-17T05:01:08.866999+0000","last_active":"2024-09-17T05:01:13.891025+0000","last_peered":"2024-09-17T05:01:13.891025+0000","last_clean":"2024-09-17T05:01:13.891025+0000","last_became_active":"2024-09-17T05:01:08.866661+0000","last_became_peered":"2024-09-17T05:01:08.866661+0000","last_unstale":"2024-09-17T05:01:13.891025+0000","last_undegraded":"2024-09-17T05:01:13.891025+0000","last_fullsized":"2024-09-17T05:01:13.891025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:40.685724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892397+0000","last_change":"2024-09-17T05:01:08.871375+0000","last_active":"2024-09-17T05:01:13.892397+0000","last_peered":"2024-09-17T05:01:13.892397+0000","last_clean":"2024-09-17T05:01:13.892397+0000","last_became_active":"2024-09-17T05:01:08.870439+0000","last_became_peered":"2024-09-17T05:01:08.870439+0000","last_unstale":"2024-09-17T05:01:13.892397+0000","last_undegraded":"2024-09-17T05:01:13.892397+0000","last_fullsized":"2024-09-17T05:01:13.892397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:49.417178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942955+0000","last_change":"2024-09-17T05:01:08.866055+0000","last_active":"2024-09-17T05:01:13.942955+0000","last_peered":"2024-09-17T05:01:13.942955+0000","last_clean":"2024-09-17T05:01:13.942955+0000","last_became_active":"2024-09-17T05:01:08.865155+0000","last_became_peered":"2024-09-17T05:01:08.865155+0000","last_unstale":"2024-09-17T05:01:13.942955+0000","last_undegraded":"2024-09-17T05:01:13.942955+0000","last_fullsized":"2024-09-17T05:01:13.942955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940728+0000","last_change":"2024-09-17T05:01:08.868372+0000","last_active":"2024-09-17T05:01:13.940728+0000","last_peered":"2024-09-17T05:01:13.940728+0000","last_clean":"2024-09-17T05:01:13.940728+0000","last_became_active":"2024-09-17T05:01:08.867742+0000","last_became_peered":"2024-09-17T05:01:08.867742+0000","last_unstale":"2024-09-17T05:01:13.940728+0000","last_undegraded":"2024-09-17T05:01:13.940728+0000","last_fullsized":"2024-09-17T05:01:13.940728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:39.401927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942609+0000","last_change":"2024-09-17T05:01:08.884233+0000","last_active":"2024-09-17T05:01:13.942609+0000","last_peered":"2024-09-17T05:01:13.942609+0000","last_clean":"2024-09-17T05:01:13.942609+0000","last_became_active":"2024-09-17T05:01:08.883484+0000","last_became_peered":"2024-09-17T05:01:08.883484+0000","last_unstale":"2024-09-17T05:01:13.942609+0000","last_undegraded":"2024-09-17T05:01:13.942609+0000","last_fullsized":"2024-09-17T05:01:13.942609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892259+0000","last_change":"2024-09-17T05:01:08.862441+0000","last_active":"2024-09-17T05:01:13.892259+0000","last_peered":"2024-09-17T05:01:13.892259+0000","last_clean":"2024-09-17T05:01:13.892259+0000","last_became_active":"2024-09-17T05:01:08.860705+0000","last_became_peered":"2024-09-17T05:01:08.860705+0000","last_unstale":"2024-09-17T05:01:13.892259+0000","last_undegraded":"2024-09-17T05:01:13.892259+0000","last_fullsized":"2024-09-17T05:01:13.892259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:47.926845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942230+0000","last_change":"2024-09-17T05:01:08.869890+0000","last_active":"2024-09-17T05:01:13.942230+0000","last_peered":"2024-09-17T05:01:13.942230+0000","last_clean":"2024-09-17T05:01:13.942230+0000","last_became_active":"2024-09-17T05:01:08.869367+0000","last_became_peered":"2024-09-17T05:01:08.869367+0000","last_unstale":"2024-09-17T05:01:13.942230+0000","last_undegraded":"2024-09-17T05:01:13.942230+0000","last_fullsized":"2024-09-17T05:01:13.942230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892235+0000","last_change":"2024-09-17T05:01:08.891532+0000","last_active":"2024-09-17T05:01:13.892235+0000","last_peered":"2024-09-17T05:01:13.892235+0000","last_clean":"2024-09-17T05:01:13.892235+0000","last_became_active":"2024-09-17T05:01:08.891207+0000","last_became_peered":"2024-09-17T05:01:08.891207+0000","last_unstale":"2024-09-17T05:01:13.892235+0000","last_undegraded":"2024-09-17T05:01:13.892235+0000","last_fullsized":"2024-09-17T05:01:13.892235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:45.826779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890942+0000","last_change":"2024-09-17T05:01:08.864037+0000","last_active":"2024-09-17T05:01:13.890942+0000","last_peered":"2024-09-17T05:01:13.890942+0000","last_clean":"2024-09-17T05:01:13.890942+0000","last_became_active":"2024-09-17T05:01:08.863691+0000","last_became_peered":"2024-09-17T05:01:08.863691+0000","last_unstale":"2024-09-17T05:01:13.890942+0000","last_undegraded":"2024-09-17T05:01:13.890942+0000","last_fullsized":"2024-09-17T05:01:13.890942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892250+0000","last_change":"2024-09-17T05:01:08.860804+0000","last_active":"2024-09-17T05:01:13.892250+0000","last_peered":"2024-09-17T05:01:13.892250+0000","last_clean":"2024-09-17T05:01:13.892250+0000","last_became_active":"2024-09-17T05:01:08.860278+0000","last_became_peered":"2024-09-17T05:01:08.860278+0000","last_unstale":"2024-09-17T05:01:13.892250+0000","last_undegraded":"2024-09-17T05:01:13.892250+0000","last_fullsized":"2024-09-17T05:01:13.892250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:41.450008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892724+0000","last_change":"2024-09-17T05:01:08.876606+0000","last_active":"2024-09-17T05:01:13.892724+0000","last_peered":"2024-09-17T05:01:13.892724+0000","last_clean":"2024-09-17T05:01:13.892724+0000","last_became_active":"2024-09-17T05:01:08.875868+0000","last_became_peered":"2024-09-17T05:01:08.875868+0000","last_unstale":"2024-09-17T05:01:13.892724+0000","last_undegraded":"2024-09-17T05:01:13.892724+0000","last_fullsized":"2024-09-17T05:01:13.892724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:36.517067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893030+0000","last_change":"2024-09-17T05:01:08.882586+0000","last_active":"2024-09-17T05:01:13.893030+0000","last_peered":"2024-09-17T05:01:13.893030+0000","last_clean":"2024-09-17T05:01:13.893030+0000","last_became_active":"2024-09-17T05:01:08.882293+0000","last_became_peered":"2024-09-17T05:01:08.882293+0000","last_unstale":"2024-09-17T05:01:13.893030+0000","last_undegraded":"2024-09-17T05:01:13.893030+0000","last_fullsized":"2024-09-17T05:01:13.893030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:55.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942578+0000","last_change":"2024-09-17T05:01:08.882456+0000","last_active":"2024-09-17T05:01:13.942578+0000","last_peered":"2024-09-17T05:01:13.942578+0000","last_clean":"2024-09-17T05:01:13.942578+0000","last_became_active":"2024-09-17T05:01:08.881935+0000","last_became_peered":"2024-09-17T05:01:08.881935+0000","last_unstale":"2024-09-17T05:01:13.942578+0000","last_undegraded":"2024-09-17T05:01:13.942578+0000","last_fullsized":"2024-09-17T05:01:13.942578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:01.625408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941814+0000","last_change":"2024-09-17T05:01:08.862457+0000","last_active":"2024-09-17T05:01:13.941814+0000","last_peered":"2024-09-17T05:01:13.941814+0000","last_clean":"2024-09-17T05:01:13.941814+0000","last_became_active":"2024-09-17T05:01:08.862099+0000","last_became_peered":"2024-09-17T05:01:08.862099+0000","last_unstale":"2024-09-17T05:01:13.941814+0000","last_undegraded":"2024-09-17T05:01:13.941814+0000","last_fullsized":"2024-09-17T05:01:13.941814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:47.337576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941761+0000","last_change":"2024-09-17T05:01:08.876965+0000","last_active":"2024-09-17T05:01:13.941761+0000","last_peered":"2024-09-17T05:01:13.941761+0000","last_clean":"2024-09-17T05:01:13.941761+0000","last_became_active":"2024-09-17T05:01:08.876594+0000","last_became_peered":"2024-09-17T05:01:08.876594+0000","last_unstale":"2024-09-17T05:01:13.941761+0000","last_undegraded":"2024-09-17T05:01:13.941761+0000","last_fullsized":"2024-09-17T05:01:13.941761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:07.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944208+0000","last_change":"2024-09-17T05:01:08.880532+0000","last_active":"2024-09-17T05:01:13.944208+0000","last_peered":"2024-09-17T05:01:13.944208+0000","last_clean":"2024-09-17T05:01:13.944208+0000","last_became_active":"2024-09-17T05:01:08.880215+0000","last_became_peered":"2024-09-17T05:01:08.880215+0000","last_unstale":"2024-09-17T05:01:13.944208+0000","last_undegraded":"2024-09-17T05:01:13.944208+0000","last_fullsized":"2024-09-17T05:01:13.944208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891730+0000","last_change":"2024-09-17T05:01:08.857708+0000","last_active":"2024-09-17T05:01:13.891730+0000","last_peered":"2024-09-17T05:01:13.891730+0000","last_clean":"2024-09-17T05:01:13.891730+0000","last_became_active":"2024-09-17T05:01:08.856723+0000","last_became_peered":"2024-09-17T05:01:08.856723+0000","last_unstale":"2024-09-17T05:01:13.891730+0000","last_undegraded":"2024-09-17T05:01:13.891730+0000","last_fullsized":"2024-09-17T05:01:13.891730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:19.705413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942123+0000","last_change":"2024-09-17T05:01:08.876806+0000","last_active":"2024-09-17T05:01:13.942123+0000","last_peered":"2024-09-17T05:01:13.942123+0000","last_clean":"2024-09-17T05:01:13.942123+0000","last_became_active":"2024-09-17T05:01:08.876453+0000","last_became_peered":"2024-09-17T05:01:08.876453+0000","last_unstale":"2024-09-17T05:01:13.942123+0000","last_undegraded":"2024-09-17T05:01:13.942123+0000","last_fullsized":"2024-09-17T05:01:13.942123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:02.271107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942651+0000","last_change":"2024-09-17T05:01:08.863011+0000","last_active":"2024-09-17T05:01:13.942651+0000","last_peered":"2024-09-17T05:01:13.942651+0000","last_clean":"2024-09-17T05:01:13.942651+0000","last_became_active":"2024-09-17T05:01:08.862561+0000","last_became_peered":"2024-09-17T05:01:08.862561+0000","last_unstale":"2024-09-17T05:01:13.942651+0000","last_undegraded":"2024-09-17T05:01:13.942651+0000","last_fullsized":"2024-09-17T05:01:13.942651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:40.251476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941484+0000","last_change":"2024-09-17T05:01:08.877286+0000","last_active":"2024-09-17T05:01:13.941484+0000","last_peered":"2024-09-17T05:01:13.941484+0000","last_clean":"2024-09-17T05:01:13.941484+0000","last_became_active":"2024-09-17T05:01:08.876336+0000","last_became_peered":"2024-09-17T05:01:08.876336+0000","last_unstale":"2024-09-17T05:01:13.941484+0000","last_undegraded":"2024-09-17T05:01:13.941484+0000","last_fullsized":"2024-09-17T05:01:13.941484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:48.065065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943117+0000","last_change":"2024-09-17T05:01:08.863862+0000","last_active":"2024-09-17T05:01:13.943117+0000","last_peered":"2024-09-17T05:01:13.943117+0000","last_clean":"2024-09-17T05:01:13.943117+0000","last_became_active":"2024-09-17T05:01:08.863514+0000","last_became_peered":"2024-09-17T05:01:08.863514+0000","last_unstale":"2024-09-17T05:01:13.943117+0000","last_undegraded":"2024-09-17T05:01:13.943117+0000","last_fullsized":"2024-09-17T05:01:13.943117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:54.036443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893288+0000","last_change":"2024-09-17T05:01:08.860436+0000","last_active":"2024-09-17T05:01:13.893288+0000","last_peered":"2024-09-17T05:01:13.893288+0000","last_clean":"2024-09-17T05:01:13.893288+0000","last_became_active":"2024-09-17T05:01:08.860010+0000","last_became_peered":"2024-09-17T05:01:08.860010+0000","last_unstale":"2024-09-17T05:01:13.893288+0000","last_undegraded":"2024-09-17T05:01:13.893288+0000","last_fullsized":"2024-09-17T05:01:13.893288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:00.968662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893613+0000","last_change":"2024-09-17T05:01:08.878717+0000","last_active":"2024-09-17T05:01:13.893613+0000","last_peered":"2024-09-17T05:01:13.893613+0000","last_clean":"2024-09-17T05:01:13.893613+0000","last_became_active":"2024-09-17T05:01:08.878392+0000","last_became_peered":"2024-09-17T05:01:08.878392+0000","last_unstale":"2024-09-17T05:01:13.893613+0000","last_undegraded":"2024-09-17T05:01:13.893613+0000","last_fullsized":"2024-09-17T05:01:13.893613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:50.747553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942379+0000","last_change":"2024-09-17T05:01:08.889143+0000","last_active":"2024-09-17T05:01:13.942379+0000","last_peered":"2024-09-17T05:01:13.942379+0000","last_clean":"2024-09-17T05:01:13.942379+0000","last_became_active":"2024-09-17T05:01:08.888428+0000","last_became_peered":"2024-09-17T05:01:08.888428+0000","last_unstale":"2024-09-17T05:01:13.942379+0000","last_undegraded":"2024-09-17T05:01:13.942379+0000","last_fullsized":"2024-09-17T05:01:13.942379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:01:32.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:01:32.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 -- 172.21.15.16:0/1239009486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 msgr2=0x7efccc0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 --2- 172.21.15.16:0/1239009486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc0b5a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efccc159520 tx=0x7efcc401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 -- 172.21.15.16:0/1239009486 shutdown_connections 2024-09-17T05:01:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 --2- 172.21.15.16:0/1239009486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc0b5a20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 --2- 172.21.15.16:0/1239009486 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efccc152540 0x7efccc15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 -- 172.21.15.16:0/1239009486 >> 172.21.15.16:0/1239009486 conn(0x7efccc0d10c0 msgr2=0x7efccc0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 -- 172.21.15.16:0/1239009486 shutdown_connections 2024-09-17T05:01:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 -- 172.21.15.16:0/1239009486 wait complete. 2024-09-17T05:01:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 Processor -- start 2024-09-17T05:01:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.207+0000 7efcd2e00640 1 -- start start 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efccc06e7a0 0x7efccc14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc139640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efccc152540 0x7efccc1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efccc168e30 con 0x7efccc06ebb0 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efccc168cb0 con 0x7efccc152540 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efccc168fb0 con 0x7efccc06e7a0 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc139640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc139640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43740/0 (socket says 172.21.15.16:43740) 2024-09-17T05:01:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 -- 172.21.15.16:0/3398382698 learned_addr learned my addr 172.21.15.16:0/3398382698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efccbe00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efccc06e7a0 0x7efccc14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd0e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efccc152540 0x7efccc1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 -- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efccc06e7a0 msgr2=0x7efccc14c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efccc06e7a0 0x7efccc14c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 -- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efccc152540 msgr2=0x7efccc1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efccc152540 0x7efccc1506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 -- 172.21.15.16:0/3398382698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc14dc30 con 0x7efccc06ebb0 2024-09-17T05:01:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc3e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc139640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efcb400b410 tx=0x7efcb400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4015030 con 0x7efccc06ebb0 2024-09-17T05:01:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcb4004020 con 0x7efccc06ebb0 2024-09-17T05:01:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc149650 con 0x7efccc06ebb0 2024-09-17T05:01:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40044b0 con 0x7efccc06ebb0 2024-09-17T05:01:32.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc14b1b0 con 0x7efccc06ebb0 2024-09-17T05:01:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.211+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc0b5a20 con 0x7efccc06ebb0 2024-09-17T05:01:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.215+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efcb4018070 con 0x7efccc06ebb0 2024-09-17T05:01:32.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.215+0000 7efcc8c00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efca8073200 0x7efca8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.215+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efcb40940a0 con 0x7efccc06ebb0 2024-09-17T05:01:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.215+0000 7efccbe00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efca8073200 0x7efca8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.215+0000 7efccbe00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efca8073200 0x7efca8075710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcbc004630 tx=0x7efcbc00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:32.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.219+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efcb405e690 con 0x7efccc06ebb0 2024-09-17T05:01:32.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.391+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efccc14d770 con 0x7efca8073200 2024-09-17T05:01:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.395+0000 7efcc8c00640 1 -- 172.21.15.16:0/3398382698 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7efcbc00b7e0 con 0x7efca8073200 2024-09-17T05:01:32.401 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:01:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efca8073200 msgr2=0x7efca8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efca8073200 0x7efca8075710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcbc004630 tx=0x7efcbc00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 msgr2=0x7efccc139640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc139640 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efcb400b410 tx=0x7efcb400b8d0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 shutdown_connections 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efca8073200 0x7efca8075710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efccc152540 0x7efccc1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efccc06ebb0 0x7efccc139640 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 --2- 172.21.15.16:0/3398382698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efccc06e7a0 0x7efccc14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 >> 172.21.15.16:0/3398382698 conn(0x7efccc0d10c0 msgr2=0x7efccc0b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 shutdown_connections 2024-09-17T05:01:32.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.399+0000 7efcd2e00640 1 -- 172.21.15.16:0/3398382698 wait complete. 2024-09-17T05:01:32.422 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-17T05:01:31.466554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236200,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738520,"statfs":{"total":773094113280,"available":772852244480,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003869"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942466+0000","last_change":"2024-09-17T05:01:07.870252+0000","last_active":"2024-09-17T05:01:13.942466+0000","last_peered":"2024-09-17T05:01:13.942466+0000","last_clean":"2024-09-17T05:01:13.942466+0000","last_became_active":"2024-09-17T05:01:07.869832+0000","last_became_peered":"2024-09-17T05:01:07.869832+0000","last_unstale":"2024-09-17T05:01:13.942466+0000","last_undegraded":"2024-09-17T05:01:13.942466+0000","last_fullsized":"2024-09-17T05:01:13.942466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893796+0000","last_change":"2024-09-17T05:01:08.860472+0000","last_active":"2024-09-17T05:01:13.893796+0000","last_peered":"2024-09-17T05:01:13.893796+0000","last_clean":"2024-09-17T05:01:13.893796+0000","last_became_active":"2024-09-17T05:01:08.860121+0000","last_became_peered":"2024-09-17T05:01:08.860121+0000","last_unstale":"2024-09-17T05:01:13.893796+0000","last_undegraded":"2024-09-17T05:01:13.893796+0000","last_fullsized":"2024-09-17T05:01:13.893796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:02.947441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941059+0000","last_change":"2024-09-17T05:01:12.917747+0000","last_active":"2024-09-17T05:01:13.941059+0000","last_peered":"2024-09-17T05:01:13.941059+0000","last_clean":"2024-09-17T05:01:13.941059+0000","last_became_active":"2024-09-17T05:01:12.916952+0000","last_became_peered":"2024-09-17T05:01:12.916952+0000","last_unstale":"2024-09-17T05:01:13.941059+0000","last_undegraded":"2024-09-17T05:01:13.941059+0000","last_fullsized":"2024-09-17T05:01:13.941059+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:00.182691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893999+0000","last_change":"2024-09-17T05:01:07.858483+0000","last_active":"2024-09-17T05:01:13.893999+0000","last_peered":"2024-09-17T05:01:13.893999+0000","last_clean":"2024-09-17T05:01:13.893999+0000","last_became_active":"2024-09-17T05:01:07.858094+0000","last_became_peered":"2024-09-17T05:01:07.858094+0000","last_unstale":"2024-09-17T05:01:13.893999+0000","last_undegraded":"2024-09-17T05:01:13.893999+0000","last_fullsized":"2024-09-17T05:01:13.893999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:31:09.683336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894263+0000","last_change":"2024-09-17T05:01:08.886624+0000","last_active":"2024-09-17T05:01:13.894263+0000","last_peered":"2024-09-17T05:01:13.894263+0000","last_clean":"2024-09-17T05:01:13.894263+0000","last_became_active":"2024-09-17T05:01:08.886308+0000","last_became_peered":"2024-09-17T05:01:08.886308+0000","last_unstale":"2024-09-17T05:01:13.894263+0000","last_undegraded":"2024-09-17T05:01:13.894263+0000","last_fullsized":"2024-09-17T05:01:13.894263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.002578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941422+0000","last_change":"2024-09-17T05:01:12.901716+0000","last_active":"2024-09-17T05:01:13.941422+0000","last_peered":"2024-09-17T05:01:13.941422+0000","last_clean":"2024-09-17T05:01:13.941422+0000","last_became_active":"2024-09-17T05:01:12.901225+0000","last_became_peered":"2024-09-17T05:01:12.901225+0000","last_unstale":"2024-09-17T05:01:13.941422+0000","last_undegraded":"2024-09-17T05:01:13.941422+0000","last_fullsized":"2024-09-17T05:01:13.941422+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:19.972140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942144+0000","last_change":"2024-09-17T05:01:07.857199+0000","last_active":"2024-09-17T05:01:13.942144+0000","last_peered":"2024-09-17T05:01:13.942144+0000","last_clean":"2024-09-17T05:01:13.942144+0000","last_became_active":"2024-09-17T05:01:07.856808+0000","last_became_peered":"2024-09-17T05:01:07.856808+0000","last_unstale":"2024-09-17T05:01:13.942144+0000","last_undegraded":"2024-09-17T05:01:13.942144+0000","last_fullsized":"2024-09-17T05:01:13.942144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943634+0000","last_change":"2024-09-17T05:01:08.880952+0000","last_active":"2024-09-17T05:01:13.943634+0000","last_peered":"2024-09-17T05:01:13.943634+0000","last_clean":"2024-09-17T05:01:13.943634+0000","last_became_active":"2024-09-17T05:01:08.880627+0000","last_became_peered":"2024-09-17T05:01:08.880627+0000","last_unstale":"2024-09-17T05:01:13.943634+0000","last_undegraded":"2024-09-17T05:01:13.943634+0000","last_fullsized":"2024-09-17T05:01:13.943634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:03.625255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943104+0000","last_change":"2024-09-17T05:01:12.903919+0000","last_active":"2024-09-17T05:01:13.943104+0000","last_peered":"2024-09-17T05:01:13.943104+0000","last_clean":"2024-09-17T05:01:13.943104+0000","last_became_active":"2024-09-17T05:01:12.901881+0000","last_became_peered":"2024-09-17T05:01:12.901881+0000","last_unstale":"2024-09-17T05:01:13.943104+0000","last_undegraded":"2024-09-17T05:01:13.943104+0000","last_fullsized":"2024-09-17T05:01:13.943104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:23.883298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942724+0000","last_change":"2024-09-17T05:01:07.859450+0000","last_active":"2024-09-17T05:01:13.942724+0000","last_peered":"2024-09-17T05:01:13.942724+0000","last_clean":"2024-09-17T05:01:13.942724+0000","last_became_active":"2024-09-17T05:01:07.857315+0000","last_became_peered":"2024-09-17T05:01:07.857315+0000","last_unstale":"2024-09-17T05:01:13.942724+0000","last_undegraded":"2024-09-17T05:01:13.942724+0000","last_fullsized":"2024-09-17T05:01:13.942724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:39.723963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892054+0000","last_change":"2024-09-17T05:01:08.893401+0000","last_active":"2024-09-17T05:01:13.892054+0000","last_peered":"2024-09-17T05:01:13.892054+0000","last_clean":"2024-09-17T05:01:13.892054+0000","last_became_active":"2024-09-17T05:01:08.892406+0000","last_became_peered":"2024-09-17T05:01:08.892406+0000","last_unstale":"2024-09-17T05:01:13.892054+0000","last_undegraded":"2024-09-17T05:01:13.892054+0000","last_fullsized":"2024-09-17T05:01:13.892054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:18.887291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942085+0000","last_change":"2024-09-17T05:01:12.918843+0000","last_active":"2024-09-17T05:01:13.942085+0000","last_peered":"2024-09-17T05:01:13.942085+0000","last_clean":"2024-09-17T05:01:13.942085+0000","last_became_active":"2024-09-17T05:01:12.916474+0000","last_became_peered":"2024-09-17T05:01:12.916474+0000","last_unstale":"2024-09-17T05:01:13.942085+0000","last_undegraded":"2024-09-17T05:01:13.942085+0000","last_fullsized":"2024-09-17T05:01:13.942085+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:05.661358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891193+0000","last_change":"2024-09-17T05:01:08.858811+0000","last_active":"2024-09-17T05:01:13.891193+0000","last_peered":"2024-09-17T05:01:13.891193+0000","last_clean":"2024-09-17T05:01:13.891193+0000","last_became_active":"2024-09-17T05:01:08.858013+0000","last_became_peered":"2024-09-17T05:01:08.858013+0000","last_unstale":"2024-09-17T05:01:13.891193+0000","last_undegraded":"2024-09-17T05:01:13.891193+0000","last_fullsized":"2024-09-17T05:01:13.891193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:06.714459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891507+0000","last_change":"2024-09-17T05:01:07.873880+0000","last_active":"2024-09-17T05:01:13.891507+0000","last_peered":"2024-09-17T05:01:13.891507+0000","last_clean":"2024-09-17T05:01:13.891507+0000","last_became_active":"2024-09-17T05:01:07.870873+0000","last_became_peered":"2024-09-17T05:01:07.870873+0000","last_unstale":"2024-09-17T05:01:13.891507+0000","last_undegraded":"2024-09-17T05:01:13.891507+0000","last_fullsized":"2024-09-17T05:01:13.891507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:02.618135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891844+0000","last_change":"2024-09-17T05:01:12.898484+0000","last_active":"2024-09-17T05:01:13.891844+0000","last_peered":"2024-09-17T05:01:13.891844+0000","last_clean":"2024-09-17T05:01:13.891844+0000","last_became_active":"2024-09-17T05:01:12.895423+0000","last_became_peered":"2024-09-17T05:01:12.895423+0000","last_unstale":"2024-09-17T05:01:13.891844+0000","last_undegraded":"2024-09-17T05:01:13.891844+0000","last_fullsized":"2024-09-17T05:01:13.891844+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:54.524295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941368+0000","last_change":"2024-09-17T05:01:08.871889+0000","last_active":"2024-09-17T05:01:13.941368+0000","last_peered":"2024-09-17T05:01:13.941368+0000","last_clean":"2024-09-17T05:01:13.941368+0000","last_became_active":"2024-09-17T05:01:08.871512+0000","last_became_peered":"2024-09-17T05:01:08.871512+0000","last_unstale":"2024-09-17T05:01:13.941368+0000","last_undegraded":"2024-09-17T05:01:13.941368+0000","last_fullsized":"2024-09-17T05:01:13.941368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892848+0000","last_change":"2024-09-17T05:01:07.865154+0000","last_active":"2024-09-17T05:01:13.892848+0000","last_peered":"2024-09-17T05:01:13.892848+0000","last_clean":"2024-09-17T05:01:13.892848+0000","last_became_active":"2024-09-17T05:01:07.864075+0000","last_became_peered":"2024-09-17T05:01:07.864075+0000","last_unstale":"2024-09-17T05:01:13.892848+0000","last_undegraded":"2024-09-17T05:01:13.892848+0000","last_fullsized":"2024-09-17T05:01:13.892848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941790+0000","last_change":"2024-09-17T05:01:12.915985+0000","last_active":"2024-09-17T05:01:13.941790+0000","last_peered":"2024-09-17T05:01:13.941790+0000","last_clean":"2024-09-17T05:01:13.941790+0000","last_became_active":"2024-09-17T05:01:12.915569+0000","last_became_peered":"2024-09-17T05:01:12.915569+0000","last_unstale":"2024-09-17T05:01:13.941790+0000","last_undegraded":"2024-09-17T05:01:13.941790+0000","last_fullsized":"2024-09-17T05:01:13.941790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:15.801136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940431+0000","last_change":"2024-09-17T05:01:08.859016+0000","last_active":"2024-09-17T05:01:13.940431+0000","last_peered":"2024-09-17T05:01:13.940431+0000","last_clean":"2024-09-17T05:01:13.940431+0000","last_became_active":"2024-09-17T05:01:08.858669+0000","last_became_peered":"2024-09-17T05:01:08.858669+0000","last_unstale":"2024-09-17T05:01:13.940431+0000","last_undegraded":"2024-09-17T05:01:13.940431+0000","last_fullsized":"2024-09-17T05:01:13.940431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:48.908013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940792+0000","last_change":"2024-09-17T05:01:07.853871+0000","last_active":"2024-09-17T05:01:13.940792+0000","last_peered":"2024-09-17T05:01:13.940792+0000","last_clean":"2024-09-17T05:01:13.940792+0000","last_became_active":"2024-09-17T05:01:07.853309+0000","last_became_peered":"2024-09-17T05:01:07.853309+0000","last_unstale":"2024-09-17T05:01:13.940792+0000","last_undegraded":"2024-09-17T05:01:13.940792+0000","last_fullsized":"2024-09-17T05:01:13.940792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941719+0000","last_change":"2024-09-17T05:01:12.917589+0000","last_active":"2024-09-17T05:01:13.941719+0000","last_peered":"2024-09-17T05:01:13.941719+0000","last_clean":"2024-09-17T05:01:13.941719+0000","last_became_active":"2024-09-17T05:01:12.917064+0000","last_became_peered":"2024-09-17T05:01:12.917064+0000","last_unstale":"2024-09-17T05:01:13.941719+0000","last_undegraded":"2024-09-17T05:01:13.941719+0000","last_fullsized":"2024-09-17T05:01:13.941719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:47.894698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890960+0000","last_change":"2024-09-17T05:01:08.865800+0000","last_active":"2024-09-17T05:01:13.890960+0000","last_peered":"2024-09-17T05:01:13.890960+0000","last_clean":"2024-09-17T05:01:13.890960+0000","last_became_active":"2024-09-17T05:01:08.865418+0000","last_became_peered":"2024-09-17T05:01:08.865418+0000","last_unstale":"2024-09-17T05:01:13.890960+0000","last_undegraded":"2024-09-17T05:01:13.890960+0000","last_fullsized":"2024-09-17T05:01:13.890960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941039+0000","last_change":"2024-09-17T05:01:07.868079+0000","last_active":"2024-09-17T05:01:13.941039+0000","last_peered":"2024-09-17T05:01:13.941039+0000","last_clean":"2024-09-17T05:01:13.941039+0000","last_became_active":"2024-09-17T05:01:07.867648+0000","last_became_peered":"2024-09-17T05:01:07.867648+0000","last_unstale":"2024-09-17T05:01:13.941039+0000","last_undegraded":"2024-09-17T05:01:13.941039+0000","last_fullsized":"2024-09-17T05:01:13.941039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943609+0000","last_change":"2024-09-17T05:01:12.913059+0000","last_active":"2024-09-17T05:01:13.943609+0000","last_peered":"2024-09-17T05:01:13.943609+0000","last_clean":"2024-09-17T05:01:13.943609+0000","last_became_active":"2024-09-17T05:01:12.912569+0000","last_became_peered":"2024-09-17T05:01:12.912569+0000","last_unstale":"2024-09-17T05:01:13.943609+0000","last_undegraded":"2024-09-17T05:01:13.943609+0000","last_fullsized":"2024-09-17T05:01:13.943609+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:23.311668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941727+0000","last_change":"2024-09-17T05:01:08.869046+0000","last_active":"2024-09-17T05:01:13.941727+0000","last_peered":"2024-09-17T05:01:13.941727+0000","last_clean":"2024-09-17T05:01:13.941727+0000","last_became_active":"2024-09-17T05:01:08.864116+0000","last_became_peered":"2024-09-17T05:01:08.864116+0000","last_unstale":"2024-09-17T05:01:13.941727+0000","last_undegraded":"2024-09-17T05:01:13.941727+0000","last_fullsized":"2024-09-17T05:01:13.941727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:51.571131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892323+0000","last_change":"2024-09-17T05:01:07.863381+0000","last_active":"2024-09-17T05:01:13.892323+0000","last_peered":"2024-09-17T05:01:13.892323+0000","last_clean":"2024-09-17T05:01:13.892323+0000","last_became_active":"2024-09-17T05:01:07.862731+0000","last_became_peered":"2024-09-17T05:01:07.862731+0000","last_unstale":"2024-09-17T05:01:13.892323+0000","last_undegraded":"2024-09-17T05:01:13.892323+0000","last_fullsized":"2024-09-17T05:01:13.892323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:39.024164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942455+0000","last_change":"2024-09-17T05:01:12.916294+0000","last_active":"2024-09-17T05:01:13.942455+0000","last_peered":"2024-09-17T05:01:13.942455+0000","last_clean":"2024-09-17T05:01:13.942455+0000","last_became_active":"2024-09-17T05:01:12.908035+0000","last_became_peered":"2024-09-17T05:01:12.908035+0000","last_unstale":"2024-09-17T05:01:13.942455+0000","last_undegraded":"2024-09-17T05:01:13.942455+0000","last_fullsized":"2024-09-17T05:01:13.942455+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:52.101015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942370+0000","last_change":"2024-09-17T05:01:08.870629+0000","last_active":"2024-09-17T05:01:13.942370+0000","last_peered":"2024-09-17T05:01:13.942370+0000","last_clean":"2024-09-17T05:01:13.942370+0000","last_became_active":"2024-09-17T05:01:08.870198+0000","last_became_peered":"2024-09-17T05:01:08.870198+0000","last_unstale":"2024-09-17T05:01:13.942370+0000","last_undegraded":"2024-09-17T05:01:13.942370+0000","last_fullsized":"2024-09-17T05:01:13.942370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:37.142368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941967+0000","last_change":"2024-09-17T05:01:07.860691+0000","last_active":"2024-09-17T05:01:13.941967+0000","last_peered":"2024-09-17T05:01:13.941967+0000","last_clean":"2024-09-17T05:01:13.941967+0000","last_became_active":"2024-09-17T05:01:07.860360+0000","last_became_peered":"2024-09-17T05:01:07.860360+0000","last_unstale":"2024-09-17T05:01:13.941967+0000","last_undegraded":"2024-09-17T05:01:13.941967+0000","last_fullsized":"2024-09-17T05:01:13.941967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940513+0000","last_change":"2024-09-17T05:01:12.900669+0000","last_active":"2024-09-17T05:01:13.940513+0000","last_peered":"2024-09-17T05:01:13.940513+0000","last_clean":"2024-09-17T05:01:13.940513+0000","last_became_active":"2024-09-17T05:01:12.900234+0000","last_became_peered":"2024-09-17T05:01:12.900234+0000","last_unstale":"2024-09-17T05:01:13.940513+0000","last_undegraded":"2024-09-17T05:01:13.940513+0000","last_fullsized":"2024-09-17T05:01:13.940513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893905+0000","last_change":"2024-09-17T05:01:08.876794+0000","last_active":"2024-09-17T05:01:13.893905+0000","last_peered":"2024-09-17T05:01:13.893905+0000","last_clean":"2024-09-17T05:01:13.893905+0000","last_became_active":"2024-09-17T05:01:08.876418+0000","last_became_peered":"2024-09-17T05:01:08.876418+0000","last_unstale":"2024-09-17T05:01:13.893905+0000","last_undegraded":"2024-09-17T05:01:13.893905+0000","last_fullsized":"2024-09-17T05:01:13.893905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:47.662092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892328+0000","last_change":"2024-09-17T05:01:07.877675+0000","last_active":"2024-09-17T05:01:13.892328+0000","last_peered":"2024-09-17T05:01:13.892328+0000","last_clean":"2024-09-17T05:01:13.892328+0000","last_became_active":"2024-09-17T05:01:07.877320+0000","last_became_peered":"2024-09-17T05:01:07.877320+0000","last_unstale":"2024-09-17T05:01:13.892328+0000","last_undegraded":"2024-09-17T05:01:13.892328+0000","last_fullsized":"2024-09-17T05:01:13.892328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:35.509947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891833+0000","last_change":"2024-09-17T05:01:12.911570+0000","last_active":"2024-09-17T05:01:13.891833+0000","last_peered":"2024-09-17T05:01:13.891833+0000","last_clean":"2024-09-17T05:01:13.891833+0000","last_became_active":"2024-09-17T05:01:12.911110+0000","last_became_peered":"2024-09-17T05:01:12.911110+0000","last_unstale":"2024-09-17T05:01:13.891833+0000","last_undegraded":"2024-09-17T05:01:13.891833+0000","last_fullsized":"2024-09-17T05:01:13.891833+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:38.085327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943250+0000","last_change":"2024-09-17T05:01:08.870024+0000","last_active":"2024-09-17T05:01:13.943250+0000","last_peered":"2024-09-17T05:01:13.943250+0000","last_clean":"2024-09-17T05:01:13.943250+0000","last_became_active":"2024-09-17T05:01:08.869679+0000","last_became_peered":"2024-09-17T05:01:08.869679+0000","last_unstale":"2024-09-17T05:01:13.943250+0000","last_undegraded":"2024-09-17T05:01:13.943250+0000","last_fullsized":"2024-09-17T05:01:13.943250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:45.964067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892819+0000","last_change":"2024-09-17T05:01:07.859295+0000","last_active":"2024-09-17T05:01:13.892819+0000","last_peered":"2024-09-17T05:01:13.892819+0000","last_clean":"2024-09-17T05:01:13.892819+0000","last_became_active":"2024-09-17T05:01:07.858949+0000","last_became_peered":"2024-09-17T05:01:07.858949+0000","last_unstale":"2024-09-17T05:01:13.892819+0000","last_undegraded":"2024-09-17T05:01:13.892819+0000","last_fullsized":"2024-09-17T05:01:13.892819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:56:29.635679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892487+0000","last_change":"2024-09-17T05:01:12.915413+0000","last_active":"2024-09-17T05:01:13.892487+0000","last_peered":"2024-09-17T05:01:13.892487+0000","last_clean":"2024-09-17T05:01:13.892487+0000","last_became_active":"2024-09-17T05:01:12.914942+0000","last_became_peered":"2024-09-17T05:01:12.914942+0000","last_unstale":"2024-09-17T05:01:13.892487+0000","last_undegraded":"2024-09-17T05:01:13.892487+0000","last_fullsized":"2024-09-17T05:01:13.892487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:57.733227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941319+0000","last_change":"2024-09-17T05:01:08.871457+0000","last_active":"2024-09-17T05:01:13.941319+0000","last_peered":"2024-09-17T05:01:13.941319+0000","last_clean":"2024-09-17T05:01:13.941319+0000","last_became_active":"2024-09-17T05:01:08.871135+0000","last_became_peered":"2024-09-17T05:01:08.871135+0000","last_unstale":"2024-09-17T05:01:13.941319+0000","last_undegraded":"2024-09-17T05:01:13.941319+0000","last_fullsized":"2024-09-17T05:01:13.941319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:15.936190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:07.873885+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:07.869214+0000","last_became_peered":"2024-09-17T05:01:07.869214+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:16.951737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893521+0000","last_change":"2024-09-17T05:01:12.908768+0000","last_active":"2024-09-17T05:01:13.893521+0000","last_peered":"2024-09-17T05:01:13.893521+0000","last_clean":"2024-09-17T05:01:13.893521+0000","last_became_active":"2024-09-17T05:01:12.908282+0000","last_became_peered":"2024-09-17T05:01:12.908282+0000","last_unstale":"2024-09-17T05:01:13.893521+0000","last_undegraded":"2024-09-17T05:01:13.893521+0000","last_fullsized":"2024-09-17T05:01:13.893521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:03:54.622089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891855+0000","last_change":"2024-09-17T05:01:08.858109+0000","last_active":"2024-09-17T05:01:13.891855+0000","last_peered":"2024-09-17T05:01:13.891855+0000","last_clean":"2024-09-17T05:01:13.891855+0000","last_became_active":"2024-09-17T05:01:08.857293+0000","last_became_peered":"2024-09-17T05:01:08.857293+0000","last_unstale":"2024-09-17T05:01:13.891855+0000","last_undegraded":"2024-09-17T05:01:13.891855+0000","last_fullsized":"2024-09-17T05:01:13.891855+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:53.180346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941193+0000","last_change":"2024-09-17T05:01:07.864654+0000","last_active":"2024-09-17T05:01:13.941193+0000","last_peered":"2024-09-17T05:01:13.941193+0000","last_clean":"2024-09-17T05:01:13.941193+0000","last_became_active":"2024-09-17T05:01:07.864166+0000","last_became_peered":"2024-09-17T05:01:07.864166+0000","last_unstale":"2024-09-17T05:01:13.941193+0000","last_undegraded":"2024-09-17T05:01:13.941193+0000","last_fullsized":"2024-09-17T05:01:13.941193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894265+0000","last_change":"2024-09-17T05:01:12.914524+0000","last_active":"2024-09-17T05:01:13.894265+0000","last_peered":"2024-09-17T05:01:13.894265+0000","last_clean":"2024-09-17T05:01:13.894265+0000","last_became_active":"2024-09-17T05:01:12.912222+0000","last_became_peered":"2024-09-17T05:01:12.912222+0000","last_unstale":"2024-09-17T05:01:13.894265+0000","last_undegraded":"2024-09-17T05:01:13.894265+0000","last_fullsized":"2024-09-17T05:01:13.894265+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892306+0000","last_change":"2024-09-17T05:01:08.875660+0000","last_active":"2024-09-17T05:01:13.892306+0000","last_peered":"2024-09-17T05:01:13.892306+0000","last_clean":"2024-09-17T05:01:13.892306+0000","last_became_active":"2024-09-17T05:01:08.875265+0000","last_became_peered":"2024-09-17T05:01:08.875265+0000","last_unstale":"2024-09-17T05:01:13.892306+0000","last_undegraded":"2024-09-17T05:01:13.892306+0000","last_fullsized":"2024-09-17T05:01:13.892306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891158+0000","last_change":"2024-09-17T05:01:07.869491+0000","last_active":"2024-09-17T05:01:13.891158+0000","last_peered":"2024-09-17T05:01:13.891158+0000","last_clean":"2024-09-17T05:01:13.891158+0000","last_became_active":"2024-09-17T05:01:07.869038+0000","last_became_peered":"2024-09-17T05:01:07.869038+0000","last_unstale":"2024-09-17T05:01:13.891158+0000","last_undegraded":"2024-09-17T05:01:13.891158+0000","last_fullsized":"2024-09-17T05:01:13.891158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:00.812725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893711+0000","last_change":"2024-09-17T05:01:12.910830+0000","last_active":"2024-09-17T05:01:13.893711+0000","last_peered":"2024-09-17T05:01:13.893711+0000","last_clean":"2024-09-17T05:01:13.893711+0000","last_became_active":"2024-09-17T05:01:12.908497+0000","last_became_peered":"2024-09-17T05:01:12.908497+0000","last_unstale":"2024-09-17T05:01:13.893711+0000","last_undegraded":"2024-09-17T05:01:13.893711+0000","last_fullsized":"2024-09-17T05:01:13.893711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:07.419369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941121+0000","last_change":"2024-09-17T05:01:08.868170+0000","last_active":"2024-09-17T05:01:13.941121+0000","last_peered":"2024-09-17T05:01:13.941121+0000","last_clean":"2024-09-17T05:01:13.941121+0000","last_became_active":"2024-09-17T05:01:08.867767+0000","last_became_peered":"2024-09-17T05:01:08.867767+0000","last_unstale":"2024-09-17T05:01:13.941121+0000","last_undegraded":"2024-09-17T05:01:13.941121+0000","last_fullsized":"2024-09-17T05:01:13.941121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891555+0000","last_change":"2024-09-17T05:01:07.873908+0000","last_active":"2024-09-17T05:01:13.891555+0000","last_peered":"2024-09-17T05:01:13.891555+0000","last_clean":"2024-09-17T05:01:13.891555+0000","last_became_active":"2024-09-17T05:01:07.871028+0000","last_became_peered":"2024-09-17T05:01:07.871028+0000","last_unstale":"2024-09-17T05:01:13.891555+0000","last_undegraded":"2024-09-17T05:01:13.891555+0000","last_fullsized":"2024-09-17T05:01:13.891555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941889+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:01:13.941889+0000","last_peered":"2024-09-17T05:01:13.941889+0000","last_clean":"2024-09-17T05:01:13.941889+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:01:13.941889+0000","last_undegraded":"2024-09-17T05:01:13.941889+0000","last_fullsized":"2024-09-17T05:01:13.941889+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892300+0000","last_change":"2024-09-17T05:01:12.904503+0000","last_active":"2024-09-17T05:01:13.892300+0000","last_peered":"2024-09-17T05:01:13.892300+0000","last_clean":"2024-09-17T05:01:13.892300+0000","last_became_active":"2024-09-17T05:01:12.903910+0000","last_became_peered":"2024-09-17T05:01:12.903910+0000","last_unstale":"2024-09-17T05:01:13.892300+0000","last_undegraded":"2024-09-17T05:01:13.892300+0000","last_fullsized":"2024-09-17T05:01:13.892300+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:47.490639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892152+0000","last_change":"2024-09-17T05:01:08.872909+0000","last_active":"2024-09-17T05:01:13.892152+0000","last_peered":"2024-09-17T05:01:13.892152+0000","last_clean":"2024-09-17T05:01:13.892152+0000","last_became_active":"2024-09-17T05:01:08.861070+0000","last_became_peered":"2024-09-17T05:01:08.861070+0000","last_unstale":"2024-09-17T05:01:13.892152+0000","last_undegraded":"2024-09-17T05:01:13.892152+0000","last_fullsized":"2024-09-17T05:01:13.892152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:44.525838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894016+0000","last_change":"2024-09-17T05:01:07.866417+0000","last_active":"2024-09-17T05:01:13.894016+0000","last_peered":"2024-09-17T05:01:13.894016+0000","last_clean":"2024-09-17T05:01:13.894016+0000","last_became_active":"2024-09-17T05:01:07.866088+0000","last_became_peered":"2024-09-17T05:01:07.866088+0000","last_unstale":"2024-09-17T05:01:13.894016+0000","last_undegraded":"2024-09-17T05:01:13.894016+0000","last_fullsized":"2024-09-17T05:01:13.894016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942333+0000","last_change":"2024-09-17T05:01:12.919691+0000","last_active":"2024-09-17T05:01:13.942333+0000","last_peered":"2024-09-17T05:01:13.942333+0000","last_clean":"2024-09-17T05:01:13.942333+0000","last_became_active":"2024-09-17T05:01:12.917951+0000","last_became_peered":"2024-09-17T05:01:12.917951+0000","last_unstale":"2024-09-17T05:01:13.942333+0000","last_undegraded":"2024-09-17T05:01:13.942333+0000","last_fullsized":"2024-09-17T05:01:13.942333+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:11.524293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942694+0000","last_change":"2024-09-17T05:01:08.864231+0000","last_active":"2024-09-17T05:01:13.942694+0000","last_peered":"2024-09-17T05:01:13.942694+0000","last_clean":"2024-09-17T05:01:13.942694+0000","last_became_active":"2024-09-17T05:01:08.863802+0000","last_became_peered":"2024-09-17T05:01:08.863802+0000","last_unstale":"2024-09-17T05:01:13.942694+0000","last_undegraded":"2024-09-17T05:01:13.942694+0000","last_fullsized":"2024-09-17T05:01:13.942694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893511+0000","last_change":"2024-09-17T05:01:07.857650+0000","last_active":"2024-09-17T05:01:13.893511+0000","last_peered":"2024-09-17T05:01:13.893511+0000","last_clean":"2024-09-17T05:01:13.893511+0000","last_became_active":"2024-09-17T05:01:07.857311+0000","last_became_peered":"2024-09-17T05:01:07.857311+0000","last_unstale":"2024-09-17T05:01:13.893511+0000","last_undegraded":"2024-09-17T05:01:13.893511+0000","last_fullsized":"2024-09-17T05:01:13.893511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:27.902116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893168+0000","last_change":"2024-09-17T05:01:12.888243+0000","last_active":"2024-09-17T05:01:13.893168+0000","last_peered":"2024-09-17T05:01:13.893168+0000","last_clean":"2024-09-17T05:01:13.893168+0000","last_became_active":"2024-09-17T05:01:12.887793+0000","last_became_peered":"2024-09-17T05:01:12.887793+0000","last_unstale":"2024-09-17T05:01:13.893168+0000","last_undegraded":"2024-09-17T05:01:13.893168+0000","last_fullsized":"2024-09-17T05:01:13.893168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:11:00.157222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942457+0000","last_change":"2024-09-17T05:01:08.889497+0000","last_active":"2024-09-17T05:01:13.942457+0000","last_peered":"2024-09-17T05:01:13.942457+0000","last_clean":"2024-09-17T05:01:13.942457+0000","last_became_active":"2024-09-17T05:01:08.887064+0000","last_became_peered":"2024-09-17T05:01:08.887064+0000","last_unstale":"2024-09-17T05:01:13.942457+0000","last_undegraded":"2024-09-17T05:01:13.942457+0000","last_fullsized":"2024-09-17T05:01:13.942457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:35.761326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941847+0000","last_change":"2024-09-17T05:01:07.851404+0000","last_active":"2024-09-17T05:01:13.941847+0000","last_peered":"2024-09-17T05:01:13.941847+0000","last_clean":"2024-09-17T05:01:13.941847+0000","last_became_active":"2024-09-17T05:01:07.851077+0000","last_became_peered":"2024-09-17T05:01:07.851077+0000","last_unstale":"2024-09-17T05:01:13.941847+0000","last_undegraded":"2024-09-17T05:01:13.941847+0000","last_fullsized":"2024-09-17T05:01:13.941847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890772+0000","last_change":"2024-09-17T05:01:12.905280+0000","last_active":"2024-09-17T05:01:13.890772+0000","last_peered":"2024-09-17T05:01:13.890772+0000","last_clean":"2024-09-17T05:01:13.890772+0000","last_became_active":"2024-09-17T05:01:12.904787+0000","last_became_peered":"2024-09-17T05:01:12.904787+0000","last_unstale":"2024-09-17T05:01:13.890772+0000","last_undegraded":"2024-09-17T05:01:13.890772+0000","last_fullsized":"2024-09-17T05:01:13.890772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:29.666258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.884835+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.884410+0000","last_became_peered":"2024-09-17T05:01:08.884410+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:42:35.960729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942469+0000","last_change":"2024-09-17T05:01:07.866602+0000","last_active":"2024-09-17T05:01:13.942469+0000","last_peered":"2024-09-17T05:01:13.942469+0000","last_clean":"2024-09-17T05:01:13.942469+0000","last_became_active":"2024-09-17T05:01:07.866061+0000","last_became_peered":"2024-09-17T05:01:07.866061+0000","last_unstale":"2024-09-17T05:01:13.942469+0000","last_undegraded":"2024-09-17T05:01:13.942469+0000","last_fullsized":"2024-09-17T05:01:13.942469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:32.970459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943528+0000","last_change":"2024-09-17T05:01:12.904969+0000","last_active":"2024-09-17T05:01:13.943528+0000","last_peered":"2024-09-17T05:01:13.943528+0000","last_clean":"2024-09-17T05:01:13.943528+0000","last_became_active":"2024-09-17T05:01:12.904665+0000","last_became_peered":"2024-09-17T05:01:12.904665+0000","last_unstale":"2024-09-17T05:01:13.943528+0000","last_undegraded":"2024-09-17T05:01:13.943528+0000","last_fullsized":"2024-09-17T05:01:13.943528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:52.943963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890912+0000","last_change":"2024-09-17T05:01:08.872790+0000","last_active":"2024-09-17T05:01:13.890912+0000","last_peered":"2024-09-17T05:01:13.890912+0000","last_clean":"2024-09-17T05:01:13.890912+0000","last_became_active":"2024-09-17T05:01:08.872387+0000","last_became_peered":"2024-09-17T05:01:08.872387+0000","last_unstale":"2024-09-17T05:01:13.890912+0000","last_undegraded":"2024-09-17T05:01:13.890912+0000","last_fullsized":"2024-09-17T05:01:13.890912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:00.607098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940575+0000","last_change":"2024-09-17T05:01:07.854980+0000","last_active":"2024-09-17T05:01:13.940575+0000","last_peered":"2024-09-17T05:01:13.940575+0000","last_clean":"2024-09-17T05:01:13.940575+0000","last_became_active":"2024-09-17T05:01:07.854665+0000","last_became_peered":"2024-09-17T05:01:07.854665+0000","last_unstale":"2024-09-17T05:01:13.940575+0000","last_undegraded":"2024-09-17T05:01:13.940575+0000","last_fullsized":"2024-09-17T05:01:13.940575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:01.359168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893909+0000","last_change":"2024-09-17T05:01:12.908923+0000","last_active":"2024-09-17T05:01:13.893909+0000","last_peered":"2024-09-17T05:01:13.893909+0000","last_clean":"2024-09-17T05:01:13.893909+0000","last_became_active":"2024-09-17T05:01:12.908398+0000","last_became_peered":"2024-09-17T05:01:12.908398+0000","last_unstale":"2024-09-17T05:01:13.893909+0000","last_undegraded":"2024-09-17T05:01:13.893909+0000","last_fullsized":"2024-09-17T05:01:13.893909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:42:45.118442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941757+0000","last_change":"2024-09-17T05:01:08.856311+0000","last_active":"2024-09-17T05:01:13.941757+0000","last_peered":"2024-09-17T05:01:13.941757+0000","last_clean":"2024-09-17T05:01:13.941757+0000","last_became_active":"2024-09-17T05:01:08.855334+0000","last_became_peered":"2024-09-17T05:01:08.855334+0000","last_unstale":"2024-09-17T05:01:13.941757+0000","last_undegraded":"2024-09-17T05:01:13.941757+0000","last_fullsized":"2024-09-17T05:01:13.941757+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942247+0000","last_change":"2024-09-17T05:01:07.872183+0000","last_active":"2024-09-17T05:01:13.942247+0000","last_peered":"2024-09-17T05:01:13.942247+0000","last_clean":"2024-09-17T05:01:13.942247+0000","last_became_active":"2024-09-17T05:01:07.871540+0000","last_became_peered":"2024-09-17T05:01:07.871540+0000","last_unstale":"2024-09-17T05:01:13.942247+0000","last_undegraded":"2024-09-17T05:01:13.942247+0000","last_fullsized":"2024-09-17T05:01:13.942247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:14.532792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894214+0000","last_change":"2024-09-17T05:01:12.906286+0000","last_active":"2024-09-17T05:01:13.894214+0000","last_peered":"2024-09-17T05:01:13.894214+0000","last_clean":"2024-09-17T05:01:13.894214+0000","last_became_active":"2024-09-17T05:01:12.905810+0000","last_became_peered":"2024-09-17T05:01:12.905810+0000","last_unstale":"2024-09-17T05:01:13.894214+0000","last_undegraded":"2024-09-17T05:01:13.894214+0000","last_fullsized":"2024-09-17T05:01:13.894214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:33.278743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891317+0000","last_change":"2024-09-17T05:01:08.893475+0000","last_active":"2024-09-17T05:01:13.891317+0000","last_peered":"2024-09-17T05:01:13.891317+0000","last_clean":"2024-09-17T05:01:13.891317+0000","last_became_active":"2024-09-17T05:01:08.893130+0000","last_became_peered":"2024-09-17T05:01:08.893130+0000","last_unstale":"2024-09-17T05:01:13.891317+0000","last_undegraded":"2024-09-17T05:01:13.891317+0000","last_fullsized":"2024-09-17T05:01:13.891317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891492+0000","last_change":"2024-09-17T05:01:07.869315+0000","last_active":"2024-09-17T05:01:13.891492+0000","last_peered":"2024-09-17T05:01:13.891492+0000","last_clean":"2024-09-17T05:01:13.891492+0000","last_became_active":"2024-09-17T05:01:07.864783+0000","last_became_peered":"2024-09-17T05:01:07.864783+0000","last_unstale":"2024-09-17T05:01:13.891492+0000","last_undegraded":"2024-09-17T05:01:13.891492+0000","last_fullsized":"2024-09-17T05:01:13.891492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893698+0000","last_change":"2024-09-17T05:01:12.921137+0000","last_active":"2024-09-17T05:01:13.893698+0000","last_peered":"2024-09-17T05:01:13.893698+0000","last_clean":"2024-09-17T05:01:13.893698+0000","last_became_active":"2024-09-17T05:01:12.920809+0000","last_became_peered":"2024-09-17T05:01:12.920809+0000","last_unstale":"2024-09-17T05:01:13.893698+0000","last_undegraded":"2024-09-17T05:01:13.893698+0000","last_fullsized":"2024-09-17T05:01:13.893698+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892745+0000","last_change":"2024-09-17T05:01:08.892262+0000","last_active":"2024-09-17T05:01:13.892745+0000","last_peered":"2024-09-17T05:01:13.892745+0000","last_clean":"2024-09-17T05:01:13.892745+0000","last_became_active":"2024-09-17T05:01:08.891946+0000","last_became_peered":"2024-09-17T05:01:08.891946+0000","last_unstale":"2024-09-17T05:01:13.892745+0000","last_undegraded":"2024-09-17T05:01:13.892745+0000","last_fullsized":"2024-09-17T05:01:13.892745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:59:40.031283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941492+0000","last_change":"2024-09-17T05:01:07.872646+0000","last_active":"2024-09-17T05:01:13.941492+0000","last_peered":"2024-09-17T05:01:13.941492+0000","last_clean":"2024-09-17T05:01:13.941492+0000","last_became_active":"2024-09-17T05:01:07.872287+0000","last_became_peered":"2024-09-17T05:01:07.872287+0000","last_unstale":"2024-09-17T05:01:13.941492+0000","last_undegraded":"2024-09-17T05:01:13.941492+0000","last_fullsized":"2024-09-17T05:01:13.941492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:56.785994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891950+0000","last_change":"2024-09-17T05:01:12.900658+0000","last_active":"2024-09-17T05:01:13.891950+0000","last_peered":"2024-09-17T05:01:13.891950+0000","last_clean":"2024-09-17T05:01:13.891950+0000","last_became_active":"2024-09-17T05:01:12.900131+0000","last_became_peered":"2024-09-17T05:01:12.900131+0000","last_unstale":"2024-09-17T05:01:13.891950+0000","last_undegraded":"2024-09-17T05:01:13.891950+0000","last_fullsized":"2024-09-17T05:01:13.891950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:54.427072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941925+0000","last_change":"2024-09-17T05:01:08.869878+0000","last_active":"2024-09-17T05:01:13.941925+0000","last_peered":"2024-09-17T05:01:13.941925+0000","last_clean":"2024-09-17T05:01:13.941925+0000","last_became_active":"2024-09-17T05:01:08.869571+0000","last_became_peered":"2024-09-17T05:01:08.869571+0000","last_unstale":"2024-09-17T05:01:13.941925+0000","last_undegraded":"2024-09-17T05:01:13.941925+0000","last_fullsized":"2024-09-17T05:01:13.941925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:29.847123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:19.977855+0000","last_change":"2024-09-17T05:01:07.862966+0000","last_active":"2024-09-17T05:01:19.977855+0000","last_peered":"2024-09-17T05:01:19.977855+0000","last_clean":"2024-09-17T05:01:19.977855+0000","last_became_active":"2024-09-17T05:01:07.862605+0000","last_became_peered":"2024-09-17T05:01:07.862605+0000","last_unstale":"2024-09-17T05:01:19.977855+0000","last_undegraded":"2024-09-17T05:01:19.977855+0000","last_fullsized":"2024-09-17T05:01:19.977855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:26.467925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941104+0000","last_change":"2024-09-17T05:01:12.897023+0000","last_active":"2024-09-17T05:01:13.941104+0000","last_peered":"2024-09-17T05:01:13.941104+0000","last_clean":"2024-09-17T05:01:13.941104+0000","last_became_active":"2024-09-17T05:01:12.896528+0000","last_became_peered":"2024-09-17T05:01:12.896528+0000","last_unstale":"2024-09-17T05:01:13.941104+0000","last_undegraded":"2024-09-17T05:01:13.941104+0000","last_fullsized":"2024-09-17T05:01:13.941104+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:04.392152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891485+0000","last_change":"2024-09-17T05:01:08.866251+0000","last_active":"2024-09-17T05:01:13.891485+0000","last_peered":"2024-09-17T05:01:13.891485+0000","last_clean":"2024-09-17T05:01:13.891485+0000","last_became_active":"2024-09-17T05:01:08.864997+0000","last_became_peered":"2024-09-17T05:01:08.864997+0000","last_unstale":"2024-09-17T05:01:13.891485+0000","last_undegraded":"2024-09-17T05:01:13.891485+0000","last_fullsized":"2024-09-17T05:01:13.891485+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:33.305210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942113+0000","last_change":"2024-09-17T05:01:07.862662+0000","last_active":"2024-09-17T05:01:13.942113+0000","last_peered":"2024-09-17T05:01:13.942113+0000","last_clean":"2024-09-17T05:01:13.942113+0000","last_became_active":"2024-09-17T05:01:07.862252+0000","last_became_peered":"2024-09-17T05:01:07.862252+0000","last_unstale":"2024-09-17T05:01:13.942113+0000","last_undegraded":"2024-09-17T05:01:13.942113+0000","last_fullsized":"2024-09-17T05:01:13.942113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:13.958450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942881+0000","last_change":"2024-09-17T05:01:12.915486+0000","last_active":"2024-09-17T05:01:13.942881+0000","last_peered":"2024-09-17T05:01:13.942881+0000","last_clean":"2024-09-17T05:01:13.942881+0000","last_became_active":"2024-09-17T05:01:12.913794+0000","last_became_peered":"2024-09-17T05:01:12.913794+0000","last_unstale":"2024-09-17T05:01:13.942881+0000","last_undegraded":"2024-09-17T05:01:13.942881+0000","last_fullsized":"2024-09-17T05:01:13.942881+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:33.883073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.894291+0000","last_change":"2024-09-17T05:01:08.863593+0000","last_active":"2024-09-17T05:01:13.894291+0000","last_peered":"2024-09-17T05:01:13.894291+0000","last_clean":"2024-09-17T05:01:13.894291+0000","last_became_active":"2024-09-17T05:01:08.863264+0000","last_became_peered":"2024-09-17T05:01:08.863264+0000","last_unstale":"2024-09-17T05:01:13.894291+0000","last_undegraded":"2024-09-17T05:01:13.894291+0000","last_fullsized":"2024-09-17T05:01:13.894291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:00.186616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942093+0000","last_change":"2024-09-17T05:01:07.860203+0000","last_active":"2024-09-17T05:01:13.942093+0000","last_peered":"2024-09-17T05:01:13.942093+0000","last_clean":"2024-09-17T05:01:13.942093+0000","last_became_active":"2024-09-17T05:01:07.858682+0000","last_became_peered":"2024-09-17T05:01:07.858682+0000","last_unstale":"2024-09-17T05:01:13.942093+0000","last_undegraded":"2024-09-17T05:01:13.942093+0000","last_fullsized":"2024-09-17T05:01:13.942093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:53.531619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940512+0000","last_change":"2024-09-17T05:01:12.912101+0000","last_active":"2024-09-17T05:01:13.940512+0000","last_peered":"2024-09-17T05:01:13.940512+0000","last_clean":"2024-09-17T05:01:13.940512+0000","last_became_active":"2024-09-17T05:01:12.911449+0000","last_became_peered":"2024-09-17T05:01:12.911449+0000","last_unstale":"2024-09-17T05:01:13.940512+0000","last_undegraded":"2024-09-17T05:01:13.940512+0000","last_fullsized":"2024-09-17T05:01:13.940512+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:51.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943817+0000","last_change":"2024-09-17T05:01:08.874496+0000","last_active":"2024-09-17T05:01:13.943817+0000","last_peered":"2024-09-17T05:01:13.943817+0000","last_clean":"2024-09-17T05:01:13.943817+0000","last_became_active":"2024-09-17T05:01:08.874209+0000","last_became_peered":"2024-09-17T05:01:08.874209+0000","last_unstale":"2024-09-17T05:01:13.943817+0000","last_undegraded":"2024-09-17T05:01:13.943817+0000","last_fullsized":"2024-09-17T05:01:13.943817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:06.383708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944328+0000","last_change":"2024-09-17T05:01:07.850788+0000","last_active":"2024-09-17T05:01:13.944328+0000","last_peered":"2024-09-17T05:01:13.944328+0000","last_clean":"2024-09-17T05:01:13.944328+0000","last_became_active":"2024-09-17T05:01:07.850154+0000","last_became_peered":"2024-09-17T05:01:07.850154+0000","last_unstale":"2024-09-17T05:01:13.944328+0000","last_undegraded":"2024-09-17T05:01:13.944328+0000","last_fullsized":"2024-09-17T05:01:13.944328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:37.351474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940756+0000","last_change":"2024-09-17T05:01:12.908537+0000","last_active":"2024-09-17T05:01:13.940756+0000","last_peered":"2024-09-17T05:01:13.940756+0000","last_clean":"2024-09-17T05:01:13.940756+0000","last_became_active":"2024-09-17T05:01:12.908060+0000","last_became_peered":"2024-09-17T05:01:12.908060+0000","last_unstale":"2024-09-17T05:01:13.940756+0000","last_undegraded":"2024-09-17T05:01:13.940756+0000","last_fullsized":"2024-09-17T05:01:13.940756+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:46:32.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941572+0000","last_change":"2024-09-17T05:01:12.895140+0000","last_active":"2024-09-17T05:01:13.941572+0000","last_peered":"2024-09-17T05:01:13.941572+0000","last_clean":"2024-09-17T05:01:13.941572+0000","last_became_active":"2024-09-17T05:01:12.894665+0000","last_became_peered":"2024-09-17T05:01:12.894665+0000","last_unstale":"2024-09-17T05:01:13.941572+0000","last_undegraded":"2024-09-17T05:01:13.941572+0000","last_fullsized":"2024-09-17T05:01:13.941572+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:41:22.431196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940721+0000","last_change":"2024-09-17T05:01:08.851860+0000","last_active":"2024-09-17T05:01:13.940721+0000","last_peered":"2024-09-17T05:01:13.940721+0000","last_clean":"2024-09-17T05:01:13.940721+0000","last_became_active":"2024-09-17T05:01:08.851546+0000","last_became_peered":"2024-09-17T05:01:08.851546+0000","last_unstale":"2024-09-17T05:01:13.940721+0000","last_undegraded":"2024-09-17T05:01:13.940721+0000","last_fullsized":"2024-09-17T05:01:13.940721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:25.615656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891094+0000","last_change":"2024-09-17T05:01:07.876275+0000","last_active":"2024-09-17T05:01:13.891094+0000","last_peered":"2024-09-17T05:01:13.891094+0000","last_clean":"2024-09-17T05:01:13.891094+0000","last_became_active":"2024-09-17T05:01:07.875993+0000","last_became_peered":"2024-09-17T05:01:07.875993+0000","last_unstale":"2024-09-17T05:01:13.891094+0000","last_undegraded":"2024-09-17T05:01:13.891094+0000","last_fullsized":"2024-09-17T05:01:13.891094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:43.841239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891826+0000","last_change":"2024-09-17T05:01:12.904195+0000","last_active":"2024-09-17T05:01:13.891826+0000","last_peered":"2024-09-17T05:01:13.891826+0000","last_clean":"2024-09-17T05:01:13.891826+0000","last_became_active":"2024-09-17T05:01:12.903808+0000","last_became_peered":"2024-09-17T05:01:12.903808+0000","last_unstale":"2024-09-17T05:01:13.891826+0000","last_undegraded":"2024-09-17T05:01:13.891826+0000","last_fullsized":"2024-09-17T05:01:13.891826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:11.189684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891943+0000","last_change":"2024-09-17T05:01:08.876556+0000","last_active":"2024-09-17T05:01:13.891943+0000","last_peered":"2024-09-17T05:01:13.891943+0000","last_clean":"2024-09-17T05:01:13.891943+0000","last_became_active":"2024-09-17T05:01:08.874581+0000","last_became_peered":"2024-09-17T05:01:08.874581+0000","last_unstale":"2024-09-17T05:01:13.891943+0000","last_undegraded":"2024-09-17T05:01:13.891943+0000","last_fullsized":"2024-09-17T05:01:13.891943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:59.333120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892433+0000","last_change":"2024-09-17T05:01:07.865323+0000","last_active":"2024-09-17T05:01:13.892433+0000","last_peered":"2024-09-17T05:01:13.892433+0000","last_clean":"2024-09-17T05:01:13.892433+0000","last_became_active":"2024-09-17T05:01:07.864832+0000","last_became_peered":"2024-09-17T05:01:07.864832+0000","last_unstale":"2024-09-17T05:01:13.892433+0000","last_undegraded":"2024-09-17T05:01:13.892433+0000","last_fullsized":"2024-09-17T05:01:13.892433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:58.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891348+0000","last_change":"2024-09-17T05:01:07.870957+0000","last_active":"2024-09-17T05:01:13.891348+0000","last_peered":"2024-09-17T05:01:13.891348+0000","last_clean":"2024-09-17T05:01:13.891348+0000","last_became_active":"2024-09-17T05:01:07.869025+0000","last_became_peered":"2024-09-17T05:01:07.869025+0000","last_unstale":"2024-09-17T05:01:13.891348+0000","last_undegraded":"2024-09-17T05:01:13.891348+0000","last_fullsized":"2024-09-17T05:01:13.891348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:17.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940880+0000","last_change":"2024-09-17T05:01:08.876853+0000","last_active":"2024-09-17T05:01:13.940880+0000","last_peered":"2024-09-17T05:01:13.940880+0000","last_clean":"2024-09-17T05:01:13.940880+0000","last_became_active":"2024-09-17T05:01:08.876525+0000","last_became_peered":"2024-09-17T05:01:08.876525+0000","last_unstale":"2024-09-17T05:01:13.940880+0000","last_undegraded":"2024-09-17T05:01:13.940880+0000","last_fullsized":"2024-09-17T05:01:13.940880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:50.685520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941188+0000","last_change":"2024-09-17T05:01:12.908492+0000","last_active":"2024-09-17T05:01:13.941188+0000","last_peered":"2024-09-17T05:01:13.941188+0000","last_clean":"2024-09-17T05:01:13.941188+0000","last_became_active":"2024-09-17T05:01:12.908032+0000","last_became_peered":"2024-09-17T05:01:12.908032+0000","last_unstale":"2024-09-17T05:01:13.941188+0000","last_undegraded":"2024-09-17T05:01:13.941188+0000","last_fullsized":"2024-09-17T05:01:13.941188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:52.016519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893141+0000","last_change":"2024-09-17T05:01:07.859924+0000","last_active":"2024-09-17T05:01:13.893141+0000","last_peered":"2024-09-17T05:01:13.893141+0000","last_clean":"2024-09-17T05:01:13.893141+0000","last_became_active":"2024-09-17T05:01:07.859692+0000","last_became_peered":"2024-09-17T05:01:07.859692+0000","last_unstale":"2024-09-17T05:01:13.893141+0000","last_undegraded":"2024-09-17T05:01:13.893141+0000","last_fullsized":"2024-09-17T05:01:13.893141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:06.820389+0000","last_clean_scrub_stamp":"2024-09-17T05:01:06.820389+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:35.935910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943063+0000","last_change":"2024-09-17T05:01:08.886919+0000","last_active":"2024-09-17T05:01:13.943063+0000","last_peered":"2024-09-17T05:01:13.943063+0000","last_clean":"2024-09-17T05:01:13.943063+0000","last_became_active":"2024-09-17T05:01:08.886565+0000","last_became_peered":"2024-09-17T05:01:08.886565+0000","last_unstale":"2024-09-17T05:01:13.943063+0000","last_undegraded":"2024-09-17T05:01:13.943063+0000","last_fullsized":"2024-09-17T05:01:13.943063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941737+0000","last_change":"2024-09-17T05:01:12.919497+0000","last_active":"2024-09-17T05:01:13.941737+0000","last_peered":"2024-09-17T05:01:13.941737+0000","last_clean":"2024-09-17T05:01:13.941737+0000","last_became_active":"2024-09-17T05:01:12.919097+0000","last_became_peered":"2024-09-17T05:01:12.919097+0000","last_unstale":"2024-09-17T05:01:13.941737+0000","last_undegraded":"2024-09-17T05:01:13.941737+0000","last_fullsized":"2024-09-17T05:01:13.941737+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:11.863458+0000","last_clean_scrub_stamp":"2024-09-17T05:01:11.863458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:33.403463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941489+0000","last_change":"2024-09-17T05:01:08.868818+0000","last_active":"2024-09-17T05:01:13.941489+0000","last_peered":"2024-09-17T05:01:13.941489+0000","last_clean":"2024-09-17T05:01:13.941489+0000","last_became_active":"2024-09-17T05:01:08.868496+0000","last_became_peered":"2024-09-17T05:01:08.868496+0000","last_unstale":"2024-09-17T05:01:13.941489+0000","last_undegraded":"2024-09-17T05:01:13.941489+0000","last_fullsized":"2024-09-17T05:01:13.941489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:17.846926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941062+0000","last_change":"2024-09-17T05:01:08.881690+0000","last_active":"2024-09-17T05:01:13.941062+0000","last_peered":"2024-09-17T05:01:13.941062+0000","last_clean":"2024-09-17T05:01:13.941062+0000","last_became_active":"2024-09-17T05:01:08.880670+0000","last_became_peered":"2024-09-17T05:01:08.880670+0000","last_unstale":"2024-09-17T05:01:13.941062+0000","last_undegraded":"2024-09-17T05:01:13.941062+0000","last_fullsized":"2024-09-17T05:01:13.941062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:44.848359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891163+0000","last_change":"2024-09-17T05:01:08.867225+0000","last_active":"2024-09-17T05:01:13.891163+0000","last_peered":"2024-09-17T05:01:13.891163+0000","last_clean":"2024-09-17T05:01:13.891163+0000","last_became_active":"2024-09-17T05:01:08.866861+0000","last_became_peered":"2024-09-17T05:01:08.866861+0000","last_unstale":"2024-09-17T05:01:13.891163+0000","last_undegraded":"2024-09-17T05:01:13.891163+0000","last_fullsized":"2024-09-17T05:01:13.891163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:40.190786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892992+0000","last_change":"2024-09-17T05:01:08.862318+0000","last_active":"2024-09-17T05:01:13.892992+0000","last_peered":"2024-09-17T05:01:13.892992+0000","last_clean":"2024-09-17T05:01:13.892992+0000","last_became_active":"2024-09-17T05:01:08.861957+0000","last_became_peered":"2024-09-17T05:01:08.861957+0000","last_unstale":"2024-09-17T05:01:13.892992+0000","last_undegraded":"2024-09-17T05:01:13.892992+0000","last_fullsized":"2024-09-17T05:01:13.892992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:26.208177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893817+0000","last_change":"2024-09-17T05:01:08.883574+0000","last_active":"2024-09-17T05:01:13.893817+0000","last_peered":"2024-09-17T05:01:13.893817+0000","last_clean":"2024-09-17T05:01:13.893817+0000","last_became_active":"2024-09-17T05:01:08.883327+0000","last_became_peered":"2024-09-17T05:01:08.883327+0000","last_unstale":"2024-09-17T05:01:13.893817+0000","last_undegraded":"2024-09-17T05:01:13.893817+0000","last_fullsized":"2024-09-17T05:01:13.893817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:36.720707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893848+0000","last_change":"2024-09-17T05:01:08.877996+0000","last_active":"2024-09-17T05:01:13.893848+0000","last_peered":"2024-09-17T05:01:13.893848+0000","last_clean":"2024-09-17T05:01:13.893848+0000","last_became_active":"2024-09-17T05:01:08.877545+0000","last_became_peered":"2024-09-17T05:01:08.877545+0000","last_unstale":"2024-09-17T05:01:13.893848+0000","last_undegraded":"2024-09-17T05:01:13.893848+0000","last_fullsized":"2024-09-17T05:01:13.893848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:43.858748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943694+0000","last_change":"2024-09-17T05:01:08.874485+0000","last_active":"2024-09-17T05:01:13.943694+0000","last_peered":"2024-09-17T05:01:13.943694+0000","last_clean":"2024-09-17T05:01:13.943694+0000","last_became_active":"2024-09-17T05:01:08.874134+0000","last_became_peered":"2024-09-17T05:01:08.874134+0000","last_unstale":"2024-09-17T05:01:13.943694+0000","last_undegraded":"2024-09-17T05:01:13.943694+0000","last_fullsized":"2024-09-17T05:01:13.943694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:10.348602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893707+0000","last_change":"2024-09-17T05:01:08.886688+0000","last_active":"2024-09-17T05:01:13.893707+0000","last_peered":"2024-09-17T05:01:13.893707+0000","last_clean":"2024-09-17T05:01:13.893707+0000","last_became_active":"2024-09-17T05:01:08.886385+0000","last_became_peered":"2024-09-17T05:01:08.886385+0000","last_unstale":"2024-09-17T05:01:13.893707+0000","last_undegraded":"2024-09-17T05:01:13.893707+0000","last_fullsized":"2024-09-17T05:01:13.893707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:29.234911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891025+0000","last_change":"2024-09-17T05:01:08.866999+0000","last_active":"2024-09-17T05:01:13.891025+0000","last_peered":"2024-09-17T05:01:13.891025+0000","last_clean":"2024-09-17T05:01:13.891025+0000","last_became_active":"2024-09-17T05:01:08.866661+0000","last_became_peered":"2024-09-17T05:01:08.866661+0000","last_unstale":"2024-09-17T05:01:13.891025+0000","last_undegraded":"2024-09-17T05:01:13.891025+0000","last_fullsized":"2024-09-17T05:01:13.891025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:40.685724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892397+0000","last_change":"2024-09-17T05:01:08.871375+0000","last_active":"2024-09-17T05:01:13.892397+0000","last_peered":"2024-09-17T05:01:13.892397+0000","last_clean":"2024-09-17T05:01:13.892397+0000","last_became_active":"2024-09-17T05:01:08.870439+0000","last_became_peered":"2024-09-17T05:01:08.870439+0000","last_unstale":"2024-09-17T05:01:13.892397+0000","last_undegraded":"2024-09-17T05:01:13.892397+0000","last_fullsized":"2024-09-17T05:01:13.892397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:49.417178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942955+0000","last_change":"2024-09-17T05:01:08.866055+0000","last_active":"2024-09-17T05:01:13.942955+0000","last_peered":"2024-09-17T05:01:13.942955+0000","last_clean":"2024-09-17T05:01:13.942955+0000","last_became_active":"2024-09-17T05:01:08.865155+0000","last_became_peered":"2024-09-17T05:01:08.865155+0000","last_unstale":"2024-09-17T05:01:13.942955+0000","last_undegraded":"2024-09-17T05:01:13.942955+0000","last_fullsized":"2024-09-17T05:01:13.942955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:17.958856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.940728+0000","last_change":"2024-09-17T05:01:08.868372+0000","last_active":"2024-09-17T05:01:13.940728+0000","last_peered":"2024-09-17T05:01:13.940728+0000","last_clean":"2024-09-17T05:01:13.940728+0000","last_became_active":"2024-09-17T05:01:08.867742+0000","last_became_peered":"2024-09-17T05:01:08.867742+0000","last_unstale":"2024-09-17T05:01:13.940728+0000","last_undegraded":"2024-09-17T05:01:13.940728+0000","last_fullsized":"2024-09-17T05:01:13.940728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:39.401927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942609+0000","last_change":"2024-09-17T05:01:08.884233+0000","last_active":"2024-09-17T05:01:13.942609+0000","last_peered":"2024-09-17T05:01:13.942609+0000","last_clean":"2024-09-17T05:01:13.942609+0000","last_became_active":"2024-09-17T05:01:08.883484+0000","last_became_peered":"2024-09-17T05:01:08.883484+0000","last_unstale":"2024-09-17T05:01:13.942609+0000","last_undegraded":"2024-09-17T05:01:13.942609+0000","last_fullsized":"2024-09-17T05:01:13.942609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892259+0000","last_change":"2024-09-17T05:01:08.862441+0000","last_active":"2024-09-17T05:01:13.892259+0000","last_peered":"2024-09-17T05:01:13.892259+0000","last_clean":"2024-09-17T05:01:13.892259+0000","last_became_active":"2024-09-17T05:01:08.860705+0000","last_became_peered":"2024-09-17T05:01:08.860705+0000","last_unstale":"2024-09-17T05:01:13.892259+0000","last_undegraded":"2024-09-17T05:01:13.892259+0000","last_fullsized":"2024-09-17T05:01:13.892259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:47.926845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942230+0000","last_change":"2024-09-17T05:01:08.869890+0000","last_active":"2024-09-17T05:01:13.942230+0000","last_peered":"2024-09-17T05:01:13.942230+0000","last_clean":"2024-09-17T05:01:13.942230+0000","last_became_active":"2024-09-17T05:01:08.869367+0000","last_became_peered":"2024-09-17T05:01:08.869367+0000","last_unstale":"2024-09-17T05:01:13.942230+0000","last_undegraded":"2024-09-17T05:01:13.942230+0000","last_fullsized":"2024-09-17T05:01:13.942230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:01.819844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892235+0000","last_change":"2024-09-17T05:01:08.891532+0000","last_active":"2024-09-17T05:01:13.892235+0000","last_peered":"2024-09-17T05:01:13.892235+0000","last_clean":"2024-09-17T05:01:13.892235+0000","last_became_active":"2024-09-17T05:01:08.891207+0000","last_became_peered":"2024-09-17T05:01:08.891207+0000","last_unstale":"2024-09-17T05:01:13.892235+0000","last_undegraded":"2024-09-17T05:01:13.892235+0000","last_fullsized":"2024-09-17T05:01:13.892235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:45.826779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.890942+0000","last_change":"2024-09-17T05:01:08.864037+0000","last_active":"2024-09-17T05:01:13.890942+0000","last_peered":"2024-09-17T05:01:13.890942+0000","last_clean":"2024-09-17T05:01:13.890942+0000","last_became_active":"2024-09-17T05:01:08.863691+0000","last_became_peered":"2024-09-17T05:01:08.863691+0000","last_unstale":"2024-09-17T05:01:13.890942+0000","last_undegraded":"2024-09-17T05:01:13.890942+0000","last_fullsized":"2024-09-17T05:01:13.890942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:19.847347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892250+0000","last_change":"2024-09-17T05:01:08.860804+0000","last_active":"2024-09-17T05:01:13.892250+0000","last_peered":"2024-09-17T05:01:13.892250+0000","last_clean":"2024-09-17T05:01:13.892250+0000","last_became_active":"2024-09-17T05:01:08.860278+0000","last_became_peered":"2024-09-17T05:01:08.860278+0000","last_unstale":"2024-09-17T05:01:13.892250+0000","last_undegraded":"2024-09-17T05:01:13.892250+0000","last_fullsized":"2024-09-17T05:01:13.892250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:41.450008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.892724+0000","last_change":"2024-09-17T05:01:08.876606+0000","last_active":"2024-09-17T05:01:13.892724+0000","last_peered":"2024-09-17T05:01:13.892724+0000","last_clean":"2024-09-17T05:01:13.892724+0000","last_became_active":"2024-09-17T05:01:08.875868+0000","last_became_peered":"2024-09-17T05:01:08.875868+0000","last_unstale":"2024-09-17T05:01:13.892724+0000","last_undegraded":"2024-09-17T05:01:13.892724+0000","last_fullsized":"2024-09-17T05:01:13.892724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:36.517067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893030+0000","last_change":"2024-09-17T05:01:08.882586+0000","last_active":"2024-09-17T05:01:13.893030+0000","last_peered":"2024-09-17T05:01:13.893030+0000","last_clean":"2024-09-17T05:01:13.893030+0000","last_became_active":"2024-09-17T05:01:08.882293+0000","last_became_peered":"2024-09-17T05:01:08.882293+0000","last_unstale":"2024-09-17T05:01:13.893030+0000","last_undegraded":"2024-09-17T05:01:13.893030+0000","last_fullsized":"2024-09-17T05:01:13.893030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:55.608285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942578+0000","last_change":"2024-09-17T05:01:08.882456+0000","last_active":"2024-09-17T05:01:13.942578+0000","last_peered":"2024-09-17T05:01:13.942578+0000","last_clean":"2024-09-17T05:01:13.942578+0000","last_became_active":"2024-09-17T05:01:08.881935+0000","last_became_peered":"2024-09-17T05:01:08.881935+0000","last_unstale":"2024-09-17T05:01:13.942578+0000","last_undegraded":"2024-09-17T05:01:13.942578+0000","last_fullsized":"2024-09-17T05:01:13.942578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:01.625408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941814+0000","last_change":"2024-09-17T05:01:08.862457+0000","last_active":"2024-09-17T05:01:13.941814+0000","last_peered":"2024-09-17T05:01:13.941814+0000","last_clean":"2024-09-17T05:01:13.941814+0000","last_became_active":"2024-09-17T05:01:08.862099+0000","last_became_peered":"2024-09-17T05:01:08.862099+0000","last_unstale":"2024-09-17T05:01:13.941814+0000","last_undegraded":"2024-09-17T05:01:13.941814+0000","last_fullsized":"2024-09-17T05:01:13.941814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:47.337576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941761+0000","last_change":"2024-09-17T05:01:08.876965+0000","last_active":"2024-09-17T05:01:13.941761+0000","last_peered":"2024-09-17T05:01:13.941761+0000","last_clean":"2024-09-17T05:01:13.941761+0000","last_became_active":"2024-09-17T05:01:08.876594+0000","last_became_peered":"2024-09-17T05:01:08.876594+0000","last_unstale":"2024-09-17T05:01:13.941761+0000","last_undegraded":"2024-09-17T05:01:13.941761+0000","last_fullsized":"2024-09-17T05:01:13.941761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:07.488342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.944208+0000","last_change":"2024-09-17T05:01:08.880532+0000","last_active":"2024-09-17T05:01:13.944208+0000","last_peered":"2024-09-17T05:01:13.944208+0000","last_clean":"2024-09-17T05:01:13.944208+0000","last_became_active":"2024-09-17T05:01:08.880215+0000","last_became_peered":"2024-09-17T05:01:08.880215+0000","last_unstale":"2024-09-17T05:01:13.944208+0000","last_undegraded":"2024-09-17T05:01:13.944208+0000","last_fullsized":"2024-09-17T05:01:13.944208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:33.977579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.891730+0000","last_change":"2024-09-17T05:01:08.857708+0000","last_active":"2024-09-17T05:01:13.891730+0000","last_peered":"2024-09-17T05:01:13.891730+0000","last_clean":"2024-09-17T05:01:13.891730+0000","last_became_active":"2024-09-17T05:01:08.856723+0000","last_became_peered":"2024-09-17T05:01:08.856723+0000","last_unstale":"2024-09-17T05:01:13.891730+0000","last_undegraded":"2024-09-17T05:01:13.891730+0000","last_fullsized":"2024-09-17T05:01:13.891730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:19.705413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942123+0000","last_change":"2024-09-17T05:01:08.876806+0000","last_active":"2024-09-17T05:01:13.942123+0000","last_peered":"2024-09-17T05:01:13.942123+0000","last_clean":"2024-09-17T05:01:13.942123+0000","last_became_active":"2024-09-17T05:01:08.876453+0000","last_became_peered":"2024-09-17T05:01:08.876453+0000","last_unstale":"2024-09-17T05:01:13.942123+0000","last_undegraded":"2024-09-17T05:01:13.942123+0000","last_fullsized":"2024-09-17T05:01:13.942123+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:02.271107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942651+0000","last_change":"2024-09-17T05:01:08.863011+0000","last_active":"2024-09-17T05:01:13.942651+0000","last_peered":"2024-09-17T05:01:13.942651+0000","last_clean":"2024-09-17T05:01:13.942651+0000","last_became_active":"2024-09-17T05:01:08.862561+0000","last_became_peered":"2024-09-17T05:01:08.862561+0000","last_unstale":"2024-09-17T05:01:13.942651+0000","last_undegraded":"2024-09-17T05:01:13.942651+0000","last_fullsized":"2024-09-17T05:01:13.942651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:40.251476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.941484+0000","last_change":"2024-09-17T05:01:08.877286+0000","last_active":"2024-09-17T05:01:13.941484+0000","last_peered":"2024-09-17T05:01:13.941484+0000","last_clean":"2024-09-17T05:01:13.941484+0000","last_became_active":"2024-09-17T05:01:08.876336+0000","last_became_peered":"2024-09-17T05:01:08.876336+0000","last_unstale":"2024-09-17T05:01:13.941484+0000","last_undegraded":"2024-09-17T05:01:13.941484+0000","last_fullsized":"2024-09-17T05:01:13.941484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:48.065065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.943117+0000","last_change":"2024-09-17T05:01:08.863862+0000","last_active":"2024-09-17T05:01:13.943117+0000","last_peered":"2024-09-17T05:01:13.943117+0000","last_clean":"2024-09-17T05:01:13.943117+0000","last_became_active":"2024-09-17T05:01:08.863514+0000","last_became_peered":"2024-09-17T05:01:08.863514+0000","last_unstale":"2024-09-17T05:01:13.943117+0000","last_undegraded":"2024-09-17T05:01:13.943117+0000","last_fullsized":"2024-09-17T05:01:13.943117+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:54.036443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893288+0000","last_change":"2024-09-17T05:01:08.860436+0000","last_active":"2024-09-17T05:01:13.893288+0000","last_peered":"2024-09-17T05:01:13.893288+0000","last_clean":"2024-09-17T05:01:13.893288+0000","last_became_active":"2024-09-17T05:01:08.860010+0000","last_became_peered":"2024-09-17T05:01:08.860010+0000","last_unstale":"2024-09-17T05:01:13.893288+0000","last_undegraded":"2024-09-17T05:01:13.893288+0000","last_fullsized":"2024-09-17T05:01:13.893288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:00.968662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.893613+0000","last_change":"2024-09-17T05:01:08.878717+0000","last_active":"2024-09-17T05:01:13.893613+0000","last_peered":"2024-09-17T05:01:13.893613+0000","last_clean":"2024-09-17T05:01:13.893613+0000","last_became_active":"2024-09-17T05:01:08.878392+0000","last_became_peered":"2024-09-17T05:01:08.878392+0000","last_unstale":"2024-09-17T05:01:13.893613+0000","last_undegraded":"2024-09-17T05:01:13.893613+0000","last_fullsized":"2024-09-17T05:01:13.893613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:10:50.747553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T05:01:13.942379+0000","last_change":"2024-09-17T05:01:08.889143+0000","last_active":"2024-09-17T05:01:13.942379+0000","last_peered":"2024-09-17T05:01:13.942379+0000","last_clean":"2024-09-17T05:01:13.942379+0000","last_became_active":"2024-09-17T05:01:08.888428+0000","last_became_peered":"2024-09-17T05:01:08.888428+0000","last_unstale":"2024-09-17T05:01:13.942379+0000","last_undegraded":"2024-09-17T05:01:13.942379+0000","last_fullsized":"2024-09-17T05:01:13.942379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:01:07.827508+0000","last_clean_scrub_stamp":"2024-09-17T05:01:07.827508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:19.275718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:01:32.427 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:01:32.427 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T05:01:32.427 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T05:01:32.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:01:32.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 -- 172.21.15.16:0/2779400070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa794003a10 msgr2=0x7fa794003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/2779400070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa794003a10 0x7fa794003e80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000c290 tx=0x7fa7a001cfa0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b1a00640 1 -- 172.21.15.16:0/2779400070 reap_dead start 2024-09-17T05:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 -- 172.21.15.16:0/2779400070 shutdown_connections 2024-09-17T05:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 -- 172.21.15.16:0/2779400070 >> 172.21.15.16:0/2779400070 conn(0x7fa7ac0cecc0 msgr2=0x7fa7ac0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:32.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 -- 172.21.15.16:0/2779400070 shutdown_connections 2024-09-17T05:01:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 -- 172.21.15.16:0/2779400070 wait complete. 2024-09-17T05:01:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.623+0000 7fa7b3a00640 1 Processor -- start 2024-09-17T05:01:32.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 -- start start 2024-09-17T05:01:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa794003a10 0x7fa7ac0b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7ac154a80 0x7fa7ac1fbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 0x7fa7ac15f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac1673e0 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ac167260 con 0x7fa7ac154a80 2024-09-17T05:01:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ac167560 con 0x7fa794003a10 2024-09-17T05:01:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 0x7fa7ac15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 0x7fa7ac15f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43756/0 (socket says 172.21.15.16:43756) 2024-09-17T05:01:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 -- 172.21.15.16:0/3780615604 learned_addr learned my addr 172.21.15.16:0/3780615604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7abe00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7ac154a80 0x7fa7ac1fbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1000640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa794003a10 0x7fa7ac0b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 -- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa794003a10 msgr2=0x7fa7ac0b7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa794003a10 0x7fa7ac0b7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 -- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7ac154a80 msgr2=0x7fa7ac1fbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7ac154a80 0x7fa7ac1fbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 -- 172.21.15.16:0/3780615604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ac15fa40 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b1a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 0x7fa7ac15f500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac075060 tx=0x7fa7a4010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a400ed50 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7a4015070 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003bc0 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ac1fc370 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.627+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ac1fc930 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.631+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ac1fc0e0 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.631+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7a4004030 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.631+0000 7fa7a8c00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa78c073230 0x7fa78c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.631+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa7a4094b80 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.631+0000 7fa7b1000640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa78c073230 0x7fa78c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.631+0000 7fa7b1000640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa78c073230 0x7fa78c075740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00ac000 tx=0x7fa7a0019de0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:32.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.635+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7ac15fa40 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.847+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa7ac0d1ce0 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.847+0000 7fa7a8c00640 1 -- 172.21.15.16:0/3780615604 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa7a405f200 con 0x7fa7ac15f0f0 2024-09-17T05:01:32.848 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:32.848 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa78c073230 msgr2=0x7fa78c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa78c073230 0x7fa78c075740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00ac000 tx=0x7fa7a0019de0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 msgr2=0x7fa7ac15f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 0x7fa7ac15f500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac075060 tx=0x7fa7a4010e60 comp rx=0 tx=0).stop 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 shutdown_connections 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa78c073230 0x7fa78c075740 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac15f0f0 0x7fa7ac15f500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7ac154a80 0x7fa7ac1fbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 --2- 172.21.15.16:0/3780615604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa794003a10 0x7fa7ac0b7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 >> 172.21.15.16:0/3780615604 conn(0x7fa7ac0cecc0 msgr2=0x7fa7ac0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 shutdown_connections 2024-09-17T05:01:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:32.851+0000 7fa7b3a00640 1 -- 172.21.15.16:0/3780615604 wait complete. 2024-09-17T05:01:32.872 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T05:01:32.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 -- 172.21.15.16:0/4291151633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44071210 msgr2=0x7fbb44071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 --2- 172.21.15.16:0/4291151633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44071210 0x7fbb44071620 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbb44072bd0 tx=0x7fbb2c01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 -- 172.21.15.16:0/4291151633 shutdown_connections 2024-09-17T05:01:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 --2- 172.21.15.16:0/4291151633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44071210 0x7fbb44071620 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 --2- 172.21.15.16:0/4291151633 >> v2:172.21.15.104:3300/0 conn(0x7fbb44152740 0x7fbb440b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 -- 172.21.15.16:0/4291151633 >> 172.21.15.16:0/4291151633 conn(0x7fbb440d10f0 msgr2=0x7fbb440d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 -- 172.21.15.16:0/4291151633 shutdown_connections 2024-09-17T05:01:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.031+0000 7fbb43200640 1 -- 172.21.15.16:0/4291151633 wait complete. 2024-09-17T05:01:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 Processor -- start 2024-09-17T05:01:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 -- start start 2024-09-17T05:01:33.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb44071210 0x7fbb441fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 0x7fbb44154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb4415f020 0x7fbb441550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44169200 con 0x7fbb44152740 2024-09-17T05:01:33.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb44169080 con 0x7fbb44071210 2024-09-17T05:01:33.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb43200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbb44169380 con 0x7fbb4415f020 2024-09-17T05:01:33.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 0x7fbb44154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 0x7fbb44154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43766/0 (socket says 172.21.15.16:43766) 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.035+0000 7fbb3be00640 1 -- 172.21.15.16:0/2471802698 learned_addr learned my addr 172.21.15.16:0/2471802698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb41e00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb44071210 0x7fbb441fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3be00640 1 -- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb4415f020 msgr2=0x7fbb441550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb42800640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb4415f020 0x7fbb441550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb4415f020 0x7fbb441550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3be00640 1 -- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb44071210 msgr2=0x7fbb441fbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb44071210 0x7fbb441fbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3be00640 1 -- 172.21.15.16:0/2471802698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb441512a0 con 0x7fbb44152740 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb41e00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb44071210 0x7fbb441fbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb42800640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb4415f020 0x7fbb441550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 0x7fbb44154b60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c01ccb0 tx=0x7fbb2c005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0b2040 con 0x7fbb44152740 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb2c0bb070 con 0x7fbb44152740 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c003600 con 0x7fbb44152740 2024-09-17T05:01:33.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb44161660 con 0x7fbb44152740 2024-09-17T05:01:33.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb44161c20 con 0x7fbb44152740 2024-09-17T05:01:33.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb441613d0 con 0x7fbb44152740 2024-09-17T05:01:33.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb2c0037a0 con 0x7fbb44152740 2024-09-17T05:01:33.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.039+0000 7fbb3a000640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb20073200 0x7fbb20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.043+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbb2c12eae0 con 0x7fbb44152740 2024-09-17T05:01:33.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.043+0000 7fbb41e00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb20073200 0x7fbb20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.043+0000 7fbb41e00640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb20073200 0x7fbb20075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009ac0 tx=0x7fbb340091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:33.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.047+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb2c0163f0 con 0x7fbb44152740 2024-09-17T05:01:33.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.251+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb44071620 con 0x7fbb44152740 2024-09-17T05:01:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.251+0000 7fbb3a000640 1 -- 172.21.15.16:0/2471802698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7fbb2c0f9160 con 0x7fbb44152740 2024-09-17T05:01:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:33.254 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:33.254 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb20073200 msgr2=0x7fbb20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb20073200 0x7fbb20075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009ac0 tx=0x7fbb340091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 msgr2=0x7fbb44154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 0x7fbb44154b60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c01ccb0 tx=0x7fbb2c005d20 comp rx=0 tx=0).stop 2024-09-17T05:01:33.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 shutdown_connections 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb20073200 0x7fbb20075710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb4415f020 0x7fbb441550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb44152740 0x7fbb44154b60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 --2- 172.21.15.16:0/2471802698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb44071210 0x7fbb441fbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 >> 172.21.15.16:0/2471802698 conn(0x7fbb440d10f0 msgr2=0x7fbb4415cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 shutdown_connections 2024-09-17T05:01:33.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.255+0000 7fbb43200640 1 -- 172.21.15.16:0/2471802698 wait complete. 2024-09-17T05:01:33.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf0ce00640 1 -- 172.21.15.16:0/3872385629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 msgr2=0x7faeec005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf0ce00640 1 --2- 172.21.15.16:0/3872385629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faeec005230 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faf00009eb0 tx=0x7faf0001d5d0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf06800640 1 -- 172.21.15.16:0/3872385629 reap_dead start 2024-09-17T05:01:33.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf0ce00640 1 -- 172.21.15.16:0/3872385629 shutdown_connections 2024-09-17T05:01:33.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf0ce00640 1 -- 172.21.15.16:0/3872385629 >> 172.21.15.16:0/3872385629 conn(0x7faf0806f900 msgr2=0x7faf080b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:33.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf0ce00640 1 -- 172.21.15.16:0/3872385629 shutdown_connections 2024-09-17T05:01:33.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.435+0000 7faf0ce00640 1 -- 172.21.15.16:0/3872385629 wait complete. 2024-09-17T05:01:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 Processor -- start 2024-09-17T05:01:33.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 -- start start 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faf0814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0814cf60 0x7faf0815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf081602a0 0x7faf08072db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf080b5280 con 0x7faeec004da0 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf0815dba0 con 0x7faf0814cf60 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf0ce00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7faf080b44b0 con 0x7faf081602a0 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.439+0000 7faf05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faf0814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faf0814c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43778/0 (socket says 172.21.15.16:43778) 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 -- 172.21.15.16:0/1062526253 learned_addr learned my addr 172.21.15.16:0/1062526253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:33.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05400640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0814cf60 0x7faf0815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf06800640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf081602a0 0x7faf08072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 -- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf081602a0 msgr2=0x7faf08072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf081602a0 0x7faf08072db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 -- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0814cf60 msgr2=0x7faf0815fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0814cf60 0x7faf0815fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 -- 172.21.15.16:0/1062526253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf081606b0 con 0x7faeec004da0 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05400640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0814cf60 0x7faf0815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf06800640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf081602a0 0x7faf08072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf05e00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faf0814c1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faef8012690 tx=0x7faef8012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8010ae0 con 0x7faeec004da0 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef8017030 con 0x7faeec004da0 2024-09-17T05:01:33.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8012e90 con 0x7faeec004da0 2024-09-17T05:01:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf08151b80 con 0x7faeec004da0 2024-09-17T05:01:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf08152140 con 0x7faeec004da0 2024-09-17T05:01:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf080ccad0 con 0x7faeec004da0 2024-09-17T05:01:33.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.443+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faef8013450 con 0x7faeec004da0 2024-09-17T05:01:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.447+0000 7faefe000640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faee80733a0 0x7faee80758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.447+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faef8098150 con 0x7faeec004da0 2024-09-17T05:01:33.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.447+0000 7faf05400640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faee80733a0 0x7faee80758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.447+0000 7faf05400640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faee80733a0 0x7faee80758b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faef4006fd0 tx=0x7faef4006ad0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.451+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf081606b0 con 0x7faeec004da0 2024-09-17T05:01:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf0815a530 con 0x7faeec004da0 2024-09-17T05:01:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faefe000640 1 -- 172.21.15.16:0/1062526253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7faef8062eb0 con 0x7faeec004da0 2024-09-17T05:01:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:33.659 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faee80733a0 msgr2=0x7faee80758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faee80733a0 0x7faee80758b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faef4006fd0 tx=0x7faef4006ad0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 msgr2=0x7faf0814c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faf0814c1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faef8012690 tx=0x7faef8012b50 comp rx=0 tx=0).stop 2024-09-17T05:01:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 shutdown_connections 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faee80733a0 0x7faee80758b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf081602a0 0x7faf08072db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0814cf60 0x7faf0815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 --2- 172.21.15.16:0/1062526253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeec004da0 0x7faf0814c1a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 >> 172.21.15.16:0/1062526253 conn(0x7faf0806f900 msgr2=0x7faf080afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 shutdown_connections 2024-09-17T05:01:33.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.659+0000 7faf0ce00640 1 -- 172.21.15.16:0/1062526253 wait complete. 2024-09-17T05:01:33.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 -- 172.21.15.16:0/4279527385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f973415c270 msgr2=0x7f973415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 --2- 172.21.15.16:0/4279527385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f973415c270 0x7f973415c680 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9720009840 tx=0x7f972001c790 comp rx=0 tx=0).stop 2024-09-17T05:01:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 -- 172.21.15.16:0/4279527385 shutdown_connections 2024-09-17T05:01:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 --2- 172.21.15.16:0/4279527385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f973415c270 0x7f973415c680 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 -- 172.21.15.16:0/4279527385 >> 172.21.15.16:0/4279527385 conn(0x7f973406f900 msgr2=0x7f97340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 -- 172.21.15.16:0/4279527385 shutdown_connections 2024-09-17T05:01:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 -- 172.21.15.16:0/4279527385 wait complete. 2024-09-17T05:01:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 Processor -- start 2024-09-17T05:01:33.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.839+0000 7f973c200640 1 -- start start 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 0x7f9734165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97340b2f50 0x7f9734150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f973415c270 0x7f973406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9734169d60 con 0x7f9734073240 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9734169be0 con 0x7f973415c270 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9734169ee0 con 0x7f97340b2f50 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 0x7f9734165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 0x7f9734165820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43782/0 (socket says 172.21.15.16:43782) 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 -- 172.21.15.16:0/1208972572 learned_addr learned my addr 172.21.15.16:0/1208972572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9738e00640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97340b2f50 0x7f9734150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973a200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f973415c270 0x7f973406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 -- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97340b2f50 msgr2=0x7f9734150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97340b2f50 0x7f9734150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 -- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f973415c270 msgr2=0x7f973406d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f973415c270 0x7f973406d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 -- 172.21.15.16:0/1208972572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97341fcdb0 con 0x7f9734073240 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9738e00640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97340b2f50 0x7f9734150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973a200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f973415c270 0x7f973406d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9739800640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 0x7f9734165820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f972800c400 tx=0x7f972800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728017070 con 0x7f9734073240 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9728005400 con 0x7f9734073240 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728003860 con 0x7f9734073240 2024-09-17T05:01:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97341fbff0 con 0x7f9734073240 2024-09-17T05:01:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97341fc5b0 con 0x7f9734073240 2024-09-17T05:01:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.843+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97340b4a50 con 0x7f9734073240 2024-09-17T05:01:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.847+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f972801a070 con 0x7f9734073240 2024-09-17T05:01:33.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.847+0000 7f9731600640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9714073200 0x7f9714075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:33.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.847+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9728093830 con 0x7f9734073240 2024-09-17T05:01:33.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.847+0000 7f9738e00640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9714073200 0x7f9714075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.847+0000 7f9738e00640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9714073200 0x7f9714075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97200af000 tx=0x7f972001afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:33.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:33.851+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f972805df40 con 0x7f9734073240 2024-09-17T05:01:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.027+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f973415c680 con 0x7f9734073240 2024-09-17T05:01:34.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.027+0000 7f9731600640 1 -- 172.21.15.16:0/1208972572 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7f9728062970 con 0x7f9734073240 2024-09-17T05:01:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:34.028 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9714073200 msgr2=0x7f9714075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9714073200 0x7f9714075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97200af000 tx=0x7f972001afd0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 msgr2=0x7f9734165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 0x7f9734165820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f972800c400 tx=0x7f972800c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 shutdown_connections 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9714073200 0x7f9714075710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f973415c270 0x7f973406d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97340b2f50 0x7f9734150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 --2- 172.21.15.16:0/1208972572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9734073240 0x7f9734165820 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 >> 172.21.15.16:0/1208972572 conn(0x7f973406f900 msgr2=0x7f9734073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 shutdown_connections 2024-09-17T05:01:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.031+0000 7f973c200640 1 -- 172.21.15.16:0/1208972572 wait complete. 2024-09-17T05:01:34.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 -- 172.21.15.16:0/3826998742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0714a0 msgr2=0x7fce4c15caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 --2- 172.21.15.16:0/3826998742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0714a0 0x7fce4c15caf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce44009bc0 tx=0x7fce4401cae0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 -- 172.21.15.16:0/3826998742 shutdown_connections 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 --2- 172.21.15.16:0/3826998742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0714a0 0x7fce4c15caf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 --2- 172.21.15.16:0/3826998742 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce4c0d0a00 0x7fce4c15c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 -- 172.21.15.16:0/3826998742 >> 172.21.15.16:0/3826998742 conn(0x7fce4c0b4160 msgr2=0x7fce4c0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 -- 172.21.15.16:0/3826998742 shutdown_connections 2024-09-17T05:01:34.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.207+0000 7fce51400640 1 -- 172.21.15.16:0/3826998742 wait complete. 2024-09-17T05:01:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 Processor -- start 2024-09-17T05:01:34.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 -- start start 2024-09-17T05:01:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce4c0714a0 0x7fce4c14adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce4c0d1aa0 0x7fce4c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 0x7fce4c0b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1699b0 con 0x7fce4c14b5d0 2024-09-17T05:01:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c169830 con 0x7fce4c0714a0 2024-09-17T05:01:34.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce51400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c169b30 con 0x7fce4c0d1aa0 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 0x7fce4c0b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 0x7fce4c0b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43800/0 (socket says 172.21.15.16:43800) 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce50a00640 1 -- 172.21.15.16:0/2479950647 learned_addr learned my addr 172.21.15.16:0/2479950647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce4be00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce4c0714a0 0x7fce4c14adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.211+0000 7fce43e00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce4c0d1aa0 0x7fce4c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce50a00640 1 -- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce4c0d1aa0 msgr2=0x7fce4c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce50a00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce4c0d1aa0 0x7fce4c155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce50a00640 1 -- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce4c0714a0 msgr2=0x7fce4c14adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce50a00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce4c0714a0 0x7fce4c14adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce50a00640 1 -- 172.21.15.16:0/2479950647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c1540e0 con 0x7fce4c14b5d0 2024-09-17T05:01:34.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce50a00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 0x7fce4c0b5500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce4401a710 tx=0x7fce44009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440bb070 con 0x7fce4c14b5d0 2024-09-17T05:01:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce440a5e50 con 0x7fce4c14b5d0 2024-09-17T05:01:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c1543c0 con 0x7fce4c14b5d0 2024-09-17T05:01:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce440b2040 con 0x7fce4c14b5d0 2024-09-17T05:01:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c154980 con 0x7fce4c14b5d0 2024-09-17T05:01:34.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4c15c5b0 con 0x7fce4c14b5d0 2024-09-17T05:01:34.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.215+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce440be070 con 0x7fce4c14b5d0 2024-09-17T05:01:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.219+0000 7fce48c00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce28073200 0x7fce28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.219+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fce4412e080 con 0x7fce4c14b5d0 2024-09-17T05:01:34.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.219+0000 7fce4be00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce28073200 0x7fce28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:34.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.219+0000 7fce4be00640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce28073200 0x7fce28075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce3c007fd0 tx=0x7fce3c007ea0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:34.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.223+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce44016790 con 0x7fce4c14b5d0 2024-09-17T05:01:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.431+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce4c0ccf40 con 0x7fce4c14b5d0 2024-09-17T05:01:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.431+0000 7fce48c00640 1 -- 172.21.15.16:0/2479950647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7fce440f8700 con 0x7fce4c14b5d0 2024-09-17T05:01:34.434 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:34.434 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce28073200 msgr2=0x7fce28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce28073200 0x7fce28075710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce3c007fd0 tx=0x7fce3c007ea0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 msgr2=0x7fce4c0b5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 0x7fce4c0b5500 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce4401a710 tx=0x7fce44009260 comp rx=0 tx=0).stop 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 shutdown_connections 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce28073200 0x7fce28075710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c14b5d0 0x7fce4c0b5500 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce4c0d1aa0 0x7fce4c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 --2- 172.21.15.16:0/2479950647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce4c0714a0 0x7fce4c14adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 >> 172.21.15.16:0/2479950647 conn(0x7fce4c0b4160 msgr2=0x7fce4c070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 shutdown_connections 2024-09-17T05:01:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:34.435+0000 7fce51400640 1 -- 172.21.15.16:0/2479950647 wait complete. 2024-09-17T05:01:34.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:01:08.278850+0000', 'modified': '2024-09-17T05:01:23.342355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4390}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4390': {'gid': 4390, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6836/636271305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6835', 'nonce': 636271305}, {'type': 'v1', 'addr': '172.21.15.104:6836', 'nonce': 636271305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4390, 'qdb_cluster': [4390]} 2024-09-17T05:01:34.859 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:01:34.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:01:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 -- 172.21.15.16:0/570643975 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5140ccd50 msgr2=0x7fe51415c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 --2- 172.21.15.16:0/570643975 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5140ccd50 0x7fe51415c9b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009870 tx=0x7fe4fc01c780 comp rx=0 tx=0).stop 2024-09-17T05:01:35.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 -- 172.21.15.16:0/570643975 shutdown_connections 2024-09-17T05:01:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 --2- 172.21.15.16:0/570643975 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5140ccd50 0x7fe51415c9b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 -- 172.21.15.16:0/570643975 >> 172.21.15.16:0/570643975 conn(0x7fe5140d23f0 msgr2=0x7fe5140d2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 -- 172.21.15.16:0/570643975 shutdown_connections 2024-09-17T05:01:35.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.023+0000 7fe519a00640 1 -- 172.21.15.16:0/570643975 wait complete. 2024-09-17T05:01:35.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 Processor -- start 2024-09-17T05:01:35.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 -- start start 2024-09-17T05:01:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe514156b30 0x7fe514153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 0x7fe5140b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe51406dfa0 0x7fe5140c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe51416ab20 con 0x7fe51415cef0 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe51416a9a0 con 0x7fe51406dfa0 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe51416aca0 con 0x7fe514156b30 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 0x7fe5140b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 0x7fe5140b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43810/0 (socket says 172.21.15.16:43810) 2024-09-17T05:01:35.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 -- 172.21.15.16:0/669464766 learned_addr learned my addr 172.21.15.16:0/669464766 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe513e00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe514156b30 0x7fe514153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519000640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe51406dfa0 0x7fe5140c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 -- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe514156b30 msgr2=0x7fe514153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe514156b30 0x7fe514153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 -- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe51406dfa0 msgr2=0x7fe5140c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe51406dfa0 0x7fe5140c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 -- 172.21.15.16:0/669464766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5141541e0 con 0x7fe51415cef0 2024-09-17T05:01:35.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe50b400640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 0x7fe5140b4e90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01d020 tx=0x7fe4fc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe510c00640 1 -- 172.21.15.16:0/669464766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0a7bb0 con 0x7fe51415cef0 2024-09-17T05:01:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe510c00640 1 -- 172.21.15.16:0/669464766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4fc0bc070 con 0x7fe51415cef0 2024-09-17T05:01:35.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5140b77a0 con 0x7fe51415cef0 2024-09-17T05:01:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe510c00640 1 -- 172.21.15.16:0/669464766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc005270 con 0x7fe51415cef0 2024-09-17T05:01:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.027+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5140c4da0 con 0x7fe51415cef0 2024-09-17T05:01:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe510c00640 1 -- 172.21.15.16:0/669464766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4fc005410 con 0x7fe51415cef0 2024-09-17T05:01:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe510c00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4f0073200 0x7fe4f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe510c00640 1 -- 172.21.15.16:0/669464766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe4fc12f130 con 0x7fe51415cef0 2024-09-17T05:01:35.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe513e00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4f0073200 0x7fe4f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe522561000 1 Processor -- start 2024-09-17T05:01:35.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe522561000 1 -- start start 2024-09-17T05:01:35.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.031+0000 7fe513e00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4f0073200 0x7fe4f0075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe504001230 tx=0x7fe504009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af15586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1407950 0x560af1558be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af1559120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560af142d410 con 0x560af13f0ca0 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560af136ab40 con 0x560af1407950 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x560af1359f70 con 0x560af1017460 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af1559120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af1559120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43812/0 (socket says 172.21.15.16:43812) 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 -- 172.21.15.16:0/4158294469 learned_addr learned my addr 172.21.15.16:0/4158294469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe513e00640 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af15586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe50b400640 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1407950 0x560af1558be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 -- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 msgr2=0x560af15586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af15586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 -- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1407950 msgr2=0x560af1558be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1407950 0x560af1558be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 -- 172.21.15.16:0/4158294469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af13215b0 con 0x560af13f0ca0 2024-09-17T05:01:35.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe519000640 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af1559120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe50c001bf0 tx=0x7fe50c001ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2892c1fed3b1fbfb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe508c00640 1 -- 172.21.15.16:0/4158294469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c019070 con 0x560af13f0ca0 2024-09-17T05:01:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe508c00640 1 -- 172.21.15.16:0/4158294469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe50c003c80 con 0x560af13f0ca0 2024-09-17T05:01:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe508c00640 1 -- 172.21.15.16:0/4158294469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c00e9a0 con 0x560af13f0ca0 2024-09-17T05:01:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- 172.21.15.16:0/4158294469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 msgr2=0x560af1559120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af1559120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe50c001bf0 tx=0x7fe50c001ff0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- 172.21.15.16:0/4158294469 shutdown_connections 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af1559120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1407950 0x560af1558be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 --2- 172.21.15.16:0/4158294469 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af15586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- 172.21.15.16:0/4158294469 >> 172.21.15.16:0/4158294469 conn(0x560af14ddd90 msgr2=0x560af150d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- 172.21.15.16:0/4158294469 shutdown_connections 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.035+0000 7fe522561000 1 -- 172.21.15.16:0/4158294469 wait complete. 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- start start 2024-09-17T05:01:35.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 Processor -- start 2024-09-17T05:01:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1017460 0x560af155f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af155fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560af12f45f0 con 0x560af13f0ca0 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560af135dd80 con 0x560af1017460 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x560af1300520 con 0x560af1407950 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 10 client.0 _ll_register_callbacks cb 0x560af1304830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af155fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af155fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43816/0 (socket says 172.21.15.16:43816) 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 -- 172.21.15.16:0/4216166985 learned_addr learned my addr 172.21.15.16:0/4216166985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe519000640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe513e00640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1017460 0x560af155f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 -- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 msgr2=0x560af15603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 -- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1017460 msgr2=0x560af155f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1017460 0x560af155f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 -- 172.21.15.16:0/4216166985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af12f8190 con 0x560af13f0ca0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe513e00640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1017460 0x560af155f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe50b400640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af155fe60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009870 tx=0x7fe4fc0f9cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc65f1035fcfe813 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe501600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe501600640 1 -- 172.21.15.16:0/4216166985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0f9210 con 0x560af13f0ca0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe501600640 1 -- 172.21.15.16:0/4216166985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe4fc11ecc0 con 0x560af13f0ca0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560af12f2110 con 0x560af13f0ca0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe501600640 1 -- 172.21.15.16:0/4216166985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fe4fc0f9e00 con 0x560af13f0ca0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 10 client.4815 fetch_fsmap learned FSMap version 9 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x560af13def10 con 0x560af13f0ca0 2024-09-17T05:01:35.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe501600640 1 -- 172.21.15.16:0/4216166985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560af12f2110 con 0x560af13f0ca0 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe501600640 1 -- 172.21.15.16:0/4216166985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fe4fc113bc0 con 0x560af13f0ca0 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 10 client.4815 fetch_fsmap finished waiting for FSMap version 9 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 10 client.4815 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 20 client.4815 populate_metadata read hostname 'smithi016' 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0ffc7d0 0x560af1569d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 4 client.4815 mds_command: new command op to 4390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.039+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 --> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af136ca90 con 0x560af0ffc7d0 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe513e00640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0ffc7d0 0x560af1569d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe513e00640 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0ffc7d0 0x560af1569d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48bcc8e1a188105 server_cookie=1f31d121826af9d5 in_seq=0 out_seq=0 2024-09-17T05:01:35.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe501600640 10 client.4815 ms_handle_connect on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe501600640 1 -- 172.21.15.16:0/4216166985 <== mds.0 v2:172.21.15.104:6835/636271305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560af136ca90 con 0x560af0ffc7d0 2024-09-17T05:01:35.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe501600640 10 client.4815 handle_command_reply: tid=0 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 client.4815 shutdown 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 msgr2=0x560af155fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af155fe60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009870 tx=0x7fe4fc0f9cd0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 shutdown_connections 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0ffc7d0 0x560af1569d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af13f0ca0 0x560af155fe60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe501600640 0 client.4815 ms_handle_reset on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 --2- 172.21.15.16:0/4216166985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af1017460 0x560af155f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 >> 172.21.15.16:0/4216166985 conn(0x560af14ddd90 msgr2=0x560af1514440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 shutdown_connections 2024-09-17T05:01:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 1 -- 172.21.15.16:0/4216166985 wait complete. 2024-09-17T05:01:35.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.043+0000 7fe522561000 20 client.4815 trim_cache size 0 max 16384 2024-09-17T05:01:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 Processor -- start 2024-09-17T05:01:35.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 -- start start 2024-09-17T05:01:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 0x560af15ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe514206410 con 0x560af1017460 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5142054b0 con 0x560af13f0ca0 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe514204810 con 0x560af1407950 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 0x560af15ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 0x560af15ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43824/0 (socket says 172.21.15.16:43824) 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 -- 172.21.15.16:0/1755199129 learned_addr learned my addr 172.21.15.16:0/1755199129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe519000640 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe50b400640 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 -- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 msgr2=0x560af15cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 -- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 msgr2=0x560af15cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 -- 172.21.15.16:0/1755199129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af13b5db0 con 0x560af1017460 2024-09-17T05:01:35.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.063+0000 7fe513e00640 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 0x560af15ca630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x560af127e1c0 tx=0x7fe504014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8a9ca48c6949 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe503e00640 1 -- 172.21.15.16:0/1755199129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504032070 con 0x560af1017460 2024-09-17T05:01:35.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe503e00640 1 -- 172.21.15.16:0/1755199129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe50400ee50 con 0x560af1017460 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe503e00640 1 -- 172.21.15.16:0/1755199129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504015880 con 0x560af1017460 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- 172.21.15.16:0/1755199129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 msgr2=0x560af15ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 0x560af15ca630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x560af127e1c0 tx=0x7fe504014000 comp rx=0 tx=0).stop 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- 172.21.15.16:0/1755199129 shutdown_connections 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1407950 0x560af15cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- 172.21.15.16:0/1755199129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1017460 0x560af15ca630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- 172.21.15.16:0/1755199129 >> 172.21.15.16:0/1755199129 conn(0x560af14ddd90 msgr2=0x560af150ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- 172.21.15.16:0/1755199129 shutdown_connections 2024-09-17T05:01:35.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- 172.21.15.16:0/1755199129 wait complete. 2024-09-17T05:01:35.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- start start 2024-09-17T05:01:35.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 Processor -- start 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af1589260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 0x560af1589ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560af13623b0 con 0x560af1407950 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560af1369910 con 0x560af13f0ca0 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x560af1419750 con 0x560af1017460 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe522561000 10 client.0 _ll_register_callbacks cb 0x7fe4fc0f55a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe519000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 0x560af1589ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe519000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 0x560af1589ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43826/0 (socket says 172.21.15.16:43826) 2024-09-17T05:01:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.067+0000 7fe519000640 1 -- 172.21.15.16:0/2255731552 learned_addr learned my addr 172.21.15.16:0/2255731552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe513e00640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af1589260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe50b400640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe519000640 1 -- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 msgr2=0x560af1589260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe519000640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af1589260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe519000640 1 -- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 msgr2=0x560af15897a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe519000640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15897a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe519000640 1 -- 172.21.15.16:0/2255731552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af1320ab0 con 0x560af1407950 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe50b400640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe519000640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 0x560af1589ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe50c002240 tx=0x7fe50c0016a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61cde48ca1a1209 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:01:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 1 -- 172.21.15.16:0/2255731552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c024070 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 1 -- 172.21.15.16:0/2255731552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe50c003c90 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50400ee50 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 1 -- 172.21.15.16:0/2255731552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fe50c02f050 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 10 client.4821 fetch_fsmap learned FSMap version 9 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x560af12ff070 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 1 -- 172.21.15.16:0/2255731552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe50400ee50 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 1 -- 172.21.15.16:0/2255731552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fe50c032020 con 0x560af1407950 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 10 client.4821 fetch_fsmap finished waiting for FSMap version 9 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 10 client.4821 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 20 client.4821 populate_metadata read hostname 'smithi016' 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0fea830 0x560af15d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 4 client.4821 mds_command: new command op to 4390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 --> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] -- command(tid 0: {"prefix": "status"}) -- 0x560af1368540 con 0x560af0fea830 2024-09-17T05:01:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe513e00640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0fea830 0x560af15d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe513e00640 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0fea830 0x560af15d66b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b3cde15256b048 server_cookie=809d3e0024303b78 in_seq=0 out_seq=0 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 10 client.4821 ms_handle_connect on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 1 -- 172.21.15.16:0/2255731552 <== mds.0 v2:172.21.15.104:6835/636271305 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x560af1368540 con 0x560af0fea830 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 10 client.4821 handle_command_reply: tid=0 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 client.4821 shutdown 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 msgr2=0x560af1589ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 0x560af1589ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe50c002240 tx=0x7fe50c0016a0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 shutdown_connections 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:6835/636271305,v1:172.21.15.104:6836/636271305] conn(0x560af0fea830 0x560af15d66b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560af1407950 0x560af1589ce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af13f0ca0 0x560af15897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 --2- 172.21.15.16:0/2255731552 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560af1017460 0x560af1589260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe501600640 0 client.4821 ms_handle_reset on v2:172.21.15.104:6835/636271305 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 >> 172.21.15.16:0/2255731552 conn(0x560af14ddd90 msgr2=0x560af15696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 shutdown_connections 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 1 -- 172.21.15.16:0/2255731552 wait complete. 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.071+0000 7fe522561000 20 client.4821 trim_cache size 0 max 16384 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:01:35.078 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4390, 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 26.236697763999999, 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 9, 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 26, 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 26, 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 32.263238790000003 2024-09-17T05:01:35.079 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:01:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4f0073200 msgr2=0x7fe4f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4f0073200 0x7fe4f0075710 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe504001230 tx=0x7fe504009040 comp rx=0 tx=0).stop 2024-09-17T05:01:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 msgr2=0x7fe5140b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 0x7fe5140b4e90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01d020 tx=0x7fe4fc005db0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 shutdown_connections 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4f0073200 0x7fe4f0075710 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe51406dfa0 0x7fe5140c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe51415cef0 0x7fe5140b4e90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 --2- 172.21.15.16:0/669464766 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe514156b30 0x7fe514153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 >> 172.21.15.16:0/669464766 conn(0x7fe5140d23f0 msgr2=0x7fe51415f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 shutdown_connections 2024-09-17T05:01:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:35.075+0000 7fe519a00640 1 -- 172.21.15.16:0/669464766 wait complete. 2024-09-17T05:01:35.093 INFO:teuthology.run_tasks:Running task kclient... 2024-09-17T05:01:35.106 INFO:tasks.kclient:Mounting kernel clients... 2024-09-17T05:01:35.107 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T05:01:35.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.107 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:01:35.117 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-17T05:01:35.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T05:01:35.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-17T05:01:35.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.769 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:01:35.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-17T05:01:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-17T05:01:35.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:01:35.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.852 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:01:35.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.862 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-09-17T05:01:35.929 INFO:teuthology.orchestra.run.smithi104.stderr:Cannot find device "ceph-brx" 2024-09-17T05:01:35.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:01:35.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.931 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:01:35.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:35.986 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-09-17T05:01:36.058 INFO:teuthology.orchestra.run.smithi104.stderr:Cannot find device "ceph-brx" 2024-09-17T05:01:36.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:01:36.060 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-17T05:01:36.061 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:01:36.061 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:01:36.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:01:36.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 -- 172.21.15.16:0/3441837806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c15f790 msgr2=0x7efd7c0b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 --2- 172.21.15.16:0/3441837806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c15f790 0x7efd7c0b2f50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd64009900 tx=0x7efd6401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 -- 172.21.15.16:0/3441837806 shutdown_connections 2024-09-17T05:01:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 --2- 172.21.15.16:0/3441837806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd7c073240 0x7efd7c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 --2- 172.21.15.16:0/3441837806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c15f790 0x7efd7c0b2f50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 -- 172.21.15.16:0/3441837806 >> 172.21.15.16:0/3441837806 conn(0x7efd7c06f900 msgr2=0x7efd7c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 -- 172.21.15.16:0/3441837806 shutdown_connections 2024-09-17T05:01:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 -- 172.21.15.16:0/3441837806 wait complete. 2024-09-17T05:01:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 Processor -- start 2024-09-17T05:01:36.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.231+0000 7efd83200640 1 -- start start 2024-09-17T05:01:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd7c073240 0x7efd7c1f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 0x7efd7c147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd7c154f70 0x7efd7c1f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c168ba0 con 0x7efd7c074b00 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c168a20 con 0x7efd7c073240 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c168d20 con 0x7efd7c154f70 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 0x7efd7c147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 0x7efd7c147200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43850/0 (socket says 172.21.15.16:43850) 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 -- 172.21.15.16:0/169351210 learned_addr learned my addr 172.21.15.16:0/169351210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd7be00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd7c073240 0x7efd7c1f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd81200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd7c154f70 0x7efd7c1f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 -- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd7c154f70 msgr2=0x7efd7c1f9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd7c154f70 0x7efd7c1f9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 -- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd7c073240 msgr2=0x7efd7c1f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd7c073240 0x7efd7c1f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 -- 172.21.15.16:0/169351210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c147740 con 0x7efd7c074b00 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd73e00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 0x7efd7c147200 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd64009850 tx=0x7efd640a5680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd640b2040 con 0x7efd7c074b00 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd640041b0 con 0x7efd7c074b00 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64005020 con 0x7efd7c074b00 2024-09-17T05:01:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c155750 con 0x7efd7c074b00 2024-09-17T05:01:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1f65b0 con 0x7efd7c074b00 2024-09-17T05:01:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.235+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c1554a0 con 0x7efd7c074b00 2024-09-17T05:01:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.239+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd640bf030 con 0x7efd7c074b00 2024-09-17T05:01:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.239+0000 7efd78c00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd58073200 0x7efd58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.239+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efd6412e6e0 con 0x7efd7c074b00 2024-09-17T05:01:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.239+0000 7efd7be00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd58073200 0x7efd58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.239+0000 7efd7be00640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd58073200 0x7efd58075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0098e0 tx=0x7efd6c009060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.243+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd640c6060 con 0x7efd7c074b00 2024-09-17T05:01:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.451+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efd7c073650 con 0x7efd7c074b00 2024-09-17T05:01:36.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.451+0000 7efd78c00640 1 -- 172.21.15.16:0/169351210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7efd640164d0 con 0x7efd7c074b00 2024-09-17T05:01:36.454 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd58073200 msgr2=0x7efd58075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd58073200 0x7efd58075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0098e0 tx=0x7efd6c009060 comp rx=0 tx=0).stop 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 msgr2=0x7efd7c147200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 0x7efd7c147200 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd64009850 tx=0x7efd640a5680 comp rx=0 tx=0).stop 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 shutdown_connections 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd58073200 0x7efd58075710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd7c154f70 0x7efd7c1f9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd7c074b00 0x7efd7c147200 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 --2- 172.21.15.16:0/169351210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd7c073240 0x7efd7c1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:36.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 >> 172.21.15.16:0/169351210 conn(0x7efd7c06f900 msgr2=0x7efd7c0cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 shutdown_connections 2024-09-17T05:01:36.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36.455+0000 7efd83200640 1 -- 172.21.15.16:0/169351210 wait complete. 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:01:36.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:01:36.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:36.477 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:01:36.486 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:36.486 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:36.486 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:01:36.487 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42396sec preferred_lft 42396sec 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:01:36.488 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:36.489 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:01:36.489 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:36.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:36.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:36.643 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:01:36.708 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:01:36.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:36.710 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:01:36.762 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:01:36.763 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:01:36.763 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:01:36.763 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:01:36.763 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:01:36.763 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:01:36.763 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:36.763 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:36.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:01:36.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:01:36.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:01:36.764 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:36.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:36.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:36.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:36.902 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:01:36.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:36.913 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-17T05:01:36.965 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:36.965 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:36.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:01:36.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T05:01:36.965 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:37.094 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T05:01:37.094 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:37.094 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:37.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T05:01:37.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:01:37.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T05:01:37.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T05:01:37.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T05:01:37.095 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:37.284 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:37.284 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:37.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:01:37.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:01:37.285 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:37.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:37.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:37.373 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:37.373 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:01:37.382 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:01:37.382 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:37.382 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:01:37.437 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:01:37.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:37.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:01:37.544 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:01:37.544 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:01:37.544 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:01:39.211 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:01:39.211 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:01:39.211 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:01:39.211 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:39.211 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:39.227 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:01:39.227 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:01:39.228 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:01:39.228 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:39.228 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:39.228 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:01:39.228 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:01:39.228 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:01:39.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:01:39.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:39.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:01:39.253 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:39.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:client4830 2024-09-17T05:01:39.359 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:39.359 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:39.369 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:39.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:39.474 INFO:teuthology.orchestra.run.smithi016.stdout:client4830 2024-09-17T05:01:39.474 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:39.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:39.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:39.533 INFO:teuthology.orchestra.run.smithi016.stdout:client4830 2024-09-17T05:01:39.533 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23ee29f8-0841-48bc-a01a-8eb97d561cfc.client4830/status') 2024-09-17T05:01:39.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:39.638 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4830 (3)192.168.0.1:0/2742769968 2024-09-17T05:01:39.638 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-17T05:01:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-17T05:01:39.638 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-17T05:01:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:69 bytes copied, 6.6694e-05 s, 1.0 MB/s 2024-09-17T05:01:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:39.647 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-17T05:01:39.647 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:01:39.647 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T05:01:39.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:01:39.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 -- 172.21.15.16:0/1989808349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f80b2f50 msgr2=0x7f48f815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 --2- 172.21.15.16:0/1989808349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f80b2f50 0x7f48f815bb60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009840 tx=0x7f48e801c790 comp rx=0 tx=0).stop 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 -- 172.21.15.16:0/1989808349 shutdown_connections 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 --2- 172.21.15.16:0/1989808349 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48f8073240 0x7f48f8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 --2- 172.21.15.16:0/1989808349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f80b2f50 0x7f48f815bb60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 -- 172.21.15.16:0/1989808349 >> 172.21.15.16:0/1989808349 conn(0x7f48f806f900 msgr2=0x7f48f80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 -- 172.21.15.16:0/1989808349 shutdown_connections 2024-09-17T05:01:39.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.803+0000 7f48fda00640 1 -- 172.21.15.16:0/1989808349 wait complete. 2024-09-17T05:01:39.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 Processor -- start 2024-09-17T05:01:39.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 -- start start 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48f8073240 0x7f48f813c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48f80b2f50 0x7f48f81f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 0x7f48f8164dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8168d60 con 0x7f48f815c0a0 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f48f8168be0 con 0x7f48f80b2f50 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48fda00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f48f8168ee0 con 0x7f48f8073240 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 0x7f48f8164dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 0x7f48f8164dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60504/0 (socket says 172.21.15.16:60504) 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 -- 172.21.15.16:0/683828144 learned_addr learned my addr 172.21.15.16:0/683828144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48ee000640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48f80b2f50 0x7f48f81f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f6a00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48f8073240 0x7f48f813c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 -- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48f8073240 msgr2=0x7f48f813c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48f8073240 0x7f48f813c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 -- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48f80b2f50 msgr2=0x7f48f81f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48f80b2f50 0x7f48f81f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 -- 172.21.15.16:0/683828144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f813ef90 con 0x7f48f815c0a0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48f7400640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 0x7f48f8164dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48f000e910 tx=0x7f48f000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.807+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0004d30 con 0x7f48f815c0a0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f000cb20 con 0x7f48f815c0a0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0005020 con 0x7f48f815c0a0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48fda00640 1 -- 172.21.15.16:0/683828144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f813f270 con 0x7f48f815c0a0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48fda00640 1 -- 172.21.15.16:0/683828144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f813f830 con 0x7f48f815c0a0 2024-09-17T05:01:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f8073650 con 0x7f48f815c0a0 2024-09-17T05:01:39.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48f00051c0 con 0x7f48f815c0a0 2024-09-17T05:01:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48ef400640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f48d4073200 0x7f48d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f48f0094e20 con 0x7f48f815c0a0 2024-09-17T05:01:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.811+0000 7f48f6a00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f48d4073200 0x7f48d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.815+0000 7f48f6a00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f48d4073200 0x7f48d4075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48e80af000 tx=0x7f48e801afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:39.819+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48f0016030 con 0x7f48f815c0a0 2024-09-17T05:01:40.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.031+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48f81fa9e0 con 0x7f48f815c0a0 2024-09-17T05:01:40.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.031+0000 7f48ef400640 1 -- 172.21.15.16:0/683828144 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f48f005f410 con 0x7f48f815c0a0 2024-09-17T05:01:40.033 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f48d4073200 msgr2=0x7f48d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f48d4073200 0x7f48d4075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f48e80af000 tx=0x7f48e801afd0 comp rx=0 tx=0).stop 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 msgr2=0x7f48f8164dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 0x7f48f8164dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48f000e910 tx=0x7f48f000edd0 comp rx=0 tx=0).stop 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 shutdown_connections 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f48d4073200 0x7f48d4075710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:40.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f815c0a0 0x7f48f8164dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48f80b2f50 0x7f48f81f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 --2- 172.21.15.16:0/683828144 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f48f8073240 0x7f48f813c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 >> 172.21.15.16:0/683828144 conn(0x7f48f806f900 msgr2=0x7f48f81516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 shutdown_connections 2024-09-17T05:01:40.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40.035+0000 7f48e7e00640 1 -- 172.21.15.16:0/683828144 wait complete. 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:01:40.055 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T05:01:40.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.056 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:40.065 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42393sec preferred_lft 42393sec 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:40.066 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:01:40.067 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.069 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:01:40.127 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:01:40.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.130 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:01:40.183 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:01:40.183 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:01:40.184 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:01:40.184 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:01:40.184 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:01:40.184 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:01:40.184 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:40.184 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:40.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:01:40.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:01:40.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:01:40.184 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:40.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:40.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:40.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.321 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:01:40.370 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:01:40.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.371 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-17T05:01:40.421 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T05:01:40.422 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:40.422 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:40.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:01:40.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T05:01:40.423 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:40.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.573 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:01:40.574 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 86:74:1c:e4:a1:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:01:40.574 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:01:40.574 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.574 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8474:1cff:fee4:a128/64 scope link 2024-09-17T05:01:40.574 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T05:01:40.642 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:01:40.644 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 86:74:1c:e4:a1:28 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:01:40.645 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:01:40.645 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.645 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::8474:1cff:fee4:a128/64 scope link 2024-09-17T05:01:40.645 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:40.646 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T05:01:40.646 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:40.646 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:40.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T05:01:40.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T05:01:40.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T05:01:40.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T05:01:40.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T05:01:40.647 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:40.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:40.833 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:40.833 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:01:40.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2024-09-17T05:01:40.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T05:01:40.833 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:01:40.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:40.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:40.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:40.922 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T05:01:40.931 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T05:01:40.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:40.931 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T05:01:40.981 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:01:40.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:40.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:01:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:01:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:01:41.083 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:01:41.101 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:01:41.101 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:01:41.101 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:01:41.101 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:41.101 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:01:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:01:41.114 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:01:41.117 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:01:41.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:41.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T05:01:41.137 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:41.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:41.240 INFO:teuthology.orchestra.run.smithi016.stdout:client4842 2024-09-17T05:01:41.240 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:41.240 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:41.249 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:41.351 INFO:teuthology.orchestra.run.smithi016.stdout:client4842 2024-09-17T05:01:41.351 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:41.410 INFO:teuthology.orchestra.run.smithi016.stdout:client4842 2024-09-17T05:01:41.411 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23ee29f8-0841-48bc-a01a-8eb97d561cfc.client4842/status') 2024-09-17T05:01:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:41.474 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.4842 (3)192.168.0.2:0/2460383573 2024-09-17T05:01:41.474 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-17T05:01:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-17T05:01:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-17T05:01:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:69 bytes copied, 6.4804e-05 s, 1.1 MB/s 2024-09-17T05:01:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:41.482 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-17T05:01:41.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:01:41.483 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:01:41.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:01:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 -- 172.21.15.16:0/2915186128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a40b4230 msgr2=0x7fa7a40b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 --2- 172.21.15.16:0/2915186128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a40b4230 0x7fa7a40b4640 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009c80 tx=0x7fa79c01cbc0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 -- 172.21.15.16:0/2915186128 shutdown_connections 2024-09-17T05:01:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 --2- 172.21.15.16:0/2915186128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a40b4230 0x7fa7a40b4640 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 -- 172.21.15.16:0/2915186128 >> 172.21.15.16:0/2915186128 conn(0x7fa7a40d0860 msgr2=0x7fa7a40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 -- 172.21.15.16:0/2915186128 shutdown_connections 2024-09-17T05:01:41.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.687+0000 7fa7ab800640 1 -- 172.21.15.16:0/2915186128 wait complete. 2024-09-17T05:01:41.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 Processor -- start 2024-09-17T05:01:41.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 -- start start 2024-09-17T05:01:41.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 0x7fa7a4165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:41.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7a40b4230 0x7fa7a4146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7a40cfa30 0x7fa7a4159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4169d80 con 0x7fa7a406ce90 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4169c00 con 0x7fa7a40b4230 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a4169f00 con 0x7fa7a40cfa30 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 0x7fa7a4165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 0x7fa7a4165b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60526/0 (socket says 172.21.15.16:60526) 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 -- 172.21.15.16:0/4235480801 learned_addr learned my addr 172.21.15.16:0/4235480801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 -- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7a40cfa30 msgr2=0x7fa7a4159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7a40cfa30 0x7fa7a4159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 -- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7a40b4230 msgr2=0x7fa7a4146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7a40b4230 0x7fa7a4146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 -- 172.21.15.16:0/4235480801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a4147990 con 0x7fa7a406ce90 2024-09-17T05:01:41.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a8e00640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 0x7fa7a4165b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c440 tx=0x7fa79800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798011790 con 0x7fa7a406ce90 2024-09-17T05:01:41.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa798017070 con 0x7fa7a406ce90 2024-09-17T05:01:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a4147c70 con 0x7fa7a406ce90 2024-09-17T05:01:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798011d00 con 0x7fa7a406ce90 2024-09-17T05:01:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.691+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a4148230 con 0x7fa7a406ce90 2024-09-17T05:01:41.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.695+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a40b4640 con 0x7fa7a406ce90 2024-09-17T05:01:41.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.695+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa79801c460 con 0x7fa7a406ce90 2024-09-17T05:01:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.695+0000 7fa7a0c00640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa784073200 0x7fa784075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.695+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa7980943e0 con 0x7fa7a406ce90 2024-09-17T05:01:41.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.695+0000 7fa7a3e00640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa784073200 0x7fa784075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:41.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.699+0000 7fa7a3e00640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa784073200 0x7fa784075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f70 tx=0x7fa7900079c0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.699+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa79805f0b0 con 0x7fa7a406ce90 2024-09-17T05:01:41.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.911+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa7a40d0e80 con 0x7fa7a406ce90 2024-09-17T05:01:41.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.911+0000 7fa7a0c00640 1 -- 172.21.15.16:0/4235480801 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa79805e9b0 con 0x7fa7a406ce90 2024-09-17T05:01:41.913 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T05:01:41.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa784073200 msgr2=0x7fa784075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa784073200 0x7fa784075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa790000f70 tx=0x7fa7900079c0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 msgr2=0x7fa7a4165b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 0x7fa7a4165b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c440 tx=0x7fa79800c900 comp rx=0 tx=0).stop 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 shutdown_connections 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa784073200 0x7fa784075710 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7a40cfa30 0x7fa7a4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7a40b4230 0x7fa7a4146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 --2- 172.21.15.16:0/4235480801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7a406ce90 0x7fa7a4165b80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 >> 172.21.15.16:0/4235480801 conn(0x7fa7a40d0860 msgr2=0x7fa7a415f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 shutdown_connections 2024-09-17T05:01:41.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:41.915+0000 7fa7ab800640 1 -- 172.21.15.16:0/4235480801 wait complete. 2024-09-17T05:01:41.935 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:01:41.936 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:01:41.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:41.937 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:01:41.946 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:41.946 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:41.947 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42386sec preferred_lft 42386sec 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:01:41.948 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:41.949 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T05:01:41.949 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:41.949 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:41.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add name ceph-brx type bridge 2024-09-17T05:01:41.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr flush dev ceph-brx 2024-09-17T05:01:41.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set ceph-brx up 2024-09-17T05:01:41.949 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T05:01:41.950 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:42.045 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:42.107 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:42.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:42.114 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:01:42.133 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:01:42.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:42.135 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:01:42.204 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:01:42.205 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:01:42.205 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:01:42.205 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:01:42.205 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:01:42.205 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:42.205 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:42.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:01:42.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:01:42.205 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:01:42.205 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:42.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:42.330 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:42.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:42.336 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:01:42.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:42.345 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2024-09-17T05:01:42.400 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:42.400 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:42.400 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:01:42.401 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T05:01:42.401 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:42.489 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:42.520 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:42.527 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T05:01:42.527 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:42.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:42.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:42.700 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:42.701 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:42.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:01:42.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T05:01:42.701 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:42.795 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:42.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:42.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:42.832 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:01:42.884 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:01:42.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:42.884 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:01:42.935 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:01:42.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:42.936 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:01:43.037 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:01:43.037 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:01:43.037 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:01:43.683 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:01:43.683 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:01:43.683 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:01:43.684 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:43.684 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:43.697 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:01:43.698 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:01:43.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:01:43.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:43.702 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:01:43.722 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:43.813 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:43.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:43.836 INFO:teuthology.orchestra.run.smithi104.stdout:client4795 2024-09-17T05:01:43.836 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:43.836 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:43.845 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:43.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:43.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:43.943 INFO:teuthology.orchestra.run.smithi104.stdout:client4795 2024-09-17T05:01:43.944 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:43.990 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:43.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:43.999 INFO:teuthology.orchestra.run.smithi104.stdout:client4795 2024-09-17T05:01:43.999 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23ee29f8-0841-48bc-a01a-8eb97d561cfc.client4795/status') 2024-09-17T05:01:44.046 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:44.057 INFO:teuthology.orchestra.run.smithi104.stdout:instance: client.4795 (3)192.168.0.1:0/4183034160 2024-09-17T05:01:44.058 INFO:teuthology.orchestra.run.smithi104.stdout:blocklisted: false 2024-09-17T05:01:44.058 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2024-09-17T05:01:44.058 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2024-09-17T05:01:44.058 INFO:teuthology.orchestra.run.smithi104.stderr:69 bytes copied, 5.9363e-05 s, 1.2 MB/s 2024-09-17T05:01:44.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:44.065 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-17T05:01:44.065 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T05:01:44.066 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T05:01:44.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:01:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.234+0000 7feae0200640 1 -- 172.21.15.16:0/2502331377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead80b2f50 msgr2=0x7fead815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.234+0000 7feae0200640 1 --2- 172.21.15.16:0/2502331377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead80b2f50 0x7fead815bb60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feacc009840 tx=0x7feacc01c790 comp rx=0 tx=0).stop 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- 172.21.15.16:0/2502331377 shutdown_connections 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 --2- 172.21.15.16:0/2502331377 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 0x7fead8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 --2- 172.21.15.16:0/2502331377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead80b2f50 0x7fead815bb60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- 172.21.15.16:0/2502331377 >> 172.21.15.16:0/2502331377 conn(0x7fead806f900 msgr2=0x7fead80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- 172.21.15.16:0/2502331377 shutdown_connections 2024-09-17T05:01:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- 172.21.15.16:0/2502331377 wait complete. 2024-09-17T05:01:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 Processor -- start 2024-09-17T05:01:44.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- start start 2024-09-17T05:01:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 0x7fead81417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 0x7fead8163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead815c0a0 0x7fead8164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fead8168c90 con 0x7fead80b2f50 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fead8168b10 con 0x7fead815c0a0 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.238+0000 7feae0200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fead8168e10 con 0x7fead8073240 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 0x7fead8163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 0x7fead8163d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60532/0 (socket says 172.21.15.16:60532) 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 -- 172.21.15.16:0/2292629713 learned_addr learned my addr 172.21.15.16:0/2292629713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadd800640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 0x7fead81417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feade200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead815c0a0 0x7fead8164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 -- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 msgr2=0x7fead81417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 0x7fead81417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 -- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead815c0a0 msgr2=0x7fead8164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead815c0a0 0x7fead8164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 -- 172.21.15.16:0/2292629713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead80cad40 con 0x7fead80b2f50 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadd800640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 0x7fead81417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feadce00640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 0x7fead8163d30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feac400c430 tx=0x7feac400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4011790 con 0x7fead80b2f50 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feac4017070 con 0x7fead80b2f50 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead80cb020 con 0x7fead80b2f50 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac4011ce0 con 0x7fead80b2f50 2024-09-17T05:01:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead80cb5e0 con 0x7fead80b2f50 2024-09-17T05:01:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead8073650 con 0x7fead80b2f50 2024-09-17T05:01:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.242+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feac401c460 con 0x7fead80b2f50 2024-09-17T05:01:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.246+0000 7fead5600640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feab8073200 0x7feab8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.246+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feac4094230 con 0x7fead80b2f50 2024-09-17T05:01:44.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.246+0000 7feadd800640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feab8073200 0x7feab8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:44.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.246+0000 7feadd800640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feab8073200 0x7feab8075710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feacc0af000 tx=0x7feacc01afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.250+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feac405e970 con 0x7fead80b2f50 2024-09-17T05:01:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.458+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fead8141ce0 con 0x7fead80b2f50 2024-09-17T05:01:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7fead5600640 1 -- 172.21.15.16:0/2292629713 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7feac40633a0 con 0x7fead80b2f50 2024-09-17T05:01:44.462 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T05:01:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feab8073200 msgr2=0x7feab8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feab8073200 0x7feab8075710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feacc0af000 tx=0x7feacc01afd0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 msgr2=0x7fead8163d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 0x7fead8163d30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feac400c430 tx=0x7feac400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 shutdown_connections 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feab8073200 0x7feab8075710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fead815c0a0 0x7fead8164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fead80b2f50 0x7fead8163d30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 --2- 172.21.15.16:0/2292629713 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fead8073240 0x7fead81417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 >> 172.21.15.16:0/2292629713 conn(0x7fead806f900 msgr2=0x7fead815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 shutdown_connections 2024-09-17T05:01:44.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:44.462+0000 7feae0200640 1 -- 172.21.15.16:0/2292629713 wait complete. 2024-09-17T05:01:44.486 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T05:01:44.486 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:01:44.486 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:01:44.487 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T05:01:44.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T05:01:44.487 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T05:01:44.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T05:01:44.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:01:44.487 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T05:01:44.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.487 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:01:44.496 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:01:44.497 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42383sec preferred_lft 42383sec 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:01:44.498 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.499 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-09-17T05:01:44.499 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.499 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:01:44.499 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:01:44.499 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:01:44.499 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.500 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:01:44.558 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:01:44.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.561 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:01:44.621 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:01:44.622 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:44.622 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:44.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:01:44.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:01:44.622 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:01:44.622 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:44.718 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:44.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:44.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.766 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:01:44.775 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:01:44.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.775 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2024-09-17T05:01:44.828 INFO:teuthology.orchestra.run.smithi104.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T05:01:44.829 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:44.829 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:44.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:01:44.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T05:01:44.829 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:44.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:44.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:44.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5a:13:e5:86:d1:4a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.983 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5813:e5ff:fe86:d14a/64 scope link 2024-09-17T05:01:44.984 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:44.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:44.985 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:45.052 INFO:teuthology.orchestra.run.smithi104.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:01:45.053 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5a:13:e5:86:d1:4a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T05:01:45.053 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T05:01:45.053 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:45.053 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5813:e5ff:fe86:d14a/64 scope link 2024-09-17T05:01:45.053 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:01:45.054 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T05:01:45.054 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:45.054 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:45.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T05:01:45.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T05:01:45.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T05:01:45.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T05:01:45.055 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T05:01:45.055 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:45.146 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:45.230 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:45.237 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:01:45.237 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:01:45.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.1 up 2024-09-17T05:01:45.237 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T05:01:45.237 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:01:45.283 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:45.314 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:45.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:45.320 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T05:01:45.376 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T05:01:45.377 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:01:45.377 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T05:01:45.432 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:01:45.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:45.432 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:01:45.531 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:01:45.532 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:01:45.532 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:01:45.550 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:01:45.550 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:01:45.550 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:01:45.550 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:45.550 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:45.560 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-17T05:01:45.560 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-17T05:01:45.560 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:01:45.560 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:01:45.560 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:01:45.560 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:01:45.564 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:01:45.564 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:01:45.565 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:01:45.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:45.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T05:01:45.584 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:45.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:45.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:45.684 INFO:teuthology.orchestra.run.smithi104.stdout:client4616 2024-09-17T05:01:45.684 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:45.684 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:45.732 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:45.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:45.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:45.831 INFO:teuthology.orchestra.run.smithi104.stdout:client4616 2024-09-17T05:01:45.831 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:01:45.877 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:01:45.880 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:45.887 INFO:teuthology.orchestra.run.smithi104.stdout:client4616 2024-09-17T05:01:45.887 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/23ee29f8-0841-48bc-a01a-8eb97d561cfc.client4616/status') 2024-09-17T05:01:45.974 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:01:45.986 INFO:teuthology.orchestra.run.smithi104.stdout:instance: client.4616 (3)192.168.0.2:0/885101725 2024-09-17T05:01:45.986 INFO:teuthology.orchestra.run.smithi104.stdout:blocklisted: false 2024-09-17T05:01:45.986 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2024-09-17T05:01:45.986 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2024-09-17T05:01:45.986 INFO:teuthology.orchestra.run.smithi104.stderr:68 bytes copied, 4.938e-05 s, 1.4 MB/s 2024-09-17T05:01:45.987 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:01:45.993 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T05:01:46.028 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2024-09-17T05:01:46.028 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:01:46.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 -- 172.21.15.16:0/1479686486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 msgr2=0x7fda8815fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 --2- 172.21.15.16:0/1479686486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 0x7fda8815fb60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fda80009bc0 tx=0x7fda8001cba0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 -- 172.21.15.16:0/1479686486 shutdown_connections 2024-09-17T05:01:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 --2- 172.21.15.16:0/1479686486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 0x7fda8815fb60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 --2- 172.21.15.16:0/1479686486 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda8806ce90 0x7fda880cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 -- 172.21.15.16:0/1479686486 >> 172.21.15.16:0/1479686486 conn(0x7fda880d0860 msgr2=0x7fda880d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 -- 172.21.15.16:0/1479686486 shutdown_connections 2024-09-17T05:01:46.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 -- 172.21.15.16:0/1479686486 wait complete. 2024-09-17T05:01:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 Processor -- start 2024-09-17T05:01:46.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.198+0000 7fda8e800640 1 -- start start 2024-09-17T05:01:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 0x7fda88154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda880cf860 0x7fda881f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 0x7fda8815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda881689e0 con 0x7fda8806ce90 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda880b5a80 con 0x7fda8815f750 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fda88168b60 con 0x7fda880cf860 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 0x7fda88154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 0x7fda88154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60544/0 (socket says 172.21.15.16:60544) 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 -- 172.21.15.16:0/3056281578 learned_addr learned my addr 172.21.15.16:0/3056281578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87e00640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 0x7fda8815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda7ea00640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda880cf860 0x7fda881f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 -- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda880cf860 msgr2=0x7fda881f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda880cf860 0x7fda881f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 -- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 msgr2=0x7fda8815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 0x7fda8815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 -- 172.21.15.16:0/3056281578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881f92d0 con 0x7fda8806ce90 2024-09-17T05:01:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda7ea00640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda880cf860 0x7fda881f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda87400640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 0x7fda88154d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e9f0 tx=0x7fda7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:46.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800cd80 con 0x7fda8806ce90 2024-09-17T05:01:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda78004020 con 0x7fda8806ce90 2024-09-17T05:01:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda881fbd50 con 0x7fda8806ce90 2024-09-17T05:01:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78004470 con 0x7fda8806ce90 2024-09-17T05:01:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881fc310 con 0x7fda8806ce90 2024-09-17T05:01:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.202+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda880cf320 con 0x7fda8806ce90 2024-09-17T05:01:46.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.206+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda78003600 con 0x7fda8806ce90 2024-09-17T05:01:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.206+0000 7fda7fe00640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda64073200 0x7fda64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.206+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fda78094fe0 con 0x7fda8806ce90 2024-09-17T05:01:46.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.206+0000 7fda7ea00640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda64073200 0x7fda64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.206+0000 7fda7ea00640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda64073200 0x7fda64075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda881596f0 tx=0x7fda70009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:46.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.210+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda7805fc40 con 0x7fda8806ce90 2024-09-17T05:01:46.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.382+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda880d0fc0 con 0x7fda8806ce90 2024-09-17T05:01:46.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda7fe00640 1 -- 172.21.15.16:0/3056281578 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18590 (secure 0 0 0) 0x7fda7805f540 con 0x7fda8806ce90 2024-09-17T05:01:46.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:46.387 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:13.324481+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda64073200 msgr2=0x7fda64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda64073200 0x7fda64075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda881596f0 tx=0x7fda70009040 comp rx=0 tx=0).stop 2024-09-17T05:01:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 msgr2=0x7fda88154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 0x7fda88154d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e9f0 tx=0x7fda7800edf0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 shutdown_connections 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda64073200 0x7fda64075710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda8815f750 0x7fda8815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda880cf860 0x7fda881f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 --2- 172.21.15.16:0/3056281578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda8806ce90 0x7fda88154d30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 >> 172.21.15.16:0/3056281578 conn(0x7fda880d0860 msgr2=0x7fda88073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 shutdown_connections 2024-09-17T05:01:46.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.386+0000 7fda8e800640 1 -- 172.21.15.16:0/3056281578 wait complete. 2024-09-17T05:01:46.409 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T05:01:06.256743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T05:01:07.232100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-17T05:01:10.875249+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-17T05:01:46.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:01:46.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 -- 172.21.15.16:0/3644643524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 msgr2=0x7f0d7c1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 --2- 172.21.15.16:0/3644643524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c1590a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009c80 tx=0x7f0d6c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:01:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 -- 172.21.15.16:0/3644643524 shutdown_connections 2024-09-17T05:01:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 --2- 172.21.15.16:0/3644643524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c1590a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 -- 172.21.15.16:0/3644643524 >> 172.21.15.16:0/3644643524 conn(0x7f0d7c0b2250 msgr2=0x7f0d7c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 -- 172.21.15.16:0/3644643524 shutdown_connections 2024-09-17T05:01:46.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 -- 172.21.15.16:0/3644643524 wait complete. 2024-09-17T05:01:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 Processor -- start 2024-09-17T05:01:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.570+0000 7f0d81a00640 1 -- start start 2024-09-17T05:01:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0d7c073f30 0x7f0d7c1fa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7c0ca6b0 0x7f0d7c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c169ed0 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c169d50 con 0x7f0d7c0ca6b0 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0d7c16a050 con 0x7f0d7c073f30 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60572/0 (socket says 172.21.15.16:60572) 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 -- 172.21.15.16:0/2562417744 learned_addr learned my addr 172.21.15.16:0/2562417744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:46.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7aa00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0d7c073f30 0x7f0d7c1fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d72000640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7c0ca6b0 0x7f0d7c1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 -- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0d7c073f30 msgr2=0x7f0d7c1fa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0d7c073f30 0x7f0d7c1fa970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 -- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7c0ca6b0 msgr2=0x7f0d7c1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7c0ca6b0 0x7f0d7c1516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 -- 172.21.15.16:0/2562417744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c0b3fa0 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d7b400640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c151c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c3c0 tx=0x7f0d7400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74017070 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d74003600 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c0b44d0 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74003a50 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c0b4a90 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.574+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c0b4240 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.578+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0d7401a070 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.578+0000 7f0d73400640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0d58073230 0x7f0d58075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.578+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0d740937a0 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.578+0000 7f0d7aa00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0d58073230 0x7f0d58075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.578+0000 7f0d7aa00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0d58073230 0x7f0d58075740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0af000 tx=0x7f0d6c01b550 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.582+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0d7405de20 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.758+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d7c1590a0 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.758+0000 7f0d73400640 1 -- 172.21.15.16:0/2562417744 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f0d74062850 con 0x7f0d7c0cb8d0 2024-09-17T05:01:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:01:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.758+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0d58073230 msgr2=0x7f0d58075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.758+0000 7f0d81a00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0d58073230 0x7f0d58075740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0af000 tx=0x7f0d6c01b550 comp rx=0 tx=0).stop 2024-09-17T05:01:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.758+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 msgr2=0x7f0d7c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.758+0000 7f0d81a00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c151c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c3c0 tx=0x7f0d7400c7c0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 shutdown_connections 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0d58073230 0x7f0d58075740 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d7c0cb8d0 0x7f0d7c151c20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d7c0ca6b0 0x7f0d7c1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 --2- 172.21.15.16:0/2562417744 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0d7c073f30 0x7f0d7c1fa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 >> 172.21.15.16:0/2562417744 conn(0x7f0d7c0b2250 msgr2=0x7f0d7c15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 shutdown_connections 2024-09-17T05:01:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.762+0000 7f0d81a00640 1 -- 172.21.15.16:0/2562417744 wait complete. 2024-09-17T05:01:46.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T05:01:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 -- 172.21.15.16:0/736720814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c15f520 msgr2=0x7fb38c15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 --2- 172.21.15.16:0/736720814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c15f520 0x7fb38c15c160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb38c072860 tx=0x7fb38401cd80 comp rx=0 tx=0).stop 2024-09-17T05:01:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 -- 172.21.15.16:0/736720814 shutdown_connections 2024-09-17T05:01:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 --2- 172.21.15.16:0/736720814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c15f520 0x7fb38c15c160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 --2- 172.21.15.16:0/736720814 >> v2:172.21.15.104:3300/0 conn(0x7fb38c074a20 0x7fb38c15efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 -- 172.21.15.16:0/736720814 >> 172.21.15.16:0/736720814 conn(0x7fb38c0b2250 msgr2=0x7fb38c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 -- 172.21.15.16:0/736720814 shutdown_connections 2024-09-17T05:01:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.942+0000 7fb394200640 1 -- 172.21.15.16:0/736720814 wait complete. 2024-09-17T05:01:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 Processor -- start 2024-09-17T05:01:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- start start 2024-09-17T05:01:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 0x7fb38c165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb38c074e30 0x7fb38c1fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb38c15f520 0x7fb38c14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c168ed0 con 0x7fb38c074a20 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c168d50 con 0x7fb38c074e30 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c169050 con 0x7fb38c15f520 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 0x7fb38c165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 0x7fb38c165b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60580/0 (socket says 172.21.15.16:60580) 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 -- 172.21.15.16:0/541660964 learned_addr learned my addr 172.21.15.16:0/541660964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb390e00640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb38c074e30 0x7fb38c1fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb392200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb38c15f520 0x7fb38c14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 -- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb38c15f520 msgr2=0x7fb38c14b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb38c15f520 0x7fb38c14b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 -- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb38c074e30 msgr2=0x7fb38c1fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb38c074e30 0x7fb38c1fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:46.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 -- 172.21.15.16:0/541660964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c154820 con 0x7fb38c074a20 2024-09-17T05:01:46.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb391800640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 0x7fb38c165b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb38000d750 tx=0x7fb38000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380017070 con 0x7fb38c074a20 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb380009d20 con 0x7fb38c074a20 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380005280 con 0x7fb38c074a20 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c154cc0 con 0x7fb38c074a20 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c1530f0 con 0x7fb38c074a20 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.946+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c152c00 con 0x7fb38c074a20 2024-09-17T05:01:46.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.950+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb380005420 con 0x7fb38c074a20 2024-09-17T05:01:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.950+0000 7fb389600640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb36c073200 0x7fb36c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:46.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.950+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb3800946d0 con 0x7fb38c074a20 2024-09-17T05:01:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.950+0000 7fb390e00640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb36c073200 0x7fb36c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.950+0000 7fb390e00640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb36c073200 0x7fb36c075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb38c14c0b0 tx=0x7fb378009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:46.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:46.954+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb38000de60 con 0x7fb38c074a20 2024-09-17T05:01:47.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.130+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb38c15efe0 con 0x7fb38c074a20 2024-09-17T05:01:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.130+0000 7fb389600640 1 -- 172.21.15.16:0/541660964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7fb38005f3a0 con 0x7fb38c074a20 2024-09-17T05:01:47.133 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb36c073200 msgr2=0x7fb36c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb36c073200 0x7fb36c075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb38c14c0b0 tx=0x7fb378009040 comp rx=0 tx=0).stop 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 msgr2=0x7fb38c165b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 0x7fb38c165b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb38000d750 tx=0x7fb38000dc10 comp rx=0 tx=0).stop 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 shutdown_connections 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb36c073200 0x7fb36c075710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb38c15f520 0x7fb38c14b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb38c074e30 0x7fb38c1fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 --2- 172.21.15.16:0/541660964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb38c074a20 0x7fb38c165b80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 >> 172.21.15.16:0/541660964 conn(0x7fb38c0b2250 msgr2=0x7fb38c0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 shutdown_connections 2024-09-17T05:01:47.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.134+0000 7fb394200640 1 -- 172.21.15.16:0/541660964 wait complete. 2024-09-17T05:01:47.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T05:01:47.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 -- 172.21.15.16:0/799977847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 msgr2=0x7fc3e00cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/799977847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e00cf140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009930 tx=0x7fc3c801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 -- 172.21.15.16:0/799977847 shutdown_connections 2024-09-17T05:01:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/799977847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e00cf140 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 -- 172.21.15.16:0/799977847 >> 172.21.15.16:0/799977847 conn(0x7fc3e00b42d0 msgr2=0x7fc3e00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 -- 172.21.15.16:0/799977847 shutdown_connections 2024-09-17T05:01:47.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.314+0000 7fc3e4e00640 1 -- 172.21.15.16:0/799977847 wait complete. 2024-09-17T05:01:47.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 Processor -- start 2024-09-17T05:01:47.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 -- start start 2024-09-17T05:01:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3e00cf680 0x7fc3e0146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e0159f70 0x7fc3e0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e016b1b0 con 0x7fc3e00d02d0 2024-09-17T05:01:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e016b030 con 0x7fc3e0159f70 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3e4e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc3e016b330 con 0x7fc3e00cf680 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3d7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e0154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53970/0 (socket says 172.21.15.16:53970) 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3d7e00640 1 -- 172.21.15.16:0/1115973059 learned_addr learned my addr 172.21.15.16:0/1115973059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3dde00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3e00cf680 0x7fc3e0146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.318+0000 7fc3de800640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e0159f70 0x7fc3e0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d7e00640 1 -- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3e00cf680 msgr2=0x7fc3e0146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d7e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3e00cf680 0x7fc3e0146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d7e00640 1 -- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e0159f70 msgr2=0x7fc3e0154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d7e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e0159f70 0x7fc3e0154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d7e00640 1 -- 172.21.15.16:0/1115973059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e0157b30 con 0x7fc3e00d02d0 2024-09-17T05:01:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d7e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e0154770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc3c801cca0 tx=0x7fc3c80a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8004230 con 0x7fc3e00d02d0 2024-09-17T05:01:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3c8003600 con 0x7fc3e00d02d0 2024-09-17T05:01:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8003a50 con 0x7fc3e00d02d0 2024-09-17T05:01:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e0156d70 con 0x7fc3e00d02d0 2024-09-17T05:01:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e0157330 con 0x7fc3e00d02d0 2024-09-17T05:01:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e00b8050 con 0x7fc3e00d02d0 2024-09-17T05:01:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc3c8003bf0 con 0x7fc3e00d02d0 2024-09-17T05:01:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.322+0000 7fc3d6000640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc3bc073200 0x7fc3bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.326+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc3c812e480 con 0x7fc3e00d02d0 2024-09-17T05:01:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.326+0000 7fc3dde00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc3bc073200 0x7fc3bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.326+0000 7fc3dde00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc3bc073200 0x7fc3bc075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0007fd0 tx=0x7fc3d0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:47.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.330+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc3c80bf070 con 0x7fc3e00d02d0 2024-09-17T05:01:47.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.510+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3e00cf140 con 0x7fc3e00d02d0 2024-09-17T05:01:47.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.510+0000 7fc3d6000640 1 -- 172.21.15.16:0/1115973059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fc3c80b6070 con 0x7fc3e00d02d0 2024-09-17T05:01:47.513 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-17T05:01:47.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc3bc073200 msgr2=0x7fc3bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc3bc073200 0x7fc3bc075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0007fd0 tx=0x7fc3d0009040 comp rx=0 tx=0).stop 2024-09-17T05:01:47.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 msgr2=0x7fc3e0154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e0154770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc3c801cca0 tx=0x7fc3c80a6600 comp rx=0 tx=0).stop 2024-09-17T05:01:47.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 shutdown_connections 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc3bc073200 0x7fc3bc075710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e0159f70 0x7fc3e0154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e00d02d0 0x7fc3e0154770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 --2- 172.21.15.16:0/1115973059 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3e00cf680 0x7fc3e0146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 >> 172.21.15.16:0/1115973059 conn(0x7fc3e00b42d0 msgr2=0x7fc3e00cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 shutdown_connections 2024-09-17T05:01:47.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.514+0000 7fc3e4e00640 1 -- 172.21.15.16:0/1115973059 wait complete. 2024-09-17T05:01:47.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-17T05:01:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.694+0000 7f551a000640 1 -- 172.21.15.16:0/528549460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54f8003a10 msgr2=0x7f54f8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.694+0000 7f551a000640 1 --2- 172.21.15.16:0/528549460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54f8003a10 0x7f54f8003e80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5504009b60 tx=0x7f550401d2b0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.694+0000 7f5513e00640 1 -- 172.21.15.16:0/528549460 reap_dead start 2024-09-17T05:01:47.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.694+0000 7f551a000640 1 -- 172.21.15.16:0/528549460 shutdown_connections 2024-09-17T05:01:47.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.694+0000 7f551a000640 1 -- 172.21.15.16:0/528549460 >> 172.21.15.16:0/528549460 conn(0x7f55140b2250 msgr2=0x7f55140d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:47.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 -- 172.21.15.16:0/528549460 shutdown_connections 2024-09-17T05:01:47.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 -- 172.21.15.16:0/528549460 wait complete. 2024-09-17T05:01:47.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 Processor -- start 2024-09-17T05:01:47.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 -- start start 2024-09-17T05:01:47.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54f8003a10 0x7f55141fca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55141555f0 0x7f551414c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 0x7f551415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5514166470 con 0x7f55140b4900 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5514075040 con 0x7f54f8003a10 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.698+0000 7f551a000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f55141665f0 con 0x7f55141555f0 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 0x7f551415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 0x7f551415a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53992/0 (socket says 172.21.15.16:53992) 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 -- 172.21.15.16:0/3094172434 learned_addr learned my addr 172.21.15.16:0/3094172434 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513400640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54f8003a10 0x7f55141fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5512a00640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55141555f0 0x7f551414c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 -- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55141555f0 msgr2=0x7f551414c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55141555f0 0x7f551414c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 -- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54f8003a10 msgr2=0x7f55141fca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54f8003a10 0x7f55141fca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 -- 172.21.15.16:0/3094172434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5514155df0 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f5513e00640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 0x7f551415a6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55140cff40 tx=0x7f550c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c003980 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f550c003b20 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55140c7210 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c004030 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55141fc260 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55141590a0 con 0x7f55140b4900 2024-09-17T05:01:47.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.702+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f550c0041d0 con 0x7f55140b4900 2024-09-17T05:01:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.706+0000 7f550b400640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f54f4073230 0x7f54f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.706+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f550c095130 con 0x7f55140b4900 2024-09-17T05:01:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.706+0000 7f5513400640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f54f4073230 0x7f54f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:47.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.706+0000 7f5513400640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f54f4073230 0x7f54f4075740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5504009b60 tx=0x7f550401a050 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:47.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.710+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f550c05f840 con 0x7f55140b4900 2024-09-17T05:01:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f551415ee70 con 0x7f55140b4900 2024-09-17T05:01:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f550b400640 1 -- 172.21.15.16:0/3094172434 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f550c064270 con 0x7f55140b4900 2024-09-17T05:01:47.887 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f54f4073230 msgr2=0x7f54f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f54f4073230 0x7f54f4075740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5504009b60 tx=0x7f550401a050 comp rx=0 tx=0).stop 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 msgr2=0x7f551415a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 0x7f551415a6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f55140cff40 tx=0x7f550c009f70 comp rx=0 tx=0).stop 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 shutdown_connections 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f54f4073230 0x7f54f4075740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55140b4900 0x7f551415a6d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f55141555f0 0x7f551414c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 --2- 172.21.15.16:0/3094172434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54f8003a10 0x7f55141fca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 >> 172.21.15.16:0/3094172434 conn(0x7f55140b2250 msgr2=0x7f5514158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 shutdown_connections 2024-09-17T05:01:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:47.886+0000 7f551a000640 1 -- 172.21.15.16:0/3094172434 wait complete. 2024-09-17T05:01:47.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-09-17T05:01:48.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 -- 172.21.15.16:0/2882512741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37fc004810 msgr2=0x7f37fc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 --2- 172.21.15.16:0/2882512741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37fc004810 0x7f37fc004c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3804009b60 tx=0x7f380401d2b0 comp rx=0 tx=0).stop 2024-09-17T05:01:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381a800640 1 -- 172.21.15.16:0/2882512741 reap_dead start 2024-09-17T05:01:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 -- 172.21.15.16:0/2882512741 shutdown_connections 2024-09-17T05:01:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 -- 172.21.15.16:0/2882512741 >> 172.21.15.16:0/2882512741 conn(0x7f381406f900 msgr2=0x7f38140b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 -- 172.21.15.16:0/2882512741 shutdown_connections 2024-09-17T05:01:48.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 -- 172.21.15.16:0/2882512741 wait complete. 2024-09-17T05:01:48.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 Processor -- start 2024-09-17T05:01:48.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.074+0000 7f381c800640 1 -- start start 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f37fc004810 0x7f381413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381413fca0 0x7f38141400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 0x7f3814155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38140cfde0 con 0x7f3814155370 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38140cfc60 con 0x7f381413fca0 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3814070ea0 con 0x7f37fc004810 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 0x7f3814155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 0x7f3814155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54008/0 (socket says 172.21.15.16:54008) 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 -- 172.21.15.16:0/654785407 learned_addr learned my addr 172.21.15.16:0/654785407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f3819e00640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f37fc004810 0x7f381413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:48.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f3819400640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381413fca0 0x7f38141400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:48.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 -- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f37fc004810 msgr2=0x7f381413d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f37fc004810 0x7f381413d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 -- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381413fca0 msgr2=0x7f38141400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381413fca0 0x7f38141400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 -- 172.21.15.16:0/654785407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3814155cc0 con 0x7f3814155370 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381a800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 0x7f3814155780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3810001340 tx=0x7f3810006410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810003b80 con 0x7f3814155370 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3810003d20 con 0x7f3814155370 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38100052d0 con 0x7f3814155370 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38141f7dc0 con 0x7f3814155370 2024-09-17T05:01:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141f72e0 con 0x7f3814155370 2024-09-17T05:01:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.078+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3814140790 con 0x7f3814155370 2024-09-17T05:01:48.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.082+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3810005470 con 0x7f3814155370 2024-09-17T05:01:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.082+0000 7f380e000640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37f8073230 0x7f37f8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.082+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6677+0+0 (secure 0 0 0) 0x7f3810097b50 con 0x7f3814155370 2024-09-17T05:01:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.082+0000 7f3819e00640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37f8073230 0x7f37f8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:48.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.082+0000 7f3819e00640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37f8073230 0x7f37f8075740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3808005ec0 tx=0x7f380800c040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:48.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.086+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3814155cc0 con 0x7f3814155370 2024-09-17T05:01:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:48.326+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f3814074dc0 con 0x7f3814155370 2024-09-17T05:01:49.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.034+0000 7f380e000640 1 -- 172.21.15.16:0/654785407 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v59) ==== 168+0+0 (secure 0 0 0) 0x7f381415ebf0 con 0x7f3814155370 2024-09-17T05:01:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.034+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37f8073230 msgr2=0x7f37f8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.034+0000 7f381c800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37f8073230 0x7f37f8075740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3808005ec0 tx=0x7f380800c040 comp rx=0 tx=0).stop 2024-09-17T05:01:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.034+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 msgr2=0x7f3814155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.034+0000 7f381c800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 0x7f3814155780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3810001340 tx=0x7f3810006410 comp rx=0 tx=0).stop 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 shutdown_connections 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37f8073230 0x7f37f8075740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3814155370 0x7f3814155780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f381413fca0 0x7f38141400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 --2- 172.21.15.16:0/654785407 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f37fc004810 0x7f381413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 >> 172.21.15.16:0/654785407 conn(0x7f381406f900 msgr2=0x7f38140cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 shutdown_connections 2024-09-17T05:01:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.038+0000 7f381c800640 1 -- 172.21.15.16:0/654785407 wait complete. 2024-09-17T05:01:49.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:01:49.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.222+0000 7f4b0ee00640 1 -- 172.21.15.16:0/3108796489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 msgr2=0x7f4b0815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.222+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/3108796489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b0815cab0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009900 tx=0x7f4af801c880 comp rx=0 tx=0).stop 2024-09-17T05:01:49.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- 172.21.15.16:0/3108796489 shutdown_connections 2024-09-17T05:01:49.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/3108796489 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b080c9440 0x7f4b080c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/3108796489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b0815cab0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- 172.21.15.16:0/3108796489 >> 172.21.15.16:0/3108796489 conn(0x7f4b080d2270 msgr2=0x7f4b080d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:49.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- 172.21.15.16:0/3108796489 shutdown_connections 2024-09-17T05:01:49.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- 172.21.15.16:0/3108796489 wait complete. 2024-09-17T05:01:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 Processor -- start 2024-09-17T05:01:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- start start 2024-09-17T05:01:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b080c9440 0x7f4b080d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b080c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b080d1580 0x7f4b08154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08169f30 con 0x7f4b080cc8c0 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08169db0 con 0x7f4b080c9440 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4b0ee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0816a0b0 con 0x7f4b080d1580 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.226+0000 7f4affe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b080c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b080c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54028/0 (socket says 172.21.15.16:54028) 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 -- 172.21.15.16:0/118803763 learned_addr learned my addr 172.21.15.16:0/118803763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b0ce00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b080d1580 0x7f4b08154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b07e00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b080c9440 0x7f4b080d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 -- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b080d1580 msgr2=0x7f4b08154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b080d1580 0x7f4b08154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 -- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b080c9440 msgr2=0x7f4b080d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b080c9440 0x7f4b080d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 -- 172.21.15.16:0/118803763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080c6110 con 0x7f4b080cc8c0 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4affe00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b080c78d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4af000c440 tx=0x7f4af0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0011790 con 0x7f4b080cc8c0 2024-09-17T05:01:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af0017070 con 0x7f4b080cc8c0 2024-09-17T05:01:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0011cc0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b080c63f0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b080c69b0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b080c9850 con 0x7f4b080cc8c0 2024-09-17T05:01:49.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.230+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4af0005080 con 0x7f4b080cc8c0 2024-09-17T05:01:49.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.234+0000 7f4b04c00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ae4073230 0x7f4ae4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.234+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f4af0093fe0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.234+0000 7f4b07e00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ae4073230 0x7f4ae4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.234+0000 7f4b07e00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ae4073230 0x7f4ae4075740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4af80af000 tx=0x7f4af801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.238+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4af005ebd0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.414+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4b080c56f0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.414+0000 7f4b04c00640 1 -- 172.21.15.16:0/118803763 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f4af005e4d0 con 0x7f4b080cc8c0 2024-09-17T05:01:49.417 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:01:49.417 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:01:49.417 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:01:49.417 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:01:49.417 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:01:49.418 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:01:49.419 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:01:49.420 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:01:49.420 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:01:49.420 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ae4073230 msgr2=0x7f4ae4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ae4073230 0x7f4ae4075740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4af80af000 tx=0x7f4af801b120 comp rx=0 tx=0).stop 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 msgr2=0x7f4b080c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b080c78d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4af000c440 tx=0x7f4af0001e80 comp rx=0 tx=0).stop 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 shutdown_connections 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ae4073230 0x7f4ae4075740 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b080d1580 0x7f4b08154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b080cc8c0 0x7f4b080c78d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 --2- 172.21.15.16:0/118803763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b080c9440 0x7f4b080d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 >> 172.21.15.16:0/118803763 conn(0x7f4b080d2270 msgr2=0x7f4b08165080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 shutdown_connections 2024-09-17T05:01:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.418+0000 7f4b0ee00640 1 -- 172.21.15.16:0/118803763 wait complete. 2024-09-17T05:01:49.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:01:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 -- 172.21.15.16:0/953880661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc160840 msgr2=0x7fc4cc1672b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/953880661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc160840 0x7fc4cc1672b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc071ce0 tx=0x7fc4c401cd50 comp rx=0 tx=0).stop 2024-09-17T05:01:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 -- 172.21.15.16:0/953880661 shutdown_connections 2024-09-17T05:01:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/953880661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc160840 0x7fc4cc1672b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/953880661 >> v2:172.21.15.104:3300/0 conn(0x7fc4cc0733e0 0x7fc4cc15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 -- 172.21.15.16:0/953880661 >> 172.21.15.16:0/953880661 conn(0x7fc4cc0b2250 msgr2=0x7fc4cc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 -- 172.21.15.16:0/953880661 shutdown_connections 2024-09-17T05:01:49.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 -- 172.21.15.16:0/953880661 wait complete. 2024-09-17T05:01:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 Processor -- start 2024-09-17T05:01:49.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.602+0000 7fc4d0e00640 1 -- start start 2024-09-17T05:01:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4cc0733e0 0x7fc4cc0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 0x7fc4cc148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4cc160840 0x7fc4cc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc16a010 con 0x7fc4cc15d680 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc0cae90 con 0x7fc4cc160840 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc16a190 con 0x7fc4cc0733e0 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 0x7fc4cc148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 0x7fc4cc148850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54038/0 (socket says 172.21.15.16:54038) 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 -- 172.21.15.16:0/1887332596 learned_addr learned my addr 172.21.15.16:0/1887332596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c9e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4cc0733e0 0x7fc4cc0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4ca800640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4cc160840 0x7fc4cc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 -- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4cc0733e0 msgr2=0x7fc4cc0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4cc0733e0 0x7fc4cc0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 -- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4cc160840 msgr2=0x7fc4cc150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4cc160840 0x7fc4cc150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 -- 172.21.15.16:0/1887332596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc0b38f0 con 0x7fc4cc15d680 2024-09-17T05:01:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c9e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4cc0733e0 0x7fc4cc0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:49.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c3e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 0x7fc4cc148850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c3a0 tx=0x7fc4b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4017070 con 0x7fc4cc15d680 2024-09-17T05:01:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4b4003600 con 0x7fc4cc15d680 2024-09-17T05:01:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4003a70 con 0x7fc4cc15d680 2024-09-17T05:01:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc0b3bd0 con 0x7fc4cc15d680 2024-09-17T05:01:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc0b4190 con 0x7fc4cc15d680 2024-09-17T05:01:49.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.606+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc0b3660 con 0x7fc4cc15d680 2024-09-17T05:01:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.610+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc4b401a070 con 0x7fc4cc15d680 2024-09-17T05:01:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.610+0000 7fc4c2000640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4a8073200 0x7fc4a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.610+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7fc4b40938b0 con 0x7fc4cc15d680 2024-09-17T05:01:49.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.610+0000 7fc4c9e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4a8073200 0x7fc4a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:49.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.610+0000 7fc4c9e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4a8073200 0x7fc4a8075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006fd0 tx=0x7fc4bc008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.614+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4b405de10 con 0x7fc4cc15d680 2024-09-17T05:01:49.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.790+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc4cc15d140 con 0x7fc4cc15d680 2024-09-17T05:01:49.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.794+0000 7fc4c2000640 1 -- 172.21.15.16:0/1887332596 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fc4b4062840 con 0x7fc4cc15d680 2024-09-17T05:01:49.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4a8073200 msgr2=0x7fc4a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4a8073200 0x7fc4a8075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006fd0 tx=0x7fc4bc008040 comp rx=0 tx=0).stop 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 msgr2=0x7fc4cc148850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 0x7fc4cc148850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400c3a0 tx=0x7fc4b400c860 comp rx=0 tx=0).stop 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 shutdown_connections 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4a8073200 0x7fc4a8075710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4cc160840 0x7fc4cc150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4cc15d680 0x7fc4cc148850 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 --2- 172.21.15.16:0/1887332596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4cc0733e0 0x7fc4cc0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 >> 172.21.15.16:0/1887332596 conn(0x7fc4cc0b2250 msgr2=0x7fc4cc0b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 shutdown_connections 2024-09-17T05:01:49.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:49.798+0000 7fc4d0e00640 1 -- 172.21.15.16:0/1887332596 wait complete. 2024-09-17T05:01:49.821 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:01:49.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:01:49.928 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:01:50.044 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:01:50.044 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:50.044 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:50.053 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:50.053 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:50.102 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:50.102 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:50.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:01:50.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 -- 172.21.15.16:0/3142648767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074a20 msgr2=0x7f5d5815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 --2- 172.21.15.16:0/3142648767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074a20 0x7f5d5815efe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d58156030 tx=0x7f5d4c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 -- 172.21.15.16:0/3142648767 shutdown_connections 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 --2- 172.21.15.16:0/3142648767 >> v2:172.21.15.104:3300/0 conn(0x7f5d5815f520 0x7f5d5815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 --2- 172.21.15.16:0/3142648767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074a20 0x7f5d5815efe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 -- 172.21.15.16:0/3142648767 >> 172.21.15.16:0/3142648767 conn(0x7f5d580b2250 msgr2=0x7f5d580d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 -- 172.21.15.16:0/3142648767 shutdown_connections 2024-09-17T05:01:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.354+0000 7f5d60000640 1 -- 172.21.15.16:0/3142648767 wait complete. 2024-09-17T05:01:50.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 Processor -- start 2024-09-17T05:01:50.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 -- start start 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d58074a20 0x7f5d581f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074e30 0x7f5d580b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 0x7f5d58159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58168ed0 con 0x7f5d5815f520 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58168d50 con 0x7f5d58074a20 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d60000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5d58169050 con 0x7f5d58074e30 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 0x7f5d58159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 0x7f5d58159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54056/0 (socket says 172.21.15.16:54056) 2024-09-17T05:01:50.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 -- 172.21.15.16:0/1864108025 learned_addr learned my addr 172.21.15.16:0/1864108025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5d600640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d58074a20 0x7f5d581f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5cc00640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074e30 0x7f5d580b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 -- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074e30 msgr2=0x7f5d580b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074e30 0x7f5d580b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 -- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d58074a20 msgr2=0x7f5d581f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d58074a20 0x7f5d581f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5e000640 1 -- 172.21.15.16:0/1864108025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d581f94a0 con 0x7f5d5815f520 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.358+0000 7f5d5cc00640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074e30 0x7f5d580b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:50.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d5e000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 0x7f5d58159d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000e940 tx=0x7f5d5000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50004d30 con 0x7f5d5815f520 2024-09-17T05:01:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d5000cb20 con 0x7f5d5815f520 2024-09-17T05:01:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d50005020 con 0x7f5d5815f520 2024-09-17T05:01:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d581f9780 con 0x7f5d5815f520 2024-09-17T05:01:50.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d58164140 con 0x7f5d5815f520 2024-09-17T05:01:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d581f9210 con 0x7f5d5815f520 2024-09-17T05:01:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d500051c0 con 0x7f5d5815f520 2024-09-17T05:01:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d55600640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.362+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f5d50095090 con 0x7f5d5815f520 2024-09-17T05:01:50.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.366+0000 7f5d5d600640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.366+0000 7f5d5d600640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af000 tx=0x7f5d4c0a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:50.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.366+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5d50016030 con 0x7f5d5815f520 2024-09-17T05:01:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.550+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d580b5290 con 0x7f5d5815f520 2024-09-17T05:01:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.550+0000 7f5d55600640 1 -- 172.21.15.16:0/1864108025 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f5d5005f700 con 0x7f5d5815f520 2024-09-17T05:01:50.553 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:01:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 msgr2=0x7f5d38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af000 tx=0x7f5d4c0a7040 comp rx=0 tx=0).stop 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 msgr2=0x7f5d58159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 0x7f5d58159d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000e940 tx=0x7f5d5000ee00 comp rx=0 tx=0).stop 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 shutdown_connections 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5815f520 0x7f5d58159d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074e30 0x7f5d580b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 --2- 172.21.15.16:0/1864108025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d58074a20 0x7f5d581f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 >> 172.21.15.16:0/1864108025 conn(0x7f5d580b2250 msgr2=0x7f5d58158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 shutdown_connections 2024-09-17T05:01:50.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.554+0000 7f5d60000640 1 -- 172.21.15.16:0/1864108025 wait complete. 2024-09-17T05:01:50.575 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:01:50.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:50.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-17T05:01:50.613 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:01:50.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:50.613 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:01:50.663 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:50.663 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:50.672 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:50.673 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:50.726 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:50.726 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:50.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:01:50.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 -- 172.21.15.16:0/2659723100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940053d0 msgr2=0x7fca9400ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 --2- 172.21.15.16:0/2659723100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940053d0 0x7fca9400ba80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009eb0 tx=0x7fcaa801d5f0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 -- 172.21.15.16:0/2659723100 shutdown_connections 2024-09-17T05:01:50.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 --2- 172.21.15.16:0/2659723100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca940053d0 0x7fca9400ba80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 -- 172.21.15.16:0/2659723100 >> 172.21.15.16:0/2659723100 conn(0x7fcab00b2250 msgr2=0x7fcab00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:50.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 -- 172.21.15.16:0/2659723100 shutdown_connections 2024-09-17T05:01:50.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 -- 172.21.15.16:0/2659723100 wait complete. 2024-09-17T05:01:50.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 Processor -- start 2024-09-17T05:01:50.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.938+0000 7fcab7c00640 1 -- start start 2024-09-17T05:01:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca940053d0 0x7fcab014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 0x7fcab0153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab014ebe0 0x7fcab0154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcab00d0b50 con 0x7fcab0158620 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcab00d09d0 con 0x7fcab014ebe0 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcab00d0cd0 con 0x7fca940053d0 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 0x7fcab0153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab5200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca940053d0 0x7fcab014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 0x7fcab0153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54070/0 (socket says 172.21.15.16:54070) 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 -- 172.21.15.16:0/1317296617 learned_addr learned my addr 172.21.15.16:0/1317296617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab5c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab014ebe0 0x7fcab0154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 -- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca940053d0 msgr2=0x7fcab014e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca940053d0 0x7fcab014e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 -- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab014ebe0 msgr2=0x7fcab0154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab014ebe0 0x7fcab0154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 -- 172.21.15.16:0/1317296617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab0158f50 con 0x7fcab0158620 2024-09-17T05:01:50.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab5c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab014ebe0 0x7fcab0154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaa7e00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 0x7fcab0153ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca98003e20 tx=0x7fca98001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98011700 con 0x7fcab0158620 2024-09-17T05:01:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca98016020 con 0x7fcab0158620 2024-09-17T05:01:50.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98011b40 con 0x7fcab0158620 2024-09-17T05:01:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab0159230 con 0x7fcab0158620 2024-09-17T05:01:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab00b4140 con 0x7fcab0158620 2024-09-17T05:01:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.942+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab015ef10 con 0x7fcab0158620 2024-09-17T05:01:50.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.946+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fca98011ce0 con 0x7fcab0158620 2024-09-17T05:01:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.946+0000 7fcaad600640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fca8c073200 0x7fca8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.946+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7fca980952f0 con 0x7fcab0158620 2024-09-17T05:01:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.946+0000 7fcab5200640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fca8c073200 0x7fca8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:50.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.946+0000 7fcab5200640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fca8c073200 0x7fca8c075710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0001840 tx=0x7fcaa0000f90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:50.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:50.950+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fca98020020 con 0x7fcab0158620 2024-09-17T05:01:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.134+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcab014fae0 con 0x7fcab0158620 2024-09-17T05:01:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcaad600640 1 -- 172.21.15.16:0/1317296617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fca9805fea0 con 0x7fcab0158620 2024-09-17T05:01:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fca8c073200 msgr2=0x7fca8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fca8c073200 0x7fca8c075710 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0001840 tx=0x7fcaa0000f90 comp rx=0 tx=0).stop 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 msgr2=0x7fcab0153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 0x7fcab0153ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca98003e20 tx=0x7fca98001810 comp rx=0 tx=0).stop 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 shutdown_connections 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fca8c073200 0x7fca8c075710 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab014ebe0 0x7fcab0154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcab0158620 0x7fcab0153ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 --2- 172.21.15.16:0/1317296617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fca940053d0 0x7fcab014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 >> 172.21.15.16:0/1317296617 conn(0x7fcab00b2250 msgr2=0x7fcab015d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 shutdown_connections 2024-09-17T05:01:51.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.138+0000 7fcab7c00640 1 -- 172.21.15.16:0/1317296617 wait complete. 2024-09-17T05:01:51.161 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:01:51.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:51.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-17T05:01:51.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:01:51.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:51.205 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:01:51.261 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:51.262 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:51.270 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:51.271 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:51.322 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:01:51.322 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:51.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:01:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 -- 172.21.15.16:0/3484294806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf4003cb0 msgr2=0x7fecf4004100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 --2- 172.21.15.16:0/3484294806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf4003cb0 0x7fecf4004100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fed0000c640 tx=0x7fed0001d260 comp rx=0 tx=0).stop 2024-09-17T05:01:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 -- 172.21.15.16:0/3484294806 shutdown_connections 2024-09-17T05:01:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 --2- 172.21.15.16:0/3484294806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecf4003cb0 0x7fecf4004100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 -- 172.21.15.16:0/3484294806 >> 172.21.15.16:0/3484294806 conn(0x7fed100d2000 msgr2=0x7fed100d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 -- 172.21.15.16:0/3484294806 shutdown_connections 2024-09-17T05:01:51.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 -- 172.21.15.16:0/3484294806 wait complete. 2024-09-17T05:01:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 Processor -- start 2024-09-17T05:01:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.586+0000 7fed0f200640 1 -- start start 2024-09-17T05:01:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fecf4003cb0 0x7fed101f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 0x7fed101fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed101fc000 0x7fed101fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed1015d1b0 con 0x7fed101437e0 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fed1015d030 con 0x7fed101fc000 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fed1015d330 con 0x7fecf4003cb0 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 0x7fed101fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 0x7fed101fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54094/0 (socket says 172.21.15.16:54094) 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 -- 172.21.15.16:0/426172968 learned_addr learned my addr 172.21.15.16:0/426172968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0e800640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed101fc000 0x7fed101fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0de00640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fecf4003cb0 0x7fed101f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 -- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fecf4003cb0 msgr2=0x7fed101f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fecf4003cb0 0x7fed101f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 -- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed101fc000 msgr2=0x7fed101fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed101fc000 0x7fed101fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 -- 172.21.15.16:0/426172968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed10143fa0 con 0x7fed101437e0 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0de00640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fecf4003cb0 0x7fed101f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0e800640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed101fc000 0x7fed101fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed05400640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 0x7fed101fbac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fecf8011a10 tx=0x7fecf8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf8004460 con 0x7fed101437e0 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecf800f9a0 con 0x7fed101437e0 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf800fdf0 con 0x7fed101437e0 2024-09-17T05:01:51.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed10144280 con 0x7fed101437e0 2024-09-17T05:01:51.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed101f5da0 con 0x7fed101437e0 2024-09-17T05:01:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.590+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed100d0b60 con 0x7fed101437e0 2024-09-17T05:01:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.594+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fecf80047b0 con 0x7fed101437e0 2024-09-17T05:01:51.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.594+0000 7fed06a00640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fecec073200 0x7fecec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:51.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.594+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7fecf80952a0 con 0x7fed101437e0 2024-09-17T05:01:51.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.594+0000 7fed0de00640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fecec073200 0x7fecec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:51.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.594+0000 7fed0de00640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fecec073200 0x7fecec075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed000ac000 tx=0x7fed0001a0a0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:51.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.598+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fecf805fee0 con 0x7fed101437e0 2024-09-17T05:01:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.786+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed10143d30 con 0x7fed101437e0 2024-09-17T05:01:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed06a00640 1 -- 172.21.15.16:0/426172968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fecf805f7e0 con 0x7fed101437e0 2024-09-17T05:01:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:01:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fecec073200 msgr2=0x7fecec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fecec073200 0x7fecec075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed000ac000 tx=0x7fed0001a0a0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 msgr2=0x7fed101fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 0x7fed101fbac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fecf8011a10 tx=0x7fecf8011ed0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 shutdown_connections 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fecec073200 0x7fecec075710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed101fc000 0x7fed101fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed101437e0 0x7fed101fbac0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 --2- 172.21.15.16:0/426172968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fecf4003cb0 0x7fed101f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 >> 172.21.15.16:0/426172968 conn(0x7fed100d2000 msgr2=0x7fed1015f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 shutdown_connections 2024-09-17T05:01:51.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:51.790+0000 7fed0f200640 1 -- 172.21.15.16:0/426172968 wait complete. 2024-09-17T05:01:51.813 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-17T05:01:51.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:51.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-17T05:01:51.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:01:51.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:51.837 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T05:01:51.887 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:51.887 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:51.896 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:51.897 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:51.949 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:01:51.949 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:01:52.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:01:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 -- 172.21.15.16:0/4023046845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a00005380 msgr2=0x7f1a0000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 --2- 172.21.15.16:0/4023046845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a00005380 0x7f1a0000ba10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c072200 tx=0x7f1a1401d5d0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1a800640 1 -- 172.21.15.16:0/4023046845 reap_dead start 2024-09-17T05:01:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 -- 172.21.15.16:0/4023046845 shutdown_connections 2024-09-17T05:01:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 -- 172.21.15.16:0/4023046845 >> 172.21.15.16:0/4023046845 conn(0x7f1a1c0d2000 msgr2=0x7f1a1c0d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 -- 172.21.15.16:0/4023046845 shutdown_connections 2024-09-17T05:01:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 -- 172.21.15.16:0/4023046845 wait complete. 2024-09-17T05:01:52.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 Processor -- start 2024-09-17T05:01:52.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.162+0000 7f1a1b200640 1 -- start start 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a00005380 0x7f1a1c1e8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c15aff0 0x7f1a1c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 0x7f1a1c146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c0d09a0 con 0x7f1a1c146800 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c0d0820 con 0x7f1a00005380 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1c0d0b20 con 0x7f1a1c15aff0 2024-09-17T05:01:52.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 0x7f1a1c146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 0x7f1a1c146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54120/0 (socket says 172.21.15.16:54120) 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 -- 172.21.15.16:0/2289470009 learned_addr learned my addr 172.21.15.16:0/2289470009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a19e00640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a00005380 0x7f1a1c1e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a13e00640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c15aff0 0x7f1a1c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 -- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c15aff0 msgr2=0x7f1a1c15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c15aff0 0x7f1a1c15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 -- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a00005380 msgr2=0x7f1a1c1e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a00005380 0x7f1a1c1e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 -- 172.21.15.16:0/2289470009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c1e85d0 con 0x7f1a1c146800 2024-09-17T05:01:52.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1a800640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 0x7f1a1c146c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a1401d040 tx=0x7f1a140aadc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a140a9410 con 0x7f1a1c146800 2024-09-17T05:01:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a14004d50 con 0x7f1a1c146800 2024-09-17T05:01:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14005250 con 0x7f1a1c146800 2024-09-17T05:01:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c06e150 con 0x7f1a1c146800 2024-09-17T05:01:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c06e710 con 0x7f1a1c146800 2024-09-17T05:01:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.166+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a1c162250 con 0x7f1a1c146800 2024-09-17T05:01:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.170+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1a140053f0 con 0x7f1a1c146800 2024-09-17T05:01:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.170+0000 7f1a12000640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19f8073200 0x7f19f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.170+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f1a1412fbd0 con 0x7f1a1c146800 2024-09-17T05:01:52.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.170+0000 7f1a19e00640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19f8073200 0x7f19f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.170+0000 7f1a19e00640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19f8073200 0x7f19f8075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c001ec0 tx=0x7f1a0c000ee0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:52.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.174+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1a140c1030 con 0x7f1a1c146800 2024-09-17T05:01:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1a1c0c9300 con 0x7f1a1c146800 2024-09-17T05:01:52.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a12000640 1 -- 172.21.15.16:0/2289470009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f1a140ad030 con 0x7f1a1c146800 2024-09-17T05:01:52.364 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19f8073200 msgr2=0x7f19f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19f8073200 0x7f19f8075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c001ec0 tx=0x7f1a0c000ee0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 msgr2=0x7f1a1c146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 0x7f1a1c146c10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a1401d040 tx=0x7f1a140aadc0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 shutdown_connections 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19f8073200 0x7f19f8075710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a1c146800 0x7f1a1c146c10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1a1c15aff0 0x7f1a1c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 --2- 172.21.15.16:0/2289470009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a00005380 0x7f1a1c1e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 >> 172.21.15.16:0/2289470009 conn(0x7f1a1c0d2000 msgr2=0x7f1a1c0d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 shutdown_connections 2024-09-17T05:01:52.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.362+0000 7f1a1b200640 1 -- 172.21.15.16:0/2289470009 wait complete. 2024-09-17T05:01:52.385 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-17T05:01:52.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:52.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-17T05:01:52.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:01:52.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:01:52.412 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T05:01:52.462 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:01:52.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 -- 172.21.15.16:0/3028742065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f00c9440 msgr2=0x7f89f00c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 --2- 172.21.15.16:0/3028742065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f00c9440 0x7f89f00c9850 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f89f00ce800 tx=0x7f89e801c800 comp rx=0 tx=0).stop 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 -- 172.21.15.16:0/3028742065 shutdown_connections 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 --2- 172.21.15.16:0/3028742065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f00c9440 0x7f89f00c9850 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 --2- 172.21.15.16:0/3028742065 >> v2:172.21.15.104:3300/0 conn(0x7f89f00d1c90 0x7f89f015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 -- 172.21.15.16:0/3028742065 >> 172.21.15.16:0/3028742065 conn(0x7f89f00d2270 msgr2=0x7f89f00d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 -- 172.21.15.16:0/3028742065 shutdown_connections 2024-09-17T05:01:52.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.618+0000 7f89f5200640 1 -- 172.21.15.16:0/3028742065 wait complete. 2024-09-17T05:01:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 Processor -- start 2024-09-17T05:01:52.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 -- start start 2024-09-17T05:01:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f00c9440 0x7f89f006eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00cc8c0 0x7f89f01531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 0x7f89f0153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0169eb0 con 0x7f89f0151c40 2024-09-17T05:01:52.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0169d30 con 0x7f89f00cc8c0 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89f5200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f89f016a030 con 0x7f89f00c9440 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 0x7f89f0153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 0x7f89f0153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54140/0 (socket says 172.21.15.16:54140) 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 -- 172.21.15.16:0/625993113 learned_addr learned my addr 172.21.15.16:0/625993113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ede00640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f00c9440 0x7f89f006eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89e5400640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00cc8c0 0x7f89f01531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 -- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f00c9440 msgr2=0x7f89f006eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f00c9440 0x7f89f006eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 -- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00cc8c0 msgr2=0x7f89f01531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00cc8c0 0x7f89f01531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.622+0000 7f89ee800640 1 -- 172.21.15.16:0/625993113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f01fbee0 con 0x7f89f0151c40 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89ee800640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 0x7f89f0153710 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89e80982f0 tx=0x7f89e8005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:52.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e80b2040 con 0x7f89f0151c40 2024-09-17T05:01:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89e80bb070 con 0x7f89f0151c40 2024-09-17T05:01:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8003600 con 0x7f89f0151c40 2024-09-17T05:01:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f01fc1c0 con 0x7f89f0151c40 2024-09-17T05:01:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f01fc780 con 0x7f89f0151c40 2024-09-17T05:01:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f00c9850 con 0x7f89f0151c40 2024-09-17T05:01:52.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.626+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89e80037a0 con 0x7f89f0151c40 2024-09-17T05:01:52.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.630+0000 7f89e6a00640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f89cc073200 0x7f89cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:52.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.630+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f89e812e880 con 0x7f89f0151c40 2024-09-17T05:01:52.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.630+0000 7f89ede00640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f89cc073200 0x7f89cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:52.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.630+0000 7f89ede00640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f89cc073200 0x7f89cc075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89e00017d0 tx=0x7f89e0001760 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:52.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.634+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f89e80163f0 con 0x7f89f0151c40 2024-09-17T05:01:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89f01fcca0 con 0x7f89f0151c40 2024-09-17T05:01:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89e6a00640 1 -- 172.21.15.16:0/625993113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f89e80f8eb0 con 0x7f89f0151c40 2024-09-17T05:01:52.844 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:52.844 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f89cc073200 msgr2=0x7f89cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f89cc073200 0x7f89cc075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f89e00017d0 tx=0x7f89e0001760 comp rx=0 tx=0).stop 2024-09-17T05:01:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 msgr2=0x7f89f0153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 0x7f89f0153710 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89e80982f0 tx=0x7f89e8005ca0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 shutdown_connections 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f89cc073200 0x7f89cc075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89f0151c40 0x7f89f0153710 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89f00cc8c0 0x7f89f01531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 --2- 172.21.15.16:0/625993113 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f89f00c9440 0x7f89f006eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.842+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 >> 172.21.15.16:0/625993113 conn(0x7f89f00d2270 msgr2=0x7f89f015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.846+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 shutdown_connections 2024-09-17T05:01:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:52.846+0000 7f89f5200640 1 -- 172.21.15.16:0/625993113 wait complete. 2024-09-17T05:01:52.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 -- 172.21.15.16:0/3184827082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b80b71a0 msgr2=0x7f08b815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 --2- 172.21.15.16:0/3184827082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b80b71a0 0x7f08b815cc00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009bf0 tx=0x7f08b001cbb0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 -- 172.21.15.16:0/3184827082 shutdown_connections 2024-09-17T05:01:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 --2- 172.21.15.16:0/3184827082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b80b71a0 0x7f08b815cc00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 -- 172.21.15.16:0/3184827082 >> 172.21.15.16:0/3184827082 conn(0x7f08b80b4600 msgr2=0x7f08b80d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 -- 172.21.15.16:0/3184827082 shutdown_connections 2024-09-17T05:01:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 -- 172.21.15.16:0/3184827082 wait complete. 2024-09-17T05:01:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 Processor -- start 2024-09-17T05:01:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.026+0000 7f08bcc00640 1 -- start start 2024-09-17T05:01:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 0x7f08b81fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b80b71a0 0x7f08b814bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b8166fe0 0x7f08b8151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08b816b2e0 con 0x7f08b80b1cb0 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08b816b160 con 0x7f08b80b71a0 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f08b816b460 con 0x7f08b8166fe0 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 0x7f08b81fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 0x7f08b81fcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54146/0 (socket says 172.21.15.16:54146) 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 -- 172.21.15.16:0/1073578702 learned_addr learned my addr 172.21.15.16:0/1073578702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08afe00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b80b71a0 0x7f08b814bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b6800640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b8166fe0 0x7f08b8151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 -- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b8166fe0 msgr2=0x7f08b8151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b8166fe0 0x7f08b8151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 -- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b80b71a0 msgr2=0x7f08b814bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b80b71a0 0x7f08b814bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 -- 172.21.15.16:0/1073578702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b806ec50 con 0x7f08b80b1cb0 2024-09-17T05:01:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08b5e00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 0x7f08b81fcee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f08a8001e40 tx=0x7f08a8014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80138e0 con 0x7f08b80b1cb0 2024-09-17T05:01:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a8013a80 con 0x7f08b80b1cb0 2024-09-17T05:01:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8014030 con 0x7f08b80b1cb0 2024-09-17T05:01:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b81f9230 con 0x7f08b80b1cb0 2024-09-17T05:01:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.030+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b81f97f0 con 0x7f08b80b1cb0 2024-09-17T05:01:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.034+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b806ee20 con 0x7f08b80b1cb0 2024-09-17T05:01:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.034+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08a80141d0 con 0x7f08b80b1cb0 2024-09-17T05:01:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.034+0000 7f08ae000640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0894073230 0x7f0894075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.034+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f08a809c750 con 0x7f08b80b1cb0 2024-09-17T05:01:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.038+0000 7f08afe00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0894073230 0x7f0894075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.038+0000 7f08afe00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0894073230 0x7f0894075740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08a0000c10 tx=0x7f08a0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.038+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08b80cee60 con 0x7f08b80b1cb0 2024-09-17T05:01:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.250+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08b815cc00 con 0x7f08b80b1cb0 2024-09-17T05:01:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.250+0000 7f08ae000640 1 -- 172.21.15.16:0/1073578702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f08a8066d40 con 0x7f08b80b1cb0 2024-09-17T05:01:53.252 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:53.253 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0894073230 msgr2=0x7f0894075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0894073230 0x7f0894075740 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08a0000c10 tx=0x7f08a0009040 comp rx=0 tx=0).stop 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 msgr2=0x7f08b81fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 0x7f08b81fcee0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f08a8001e40 tx=0x7f08a8014f90 comp rx=0 tx=0).stop 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 shutdown_connections 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0894073230 0x7f0894075740 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f08b8166fe0 0x7f08b8151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b80b71a0 0x7f08b814bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 --2- 172.21.15.16:0/1073578702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08b80b1cb0 0x7f08b81fcee0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 >> 172.21.15.16:0/1073578702 conn(0x7f08b80b4600 msgr2=0x7f08b8072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 shutdown_connections 2024-09-17T05:01:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.254+0000 7f08bcc00640 1 -- 172.21.15.16:0/1073578702 wait complete. 2024-09-17T05:01:53.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 -- 172.21.15.16:0/1023473876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac581607f0 msgr2=0x7fac580b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1023473876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac581607f0 0x7fac580b1bb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fac44009840 tx=0x7fac4401c790 comp rx=0 tx=0).stop 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 -- 172.21.15.16:0/1023473876 shutdown_connections 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1023473876 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac580d1ce0 0x7fac580b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1023473876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac581607f0 0x7fac580b1bb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 -- 172.21.15.16:0/1023473876 >> 172.21.15.16:0/1023473876 conn(0x7fac580cecc0 msgr2=0x7fac580cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 -- 172.21.15.16:0/1023473876 shutdown_connections 2024-09-17T05:01:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.430+0000 7fac5fe00640 1 -- 172.21.15.16:0/1023473876 wait complete. 2024-09-17T05:01:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 Processor -- start 2024-09-17T05:01:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 -- start start 2024-09-17T05:01:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac5815a300 0x7fac580b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 0x7fac580b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac580b6bc0 0x7fac581418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac58169d50 con 0x7fac581545f0 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac58169bd0 con 0x7fac5815a300 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5fe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fac58169ed0 con 0x7fac580b6bc0 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 0x7fac580b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac5815a300 0x7fac580b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 0x7fac580b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54158/0 (socket says 172.21.15.16:54158) 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 -- 172.21.15.16:0/1330592536 learned_addr learned my addr 172.21.15.16:0/1330592536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5de00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac580b6bc0 0x7fac581418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 -- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac580b6bc0 msgr2=0x7fac581418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac580b6bc0 0x7fac581418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 -- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac5815a300 msgr2=0x7fac580b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac5815a300 0x7fac580b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.434+0000 7fac5ca00640 1 -- 172.21.15.16:0/1330592536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac58156630 con 0x7fac581545f0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac5ca00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 0x7fac580b6680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac44009790 tx=0x7fac44005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac440b2040 con 0x7fac581545f0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac44004320 con 0x7fac581545f0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac58159330 con 0x7fac581545f0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac440052d0 con 0x7fac581545f0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac58156bb0 con 0x7fac581545f0 2024-09-17T05:01:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac580d1ce0 con 0x7fac581545f0 2024-09-17T05:01:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac44005470 con 0x7fac581545f0 2024-09-17T05:01:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac55600640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac38073200 0x7fac38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.438+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7fac4412dc00 con 0x7fac581545f0 2024-09-17T05:01:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.442+0000 7fac5d400640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac38073200 0x7fac38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.442+0000 7fac5d400640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac38073200 0x7fac38075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac4c007fd0 tx=0x7fac4c007ea0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.446+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac440bc060 con 0x7fac581545f0 2024-09-17T05:01:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.618+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac58073060 con 0x7fac581545f0 2024-09-17T05:01:53.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac55600640 1 -- 172.21.15.16:0/1330592536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18825 (secure 0 0 0) 0x7fac44016410 con 0x7fac581545f0 2024-09-17T05:01:53.622 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:53.623 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:49.027397+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac38073200 msgr2=0x7fac38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac38073200 0x7fac38075710 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fac4c007fd0 tx=0x7fac4c007ea0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 msgr2=0x7fac580b6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 0x7fac580b6680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac44009790 tx=0x7fac44005db0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 shutdown_connections 2024-09-17T05:01:53.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac38073200 0x7fac38075710 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac580b6bc0 0x7fac581418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac581545f0 0x7fac580b6680 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 --2- 172.21.15.16:0/1330592536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac5815a300 0x7fac580b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 >> 172.21.15.16:0/1330592536 conn(0x7fac580cecc0 msgr2=0x7fac581668e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 shutdown_connections 2024-09-17T05:01:53.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.622+0000 7fac5fe00640 1 -- 172.21.15.16:0/1330592536 wait complete. 2024-09-17T05:01:53.645 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:01:53.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 -- 172.21.15.16:0/3875389943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc0cf860 msgr2=0x7fa0fc15f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 --2- 172.21.15.16:0/3875389943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc0cf860 0x7fa0fc15f040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0735a0 tx=0x7fa0e401c890 comp rx=0 tx=0).stop 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 -- 172.21.15.16:0/3875389943 shutdown_connections 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 --2- 172.21.15.16:0/3875389943 >> v2:172.21.15.104:3300/0 conn(0x7fa0fc15f580 0x7fa0fc15f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 --2- 172.21.15.16:0/3875389943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc0cf860 0x7fa0fc15f040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 --2- 172.21.15.16:0/3875389943 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0fc06ce90 0x7fa0fc0cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 -- 172.21.15.16:0/3875389943 >> 172.21.15.16:0/3875389943 conn(0x7fa0fc0d0860 msgr2=0x7fa0fc0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:53.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 -- 172.21.15.16:0/3875389943 shutdown_connections 2024-09-17T05:01:53.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.802+0000 7fa102200640 1 -- 172.21.15.16:0/3875389943 wait complete. 2024-09-17T05:01:53.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 Processor -- start 2024-09-17T05:01:53.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 -- start start 2024-09-17T05:01:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 0x7fa0fc1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0fc0cf860 0x7fa0fc15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0fc15f580 0x7fa0fc1edde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0cfe70 con 0x7fa0fc06ce90 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0b5a80 con 0x7fa0fc15f580 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc0b4330 con 0x7fa0fc0cf860 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 0x7fa0fc1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 0x7fa0fc1483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54180/0 (socket says 172.21.15.16:54180) 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 -- 172.21.15.16:0/2625662123 learned_addr learned my addr 172.21.15.16:0/2625662123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 -- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0fc0cf860 msgr2=0x7fa0fc15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0fc0cf860 0x7fa0fc15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 -- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0fc15f580 msgr2=0x7fa0fc1edde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0fc15f580 0x7fa0fc1edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:53.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 -- 172.21.15.16:0/2625662123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc15d020 con 0x7fa0fc06ce90 2024-09-17T05:01:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0fb400640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 0x7fa0fc1483f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e7c0 tx=0x7fa0ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00ccc0 con 0x7fa0fc06ce90 2024-09-17T05:01:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0ec00ce80 con 0x7fa0fc06ce90 2024-09-17T05:01:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec005280 con 0x7fa0fc06ce90 2024-09-17T05:01:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc15dee0 con 0x7fa0fc06ce90 2024-09-17T05:01:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.806+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc159180 con 0x7fa0fc06ce90 2024-09-17T05:01:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.810+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc0b4820 con 0x7fa0fc06ce90 2024-09-17T05:01:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.810+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa0ec005420 con 0x7fa0fc06ce90 2024-09-17T05:01:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.810+0000 7fa0f3400640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa0d8073200 0x7fa0d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.810+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7fa0ec0950a0 con 0x7fa0fc06ce90 2024-09-17T05:01:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.810+0000 7fa0f3e00640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa0d8073200 0x7fa0d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.810+0000 7fa0f3e00640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa0d8073200 0x7fa0d8075710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40af000 tx=0x7fa0e4019630 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:53.814+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa0ec05fc50 con 0x7fa0fc06ce90 2024-09-17T05:01:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0fc0b5310 con 0x7fa0fc06ce90 2024-09-17T05:01:54.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa0f3400640 1 -- 172.21.15.16:0/2625662123 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7fa0ec05f550 con 0x7fa0fc06ce90 2024-09-17T05:01:54.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:54.028 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa0d8073200 msgr2=0x7fa0d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa0d8073200 0x7fa0d8075710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0e40af000 tx=0x7fa0e4019630 comp rx=0 tx=0).stop 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 msgr2=0x7fa0fc1483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 0x7fa0fc1483f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e7c0 tx=0x7fa0ec00ec80 comp rx=0 tx=0).stop 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 shutdown_connections 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa0d8073200 0x7fa0d8075710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0fc15f580 0x7fa0fc1edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0fc0cf860 0x7fa0fc15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 --2- 172.21.15.16:0/2625662123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0fc06ce90 0x7fa0fc1483f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 >> 172.21.15.16:0/2625662123 conn(0x7fa0fc0d0860 msgr2=0x7fa0fc073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 shutdown_connections 2024-09-17T05:01:54.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.026+0000 7fa102200640 1 -- 172.21.15.16:0/2625662123 wait complete. 2024-09-17T05:01:54.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 -- 172.21.15.16:0/2310215711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00749d0 msgr2=0x7f15c0152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 --2- 172.21.15.16:0/2310215711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00749d0 0x7f15c0152250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15c006f7a0 tx=0x7f15b001c830 comp rx=0 tx=0).stop 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 -- 172.21.15.16:0/2310215711 shutdown_connections 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 --2- 172.21.15.16:0/2310215711 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15c00cc6f0 0x7f15c00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 --2- 172.21.15.16:0/2310215711 >> v2:172.21.15.104:3300/0 conn(0x7f15c006dfa0 0x7f15c006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 --2- 172.21.15.16:0/2310215711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00749d0 0x7f15c0152250 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 -- 172.21.15.16:0/2310215711 >> 172.21.15.16:0/2310215711 conn(0x7f15c00cdb80 msgr2=0x7f15c00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 -- 172.21.15.16:0/2310215711 shutdown_connections 2024-09-17T05:01:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.206+0000 7f15c7c00640 1 -- 172.21.15.16:0/2310215711 wait complete. 2024-09-17T05:01:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 Processor -- start 2024-09-17T05:01:54.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 -- start start 2024-09-17T05:01:54.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c006dfa0 0x7f15c02010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15c00749d0 0x7f15c014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 0x7f15c01fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15c006ce90 con 0x7f15c00cc6f0 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f15c015fdd0 con 0x7f15c006dfa0 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c7c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f15c006d010 con 0x7f15c00749d0 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 0x7f15c01fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 0x7f15c01fa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54200/0 (socket says 172.21.15.16:54200) 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c5c00640 1 -- 172.21.15.16:0/298440376 learned_addr learned my addr 172.21.15.16:0/298440376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15c5200640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c006dfa0 0x7f15c02010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.210+0000 7f15b7e00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15c00749d0 0x7f15c014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c5c00640 1 -- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15c00749d0 msgr2=0x7f15c014c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c5c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15c00749d0 0x7f15c014c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c5c00640 1 -- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c006dfa0 msgr2=0x7f15c02010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c5c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c006dfa0 0x7f15c02010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c5c00640 1 -- 172.21.15.16:0/298440376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c01fd2b0 con 0x7f15c00cc6f0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c5c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 0x7f15c01fa0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15b800d9d0 tx=0x7f15b800de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8017070 con 0x7f15c00cc6f0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15b8004b30 con 0x7f15c00cc6f0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c01fd590 con 0x7f15c00cc6f0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b800be60 con 0x7f15c00cc6f0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c01fdb50 con 0x7f15c00cc6f0 2024-09-17T05:01:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c0152250 con 0x7f15c00cc6f0 2024-09-17T05:01:54.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.214+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f15b8003980 con 0x7f15c00cc6f0 2024-09-17T05:01:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.218+0000 7f15bd600640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f159c073200 0x7f159c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.218+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f15b8094d00 con 0x7f15c00cc6f0 2024-09-17T05:01:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.218+0000 7f15c5200640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f159c073200 0x7f159c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.218+0000 7f15c5200640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f159c073200 0x7f159c075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f15b00af000 tx=0x7f15b00a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.222+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f15b8020020 con 0x7f15c00cc6f0 2024-09-17T05:01:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.426+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15c00b0ce0 con 0x7f15c00cc6f0 2024-09-17T05:01:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15bd600640 1 -- 172.21.15.16:0/298440376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f15b805f1e0 con 0x7f15c00cc6f0 2024-09-17T05:01:54.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:54.431 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-17T05:01:23:926323+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2060571567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2060571567},{"type":"v1","addr":"172.21.15.16:6835","nonce":2060571567}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4316,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2436348016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2436348016},{"type":"v1","addr":"172.21.15.104:6837","nonce":2436348016}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4387,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2346094270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":2346094270},{"type":"v1","addr":"172.21.15.16:6834","nonce":2346094270}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:01:08.278850+0000","modified":"2024-09-17T05:01:23.342355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/636271305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":636271305},{"type":"v1","addr":"172.21.15.104:6836","nonce":636271305}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-17T05:01:54.431 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f159c073200 msgr2=0x7f159c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f159c073200 0x7f159c075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f15b00af000 tx=0x7f15b00a7040 comp rx=0 tx=0).stop 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 msgr2=0x7f15c01fa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 0x7f15c01fa0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f15b800d9d0 tx=0x7f15b800de90 comp rx=0 tx=0).stop 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 shutdown_connections 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f159c073200 0x7f159c075710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15c00cc6f0 0x7f15c01fa0d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f15c00749d0 0x7f15c014c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 --2- 172.21.15.16:0/298440376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15c006dfa0 0x7f15c02010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 >> 172.21.15.16:0/298440376 conn(0x7f15c00cdb80 msgr2=0x7f15c00b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 shutdown_connections 2024-09-17T05:01:54.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.430+0000 7f15c7c00640 1 -- 172.21.15.16:0/298440376 wait complete. 2024-09-17T05:01:54.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:01:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 -- 172.21.15.16:0/2030659454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8003a60 msgr2=0x7f3bb8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 --2- 172.21.15.16:0/2030659454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8003a60 0x7f3bb8003ed0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40016c0 tx=0x7f3bc401c770 comp rx=0 tx=0).stop 2024-09-17T05:01:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 -- 172.21.15.16:0/2030659454 shutdown_connections 2024-09-17T05:01:54.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 --2- 172.21.15.16:0/2030659454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bb8003a60 0x7f3bb8003ed0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 -- 172.21.15.16:0/2030659454 >> 172.21.15.16:0/2030659454 conn(0x7f3bd00d0860 msgr2=0x7f3bd00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 -- 172.21.15.16:0/2030659454 shutdown_connections 2024-09-17T05:01:54.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.610+0000 7f3bd8200640 1 -- 172.21.15.16:0/2030659454 wait complete. 2024-09-17T05:01:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 Processor -- start 2024-09-17T05:01:54.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 -- start start 2024-09-17T05:01:54.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb8003a60 0x7f3bd00aa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd00aae20 0x7f3bd01fc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 0x7f3bd0152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd00739a0 con 0x7f3bd01fcad0 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd0073820 con 0x7f3bb8003a60 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd0073b20 con 0x7f3bd00aae20 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 0x7f3bd0152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 0x7f3bd0152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54220/0 (socket says 172.21.15.16:54220) 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 -- 172.21.15.16:0/3214460067 learned_addr learned my addr 172.21.15.16:0/3214460067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd5800640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb8003a60 0x7f3bd00aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd4e00640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd00aae20 0x7f3bd01fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 -- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd00aae20 msgr2=0x7f3bd01fc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd00aae20 0x7f3bd01fc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 -- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb8003a60 msgr2=0x7f3bd00aa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb8003a60 0x7f3bd00aa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 -- 172.21.15.16:0/3214460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd00ab870 con 0x7f3bd01fcad0 2024-09-17T05:01:54.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd6200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 0x7f3bd0152e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00735a0 tx=0x7f3bc8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800ec30 con 0x7f3bd01fcad0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bc8012450 con 0x7f3bd01fcad0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd00a96a0 con 0x7f3bd01fcad0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc80128a0 con 0x7f3bd01fcad0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.614+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd00a9c60 con 0x7f3bd01fcad0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.618+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd006ce90 con 0x7f3bd01fcad0 2024-09-17T05:01:54.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.618+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bc801a020 con 0x7f3bd01fcad0 2024-09-17T05:01:54.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.618+0000 7f3bcd600640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb0073200 0x7f3bb0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.618+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f3bc8094f40 con 0x7f3bd01fcad0 2024-09-17T05:01:54.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.618+0000 7f3bd5800640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb0073200 0x7f3bb0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.618+0000 7f3bd5800640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb0073200 0x7f3bb0075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006ea50 tx=0x7f3bc4019560 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:54.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.622+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3bc8023020 con 0x7f3bd01fcad0 2024-09-17T05:01:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.798+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bd00ab600 con 0x7f3bd01fcad0 2024-09-17T05:01:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bcd600640 1 -- 172.21.15.16:0/3214460067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18825 (secure 0 0 0) 0x7f3bc805f5e0 con 0x7f3bd01fcad0 2024-09-17T05:01:54.802 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:01:54.802 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:01:49.027397+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:01:06.256743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T05:01:07.232100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T05:01:10.875249+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":5}]},{"pgid":"4.1a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:01:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb0073200 msgr2=0x7f3bb0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb0073200 0x7f3bb0075710 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006ea50 tx=0x7f3bc4019560 comp rx=0 tx=0).stop 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 msgr2=0x7f3bd0152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 0x7f3bd0152e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00735a0 tx=0x7f3bc8010da0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 shutdown_connections 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb0073200 0x7f3bb0075710 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd01fcad0 0x7f3bd0152e50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd00aae20 0x7f3bd01fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 --2- 172.21.15.16:0/3214460067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bb8003a60 0x7f3bd00aa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 >> 172.21.15.16:0/3214460067 conn(0x7f3bd00d0860 msgr2=0x7f3bd00b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 shutdown_connections 2024-09-17T05:01:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.802+0000 7f3bd8200640 1 -- 172.21.15.16:0/3214460067 wait complete. 2024-09-17T05:01:54.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:01:54.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 -- 172.21.15.16:0/1426656195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83480ce4d0 msgr2=0x7f8348155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 --2- 172.21.15.16:0/1426656195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83480ce4d0 0x7f8348155500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8338009900 tx=0x7f833801c700 comp rx=0 tx=0).stop 2024-09-17T05:01:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 -- 172.21.15.16:0/1426656195 shutdown_connections 2024-09-17T05:01:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 --2- 172.21.15.16:0/1426656195 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8348155a40 0x7f83480af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 --2- 172.21.15.16:0/1426656195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83480ce4d0 0x7f8348155500 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 -- 172.21.15.16:0/1426656195 >> 172.21.15.16:0/1426656195 conn(0x7f83480cd980 msgr2=0x7f83480cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 -- 172.21.15.16:0/1426656195 shutdown_connections 2024-09-17T05:01:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.982+0000 7f834d600640 1 -- 172.21.15.16:0/1426656195 wait complete. 2024-09-17T05:01:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 Processor -- start 2024-09-17T05:01:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 -- start start 2024-09-17T05:01:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83480afa00 0x7f83480aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83480ce4d0 0x7f834815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 0x7f8348153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8348071ce0 con 0x7f8348155a40 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83480b0080 con 0x7f83480afa00 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f834d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8348071e60 con 0x7f83480ce4d0 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 0x7f8348153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 0x7f8348153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54240/0 (socket says 172.21.15.16:54240) 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 -- 172.21.15.16:0/1230200208 learned_addr learned my addr 172.21.15.16:0/1230200208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8346000640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83480ce4d0 0x7f834815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8346a00640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83480afa00 0x7f83480aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 -- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83480ce4d0 msgr2=0x7f834815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83480ce4d0 0x7f834815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 -- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83480afa00 msgr2=0x7f83480aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83480afa00 0x7f83480aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 -- 172.21.15.16:0/1230200208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8348158b70 con 0x7f8348155a40 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.986+0000 7f8347400640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 0x7f8348153ec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f834000c730 tx=0x7f834000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340016020 con 0x7f8348155a40 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f834000ab40 con 0x7f8348155a40 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8348158e50 con 0x7f8348155a40 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834000ce40 con 0x7f8348155a40 2024-09-17T05:01:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481fa0d0 con 0x7f8348155a40 2024-09-17T05:01:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348155500 con 0x7f8348155a40 2024-09-17T05:01:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8340019020 con 0x7f8348155a40 2024-09-17T05:01:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f833ea00640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8328073200 0x7f8328075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6677+0+0 (secure 0 0 0) 0x7f8340094030 con 0x7f8348155a40 2024-09-17T05:01:54.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.990+0000 7f8346a00640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8328073200 0x7f8328075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:54.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.994+0000 7f8346a00640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8328073200 0x7f8328075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83380af000 tx=0x7f833801af40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:54.994+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8340020020 con 0x7f8348155a40 2024-09-17T05:01:55.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.206+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f834806a180 con 0x7f8348155a40 2024-09-17T05:01:55.209 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:01:55.206+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:01:55.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.210+0000 7f833ea00640 1 -- 172.21.15.16:0/1230200208 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f834005ec70 con 0x7f8348155a40 2024-09-17T05:01:55.212 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:01:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8328073200 msgr2=0x7f8328075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8328073200 0x7f8328075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83380af000 tx=0x7f833801af40 comp rx=0 tx=0).stop 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 msgr2=0x7f8348153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 0x7f8348153ec0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f834000c730 tx=0x7f834000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 shutdown_connections 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8328073200 0x7f8328075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8348155a40 0x7f8348153ec0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83480ce4d0 0x7f834815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 --2- 172.21.15.16:0/1230200208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83480afa00 0x7f83480aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 >> 172.21.15.16:0/1230200208 conn(0x7f83480cd980 msgr2=0x7f834815f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 shutdown_connections 2024-09-17T05:01:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.214+0000 7f834d600640 1 -- 172.21.15.16:0/1230200208 wait complete. 2024-09-17T05:01:55.227 INFO:tasks.ceph.mds.d.smithi104.stderr: -4614> 2024-09-17T05:01:13.816+0000 7f3c6ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:01:55.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:01:55.348 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:01:55.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 -- 172.21.15.16:0/778998901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 msgr2=0x7f524c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 --2- 172.21.15.16:0/778998901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0ce8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f524c071ce0 tx=0x7f524401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 -- 172.21.15.16:0/778998901 shutdown_connections 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 --2- 172.21.15.16:0/778998901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0ce8a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 --2- 172.21.15.16:0/778998901 >> v2:172.21.15.104:3300/0 conn(0x7f524c06e1d0 0x7f524c0d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 -- 172.21.15.16:0/778998901 >> 172.21.15.16:0/778998901 conn(0x7f524c0b2250 msgr2=0x7f524c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 -- 172.21.15.16:0/778998901 shutdown_connections 2024-09-17T05:01:55.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.394+0000 7f5251600640 1 -- 172.21.15.16:0/778998901 wait complete. 2024-09-17T05:01:55.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 Processor -- start 2024-09-17T05:01:55.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 -- start start 2024-09-17T05:01:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f524c06e1d0 0x7f524c1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:55.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c15e780 0x7f524c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f524c168cb0 con 0x7f524c0ce490 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f524c168b30 con 0x7f524c15e780 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5251600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f524c168e30 con 0x7f524c06e1d0 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54268/0 (socket says 172.21.15.16:54268) 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 -- 172.21.15.16:0/2150076269 learned_addr learned my addr 172.21.15.16:0/2150076269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:55.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f524aa00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f524c06e1d0 0x7f524c1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f524b400640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c15e780 0x7f524c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 -- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f524c06e1d0 msgr2=0x7f524c1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f524c06e1d0 0x7f524c1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 -- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c15e780 msgr2=0x7f524c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c15e780 0x7f524c158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 -- 172.21.15.16:0/2150076269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c158660 con 0x7f524c0ce490 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.398+0000 7f5243e00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0b5120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f523400c3f0 tx=0x7f523400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234017070 con 0x7f524c0ce490 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52340053c0 con 0x7f524c0ce490 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c158940 con 0x7f524c0ce490 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234003860 con 0x7f524c0ce490 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c0c6270 con 0x7f524c0ce490 2024-09-17T05:01:55.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c158b60 con 0x7f524c0ce490 2024-09-17T05:01:55.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f523401a070 con 0x7f524c0ce490 2024-09-17T05:01:55.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5242a00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5228073200 0x7f5228075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6763+0+0 (secure 0 0 0) 0x7f5234093980 con 0x7f524c0ce490 2024-09-17T05:01:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.402+0000 7f524aa00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5228073200 0x7f5228075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:55.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.406+0000 7f524aa00640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5228073200 0x7f5228075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f523c009b40 tx=0x7f523c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:55.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.410+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5234001090 con 0x7f524c0ce490 2024-09-17T05:01:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:55.618+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f524c0d0770 con 0x7f524c0ce490 2024-09-17T05:01:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.214+0000 7f5242a00640 1 -- 172.21.15.16:0/2150076269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f523405de90 con 0x7f524c0ce490 2024-09-17T05:01:56.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5228073200 msgr2=0x7f5228075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:56.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5228073200 0x7f5228075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f523c009b40 tx=0x7f523c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 msgr2=0x7f524c0b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0b5120 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f523400c3f0 tx=0x7f523400c8b0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 shutdown_connections 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5228073200 0x7f5228075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c15e780 0x7f524c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c0ce490 0x7f524c0b5120 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 --2- 172.21.15.16:0/2150076269 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f524c06e1d0 0x7f524c1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 >> 172.21.15.16:0/2150076269 conn(0x7f524c0b2250 msgr2=0x7f524c06d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 shutdown_connections 2024-09-17T05:01:56.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.218+0000 7f5251600640 1 -- 172.21.15.16:0/2150076269 wait complete. 2024-09-17T05:01:56.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 -- 172.21.15.16:0/1836027200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f8160510 msgr2=0x7f11f815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 --2- 172.21.15.16:0/1836027200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f8160510 0x7f11f815daf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009c80 tx=0x7f11f001cbc0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 -- 172.21.15.16:0/1836027200 shutdown_connections 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 --2- 172.21.15.16:0/1836027200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f8160510 0x7f11f815daf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 --2- 172.21.15.16:0/1836027200 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f80d1ae0 0x7f11f815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 -- 172.21.15.16:0/1836027200 >> 172.21.15.16:0/1836027200 conn(0x7f11f80d2240 msgr2=0x7f11f80d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 -- 172.21.15.16:0/1836027200 shutdown_connections 2024-09-17T05:01:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 -- 172.21.15.16:0/1836027200 wait complete. 2024-09-17T05:01:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 Processor -- start 2024-09-17T05:01:56.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 -- start start 2024-09-17T05:01:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 0x7f11f815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f8153490 0x7f11f8146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.410+0000 7f11ff000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f8160510 0x7f11f81500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:56.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11ff000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8169d00 con 0x7f11f80d1ae0 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11ff000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8169b80 con 0x7f11f8160510 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11ff000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f11f8169e80 con 0x7f11f8153490 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 0x7f11f815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 0x7f11f815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54278/0 (socket says 172.21.15.16:54278) 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 -- 172.21.15.16:0/1461931454 learned_addr learned my addr 172.21.15.16:0/1461931454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11fd000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f8160510 0x7f11f81500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7400640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f8153490 0x7f11f8146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 -- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f8153490 msgr2=0x7f11f8146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f8153490 0x7f11f8146970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 -- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f8160510 msgr2=0x7f11f81500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f8160510 0x7f11f81500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:56.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 -- 172.21.15.16:0/1461931454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f8147c30 con 0x7f11f80d1ae0 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7400640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f8153490 0x7f11f8146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11f7e00640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 0x7f11f815be10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b440 tx=0x7f11e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8015020 con 0x7f11f80d1ae0 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e8011980 con 0x7f11f80d1ae0 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8011dd0 con 0x7f11f80d1ae0 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f8147f10 con 0x7f11f80d1ae0 2024-09-17T05:01:56.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f814b9d0 con 0x7f11f80d1ae0 2024-09-17T05:01:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f814bc30 con 0x7f11f80d1ae0 2024-09-17T05:01:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.414+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f11e8018020 con 0x7f11f80d1ae0 2024-09-17T05:01:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.418+0000 7f11efe00640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11d8073200 0x7f11d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.418+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6763+0+0 (secure 0 0 0) 0x7f11e8094440 con 0x7f11f80d1ae0 2024-09-17T05:01:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.418+0000 7f11f7400640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11d8073200 0x7f11d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.418+0000 7f11f7400640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11d8073200 0x7f11d8075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f11e4005fd0 tx=0x7f11e4005f20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.422+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11e801f030 con 0x7f11f80d1ae0 2024-09-17T05:01:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:56.598+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11f815ffd0 con 0x7f11f80d1ae0 2024-09-17T05:01:57.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.218+0000 7f11efe00640 1 -- 172.21.15.16:0/1461931454 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f11e800d040 con 0x7f11f80d1ae0 2024-09-17T05:01:57.219 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:01:57.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11d8073200 msgr2=0x7f11d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:57.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11d8073200 0x7f11d8075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f11e4005fd0 tx=0x7f11e4005f20 comp rx=0 tx=0).stop 2024-09-17T05:01:57.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 msgr2=0x7f11f815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:57.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 0x7f11f815be10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f11e800b440 tx=0x7f11e800b900 comp rx=0 tx=0).stop 2024-09-17T05:01:57.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 shutdown_connections 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11d8073200 0x7f11d8075710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11f8160510 0x7f11f81500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11f8153490 0x7f11f8146970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 --2- 172.21.15.16:0/1461931454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11f80d1ae0 0x7f11f815be10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 >> 172.21.15.16:0/1461931454 conn(0x7f11f80d2240 msgr2=0x7f11f81499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 shutdown_connections 2024-09-17T05:01:57.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.222+0000 7f11ff000640 1 -- 172.21.15.16:0/1461931454 wait complete. 2024-09-17T05:01:57.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 -- 172.21.15.16:0/2983197017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c06e4e0 msgr2=0x7f4f3c15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 --2- 172.21.15.16:0/2983197017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c06e4e0 0x7f4f3c15efa0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0b2c20 tx=0x7f4f2401c890 comp rx=0 tx=0).stop 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 -- 172.21.15.16:0/2983197017 shutdown_connections 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 --2- 172.21.15.16:0/2983197017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c15f4e0 0x7f4f3c15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 --2- 172.21.15.16:0/2983197017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c06e4e0 0x7f4f3c15efa0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 --2- 172.21.15.16:0/2983197017 >> v2:172.21.15.104:3300/0 conn(0x7f4f3c0b4620 0x7f4f3c06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 -- 172.21.15.16:0/2983197017 >> 172.21.15.16:0/2983197017 conn(0x7f4f3c0d0050 msgr2=0x7f4f3c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 -- 172.21.15.16:0/2983197017 shutdown_connections 2024-09-17T05:01:57.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.406+0000 7f4f41400640 1 -- 172.21.15.16:0/2983197017 wait complete. 2024-09-17T05:01:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 Processor -- start 2024-09-17T05:01:57.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 -- start start 2024-09-17T05:01:57.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f3c06e4e0 0x7f4f3c148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:57.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c0b4620 0x7f4f3c148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:57.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 0x7f4f3c146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:57.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c06d6a0 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c06d820 con 0x7f4f3c06e4e0 2024-09-17T05:01:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c06d9a0 con 0x7f4f3c0b4620 2024-09-17T05:01:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 0x7f4f3c146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 0x7f4f3c146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33118/0 (socket says 172.21.15.16:33118) 2024-09-17T05:01:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 -- 172.21.15.16:0/1386187377 learned_addr learned my addr 172.21.15.16:0/1386187377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3a800640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f3c06e4e0 0x7f4f3c148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:57.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f33e00640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c0b4620 0x7f4f3c148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 -- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c0b4620 msgr2=0x7f4f3c148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c0b4620 0x7f4f3c148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 -- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f3c06e4e0 msgr2=0x7f4f3c148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f3c06e4e0 0x7f4f3c148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 -- 172.21.15.16:0/1386187377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c14b030 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f33e00640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c0b4620 0x7f4f3c148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f3b200640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 0x7f4f3c146270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400ea80 tx=0x7f4f3400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34004b30 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f34004cf0 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c14b310 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34005280 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.410+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c14cdb0 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.414+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c06dfa0 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.414+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4f34005420 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.414+0000 7f4f32a00640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f18073200 0x7f4f18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.414+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6325+0+0 (secure 0 0 0) 0x7f4f340950f0 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.414+0000 7f4f3a800640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f18073200 0x7f4f18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:57.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.418+0000 7f4f3a800640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f18073200 0x7f4f18075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0058d0 tx=0x7f4f2c005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:57.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.418+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4f3405fe90 con 0x7f4f3c15f4e0 2024-09-17T05:01:57.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:57.594+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4f3c069770 con 0x7f4f3c15f4e0 2024-09-17T05:01:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.222+0000 7f4f32a00640 1 -- 172.21.15.16:0/1386187377 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f4f3405f790 con 0x7f4f3c15f4e0 2024-09-17T05:01:58.225 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:01:58.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.226+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f18073200 msgr2=0x7f4f18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:58.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f18073200 0x7f4f18075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0058d0 tx=0x7f4f2c005860 comp rx=0 tx=0).stop 2024-09-17T05:01:58.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 msgr2=0x7f4f3c146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:58.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 0x7f4f3c146270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400ea80 tx=0x7f4f3400ef40 comp rx=0 tx=0).stop 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 shutdown_connections 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f18073200 0x7f4f18075710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f3c15f4e0 0x7f4f3c146270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f3c0b4620 0x7f4f3c148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 --2- 172.21.15.16:0/1386187377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f3c06e4e0 0x7f4f3c148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 >> 172.21.15.16:0/1386187377 conn(0x7f4f3c0d0050 msgr2=0x7f4f3c14ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 shutdown_connections 2024-09-17T05:01:58.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.230+0000 7f4f41400640 1 -- 172.21.15.16:0/1386187377 wait complete. 2024-09-17T05:01:58.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 -- 172.21.15.16:0/3339851176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c8003a60 msgr2=0x7fc4c8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 --2- 172.21.15.16:0/3339851176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c8003a60 0x7fc4c8003ed0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4006fd0 tx=0x7fc4d401ceb0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 -- 172.21.15.16:0/3339851176 shutdown_connections 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 --2- 172.21.15.16:0/3339851176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4c8003a60 0x7fc4c8003ed0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 -- 172.21.15.16:0/3339851176 >> 172.21.15.16:0/3339851176 conn(0x7fc4e40d0050 msgr2=0x7fc4e40d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 -- 172.21.15.16:0/3339851176 shutdown_connections 2024-09-17T05:01:58.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.422+0000 7fc4eb200640 1 -- 172.21.15.16:0/3339851176 wait complete. 2024-09-17T05:01:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 Processor -- start 2024-09-17T05:01:58.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 -- start start 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4c8003a60 0x7fc4e41f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 0x7fc4e4151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e40cb4c0 0x7fc4e40cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e406dfa0 con 0x7fc4e4151180 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e406e120 con 0x7fc4e40cb4c0 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4eb200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e406e2a0 con 0x7fc4c8003a60 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 0x7fc4e4151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 0x7fc4e4151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33124/0 (socket says 172.21.15.16:33124) 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4dbe00640 1 -- 172.21.15.16:0/1413311956 learned_addr learned my addr 172.21.15.16:0/1413311956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4e3e00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4c8003a60 0x7fc4e41f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.426+0000 7fc4e9200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e40cb4c0 0x7fc4e40cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:58.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4dbe00640 1 -- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4c8003a60 msgr2=0x7fc4e41f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4dbe00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4c8003a60 0x7fc4e41f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4dbe00640 1 -- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e40cb4c0 msgr2=0x7fc4e40cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4dbe00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e40cb4c0 0x7fc4e40cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4dbe00640 1 -- 172.21.15.16:0/1413311956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e40c94d0 con 0x7fc4e4151180 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e3e00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4c8003a60 0x7fc4e41f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e9200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e40cb4c0 0x7fc4e40cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4dbe00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 0x7fc4e4151590 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc003eb0 tx=0x7fc4cc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc015070 con 0x7fc4e4151180 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4cc011840 con 0x7fc4e4151180 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc011c50 con 0x7fc4e4151180 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e40c97b0 con 0x7fc4e4151180 2024-09-17T05:01:58.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41fbaa0 con 0x7fc4e4151180 2024-09-17T05:01:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e406e7a0 con 0x7fc4e4151180 2024-09-17T05:01:58.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc4cc018070 con 0x7fc4e4151180 2024-09-17T05:01:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.430+0000 7fc4e0c00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4c0073200 0x7fc4c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.434+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5882+0+0 (secure 0 0 0) 0x7fc4cc094f20 con 0x7fc4e4151180 2024-09-17T05:01:58.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.434+0000 7fc4e3e00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4c0073200 0x7fc4c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:58.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.434+0000 7fc4e3e00640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4c0073200 0x7fc4c0075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ab000 tx=0x7fc4d4019cf0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:58.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.438+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc4cc05fe80 con 0x7fc4e4151180 2024-09-17T05:01:58.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:58.610+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc4e4069500 con 0x7fc4e4151180 2024-09-17T05:01:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.226+0000 7fc4e0c00640 1 -- 172.21.15.16:0/1413311956 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7fc4cc05f780 con 0x7fc4e4151180 2024-09-17T05:01:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-09-17T05:01:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4c0073200 msgr2=0x7fc4c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4c0073200 0x7fc4c0075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40ab000 tx=0x7fc4d4019cf0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 msgr2=0x7fc4e4151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 0x7fc4e4151590 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc003eb0 tx=0x7fc4cc001640 comp rx=0 tx=0).stop 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 shutdown_connections 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc4c0073200 0x7fc4c0075710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e40cb4c0 0x7fc4e40cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4e4151180 0x7fc4e4151590 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 --2- 172.21.15.16:0/1413311956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc4c8003a60 0x7fc4e41f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 >> 172.21.15.16:0/1413311956 conn(0x7fc4e40d0050 msgr2=0x7fc4e40ad630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 shutdown_connections 2024-09-17T05:01:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.230+0000 7fc4eb200640 1 -- 172.21.15.16:0/1413311956 wait complete. 2024-09-17T05:01:59.257 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:01:59.257 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:01:59.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:01:59.257 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:01:59.257 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:01:59.257 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:01:59.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:01:59.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:01:59.258 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:01:59.258 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:01:59.259 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:01:59.259 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:01:59.259 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:01:59.259 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:01:59.259 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:01:59.259 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:01:59.273 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:01:59.273 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:01:59.274 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:01:59.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:01:59.275 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:01:59.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:01:59.280 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:01:59.280 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:01:59.321 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:01:59.321 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:59.322 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:01:59.322 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:59.323 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:01:59.323 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:59.325 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:01:59.325 DEBUG:teuthology.parallel:result is None 2024-09-17T05:01:59.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:01:59.432 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:01:59.434 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:01:59.435 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:01:59.435 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:01:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 -- 172.21.15.16:0/4211808003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 msgr2=0x7fe9fc0741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 --2- 172.21.15.16:0/4211808003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc0741d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009840 tx=0x7fe9e401c820 comp rx=0 tx=0).stop 2024-09-17T05:01:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 -- 172.21.15.16:0/4211808003 shutdown_connections 2024-09-17T05:01:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 --2- 172.21.15.16:0/4211808003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc0741d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 --2- 172.21.15.16:0/4211808003 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe9fc155400 0x7fe9fc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 -- 172.21.15.16:0/4211808003 >> 172.21.15.16:0/4211808003 conn(0x7fe9fc0b2dd0 msgr2=0x7fe9fc0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:01:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 -- 172.21.15.16:0/4211808003 shutdown_connections 2024-09-17T05:01:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.498+0000 7fea03000640 1 -- 172.21.15.16:0/4211808003 wait complete. 2024-09-17T05:01:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 Processor -- start 2024-09-17T05:01:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 -- start start 2024-09-17T05:01:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc14a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe9fc0cfcd0 0x7fe9fc154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc155400 0x7fe9fc14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc0cc810 con 0x7fe9fc073dc0 2024-09-17T05:01:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc0cc690 con 0x7fe9fc155400 2024-09-17T05:01:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea03000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0cc990 con 0x7fe9fc0cfcd0 2024-09-17T05:01:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc14a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc14a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33162/0 (socket says 172.21.15.16:33162) 2024-09-17T05:01:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fe9fbe00640 1 -- 172.21.15.16:0/87508504 learned_addr learned my addr 172.21.15.16:0/87508504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:01:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fe9f3400640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe9fc0cfcd0 0x7fe9fc154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.502+0000 7fea01000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc155400 0x7fe9fc14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9fbe00640 1 -- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe9fc0cfcd0 msgr2=0x7fe9fc154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9fbe00640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe9fc0cfcd0 0x7fe9fc154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9fbe00640 1 -- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc155400 msgr2=0x7fe9fc14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:01:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9fbe00640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc155400 0x7fe9fc14d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:01:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9fbe00640 1 -- 172.21.15.16:0/87508504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc150af0 con 0x7fe9fc073dc0 2024-09-17T05:01:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9fbe00640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc14a750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00d750 tx=0x7fe9ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec017070 con 0x7fe9fc073dc0 2024-09-17T05:01:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9ec009cc0 con 0x7fe9fc073dc0 2024-09-17T05:01:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc150dd0 con 0x7fe9fc073dc0 2024-09-17T05:01:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec005280 con 0x7fe9fc073dc0 2024-09-17T05:01:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc151390 con 0x7fe9fc073dc0 2024-09-17T05:01:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc0cce40 con 0x7fe9fc073dc0 2024-09-17T05:01:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.506+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe9ec005420 con 0x7fe9fc073dc0 2024-09-17T05:01:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.510+0000 7fe9f8c00640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe9d8073230 0x7fe9d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:01:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.510+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7fe9ec0944a0 con 0x7fe9fc073dc0 2024-09-17T05:01:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.510+0000 7fe9f3400640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe9d8073230 0x7fe9d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:01:59.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.510+0000 7fe9f3400640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe9d8073230 0x7fe9d8075740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40af000 tx=0x7fe9e401b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:01:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.510+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe9ec00de60 con 0x7fe9fc073dc0 2024-09-17T05:01:59.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:01:59.694+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe9fc0cd500 con 0x7fe9fc073dc0 2024-09-17T05:01:59.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:01:59.838 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:02:00.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fe9f8c00640 1 -- 172.21.15.16:0/87508504 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7fe9ec05f050 con 0x7fe9fc073dc0 2024-09-17T05:02:00.247 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe9d8073230 msgr2=0x7fe9d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe9d8073230 0x7fe9d8075740 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40af000 tx=0x7fe9e401b060 comp rx=0 tx=0).stop 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 msgr2=0x7fe9fc14a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc14a750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00d750 tx=0x7fe9ec00dc10 comp rx=0 tx=0).stop 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 shutdown_connections 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe9d8073230 0x7fe9d8075740 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc155400 0x7fe9fc14d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe9fc0cfcd0 0x7fe9fc154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 --2- 172.21.15.16:0/87508504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9fc073dc0 0x7fe9fc14a750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 >> 172.21.15.16:0/87508504 conn(0x7fe9fc0b2dd0 msgr2=0x7fe9fc152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:00.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 shutdown_connections 2024-09-17T05:02:00.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.246+0000 7fea03000640 1 -- 172.21.15.16:0/87508504 wait complete. 2024-09-17T05:02:00.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:02:00.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 -- 172.21.15.16:0/390545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc0b2f50 msgr2=0x7fcddc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 --2- 172.21.15.16:0/390545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc0b2f50 0x7fcddc15bb60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcddc15cc70 tx=0x7fcdd001c770 comp rx=0 tx=0).stop 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 -- 172.21.15.16:0/390545762 shutdown_connections 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 --2- 172.21.15.16:0/390545762 >> v2:172.21.15.104:3300/0 conn(0x7fcddc073240 0x7fcddc073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 --2- 172.21.15.16:0/390545762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc0b2f50 0x7fcddc15bb60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 -- 172.21.15.16:0/390545762 >> 172.21.15.16:0/390545762 conn(0x7fcddc06f900 msgr2=0x7fcddc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 -- 172.21.15.16:0/390545762 shutdown_connections 2024-09-17T05:02:00.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 -- 172.21.15.16:0/390545762 wait complete. 2024-09-17T05:02:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 Processor -- start 2024-09-17T05:02:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 -- start start 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.430+0000 7fcde4800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 0x7fcddc160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc0b2f50 0x7fcddc1428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc15c0a0 0x7fcddc160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc168c40 con 0x7fcddc073240 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc168ac0 con 0x7fcddc15c0a0 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcddc168dc0 con 0x7fcddc0b2f50 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 0x7fcddc160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 0x7fcddc160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33174/0 (socket says 172.21.15.16:33174) 2024-09-17T05:02:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 -- 172.21.15.16:0/1132726473 learned_addr learned my addr 172.21.15.16:0/1132726473 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde2800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc15c0a0 0x7fcddc160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1400640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc0b2f50 0x7fcddc1428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 -- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc0b2f50 msgr2=0x7fcddc1428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc0b2f50 0x7fcddc1428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 -- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc15c0a0 msgr2=0x7fcddc160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc15c0a0 0x7fcddc160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 -- 172.21.15.16:0/1132726473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc1607c0 con 0x7fcddc073240 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1400640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc0b2f50 0x7fcddc1428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde1e00640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 0x7fcddc160280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0098370 tx=0x7fcdd0005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00b2040 con 0x7fcddc073240 2024-09-17T05:02:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd00bb070 con 0x7fcddc073240 2024-09-17T05:02:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc142e20 con 0x7fcddc073240 2024-09-17T05:02:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0003600 con 0x7fcddc073240 2024-09-17T05:02:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc15ac80 con 0x7fcddc073240 2024-09-17T05:02:00.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc073650 con 0x7fcddc073240 2024-09-17T05:02:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.434+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcdd00037a0 con 0x7fcddc073240 2024-09-17T05:02:00.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.438+0000 7fcdd6000640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcdc0073230 0x7fcdc0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.438+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fcdd012ed20 con 0x7fcddc073240 2024-09-17T05:02:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.438+0000 7fcde1400640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcdc0073230 0x7fcdc0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.438+0000 7fcde1400640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcdc0073230 0x7fcdc0075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcddc151070 tx=0x7fcdcc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.438+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcdd00163f0 con 0x7fcddc073240 2024-09-17T05:02:00.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.670+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcddc06de30 con 0x7fcddc073240 2024-09-17T05:02:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcdd6000640 1 -- 172.21.15.16:0/1132726473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+6645 (secure 0 0 0) 0x7fcdd00fa120 con 0x7fcddc073240 2024-09-17T05:02:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:00.674 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:02:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:02:00.675 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:02:00.676 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.677 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.679 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:02:00.681 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:02:00.683 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.684 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:02:00.685 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.686 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD6DOlmeJGtCRAApI1z85wRjv7OdkLRT01Isw==", 2024-09-17T05:02:00.687 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD6DOlmWJ2tCRAALtrIUX6VAOmVAMYlubkjoQ==", 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2024-09-17T05:02:00.688 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD6DOlmcqatCRAA6W8dU/NNwC8GnSCmChxJWg==", 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD6DOlm9rGtCRAAlK7b7loKTIxnGJNC8Kp7gQ==", 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.689 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD6DOlm9LqtCRAAnBGmzH3CVEhiBdBSolypcw==", 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQD6DOlm38OtCRAA4Y9s2vhLViXs7rCoIHuQrg==", 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.690 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:02:00.691 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:02:00.692 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:02:00.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:00.692 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:02:00.692 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcdc0073230 msgr2=0x7fcdc0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcdc0073230 0x7fcdc0075740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcddc151070 tx=0x7fcdcc009040 comp rx=0 tx=0).stop 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 msgr2=0x7fcddc160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 0x7fcddc160280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0098370 tx=0x7fcdd0005ca0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 shutdown_connections 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcdc0073230 0x7fcdc0075740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcddc15c0a0 0x7fcddc160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcddc0b2f50 0x7fcddc1428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 --2- 172.21.15.16:0/1132726473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcddc073240 0x7fcddc160280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 >> 172.21.15.16:0/1132726473 conn(0x7fcddc06f900 msgr2=0x7fcddc155630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:00.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 shutdown_connections 2024-09-17T05:02:00.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.674+0000 7fcde4800640 1 -- 172.21.15.16:0/1132726473 wait complete. 2024-09-17T05:02:00.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T05:02:00.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 -- 172.21.15.16:0/3097478136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00cf850 msgr2=0x7f9ae00b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3097478136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00cf850 0x7f9ae00b2100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009bf0 tx=0x7f9ad801ca50 comp rx=0 tx=0).stop 2024-09-17T05:02:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 -- 172.21.15.16:0/3097478136 shutdown_connections 2024-09-17T05:02:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3097478136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00cf850 0x7f9ae00b2100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 -- 172.21.15.16:0/3097478136 >> 172.21.15.16:0/3097478136 conn(0x7f9ae00b42d0 msgr2=0x7f9ae00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 -- 172.21.15.16:0/3097478136 shutdown_connections 2024-09-17T05:02:00.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 -- 172.21.15.16:0/3097478136 wait complete. 2024-09-17T05:02:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 Processor -- start 2024-09-17T05:02:00.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.854+0000 7f9ae8400640 1 -- start start 2024-09-17T05:02:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 0x7f9ae014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ae0159f70 0x7f9ae00b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ae00b6c80 0x7f9ae006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae016af70 con 0x7f9ae00d02d0 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae016adf0 con 0x7f9ae0159f70 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae016b0f0 con 0x7f9ae00b6c80 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 0x7f9ae014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 0x7f9ae014ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33188/0 (socket says 172.21.15.16:33188) 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 -- 172.21.15.16:0/3029496019 learned_addr learned my addr 172.21.15.16:0/3029496019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae6400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ae00b6c80 0x7f9ae006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5000640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ae0159f70 0x7f9ae00b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 -- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ae00b6c80 msgr2=0x7f9ae006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ae00b6c80 0x7f9ae006f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 -- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ae0159f70 msgr2=0x7f9ae00b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ae0159f70 0x7f9ae00b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 -- 172.21.15.16:0/3029496019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae00b5370 con 0x7f9ae00d02d0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5000640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ae0159f70 0x7f9ae00b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae5a00640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 0x7f9ae014ef40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c430 tx=0x7f9ad400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4011790 con 0x7f9ae00d02d0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad4017070 con 0x7f9ae00d02d0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4011cc0 con 0x7f9ae00d02d0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae00b5610 con 0x7f9ae00d02d0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae01fcc30 con 0x7f9ae00d02d0 2024-09-17T05:02:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.858+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae00b2100 con 0x7f9ae00d02d0 2024-09-17T05:02:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.862+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ad401c460 con 0x7f9ae00d02d0 2024-09-17T05:02:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.862+0000 7f9add600640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ac0073230 0x7f9ac0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.862+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f9ad4093e90 con 0x7f9ae00d02d0 2024-09-17T05:02:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.862+0000 7f9ae5000640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ac0073230 0x7f9ac0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:00.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.862+0000 7f9ae5000640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ac0073230 0x7f9ac0075740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0151c40 tx=0x7f9acc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:00.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:00.866+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ad405f060 con 0x7f9ae00d02d0 2024-09-17T05:02:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.106+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f9ae00d1e40 con 0x7f9ae00d02d0 2024-09-17T05:02:01.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9add600640 1 -- 172.21.15.16:0/3029496019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v25) ==== 90+0+0 (secure 0 0 0) 0x7f9ad405e960 con 0x7f9ae00d02d0 2024-09-17T05:02:01.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ac0073230 msgr2=0x7f9ac0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ac0073230 0x7f9ac0075740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0151c40 tx=0x7f9acc009040 comp rx=0 tx=0).stop 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 msgr2=0x7f9ae014ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 0x7f9ae014ef40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400c430 tx=0x7f9ad400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 shutdown_connections 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ac0073230 0x7f9ac0075740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9ae00b6c80 0x7f9ae006f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ae0159f70 0x7f9ae00b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 --2- 172.21.15.16:0/3029496019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae00d02d0 0x7f9ae014ef40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 >> 172.21.15.16:0/3029496019 conn(0x7f9ae00b42d0 msgr2=0x7f9ae00d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 shutdown_connections 2024-09-17T05:02:01.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.110+0000 7f9ae8400640 1 -- 172.21.15.16:0/3029496019 wait complete. 2024-09-17T05:02:01.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T05:02:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 -- 172.21.15.16:0/3597068910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 msgr2=0x7fd35c15bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 --2- 172.21.15.16:0/3597068910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c15bc60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd354009c80 tx=0x7fd35401ccd0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 -- 172.21.15.16:0/3597068910 shutdown_connections 2024-09-17T05:02:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 --2- 172.21.15.16:0/3597068910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c15bc60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 --2- 172.21.15.16:0/3597068910 >> v2:172.21.15.104:3300/0 conn(0x7fd35c073a20 0x7fd35c0cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 -- 172.21.15.16:0/3597068910 >> 172.21.15.16:0/3597068910 conn(0x7fd35c0b2250 msgr2=0x7fd35c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 -- 172.21.15.16:0/3597068910 shutdown_connections 2024-09-17T05:02:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.294+0000 7fd361200640 1 -- 172.21.15.16:0/3597068910 wait complete. 2024-09-17T05:02:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 Processor -- start 2024-09-17T05:02:01.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 -- start start 2024-09-17T05:02:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd35c073610 0x7fd35c1fa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd35c073a20 0x7fd35c151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c0afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c167c30 con 0x7fd35c15b850 2024-09-17T05:02:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c167ab0 con 0x7fd35c073610 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c167db0 con 0x7fd35c073a20 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c0afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c0afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33216/0 (socket says 172.21.15.16:33216) 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 -- 172.21.15.16:0/830278252 learned_addr learned my addr 172.21.15.16:0/830278252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd353e00640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd35c073a20 0x7fd35c151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd359e00640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd35c073610 0x7fd35c1fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 -- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd35c073a20 msgr2=0x7fd35c151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd35c073a20 0x7fd35c151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 -- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd35c073610 msgr2=0x7fd35c1fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd35c073610 0x7fd35c1fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 -- 172.21.15.16:0/830278252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1faa80 con 0x7fd35c15b850 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd35a800640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c0afd50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd354009bd0 tx=0x7fd3540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3540bb070 con 0x7fd35c15b850 2024-09-17T05:02:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd354003e70 con 0x7fd35c15b850 2024-09-17T05:02:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3540b2040 con 0x7fd35c15b850 2024-09-17T05:02:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c150790 con 0x7fd35c15b850 2024-09-17T05:02:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.298+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c150d50 con 0x7fd35c15b850 2024-09-17T05:02:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.302+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c15bc60 con 0x7fd35c15b850 2024-09-17T05:02:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.302+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd354004a50 con 0x7fd35c15b850 2024-09-17T05:02:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.302+0000 7fd352000640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd338073200 0x7fd338075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.302+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fd35412e6a0 con 0x7fd35c15b850 2024-09-17T05:02:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.302+0000 7fd359e00640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd338073200 0x7fd338075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.302+0000 7fd359e00640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd338073200 0x7fd338075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd34c001650 tx=0x7fd34c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.306+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd3540c4020 con 0x7fd35c15b850 2024-09-17T05:02:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.538+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fd35c069770 con 0x7fd35c15b850 2024-09-17T05:02:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd352000640 1 -- 172.21.15.16:0/830278252 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v26) ==== 90+0+0 (secure 0 0 0) 0x7fd354016950 con 0x7fd35c15b850 2024-09-17T05:02:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd338073200 msgr2=0x7fd338075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd338073200 0x7fd338075710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd34c001650 tx=0x7fd34c009040 comp rx=0 tx=0).stop 2024-09-17T05:02:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 msgr2=0x7fd35c0afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c0afd50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd354009bd0 tx=0x7fd3540045e0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 shutdown_connections 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd338073200 0x7fd338075710 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd35c15b850 0x7fd35c0afd50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd35c073a20 0x7fd35c151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 --2- 172.21.15.16:0/830278252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd35c073610 0x7fd35c1fa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 >> 172.21.15.16:0/830278252 conn(0x7fd35c0b2250 msgr2=0x7fd35c074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 shutdown_connections 2024-09-17T05:02:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.542+0000 7fd361200640 1 -- 172.21.15.16:0/830278252 wait complete. 2024-09-17T05:02:01.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 -- 172.21.15.16:0/2089808024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 msgr2=0x7f5dec15b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 --2- 172.21.15.16:0/2089808024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec15b680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5dec071ce0 tx=0x7f5dd401c770 comp rx=0 tx=0).stop 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 -- 172.21.15.16:0/2089808024 shutdown_connections 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 --2- 172.21.15.16:0/2089808024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec15b680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 --2- 172.21.15.16:0/2089808024 >> v2:172.21.15.104:3300/0 conn(0x7f5dec073610 0x7f5dec15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 -- 172.21.15.16:0/2089808024 >> 172.21.15.16:0/2089808024 conn(0x7f5dec0b2250 msgr2=0x7f5dec0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 -- 172.21.15.16:0/2089808024 shutdown_connections 2024-09-17T05:02:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.722+0000 7f5df0e00640 1 -- 172.21.15.16:0/2089808024 wait complete. 2024-09-17T05:02:01.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 Processor -- start 2024-09-17T05:02:01.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 -- start start 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5dec073610 0x7f5dec0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec1549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec15bbc0 0x7f5dec154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec165da0 con 0x7f5dec073a20 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec165c20 con 0x7f5dec15bbc0 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5df0e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec165f20 con 0x7f5dec073610 2024-09-17T05:02:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec1549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec1549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33242/0 (socket says 172.21.15.16:33242) 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 -- 172.21.15.16:0/3745710805 learned_addr learned my addr 172.21.15.16:0/3745710805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de9e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5dec073610 0x7f5dec0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5dea800640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec15bbc0 0x7f5dec154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 -- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5dec073610 msgr2=0x7f5dec0aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5dec073610 0x7f5dec0aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 -- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec15bbc0 msgr2=0x7f5dec154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec15bbc0 0x7f5dec154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5de3e00640 1 -- 172.21.15.16:0/3745710805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec0add60 con 0x7f5dec073a20 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.726+0000 7f5dea800640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec15bbc0 0x7f5dec154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de3e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec1549b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4000c10 tx=0x7f5dd40a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:01.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40042d0 con 0x7f5dec073a20 2024-09-17T05:02:01.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dd40be070 con 0x7f5dec073a20 2024-09-17T05:02:01.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec0ae040 con 0x7f5dec073a20 2024-09-17T05:02:01.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4005290 con 0x7f5dec073a20 2024-09-17T05:02:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec0ae600 con 0x7f5dec073a20 2024-09-17T05:02:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec166480 con 0x7f5dec073a20 2024-09-17T05:02:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5dd4005430 con 0x7f5dec073a20 2024-09-17T05:02:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de2000640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5dc8073200 0x7f5dc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5dd412ea60 con 0x7f5dec073a20 2024-09-17T05:02:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.730+0000 7f5de9e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5dc8073200 0x7f5dc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.734+0000 7f5de9e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5dc8073200 0x7f5dc8075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009ad0 tx=0x7f5ddc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:01.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.734+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5dd40c7020 con 0x7f5dec073a20 2024-09-17T05:02:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.978+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f5dec0b18c0 con 0x7f5dec073a20 2024-09-17T05:02:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.978+0000 7f5de2000640 1 -- 172.21.15.16:0/3745710805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v27) ==== 90+0+0 (secure 0 0 0) 0x7f5dd4100070 con 0x7f5dec073a20 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5dc8073200 msgr2=0x7f5dc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5dc8073200 0x7f5dc8075710 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009ad0 tx=0x7f5ddc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 msgr2=0x7f5dec1549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec1549b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4000c10 tx=0x7f5dd40a66f0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 shutdown_connections 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5dc8073200 0x7f5dc8075710 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec15bbc0 0x7f5dec154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5dec073a20 0x7f5dec1549b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 --2- 172.21.15.16:0/3745710805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5dec073610 0x7f5dec0aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 >> 172.21.15.16:0/3745710805 conn(0x7f5dec0b2250 msgr2=0x7f5dec0b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 shutdown_connections 2024-09-17T05:02:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:01.982+0000 7f5df0e00640 1 -- 172.21.15.16:0/3745710805 wait complete. 2024-09-17T05:02:02.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T05:02:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 -- 172.21.15.16:0/3617925977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc15ee90 msgr2=0x7f14dc0ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 --2- 172.21.15.16:0/3617925977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc15ee90 0x7f14dc0ca6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f14dc152250 tx=0x7f14c801c830 comp rx=0 tx=0).stop 2024-09-17T05:02:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 -- 172.21.15.16:0/3617925977 shutdown_connections 2024-09-17T05:02:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 --2- 172.21.15.16:0/3617925977 >> v2:172.21.15.104:3300/0 conn(0x7f14dc0cabf0 0x7f14dc0cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 --2- 172.21.15.16:0/3617925977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc15ee90 0x7f14dc0ca6b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 -- 172.21.15.16:0/3617925977 >> 172.21.15.16:0/3617925977 conn(0x7f14dc0b2250 msgr2=0x7f14dc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 -- 172.21.15.16:0/3617925977 shutdown_connections 2024-09-17T05:02:02.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 -- 172.21.15.16:0/3617925977 wait complete. 2024-09-17T05:02:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.162+0000 7f14e3800640 1 Processor -- start 2024-09-17T05:02:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 -- start start 2024-09-17T05:02:02.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14dc06e1d0 0x7f14dc0c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14dc0cabf0 0x7f14dc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 0x7f14dc0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0ceca0 con 0x7f14dc14aa20 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0d0e80 con 0x7f14dc06e1d0 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc0cee20 con 0x7f14dc0cabf0 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 0x7f14dc0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 0x7f14dc0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33246/0 (socket says 172.21.15.16:33246) 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 -- 172.21.15.16:0/3186914696 learned_addr learned my addr 172.21.15.16:0/3186914696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e0e00640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14dc06e1d0 0x7f14dc0c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14dbe00640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14dc0cabf0 0x7f14dc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 -- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14dc0cabf0 msgr2=0x7f14dc0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14dc0cabf0 0x7f14dc0cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 -- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14dc06e1d0 msgr2=0x7f14dc0c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14dc06e1d0 0x7f14dc0c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 -- 172.21.15.16:0/3186914696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc0c3530 con 0x7f14dc14aa20 2024-09-17T05:02:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e1800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 0x7f14dc0cce70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f14d400ea80 tx=0x7f14d400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d400cea0 con 0x7f14dc14aa20 2024-09-17T05:02:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14d4004b30 con 0x7f14dc14aa20 2024-09-17T05:02:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc0aed90 con 0x7f14dc14aa20 2024-09-17T05:02:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4005020 con 0x7f14dc14aa20 2024-09-17T05:02:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.166+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc0b3080 con 0x7f14dc14aa20 2024-09-17T05:02:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.170+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc0cb000 con 0x7f14dc14aa20 2024-09-17T05:02:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.170+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f14d40051c0 con 0x7f14dc14aa20 2024-09-17T05:02:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.170+0000 7f14d8c00640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14bc073230 0x7f14bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.170+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f14d4094bd0 con 0x7f14dc14aa20 2024-09-17T05:02:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.170+0000 7f14e0e00640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14bc073230 0x7f14bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.170+0000 7f14e0e00640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14bc073230 0x7f14bc075740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14d0001650 tx=0x7f14d0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.174+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14d405fe30 con 0x7f14dc14aa20 2024-09-17T05:02:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.410+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f14dc14a840 con 0x7f14dc14aa20 2024-09-17T05:02:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14d8c00640 1 -- 172.21.15.16:0/3186914696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7f14dc14a840 con 0x7f14dc14aa20 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14bc073230 msgr2=0x7f14bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14bc073230 0x7f14bc075740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14d0001650 tx=0x7f14d0009040 comp rx=0 tx=0).stop 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 msgr2=0x7f14dc0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 0x7f14dc0cce70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f14d400ea80 tx=0x7f14d400ef40 comp rx=0 tx=0).stop 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 shutdown_connections 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14bc073230 0x7f14bc075740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14dc14aa20 0x7f14dc0cce70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14dc0cabf0 0x7f14dc0cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 --2- 172.21.15.16:0/3186914696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14dc06e1d0 0x7f14dc0c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 >> 172.21.15.16:0/3186914696 conn(0x7f14dc0b2250 msgr2=0x7f14dc0cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 shutdown_connections 2024-09-17T05:02:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.414+0000 7f14e3800640 1 -- 172.21.15.16:0/3186914696 wait complete. 2024-09-17T05:02:02.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T05:02:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 -- 172.21.15.16:0/1532279041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074a20 msgr2=0x7f5b6415efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 --2- 172.21.15.16:0/1532279041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074a20 0x7f5b6415efe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009840 tx=0x7f5b5401c820 comp rx=0 tx=0).stop 2024-09-17T05:02:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 -- 172.21.15.16:0/1532279041 shutdown_connections 2024-09-17T05:02:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 --2- 172.21.15.16:0/1532279041 >> v2:172.21.15.104:3300/0 conn(0x7f5b64074e30 0x7f5b6415f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 --2- 172.21.15.16:0/1532279041 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074a20 0x7f5b6415efe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 -- 172.21.15.16:0/1532279041 >> 172.21.15.16:0/1532279041 conn(0x7f5b640b2250 msgr2=0x7f5b640d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:02.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 -- 172.21.15.16:0/1532279041 shutdown_connections 2024-09-17T05:02:02.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.594+0000 7f5b68e00640 1 -- 172.21.15.16:0/1532279041 wait complete. 2024-09-17T05:02:02.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 Processor -- start 2024-09-17T05:02:02.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 -- start start 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b64074a20 0x7f5b6414f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074e30 0x7f5b641551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 0x7f5b64155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64168ed0 con 0x7f5b6415ba50 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64168d50 con 0x7f5b64074a20 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b68e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5b64169050 con 0x7f5b64074e30 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b62800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 0x7f5b64155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b62800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 0x7f5b64155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33262/0 (socket says 172.21.15.16:33262) 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b62800640 1 -- 172.21.15.16:0/4281962797 learned_addr learned my addr 172.21.15.16:0/4281962797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b61e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b64074a20 0x7f5b6414f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.598+0000 7f5b61400640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074e30 0x7f5b641551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b62800640 1 -- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074e30 msgr2=0x7f5b641551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b62800640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074e30 0x7f5b641551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b62800640 1 -- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b64074a20 msgr2=0x7f5b6414f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b62800640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b64074a20 0x7f5b6414f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b62800640 1 -- 172.21.15.16:0/4281962797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b641fb2b0 con 0x7f5b6415ba50 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b62800640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 0x7f5b64155710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c9f0 tx=0x7f5b5c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c016020 con 0x7f5b6415ba50 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b5c012920 con 0x7f5b6415ba50 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b640c6bf0 con 0x7f5b6415ba50 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c012d70 con 0x7f5b6415ba50 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b640c7f00 con 0x7f5b6415ba50 2024-09-17T05:02:02.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b640c6960 con 0x7f5b6415ba50 2024-09-17T05:02:02.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5b5c019020 con 0x7f5b6415ba50 2024-09-17T05:02:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.602+0000 7f5b5a000640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b44073200 0x7f5b44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.606+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5b5c0941b0 con 0x7f5b6415ba50 2024-09-17T05:02:02.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.606+0000 7f5b61e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b44073200 0x7f5b44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:02.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.606+0000 7f5b61e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b44073200 0x7f5b44075710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b540af000 tx=0x7f5b5401b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.610+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5b5c020020 con 0x7f5b6415ba50 2024-09-17T05:02:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.846+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f5b6415be60 con 0x7f5b6415ba50 2024-09-17T05:02:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b5a000640 1 -- 172.21.15.16:0/4281962797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v29) ==== 97+0+0 (secure 0 0 0) 0x7f5b5c05ecb0 con 0x7f5b6415ba50 2024-09-17T05:02:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b44073200 msgr2=0x7f5b44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b44073200 0x7f5b44075710 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b540af000 tx=0x7f5b5401b060 comp rx=0 tx=0).stop 2024-09-17T05:02:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 msgr2=0x7f5b64155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 0x7f5b64155710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c9f0 tx=0x7f5b5c00cae0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 shutdown_connections 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b44073200 0x7f5b44075710 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b6415ba50 0x7f5b64155710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b64074e30 0x7f5b641551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 --2- 172.21.15.16:0/4281962797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b64074a20 0x7f5b6414f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.850+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 >> 172.21.15.16:0/4281962797 conn(0x7f5b640b2250 msgr2=0x7f5b64154110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.854+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 shutdown_connections 2024-09-17T05:02:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:02.854+0000 7f5b68e00640 1 -- 172.21.15.16:0/4281962797 wait complete. 2024-09-17T05:02:02.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T05:02:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.030+0000 7f5686a00640 1 -- 172.21.15.16:0/1421088365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680074e30 msgr2=0x7f56800ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.030+0000 7f5686a00640 1 --2- 172.21.15.16:0/1421088365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680074e30 0x7f56800ce5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f56800ce2c0 tx=0x7f566801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.030+0000 7f5686a00640 1 -- 172.21.15.16:0/1421088365 shutdown_connections 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.030+0000 7f5686a00640 1 --2- 172.21.15.16:0/1421088365 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f568015f6f0 0x7f56800ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.030+0000 7f5686a00640 1 --2- 172.21.15.16:0/1421088365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5680074e30 0x7f56800ce5d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.030+0000 7f5686a00640 1 -- 172.21.15.16:0/1421088365 >> 172.21.15.16:0/1421088365 conn(0x7f56800b2250 msgr2=0x7f56800d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 -- 172.21.15.16:0/1421088365 shutdown_connections 2024-09-17T05:02:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 -- 172.21.15.16:0/1421088365 wait complete. 2024-09-17T05:02:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 Processor -- start 2024-09-17T05:02:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 -- start start 2024-09-17T05:02:03.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5680074a20 0x7f56800c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5680074e30 0x7f5680159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 0x7f56800c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5680168f30 con 0x7f568015f6f0 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5680168db0 con 0x7f5680074a20 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.034+0000 7f5686a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f56801690b0 con 0x7f5680074e30 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 0x7f56800c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 0x7f56800c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33282/0 (socket says 172.21.15.16:33282) 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 -- 172.21.15.16:0/269318878 learned_addr learned my addr 172.21.15.16:0/269318878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5677e00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5680074e30 0x7f5680159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f567fe00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5680074a20 0x7f56800c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 -- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5680074e30 msgr2=0x7f5680159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5680074e30 0x7f5680159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 -- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5680074a20 msgr2=0x7f56800c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5680074a20 0x7f56800c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 -- 172.21.15.16:0/269318878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56801fcdf0 con 0x7f568015f6f0 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5684a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 0x7f56800c7840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f567800e940 tx=0x7f567800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678004d30 con 0x7f568015f6f0 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f567800cb20 con 0x7f568015f6f0 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678005020 con 0x7f568015f6f0 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56801fd0d0 con 0x7f568015f6f0 2024-09-17T05:02:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56800c7320 con 0x7f568015f6f0 2024-09-17T05:02:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56800c7110 con 0x7f568015f6f0 2024-09-17T05:02:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.038+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f56780051c0 con 0x7f568015f6f0 2024-09-17T05:02:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.042+0000 7f567cc00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f565c073200 0x7f565c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.042+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f5678094990 con 0x7f568015f6f0 2024-09-17T05:02:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.042+0000 7f567fe00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f565c073200 0x7f565c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.042+0000 7f567fe00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f565c073200 0x7f565c075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5670007fd0 tx=0x7f5670009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:03.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.046+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5678016030 con 0x7f568015f6f0 2024-09-17T05:02:03.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.282+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f568015efe0 con 0x7f568015f6f0 2024-09-17T05:02:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f567cc00640 1 -- 172.21.15.16:0/269318878 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v30) ==== 90+0+0 (secure 0 0 0) 0x7f567805f480 con 0x7f568015f6f0 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f565c073200 msgr2=0x7f565c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f565c073200 0x7f565c075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5670007fd0 tx=0x7f5670009040 comp rx=0 tx=0).stop 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 msgr2=0x7f56800c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 0x7f56800c7840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f567800e940 tx=0x7f567800ee00 comp rx=0 tx=0).stop 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 shutdown_connections 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f565c073200 0x7f565c075710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f568015f6f0 0x7f56800c7840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5680074e30 0x7f5680159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 --2- 172.21.15.16:0/269318878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5680074a20 0x7f56800c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 >> 172.21.15.16:0/269318878 conn(0x7f56800b2250 msgr2=0x7f56800ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 shutdown_connections 2024-09-17T05:02:03.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.286+0000 7f5686a00640 1 -- 172.21.15.16:0/269318878 wait complete. 2024-09-17T05:02:03.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:02:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 -- 172.21.15.16:0/3777648398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180cfa30 msgr2=0x7fba180b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 --2- 172.21.15.16:0/3777648398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180cfa30 0x7fba180b3b20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fba180698b0 tx=0x7fba0001c800 comp rx=0 tx=0).stop 2024-09-17T05:02:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 -- 172.21.15.16:0/3777648398 shutdown_connections 2024-09-17T05:02:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 --2- 172.21.15.16:0/3777648398 >> v2:172.21.15.104:3300/0 conn(0x7fba180b4060 0x7fba180b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 --2- 172.21.15.16:0/3777648398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180cfa30 0x7fba180b3b20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 -- 172.21.15.16:0/3777648398 >> 172.21.15.16:0/3777648398 conn(0x7fba180d0860 msgr2=0x7fba180d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 -- 172.21.15.16:0/3777648398 shutdown_connections 2024-09-17T05:02:03.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 -- 172.21.15.16:0/3777648398 wait complete. 2024-09-17T05:02:03.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 Processor -- start 2024-09-17T05:02:03.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.466+0000 7fba1d000640 1 -- start start 2024-09-17T05:02:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba1806ce90 0x7fba180aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 0x7fba18156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba180cfa30 0x7fba18158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba181689e0 con 0x7fba180b4060 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba180b5a80 con 0x7fba180cfa30 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fba18168b60 con 0x7fba1806ce90 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 0x7fba18156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 0x7fba18156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33298/0 (socket says 172.21.15.16:33298) 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 -- 172.21.15.16:0/1810494617 learned_addr learned my addr 172.21.15.16:0/1810494617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba15e00640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba1806ce90 0x7fba180aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba16800640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba180cfa30 0x7fba18158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 -- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba1806ce90 msgr2=0x7fba180aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba1806ce90 0x7fba180aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 -- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba180cfa30 msgr2=0x7fba18158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba180cfa30 0x7fba18158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 -- 172.21.15.16:0/1810494617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba18164cf0 con 0x7fba180b4060 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba16800640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba180cfa30 0x7fba18158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:03.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0fe00640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 0x7fba18156060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fba0001cce0 tx=0x7fba000a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:03.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b2040 con 0x7fba180b4060 2024-09-17T05:02:03.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba00004230 con 0x7fba180b4060 2024-09-17T05:02:03.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba00003600 con 0x7fba180b4060 2024-09-17T05:02:03.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba18164fd0 con 0x7fba180b4060 2024-09-17T05:02:03.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba180ae5f0 con 0x7fba180b4060 2024-09-17T05:02:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.470+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba180b4470 con 0x7fba180b4060 2024-09-17T05:02:03.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.474+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba000bd030 con 0x7fba180b4060 2024-09-17T05:02:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.474+0000 7fba0e000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb9f4073230 0x7fb9f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.474+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fba0012dbb0 con 0x7fba180b4060 2024-09-17T05:02:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.474+0000 7fba15e00640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb9f4073230 0x7fb9f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.474+0000 7fba15e00640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb9f4073230 0x7fb9f4075740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba08000f70 tx=0x7fba08009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:03.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.478+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba000c4060 con 0x7fba180b4060 2024-09-17T05:02:03.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fba1815f850 con 0x7fba180b4060 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba0e000640 1 -- 172.21.15.16:0/1810494617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7fba000163f0 con 0x7fba180b4060 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:02:03.655 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:02:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb9f4073230 msgr2=0x7fb9f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb9f4073230 0x7fb9f4075740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fba08000f70 tx=0x7fba08009040 comp rx=0 tx=0).stop 2024-09-17T05:02:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 msgr2=0x7fba18156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 0x7fba18156060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fba0001cce0 tx=0x7fba000a57d0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 shutdown_connections 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb9f4073230 0x7fb9f4075740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba180cfa30 0x7fba18158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba180b4060 0x7fba18156060 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 --2- 172.21.15.16:0/1810494617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba1806ce90 0x7fba180aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 >> 172.21.15.16:0/1810494617 conn(0x7fba180d0860 msgr2=0x7fba180d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 shutdown_connections 2024-09-17T05:02:03.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.654+0000 7fba1d000640 1 -- 172.21.15.16:0/1810494617 wait complete. 2024-09-17T05:02:03.677 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:02:03.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:02:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 -- 172.21.15.16:0/1548485367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c074bf0 msgr2=0x7f6b9c15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 --2- 172.21.15.16:0/1548485367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c074bf0 0x7f6b9c15f6b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009bc0 tx=0x7f6b9401cb70 comp rx=0 tx=0).stop 2024-09-17T05:02:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 -- 172.21.15.16:0/1548485367 shutdown_connections 2024-09-17T05:02:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 --2- 172.21.15.16:0/1548485367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c074bf0 0x7f6b9c15f6b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 --2- 172.21.15.16:0/1548485367 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9c0b4620 0x7f6b9c06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 -- 172.21.15.16:0/1548485367 >> 172.21.15.16:0/1548485367 conn(0x7f6b9c0d0050 msgr2=0x7f6b9c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 -- 172.21.15.16:0/1548485367 shutdown_connections 2024-09-17T05:02:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 -- 172.21.15.16:0/1548485367 wait complete. 2024-09-17T05:02:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 Processor -- start 2024-09-17T05:02:03.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.838+0000 7f6ba1200640 1 -- start start 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 0x7f6b9c1f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9c074bf0 0x7f6b9c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b9c0b4620 0x7f6b9c15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c152890 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c15fe00 con 0x7f6b9c0b4620 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c168920 con 0x7f6b9c074bf0 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 0x7f6b9c1f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 0x7f6b9c1f9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33320/0 (socket says 172.21.15.16:33320) 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 -- 172.21.15.16:0/290165049 learned_addr learned my addr 172.21.15.16:0/290165049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b9a800640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b9c0b4620 0x7f6b9c15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b93e00640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9c074bf0 0x7f6b9c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 -- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9c074bf0 msgr2=0x7f6b9c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9c074bf0 0x7f6b9c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 -- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b9c0b4620 msgr2=0x7f6b9c15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b9c0b4620 0x7f6b9c15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 -- 172.21.15.16:0/290165049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c0ae200 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b9a800640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b9c0b4620 0x7f6b9c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b99e00640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 0x7f6b9c1f9230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00d720 tx=0x7f6b8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:03.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c017070 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b8c009d20 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c005280 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c0b15f0 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1f8a50 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.842+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c06dfa0 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.846+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b8c005420 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.846+0000 7f6b92000640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b78073230 0x7f6b78075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:03.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.846+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f6b8c0941a0 con 0x7f6b9c06e4e0 2024-09-17T05:02:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.846+0000 7f6b93e00640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b78073230 0x7f6b78075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:03.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.846+0000 7f6b93e00640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b78073230 0x7f6b78075740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b840046c0 tx=0x7f6b84009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:03.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:03.850+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b8c05f370 con 0x7f6b9c06e4e0 2024-09-17T05:02:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.022+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6b9c09dee0 con 0x7f6b9c06e4e0 2024-09-17T05:02:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.290+0000 7f6b92000640 1 -- 172.21.15.16:0/290165049 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7f6b8c05ec70 con 0x7f6b9c06e4e0 2024-09-17T05:02:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:02:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b78073230 msgr2=0x7f6b78075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b78073230 0x7f6b78075740 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b840046c0 tx=0x7f6b84009040 comp rx=0 tx=0).stop 2024-09-17T05:02:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 msgr2=0x7f6b9c1f9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 0x7f6b9c1f9230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00d720 tx=0x7f6b8c00dbe0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 shutdown_connections 2024-09-17T05:02:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b78073230 0x7f6b78075740 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b9c0b4620 0x7f6b9c15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9c074bf0 0x7f6b9c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 --2- 172.21.15.16:0/290165049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b9c06e4e0 0x7f6b9c1f9230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 >> 172.21.15.16:0/290165049 conn(0x7f6b9c0d0050 msgr2=0x7f6b9c15cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 shutdown_connections 2024-09-17T05:02:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.294+0000 7f6ba1200640 1 -- 172.21.15.16:0/290165049 wait complete. 2024-09-17T05:02:04.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:02:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 -- 172.21.15.16:0/3947452913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c15c680 msgr2=0x7ffb1c073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 --2- 172.21.15.16:0/3947452913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c15c680 0x7ffb1c073950 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009bc0 tx=0x7ffb1401cb10 comp rx=0 tx=0).stop 2024-09-17T05:02:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 -- 172.21.15.16:0/3947452913 shutdown_connections 2024-09-17T05:02:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 --2- 172.21.15.16:0/3947452913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c15c680 0x7ffb1c073950 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 -- 172.21.15.16:0/3947452913 >> 172.21.15.16:0/3947452913 conn(0x7ffb1c06f900 msgr2=0x7ffb1c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 -- 172.21.15.16:0/3947452913 shutdown_connections 2024-09-17T05:02:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 -- 172.21.15.16:0/3947452913 wait complete. 2024-09-17T05:02:04.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 Processor -- start 2024-09-17T05:02:04.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.486+0000 7ffb21400640 1 -- start start 2024-09-17T05:02:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 0x7ffb1c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb1c15c270 0x7ffb1c14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb1c15c680 0x7ffb1c14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c169c30 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c169ab0 con 0x7ffb1c15c270 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ffb1c169db0 con 0x7ffb1c15c680 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 0x7ffb1c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 0x7ffb1c0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33346/0 (socket says 172.21.15.16:33346) 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 -- 172.21.15.16:0/3597740055 learned_addr learned my addr 172.21.15.16:0/3597740055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1b200640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb1c15c680 0x7ffb1c14c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb13e00640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb1c15c270 0x7ffb1c14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 -- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb1c15c680 msgr2=0x7ffb1c14c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb1c15c680 0x7ffb1c14c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 -- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb1c15c270 msgr2=0x7ffb1c14d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb1c15c270 0x7ffb1c14d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 -- 172.21.15.16:0/3597740055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c161770 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb1a800640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 0x7ffb1c0aec10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c400 tx=0x7ffb0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c011790 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb0c017070 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c011cc0 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c161a50 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c162360 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.490+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c161c90 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.494+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffb0c01c460 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.494+0000 7ffb12a00640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffaf8073230 0x7ffaf8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.494+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7ffb0c094790 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.494+0000 7ffb13e00640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffaf8073230 0x7ffaf8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:04.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.494+0000 7ffb13e00640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffaf8073230 0x7ffaf8075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0cfdb0 tx=0x7ffb04009060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.498+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffb1c161c90 con 0x7ffb1c0b2f50 2024-09-17T05:02:04.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:04.674+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffb1c15bae0 con 0x7ffb1c0b2f50 2024-09-17T05:02:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.294+0000 7ffb12a00640 1 -- 172.21.15.16:0/3597740055 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7ffb0c05f320 con 0x7ffb1c0b2f50 2024-09-17T05:02:05.297 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffaf8073230 msgr2=0x7ffaf8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffaf8073230 0x7ffaf8075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0cfdb0 tx=0x7ffb04009060 comp rx=0 tx=0).stop 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 msgr2=0x7ffb1c0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 0x7ffb1c0aec10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c400 tx=0x7ffb0c00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 shutdown_connections 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffaf8073230 0x7ffaf8075740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb1c15c680 0x7ffb1c14c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb1c15c270 0x7ffb1c14d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 --2- 172.21.15.16:0/3597740055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb1c0b2f50 0x7ffb1c0aec10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 >> 172.21.15.16:0/3597740055 conn(0x7ffb1c06f900 msgr2=0x7ffb1c0d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:05.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 shutdown_connections 2024-09-17T05:02:05.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.298+0000 7ffb21400640 1 -- 172.21.15.16:0/3597740055 wait complete. 2024-09-17T05:02:05.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:02:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 -- 172.21.15.16:0/1446292467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd8815f6f0 msgr2=0x7efd880ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 --2- 172.21.15.16:0/1446292467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd8815f6f0 0x7efd880ceb10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd880cca90 tx=0x7efd8001cb80 comp rx=0 tx=0).stop 2024-09-17T05:02:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 -- 172.21.15.16:0/1446292467 shutdown_connections 2024-09-17T05:02:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 --2- 172.21.15.16:0/1446292467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd8815f6f0 0x7efd880ceb10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 --2- 172.21.15.16:0/1446292467 >> v2:172.21.15.104:3300/0 conn(0x7efd88074e30 0x7efd880ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 -- 172.21.15.16:0/1446292467 >> 172.21.15.16:0/1446292467 conn(0x7efd880b2250 msgr2=0x7efd880d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:05.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 -- 172.21.15.16:0/1446292467 shutdown_connections 2024-09-17T05:02:05.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.502+0000 7efd8d800640 1 -- 172.21.15.16:0/1446292467 wait complete. 2024-09-17T05:02:05.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 Processor -- start 2024-09-17T05:02:05.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 -- start start 2024-09-17T05:02:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd88074a20 0x7efd880b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 0x7efd88153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd8815f6f0 0x7efd880c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd88168e20 con 0x7efd88074e30 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efd88168ca0 con 0x7efd8815f6f0 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd8d800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efd88168fa0 con 0x7efd88074a20 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd7e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 0x7efd88153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd7e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 0x7efd88153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33366/0 (socket says 172.21.15.16:33366) 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd7e000640 1 -- 172.21.15.16:0/9822199 learned_addr learned my addr 172.21.15.16:0/9822199 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd86a00640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd88074a20 0x7efd880b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.506+0000 7efd87400640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd8815f6f0 0x7efd880c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7e000640 1 -- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd88074a20 msgr2=0x7efd880b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7e000640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd88074a20 0x7efd880b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7e000640 1 -- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd8815f6f0 msgr2=0x7efd880c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7e000640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd8815f6f0 0x7efd880c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:05.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7e000640 1 -- 172.21.15.16:0/9822199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880b3200 con 0x7efd88074e30 2024-09-17T05:02:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd86a00640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd88074a20 0x7efd880b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7e000640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 0x7efd88153ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d750 tx=0x7efd7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70017070 con 0x7efd88074e30 2024-09-17T05:02:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd70009d20 con 0x7efd88074e30 2024-09-17T05:02:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd880b34e0 con 0x7efd88074e30 2024-09-17T05:02:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70005280 con 0x7efd88074e30 2024-09-17T05:02:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd880b3aa0 con 0x7efd88074e30 2024-09-17T05:02:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd880b5ea0 con 0x7efd88074e30 2024-09-17T05:02:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.510+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd70005420 con 0x7efd88074e30 2024-09-17T05:02:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.514+0000 7efd7f400640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd64073200 0x7efd64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.514+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7efd700948d0 con 0x7efd88074e30 2024-09-17T05:02:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.514+0000 7efd86a00640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd64073200 0x7efd64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:05.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.514+0000 7efd86a00640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd64073200 0x7efd64075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd780046c0 tx=0x7efd78009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:05.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.514+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd880b5ea0 con 0x7efd88074e30 2024-09-17T05:02:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:05.734+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efd8815efe0 con 0x7efd88074e30 2024-09-17T05:02:06.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.322+0000 7efd7f400640 1 -- 172.21.15.16:0/9822199 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7efd7000de60 con 0x7efd88074e30 2024-09-17T05:02:06.326 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 5 and data pool 6 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.338+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd64073200 msgr2=0x7efd64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.338+0000 7efd8d800640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd64073200 0x7efd64075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd780046c0 tx=0x7efd78009040 comp rx=0 tx=0).stop 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.338+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 msgr2=0x7efd88153ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.338+0000 7efd8d800640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 0x7efd88153ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d750 tx=0x7efd7000dc10 comp rx=0 tx=0).stop 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.342+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 shutdown_connections 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.342+0000 7efd8d800640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd64073200 0x7efd64075710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.342+0000 7efd8d800640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd8815f6f0 0x7efd880c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.342+0000 7efd8d800640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd88074e30 0x7efd88153ea0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.342+0000 7efd8d800640 1 --2- 172.21.15.16:0/9822199 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd88074a20 0x7efd880b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.342+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 >> 172.21.15.16:0/9822199 conn(0x7efd880b2250 msgr2=0x7efd8815d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:06.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.346+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 shutdown_connections 2024-09-17T05:02:06.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.350+0000 7efd8d800640 1 -- 172.21.15.16:0/9822199 wait complete. 2024-09-17T05:02:06.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:02:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 -- 172.21.15.16:0/1673677233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15be10 msgr2=0x7f059c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 --2- 172.21.15.16:0/1673677233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15be10 0x7f059c06dc60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f059c072860 tx=0x7f059401cd80 comp rx=0 tx=0).stop 2024-09-17T05:02:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 -- 172.21.15.16:0/1673677233 shutdown_connections 2024-09-17T05:02:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 --2- 172.21.15.16:0/1673677233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15be10 0x7f059c06dc60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 --2- 172.21.15.16:0/1673677233 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f059c15ba00 0x7f059c06d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 --2- 172.21.15.16:0/1673677233 >> v2:172.21.15.104:3300/0 conn(0x7f059c0cc550 0x7f059c15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 -- 172.21.15.16:0/1673677233 >> 172.21.15.16:0/1673677233 conn(0x7f059c06f900 msgr2=0x7f059c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 -- 172.21.15.16:0/1673677233 shutdown_connections 2024-09-17T05:02:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.542+0000 7f05a1800640 1 -- 172.21.15.16:0/1673677233 wait complete. 2024-09-17T05:02:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 Processor -- start 2024-09-17T05:02:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 -- start start 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f059c0cc550 0x7f059c14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 0x7f059c1684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f059c15be10 0x7f059c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f059c0737d0 con 0x7f059c15ba00 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f059c073650 con 0x7f059c0cc550 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f059c0b2f50 con 0x7f059c15be10 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 0x7f059c1684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 0x7f059c1684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33384/0 (socket says 172.21.15.16:33384) 2024-09-17T05:02:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 -- 172.21.15.16:0/816572188 learned_addr learned my addr 172.21.15.16:0/816572188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f059aa00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f059c0cc550 0x7f059c14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f059b400640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f059c15be10 0x7f059c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 -- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f059c15be10 msgr2=0x7f059c0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f059c15be10 0x7f059c0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 -- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f059c0cc550 msgr2=0x7f059c14dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f059c0cc550 0x7f059c14dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 -- 172.21.15.16:0/816572188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f059c14e530 con 0x7f059c15ba00 2024-09-17T05:02:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0593e00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 0x7f059c1684e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f058400d7b0 tx=0x7f058400dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584017070 con 0x7f059c15ba00 2024-09-17T05:02:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0584004310 con 0x7f059c15ba00 2024-09-17T05:02:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f059c167440 con 0x7f059c15ba00 2024-09-17T05:02:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584009c30 con 0x7f059c15ba00 2024-09-17T05:02:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.546+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f059c167a00 con 0x7f059c15ba00 2024-09-17T05:02:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.550+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f059c14e700 con 0x7f059c15ba00 2024-09-17T05:02:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.550+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0584005020 con 0x7f059c15ba00 2024-09-17T05:02:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.550+0000 7f0592a00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0578073230 0x7f0578075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.550+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0584094630 con 0x7f059c15ba00 2024-09-17T05:02:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.550+0000 7f059aa00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0578073230 0x7f0578075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.554+0000 7f059aa00640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0578073230 0x7f0578075740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f058c00bad0 tx=0x7f058c00b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.554+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0584020020 con 0x7f059c15ba00 2024-09-17T05:02:06.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.730+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f059c15e810 con 0x7f059c15ba00 2024-09-17T05:02:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.730+0000 7f0592a00640 1 -- 172.21.15.16:0/816572188 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17003 (secure 0 0 0) 0x7f058405ee70 con 0x7f059c15ba00 2024-09-17T05:02:06.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:06.734 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:02:05.738315+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:02:04.037604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T05:02:04.686294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:02:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0578073230 msgr2=0x7f0578075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0578073230 0x7f0578075740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f058c00bad0 tx=0x7f058c00b260 comp rx=0 tx=0).stop 2024-09-17T05:02:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 msgr2=0x7f059c1684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 0x7f059c1684e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f058400d7b0 tx=0x7f058400dc70 comp rx=0 tx=0).stop 2024-09-17T05:02:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 shutdown_connections 2024-09-17T05:02:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0578073230 0x7f0578075740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f059c15be10 0x7f059c0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f059c15ba00 0x7f059c1684e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 --2- 172.21.15.16:0/816572188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f059c0cc550 0x7f059c14dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 >> 172.21.15.16:0/816572188 conn(0x7f059c06f900 msgr2=0x7f059c15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 shutdown_connections 2024-09-17T05:02:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.734+0000 7f05a1800640 1 -- 172.21.15.16:0/816572188 wait complete. 2024-09-17T05:02:06.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:02:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 -- 172.21.15.16:0/3195380657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d0830 msgr2=0x7fe800070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 --2- 172.21.15.16:0/3195380657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d0830 0x7fe800070490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009900 tx=0x7fe7e801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 -- 172.21.15.16:0/3195380657 shutdown_connections 2024-09-17T05:02:06.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 --2- 172.21.15.16:0/3195380657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d0830 0x7fe800070490 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 --2- 172.21.15.16:0/3195380657 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8000d1aa0 0x7fe8000d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 -- 172.21.15.16:0/3195380657 >> 172.21.15.16:0/3195380657 conn(0x7fe8000b4160 msgr2=0x7fe8000b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 -- 172.21.15.16:0/3195380657 shutdown_connections 2024-09-17T05:02:06.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 -- 172.21.15.16:0/3195380657 wait complete. 2024-09-17T05:02:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 Processor -- start 2024-09-17T05:02:06.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.914+0000 7fe805600640 1 -- start start 2024-09-17T05:02:06.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8000709d0 0x7fe8001574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8000d0830 0x7fe8001fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 0x7fe8001fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe80016a110 con 0x7fe8000d1aa0 2024-09-17T05:02:06.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe800169f90 con 0x7fe8000d0830 2024-09-17T05:02:06.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe80016a290 con 0x7fe8000709d0 2024-09-17T05:02:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 0x7fe8001fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 0x7fe8001fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33402/0 (socket says 172.21.15.16:33402) 2024-09-17T05:02:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 -- 172.21.15.16:0/1291595439 learned_addr learned my addr 172.21.15.16:0/1291595439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe7ffe00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8000709d0 0x7fe8001574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe7f7400640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8000d0830 0x7fe8001fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 -- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8000709d0 msgr2=0x7fe8001574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8000709d0 0x7fe8001574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 -- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8000d0830 msgr2=0x7fe8001fcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8000d0830 0x7fe8001fcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 -- 172.21.15.16:0/1291595439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8001f95b0 con 0x7fe8000d1aa0 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe804c00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 0x7fe8001fd070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c760 tx=0x7fe7f800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:06.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8003b80 con 0x7fe8000d1aa0 2024-09-17T05:02:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7f8003d20 con 0x7fe8000d1aa0 2024-09-17T05:02:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f800aab0 con 0x7fe8000d1aa0 2024-09-17T05:02:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8001f9890 con 0x7fe8000d1aa0 2024-09-17T05:02:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001f9e50 con 0x7fe8000d1aa0 2024-09-17T05:02:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.918+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe800070490 con 0x7fe8000d1aa0 2024-09-17T05:02:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.922+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe7f800ac50 con 0x7fe8000d1aa0 2024-09-17T05:02:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.922+0000 7fe7fcc00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.922+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe7f8094090 con 0x7fe8000d1aa0 2024-09-17T05:02:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.922+0000 7fe7ffe00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.922+0000 7fe7ffe00640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009b40 tx=0x7fe7f00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:06.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:06.926+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7f805e830 con 0x7fe8000d1aa0 2024-09-17T05:02:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.106+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe800157a10 con 0x7fe8000d1aa0 2024-09-17T05:02:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.106+0000 7fe7fcc00640 1 -- 172.21.15.16:0/1291595439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17003 (secure 0 0 0) 0x7fe7f8063260 con 0x7fe8000d1aa0 2024-09-17T05:02:07.110 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:07.110 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:02:05.738315+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:02:04.037604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T05:02:04.686294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:02:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 msgr2=0x7fe7dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009b40 tx=0x7fe7f00091e0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 msgr2=0x7fe8001fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 0x7fe8001fd070 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c760 tx=0x7fe7f800cc20 comp rx=0 tx=0).stop 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 shutdown_connections 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8000d1aa0 0x7fe8001fd070 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8000d0830 0x7fe8001fcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 --2- 172.21.15.16:0/1291595439 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8000709d0 0x7fe8001574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 >> 172.21.15.16:0/1291595439 conn(0x7fe8000b4160 msgr2=0x7fe80015b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 shutdown_connections 2024-09-17T05:02:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.110+0000 7fe805600640 1 -- 172.21.15.16:0/1291595439 wait complete. 2024-09-17T05:02:07.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:02:07.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 -- 172.21.15.16:0/3243552327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c074e30 msgr2=0x7fa41c15f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:07.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 --2- 172.21.15.16:0/3243552327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c074e30 0x7fa41c15f520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa41c072860 tx=0x7fa40401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 -- 172.21.15.16:0/3243552327 shutdown_connections 2024-09-17T05:02:07.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 --2- 172.21.15.16:0/3243552327 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa41c15ba50 0x7fa41c15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 --2- 172.21.15.16:0/3243552327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c074e30 0x7fa41c15f520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 --2- 172.21.15.16:0/3243552327 >> v2:172.21.15.104:3300/0 conn(0x7fa41c074a20 0x7fa41c15efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 -- 172.21.15.16:0/3243552327 >> 172.21.15.16:0/3243552327 conn(0x7fa41c0b2250 msgr2=0x7fa41c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:07.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 -- 172.21.15.16:0/3243552327 shutdown_connections 2024-09-17T05:02:07.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 -- 172.21.15.16:0/3243552327 wait complete. 2024-09-17T05:02:07.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 Processor -- start 2024-09-17T05:02:07.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.294+0000 7fa421c00640 1 -- start start 2024-09-17T05:02:07.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa41c074a20 0x7fa41c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa41c074e30 0x7fa41c164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 0x7fa41c165140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0cf360 con 0x7fa41c15ba50 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0cf1e0 con 0x7fa41c074a20 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c1681d0 con 0x7fa41c074e30 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 0x7fa41c165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 0x7fa41c165140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47936/0 (socket says 172.21.15.16:47936) 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 -- 172.21.15.16:0/1166580885 learned_addr learned my addr 172.21.15.16:0/1166580885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa413e00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa41c074e30 0x7fa41c164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41aa00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa41c074a20 0x7fa41c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:07.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 -- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa41c074e30 msgr2=0x7fa41c164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa41c074e30 0x7fa41c164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 -- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa41c074a20 msgr2=0x7fa41c0c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa41c074a20 0x7fa41c0c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 -- 172.21.15.16:0/1166580885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c1576b0 con 0x7fa41c15ba50 2024-09-17T05:02:07.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa41b400640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 0x7fa41c165140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e940 tx=0x7fa41400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414004d30 con 0x7fa41c15ba50 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa41400cb20 con 0x7fa41c15ba50 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c151750 con 0x7fa41c15ba50 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414005020 con 0x7fa41c15ba50 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c1573d0 con 0x7fa41c15ba50 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.298+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c15be60 con 0x7fa41c15ba50 2024-09-17T05:02:07.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.302+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4140051c0 con 0x7fa41c15ba50 2024-09-17T05:02:07.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.302+0000 7fa412a00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa3f8073200 0x7fa3f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:07.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.302+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa414094cb0 con 0x7fa41c15ba50 2024-09-17T05:02:07.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.302+0000 7fa41aa00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa3f8073200 0x7fa3f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:07.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.302+0000 7fa41aa00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa3f8073200 0x7fa3f8075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00bad0 tx=0x7fa40c00b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:07.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.306+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa41c1576b0 con 0x7fa41c15ba50 2024-09-17T05:02:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:07.522+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa41c0ca0e0 con 0x7fa41c15ba50 2024-09-17T05:02:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.326+0000 7fa412a00640 1 -- 172.21.15.16:0/1166580885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa414016030 con 0x7fa41c15ba50 2024-09-17T05:02:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa3f8073200 msgr2=0x7fa3f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa3f8073200 0x7fa3f8075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00bad0 tx=0x7fa40c00b260 comp rx=0 tx=0).stop 2024-09-17T05:02:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 msgr2=0x7fa41c165140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 0x7fa41c165140 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa41400e940 tx=0x7fa41400ee00 comp rx=0 tx=0).stop 2024-09-17T05:02:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 shutdown_connections 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa3f8073200 0x7fa3f8075710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa41c15ba50 0x7fa41c165140 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa41c074e30 0x7fa41c164c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 --2- 172.21.15.16:0/1166580885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa41c074a20 0x7fa41c0c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 >> 172.21.15.16:0/1166580885 conn(0x7fa41c0b2250 msgr2=0x7fa41c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 shutdown_connections 2024-09-17T05:02:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.330+0000 7fa421c00640 1 -- 172.21.15.16:0/1166580885 wait complete. 2024-09-17T05:02:08.358 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:02:08.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:02:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 -- 172.21.15.16:0/1251170477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde400cf4a0 msgr2=0x7fde40166910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 --2- 172.21.15.16:0/1251170477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde400cf4a0 0x7fde40166910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fde38009bc0 tx=0x7fde3801ca80 comp rx=0 tx=0).stop 2024-09-17T05:02:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 -- 172.21.15.16:0/1251170477 shutdown_connections 2024-09-17T05:02:08.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 --2- 172.21.15.16:0/1251170477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde400cf4a0 0x7fde40166910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 -- 172.21.15.16:0/1251170477 >> 172.21.15.16:0/1251170477 conn(0x7fde400b2250 msgr2=0x7fde400d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 -- 172.21.15.16:0/1251170477 shutdown_connections 2024-09-17T05:02:08.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.522+0000 7fde46400640 1 -- 172.21.15.16:0/1251170477 wait complete. 2024-09-17T05:02:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 Processor -- start 2024-09-17T05:02:08.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 -- start start 2024-09-17T05:02:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 0x7fde4014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde400cf4a0 0x7fde4014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde4015e780 0x7fde401504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde40169b60 con 0x7fde4006e1d0 2024-09-17T05:02:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde401699e0 con 0x7fde4015e780 2024-09-17T05:02:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde46400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fde40169ce0 con 0x7fde400cf4a0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 0x7fde4014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 0x7fde4014de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47950/0 (socket says 172.21.15.16:47950) 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 -- 172.21.15.16:0/1800547344 learned_addr learned my addr 172.21.15.16:0/1800547344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3fe00640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde4015e780 0x7fde401504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3ea00640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde400cf4a0 0x7fde4014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 -- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde400cf4a0 msgr2=0x7fde4014ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde400cf4a0 0x7fde4014ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 -- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde4015e780 msgr2=0x7fde401504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde4015e780 0x7fde401504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 -- 172.21.15.16:0/1800547344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde401fc5e0 con 0x7fde4006e1d0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde3f400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 0x7fde4014de20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fde3000c3c0 tx=0x7fde3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30017070 con 0x7fde4006e1d0 2024-09-17T05:02:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.526+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde300053c0 con 0x7fde4006e1d0 2024-09-17T05:02:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30003860 con 0x7fde4006e1d0 2024-09-17T05:02:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde401fc8c0 con 0x7fde4006e1d0 2024-09-17T05:02:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde401fce80 con 0x7fde4006e1d0 2024-09-17T05:02:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde400d0770 con 0x7fde4006e1d0 2024-09-17T05:02:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fde3001a070 con 0x7fde4006e1d0 2024-09-17T05:02:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde37400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fde20073200 0x7fde20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fde30093830 con 0x7fde4006e1d0 2024-09-17T05:02:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.530+0000 7fde3ea00640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fde20073200 0x7fde20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.534+0000 7fde3ea00640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fde20073200 0x7fde20075710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fde400d0d60 tx=0x7fde2c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.534+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fde3005e620 con 0x7fde4006e1d0 2024-09-17T05:02:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:08.746+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fde400d0ab0 con 0x7fde4006e1d0 2024-09-17T05:02:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.330+0000 7fde37400640 1 -- 172.21.15.16:0/1800547344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fde3005df20 con 0x7fde4006e1d0 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fde20073200 msgr2=0x7fde20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fde20073200 0x7fde20075710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fde400d0d60 tx=0x7fde2c009040 comp rx=0 tx=0).stop 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 msgr2=0x7fde4014de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 0x7fde4014de20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fde3000c3c0 tx=0x7fde3000c880 comp rx=0 tx=0).stop 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 shutdown_connections 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fde20073200 0x7fde20075710 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde4015e780 0x7fde401504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde400cf4a0 0x7fde4014ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 --2- 172.21.15.16:0/1800547344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4006e1d0 0x7fde4014de20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 >> 172.21.15.16:0/1800547344 conn(0x7fde400b2250 msgr2=0x7fde40151050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 shutdown_connections 2024-09-17T05:02:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.334+0000 7fde46400640 1 -- 172.21.15.16:0/1800547344 wait complete. 2024-09-17T05:02:09.362 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:02:09.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 -- 172.21.15.16:0/4175196139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f247815c0a0 msgr2=0x7f247815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 --2- 172.21.15.16:0/4175196139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f247815c0a0 0x7f247815c4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2464009900 tx=0x7f246401c850 comp rx=0 tx=0).stop 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 -- 172.21.15.16:0/4175196139 shutdown_connections 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 --2- 172.21.15.16:0/4175196139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f247815c0a0 0x7f247815c4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 --2- 172.21.15.16:0/4175196139 >> v2:172.21.15.104:3300/0 conn(0x7f24780b2f50 0x7f247815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 -- 172.21.15.16:0/4175196139 >> 172.21.15.16:0/4175196139 conn(0x7f247806f900 msgr2=0x7f24780b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:09.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 -- 172.21.15.16:0/4175196139 shutdown_connections 2024-09-17T05:02:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 -- 172.21.15.16:0/4175196139 wait complete. 2024-09-17T05:02:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 Processor -- start 2024-09-17T05:02:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.526+0000 7f247f400640 1 -- start start 2024-09-17T05:02:09.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24780b2f50 0x7f247813da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 0x7f24781659a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247815f150 0x7f247813d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2478169150 con 0x7f247815c0a0 2024-09-17T05:02:09.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2478168fd0 con 0x7f247815f150 2024-09-17T05:02:09.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f24781692d0 con 0x7f24780b2f50 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 0x7f24781659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 0x7f24781659a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47956/0 (socket says 172.21.15.16:47956) 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 -- 172.21.15.16:0/3751371721 learned_addr learned my addr 172.21.15.16:0/3751371721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247ca00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24780b2f50 0x7f247813da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247d400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247815f150 0x7f247813d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 -- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24780b2f50 msgr2=0x7f247813da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24780b2f50 0x7f247813da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 -- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247815f150 msgr2=0x7f247813d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247815f150 0x7f247813d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 -- 172.21.15.16:0/3751371721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247813df90 con 0x7f247815c0a0 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247ca00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24780b2f50 0x7f247813da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247d400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247815f150 0x7f247813d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:09.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2477e00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 0x7f24781659a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2464009850 tx=0x7f24640040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:09.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24640b2040 con 0x7f247815c0a0 2024-09-17T05:02:09.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2464005020 con 0x7f247815c0a0 2024-09-17T05:02:09.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464005450 con 0x7f247815c0a0 2024-09-17T05:02:09.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24780cf0d0 con 0x7f247815c0a0 2024-09-17T05:02:09.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24780cf690 con 0x7f247815c0a0 2024-09-17T05:02:09.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.530+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247815bb60 con 0x7f247815c0a0 2024-09-17T05:02:09.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.534+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24640046d0 con 0x7f247815c0a0 2024-09-17T05:02:09.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.534+0000 7f2474c00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2458073230 0x7f2458075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.534+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f246412df80 con 0x7f247815c0a0 2024-09-17T05:02:09.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.534+0000 7f247ca00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2458073230 0x7f2458075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:09.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.534+0000 7f247ca00640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2458073230 0x7f2458075740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f246c00bbd0 tx=0x7f246c00b5b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:09.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.538+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f24640164d0 con 0x7f247815c0a0 2024-09-17T05:02:09.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.746+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f247815f770 con 0x7f247815c0a0 2024-09-17T05:02:09.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.746+0000 7f2474c00640 1 -- 172.21.15.16:0/3751371721 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f24640f8d70 con 0x7f247815c0a0 2024-09-17T05:02:09.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:09.748 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:02:09.748 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:02:09.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2458073230 msgr2=0x7f2458075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2458073230 0x7f2458075740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f246c00bbd0 tx=0x7f246c00b5b0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 msgr2=0x7f24781659a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 0x7f24781659a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2464009850 tx=0x7f24640040c0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 shutdown_connections 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2458073230 0x7f2458075740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f247815f150 0x7f247813d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f247815c0a0 0x7f24781659a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 --2- 172.21.15.16:0/3751371721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24780b2f50 0x7f247813da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 >> 172.21.15.16:0/3751371721 conn(0x7f247806f900 msgr2=0x7f24781518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 shutdown_connections 2024-09-17T05:02:09.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.750+0000 7f247f400640 1 -- 172.21.15.16:0/3751371721 wait complete. 2024-09-17T05:02:09.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:02:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 -- 172.21.15.16:0/1592448617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 msgr2=0x7f18e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1592448617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 0x7f18e015bb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009bf0 tx=0x7f18d001ca70 comp rx=0 tx=0).stop 2024-09-17T05:02:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 -- 172.21.15.16:0/1592448617 shutdown_connections 2024-09-17T05:02:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1592448617 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 0x7f18e015bb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 -- 172.21.15.16:0/1592448617 >> 172.21.15.16:0/1592448617 conn(0x7f18e006f900 msgr2=0x7f18e00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 -- 172.21.15.16:0/1592448617 shutdown_connections 2024-09-17T05:02:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.926+0000 7f18e4e00640 1 -- 172.21.15.16:0/1592448617 wait complete. 2024-09-17T05:02:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 Processor -- start 2024-09-17T05:02:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 -- start start 2024-09-17T05:02:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 0x7f18e00b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 0x7f18e00b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18e015c0a0 0x7f18e0161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0169e50 con 0x7f18e0073240 2024-09-17T05:02:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0169cd0 con 0x7f18e015c0a0 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18e4e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f18e0169fd0 con 0x7f18e00b2f50 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 0x7f18e00b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 0x7f18e00b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47972/0 (socket says 172.21.15.16:47972) 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 -- 172.21.15.16:0/1608133595 learned_addr learned my addr 172.21.15.16:0/1608133595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 -- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 msgr2=0x7f18e00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 0x7f18e00b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 -- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18e015c0a0 msgr2=0x7f18e0161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18e015c0a0 0x7f18e0161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.930+0000 7f18dde00640 1 -- 172.21.15.16:0/1608133595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e0160660 con 0x7f18e0073240 2024-09-17T05:02:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18dde00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 0x7f18e00b0a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f18d0019680 tx=0x7f18d001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d00b2040 con 0x7f18e0073240 2024-09-17T05:02:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18d0005040 con 0x7f18e0073240 2024-09-17T05:02:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e0160940 con 0x7f18e0073240 2024-09-17T05:02:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8001700 con 0x7f18e0073240 2024-09-17T05:02:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e0160f00 con 0x7f18e0073240 2024-09-17T05:02:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e00b0500 con 0x7f18e0073240 2024-09-17T05:02:09.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f18d0005490 con 0x7f18e0073240 2024-09-17T05:02:09.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18d6a00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f18bc073200 0x7f18bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:09.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.934+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f18d012ef60 con 0x7f18e0073240 2024-09-17T05:02:09.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.938+0000 7f18d5400640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f18bc073200 0x7f18bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.938+0000 7f18d5400640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f18bc073200 0x7f18bc075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f18c80046c0 tx=0x7f18c80015c0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:09.938+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f18d00c6060 con 0x7f18e0073240 2024-09-17T05:02:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.114+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18e00b0220 con 0x7f18e0073240 2024-09-17T05:02:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18d6a00640 1 -- 172.21.15.16:0/1608133595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16990 (secure 0 0 0) 0x7f18d00166f0 con 0x7f18e0073240 2024-09-17T05:02:10.119 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:10.119 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:02:07.310460+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:02:04.037604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T05:02:04.686294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:02:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f18bc073200 msgr2=0x7f18bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f18bc073200 0x7f18bc075710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f18c80046c0 tx=0x7f18c80015c0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 msgr2=0x7f18e00b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 0x7f18e00b0a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f18d0019680 tx=0x7f18d001ccc0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 shutdown_connections 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f18bc073200 0x7f18bc075710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18e015c0a0 0x7f18e0161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f18e00b2f50 0x7f18e00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 --2- 172.21.15.16:0/1608133595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18e0073240 0x7f18e00b0a20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 >> 172.21.15.16:0/1608133595 conn(0x7f18e006f900 msgr2=0x7f18e0166950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 shutdown_connections 2024-09-17T05:02:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.118+0000 7f18e4e00640 1 -- 172.21.15.16:0/1608133595 wait complete. 2024-09-17T05:02:10.142 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:02:10.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:02:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 -- 172.21.15.16:0/661944594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15c270 msgr2=0x7f28fc15c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 --2- 172.21.15.16:0/661944594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15c270 0x7f28fc15c680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f28fc15cc70 tx=0x7f28e401c830 comp rx=0 tx=0).stop 2024-09-17T05:02:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 -- 172.21.15.16:0/661944594 shutdown_connections 2024-09-17T05:02:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 --2- 172.21.15.16:0/661944594 >> v2:172.21.15.104:3300/0 conn(0x7f28fc073240 0x7f28fc073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 --2- 172.21.15.16:0/661944594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc15c270 0x7f28fc15c680 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 -- 172.21.15.16:0/661944594 >> 172.21.15.16:0/661944594 conn(0x7f28fc06f900 msgr2=0x7f28fc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 -- 172.21.15.16:0/661944594 shutdown_connections 2024-09-17T05:02:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.298+0000 7f2900e00640 1 -- 172.21.15.16:0/661944594 wait complete. 2024-09-17T05:02:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 Processor -- start 2024-09-17T05:02:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 -- start start 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28fc073240 0x7f28fc160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 0x7f28fc164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f28fc15c270 0x7f28fc0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc168d10 con 0x7f28fc0b2f50 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc168b90 con 0x7f28fc073240 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f28fc168e90 con 0x7f28fc15c270 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 0x7f28fc164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 0x7f28fc164940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47982/0 (socket says 172.21.15.16:47982) 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 -- 172.21.15.16:0/3728723540 learned_addr learned my addr 172.21.15.16:0/3728723540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f9e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28fc073240 0x7f28fc160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28fa800640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f28fc15c270 0x7f28fc0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 -- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f28fc15c270 msgr2=0x7f28fc0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f28fc15c270 0x7f28fc0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 -- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28fc073240 msgr2=0x7f28fc160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28fc073240 0x7f28fc160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 -- 172.21.15.16:0/3728723540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc142bf0 con 0x7f28fc0b2f50 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28fa800640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f28fc15c270 0x7f28fc0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f3e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 0x7f28fc164940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28e40b0040 tx=0x7f28e4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40bb070 con 0x7f28fc0b2f50 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28e4005030 con 0x7f28fc0b2f50 2024-09-17T05:02:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40b2040 con 0x7f28fc0b2f50 2024-09-17T05:02:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc142ed0 con 0x7f28fc0b2f50 2024-09-17T05:02:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.302+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc163e10 con 0x7f28fc0b2f50 2024-09-17T05:02:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.306+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc073650 con 0x7f28fc0b2f50 2024-09-17T05:02:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.306+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f28e4005460 con 0x7f28fc0b2f50 2024-09-17T05:02:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.306+0000 7f28f2000640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f28d8073200 0x7f28d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.306+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f28e412d930 con 0x7f28fc0b2f50 2024-09-17T05:02:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.306+0000 7f28f9e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f28d8073200 0x7f28d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.310+0000 7f28f9e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f28d8073200 0x7f28d8075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28ec007fd0 tx=0x7f28ec009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:10.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.310+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f28e40164b0 con 0x7f28fc0b2f50 2024-09-17T05:02:10.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.486+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28fc0ce680 con 0x7f28d8073200 2024-09-17T05:02:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.490+0000 7f28f2000640 1 -- 172.21.15.16:0/3728723540 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251694 (secure 0 0 0) 0x7f28fc0ce680 con 0x7f28d8073200 2024-09-17T05:02:10.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:10.498 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:02:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f28d8073200 msgr2=0x7f28d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f28d8073200 0x7f28d8075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28ec007fd0 tx=0x7f28ec009040 comp rx=0 tx=0).stop 2024-09-17T05:02:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 msgr2=0x7f28fc164940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 0x7f28fc164940 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f28e40b0040 tx=0x7f28e4004040 comp rx=0 tx=0).stop 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 shutdown_connections 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f28d8073200 0x7f28d8075710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f28fc15c270 0x7f28fc0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28fc0b2f50 0x7f28fc164940 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 --2- 172.21.15.16:0/3728723540 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28fc073240 0x7f28fc160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 >> 172.21.15.16:0/3728723540 conn(0x7f28fc06f900 msgr2=0x7f28fc0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 shutdown_connections 2024-09-17T05:02:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.494+0000 7f2900e00640 1 -- 172.21.15.16:0/3728723540 wait complete. 2024-09-17T05:02:10.518 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-17T05:02:09.476186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237588,"kb_used_data":4676,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737132,"statfs":{"total":773094113280,"available":772850823168,"internally_reserved":0,"allocated":4788224,"data_stored":2693931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181300"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321351+0000","last_change":"2024-09-17T05:02:06.338525+0000","last_active":"2024-09-17T05:02:07.321351+0000","last_peered":"2024-09-17T05:02:07.321351+0000","last_clean":"2024-09-17T05:02:07.321351+0000","last_became_active":"2024-09-17T05:02:06.338232+0000","last_became_peered":"2024-09-17T05:02:06.338232+0000","last_unstale":"2024-09-17T05:02:07.321351+0000","last_undegraded":"2024-09-17T05:02:07.321351+0000","last_fullsized":"2024-09-17T05:02:07.321351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:56.646622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420620+0000","last_change":"2024-09-17T05:02:06.332250+0000","last_active":"2024-09-17T05:02:07.420620+0000","last_peered":"2024-09-17T05:02:07.420620+0000","last_clean":"2024-09-17T05:02:07.420620+0000","last_became_active":"2024-09-17T05:02:06.331927+0000","last_became_peered":"2024-09-17T05:02:06.331927+0000","last_unstale":"2024-09-17T05:02:07.420620+0000","last_undegraded":"2024-09-17T05:02:07.420620+0000","last_fullsized":"2024-09-17T05:02:07.420620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320879+0000","last_change":"2024-09-17T05:02:06.336648+0000","last_active":"2024-09-17T05:02:07.320879+0000","last_peered":"2024-09-17T05:02:07.320879+0000","last_clean":"2024-09-17T05:02:07.320879+0000","last_became_active":"2024-09-17T05:02:06.336337+0000","last_became_peered":"2024-09-17T05:02:06.336337+0000","last_unstale":"2024-09-17T05:02:07.320879+0000","last_undegraded":"2024-09-17T05:02:07.320879+0000","last_fullsized":"2024-09-17T05:02:07.320879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:44.810456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321127+0000","last_change":"2024-09-17T05:02:06.348672+0000","last_active":"2024-09-17T05:02:07.321127+0000","last_peered":"2024-09-17T05:02:07.321127+0000","last_clean":"2024-09-17T05:02:07.321127+0000","last_became_active":"2024-09-17T05:02:06.348392+0000","last_became_peered":"2024-09-17T05:02:06.348392+0000","last_unstale":"2024-09-17T05:02:07.321127+0000","last_undegraded":"2024-09-17T05:02:07.321127+0000","last_fullsized":"2024-09-17T05:02:07.321127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:06.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346718+0000","last_change":"2024-09-17T05:02:06.343632+0000","last_active":"2024-09-17T05:02:07.346718+0000","last_peered":"2024-09-17T05:02:07.346718+0000","last_clean":"2024-09-17T05:02:07.346718+0000","last_became_active":"2024-09-17T05:02:06.343257+0000","last_became_peered":"2024-09-17T05:02:06.343257+0000","last_unstale":"2024-09-17T05:02:07.346718+0000","last_undegraded":"2024-09-17T05:02:07.346718+0000","last_fullsized":"2024-09-17T05:02:07.346718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:00.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420107+0000","last_change":"2024-09-17T05:02:06.330126+0000","last_active":"2024-09-17T05:02:07.420107+0000","last_peered":"2024-09-17T05:02:07.420107+0000","last_clean":"2024-09-17T05:02:07.420107+0000","last_became_active":"2024-09-17T05:02:06.329825+0000","last_became_peered":"2024-09-17T05:02:06.329825+0000","last_unstale":"2024-09-17T05:02:07.420107+0000","last_undegraded":"2024-09-17T05:02:07.420107+0000","last_fullsized":"2024-09-17T05:02:07.420107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:46.368942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420406+0000","last_change":"2024-09-17T05:02:06.343766+0000","last_active":"2024-09-17T05:02:07.420406+0000","last_peered":"2024-09-17T05:02:07.420406+0000","last_clean":"2024-09-17T05:02:07.420406+0000","last_became_active":"2024-09-17T05:02:06.343207+0000","last_became_peered":"2024-09-17T05:02:06.343207+0000","last_unstale":"2024-09-17T05:02:07.420406+0000","last_undegraded":"2024-09-17T05:02:07.420406+0000","last_fullsized":"2024-09-17T05:02:07.420406+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:51.030820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320027+0000","last_change":"2024-09-17T05:02:06.338244+0000","last_active":"2024-09-17T05:02:07.320027+0000","last_peered":"2024-09-17T05:02:07.320027+0000","last_clean":"2024-09-17T05:02:07.320027+0000","last_became_active":"2024-09-17T05:02:06.337859+0000","last_became_peered":"2024-09-17T05:02:06.337859+0000","last_unstale":"2024-09-17T05:02:07.320027+0000","last_undegraded":"2024-09-17T05:02:07.320027+0000","last_fullsized":"2024-09-17T05:02:07.320027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:36.957053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.347110+0000","last_change":"2024-09-17T05:02:06.336573+0000","last_active":"2024-09-17T05:02:07.347110+0000","last_peered":"2024-09-17T05:02:07.347110+0000","last_clean":"2024-09-17T05:02:07.347110+0000","last_became_active":"2024-09-17T05:02:06.336253+0000","last_became_peered":"2024-09-17T05:02:06.336253+0000","last_unstale":"2024-09-17T05:02:07.347110+0000","last_undegraded":"2024-09-17T05:02:07.347110+0000","last_fullsized":"2024-09-17T05:02:07.347110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:47.955020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420328+0000","last_change":"2024-09-17T05:02:06.362653+0000","last_active":"2024-09-17T05:02:07.420328+0000","last_peered":"2024-09-17T05:02:07.420328+0000","last_clean":"2024-09-17T05:02:07.420328+0000","last_became_active":"2024-09-17T05:02:06.362345+0000","last_became_peered":"2024-09-17T05:02:06.362345+0000","last_unstale":"2024-09-17T05:02:07.420328+0000","last_undegraded":"2024-09-17T05:02:07.420328+0000","last_fullsized":"2024-09-17T05:02:07.420328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:12.091714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350026+0000","last_change":"2024-09-17T05:02:06.341127+0000","last_active":"2024-09-17T05:02:07.350026+0000","last_peered":"2024-09-17T05:02:07.350026+0000","last_clean":"2024-09-17T05:02:07.350026+0000","last_became_active":"2024-09-17T05:02:06.340796+0000","last_became_peered":"2024-09-17T05:02:06.340796+0000","last_unstale":"2024-09-17T05:02:07.350026+0000","last_undegraded":"2024-09-17T05:02:07.350026+0000","last_fullsized":"2024-09-17T05:02:07.350026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:54.880594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346735+0000","last_change":"2024-09-17T05:02:06.336217+0000","last_active":"2024-09-17T05:02:07.346735+0000","last_peered":"2024-09-17T05:02:07.346735+0000","last_clean":"2024-09-17T05:02:07.346735+0000","last_became_active":"2024-09-17T05:02:06.333449+0000","last_became_peered":"2024-09-17T05:02:06.333449+0000","last_unstale":"2024-09-17T05:02:07.346735+0000","last_undegraded":"2024-09-17T05:02:07.346735+0000","last_fullsized":"2024-09-17T05:02:07.346735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:23.096982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420312+0000","last_change":"2024-09-17T05:02:06.330216+0000","last_active":"2024-09-17T05:02:07.420312+0000","last_peered":"2024-09-17T05:02:07.420312+0000","last_clean":"2024-09-17T05:02:07.420312+0000","last_became_active":"2024-09-17T05:02:06.329869+0000","last_became_peered":"2024-09-17T05:02:06.329869+0000","last_unstale":"2024-09-17T05:02:07.420312+0000","last_undegraded":"2024-09-17T05:02:07.420312+0000","last_fullsized":"2024-09-17T05:02:07.420312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419668+0000","last_change":"2024-09-17T05:02:06.337107+0000","last_active":"2024-09-17T05:02:07.419668+0000","last_peered":"2024-09-17T05:02:07.419668+0000","last_clean":"2024-09-17T05:02:07.419668+0000","last_became_active":"2024-09-17T05:02:06.336774+0000","last_became_peered":"2024-09-17T05:02:06.336774+0000","last_unstale":"2024-09-17T05:02:07.419668+0000","last_undegraded":"2024-09-17T05:02:07.419668+0000","last_fullsized":"2024-09-17T05:02:07.419668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:26.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314512+0000","last_change":"2024-09-17T05:02:06.321133+0000","last_active":"2024-09-17T05:02:07.314512+0000","last_peered":"2024-09-17T05:02:07.314512+0000","last_clean":"2024-09-17T05:02:07.314512+0000","last_became_active":"2024-09-17T05:02:06.320714+0000","last_became_peered":"2024-09-17T05:02:06.320714+0000","last_unstale":"2024-09-17T05:02:07.314512+0000","last_undegraded":"2024-09-17T05:02:07.314512+0000","last_fullsized":"2024-09-17T05:02:07.314512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:18.256704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420854+0000","last_change":"2024-09-17T05:02:06.335127+0000","last_active":"2024-09-17T05:02:07.420854+0000","last_peered":"2024-09-17T05:02:07.420854+0000","last_clean":"2024-09-17T05:02:07.420854+0000","last_became_active":"2024-09-17T05:02:06.334693+0000","last_became_peered":"2024-09-17T05:02:06.334693+0000","last_unstale":"2024-09-17T05:02:07.420854+0000","last_undegraded":"2024-09-17T05:02:07.420854+0000","last_fullsized":"2024-09-17T05:02:07.420854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:20.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315318+0000","last_change":"2024-09-17T05:02:06.352349+0000","last_active":"2024-09-17T05:02:07.315318+0000","last_peered":"2024-09-17T05:02:07.315318+0000","last_clean":"2024-09-17T05:02:07.315318+0000","last_became_active":"2024-09-17T05:02:06.352010+0000","last_became_peered":"2024-09-17T05:02:06.352010+0000","last_unstale":"2024-09-17T05:02:07.315318+0000","last_undegraded":"2024-09-17T05:02:07.315318+0000","last_fullsized":"2024-09-17T05:02:07.315318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.149726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314938+0000","last_change":"2024-09-17T05:02:06.336278+0000","last_active":"2024-09-17T05:02:07.314938+0000","last_peered":"2024-09-17T05:02:07.314938+0000","last_clean":"2024-09-17T05:02:07.314938+0000","last_became_active":"2024-09-17T05:02:06.334917+0000","last_became_peered":"2024-09-17T05:02:06.334917+0000","last_unstale":"2024-09-17T05:02:07.314938+0000","last_undegraded":"2024-09-17T05:02:07.314938+0000","last_fullsized":"2024-09-17T05:02:07.314938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:51.224167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.314200+0000","last_change":"2024-09-17T05:02:05.314200+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.314200+0000","last_undegraded":"2024-09-17T05:02:05.314200+0000","last_fullsized":"2024-09-17T05:02:05.314200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349734+0000","last_change":"2024-09-17T05:02:06.353747+0000","last_active":"2024-09-17T05:02:07.349734+0000","last_peered":"2024-09-17T05:02:07.349734+0000","last_clean":"2024-09-17T05:02:07.349734+0000","last_became_active":"2024-09-17T05:02:06.353157+0000","last_became_peered":"2024-09-17T05:02:06.353157+0000","last_unstale":"2024-09-17T05:02:07.349734+0000","last_undegraded":"2024-09-17T05:02:07.349734+0000","last_fullsized":"2024-09-17T05:02:07.349734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349652+0000","last_change":"2024-09-17T05:02:06.352255+0000","last_active":"2024-09-17T05:02:07.349652+0000","last_peered":"2024-09-17T05:02:07.349652+0000","last_clean":"2024-09-17T05:02:07.349652+0000","last_became_active":"2024-09-17T05:02:06.351865+0000","last_became_peered":"2024-09-17T05:02:06.351865+0000","last_unstale":"2024-09-17T05:02:07.349652+0000","last_undegraded":"2024-09-17T05:02:07.349652+0000","last_fullsized":"2024-09-17T05:02:07.349652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:00.377367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349479+0000","last_change":"2024-09-17T05:02:06.347103+0000","last_active":"2024-09-17T05:02:07.349479+0000","last_peered":"2024-09-17T05:02:07.349479+0000","last_clean":"2024-09-17T05:02:07.349479+0000","last_became_active":"2024-09-17T05:02:06.346592+0000","last_became_peered":"2024-09-17T05:02:06.346592+0000","last_unstale":"2024-09-17T05:02:07.349479+0000","last_undegraded":"2024-09-17T05:02:07.349479+0000","last_fullsized":"2024-09-17T05:02:07.349479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314660+0000","last_change":"2024-09-17T05:02:06.334073+0000","last_active":"2024-09-17T05:02:07.314660+0000","last_peered":"2024-09-17T05:02:07.314660+0000","last_clean":"2024-09-17T05:02:07.314660+0000","last_became_active":"2024-09-17T05:02:06.333712+0000","last_became_peered":"2024-09-17T05:02:06.333712+0000","last_unstale":"2024-09-17T05:02:07.314660+0000","last_undegraded":"2024-09-17T05:02:07.314660+0000","last_fullsized":"2024-09-17T05:02:07.314660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:51.877414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321024+0000","last_change":"2024-09-17T05:02:06.345627+0000","last_active":"2024-09-17T05:02:07.321024+0000","last_peered":"2024-09-17T05:02:07.321024+0000","last_clean":"2024-09-17T05:02:07.321024+0000","last_became_active":"2024-09-17T05:02:06.345225+0000","last_became_peered":"2024-09-17T05:02:06.345225+0000","last_unstale":"2024-09-17T05:02:07.321024+0000","last_undegraded":"2024-09-17T05:02:07.321024+0000","last_fullsized":"2024-09-17T05:02:07.321024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:59.047797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420455+0000","last_change":"2024-09-17T05:02:06.338993+0000","last_active":"2024-09-17T05:02:07.420455+0000","last_peered":"2024-09-17T05:02:07.420455+0000","last_clean":"2024-09-17T05:02:07.420455+0000","last_became_active":"2024-09-17T05:02:06.337798+0000","last_became_peered":"2024-09-17T05:02:06.337798+0000","last_unstale":"2024-09-17T05:02:07.420455+0000","last_undegraded":"2024-09-17T05:02:07.420455+0000","last_fullsized":"2024-09-17T05:02:07.420455+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:01.425057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320694+0000","last_change":"2024-09-17T05:02:06.350077+0000","last_active":"2024-09-17T05:02:07.320694+0000","last_peered":"2024-09-17T05:02:07.320694+0000","last_clean":"2024-09-17T05:02:07.320694+0000","last_became_active":"2024-09-17T05:02:06.349773+0000","last_became_peered":"2024-09-17T05:02:06.349773+0000","last_unstale":"2024-09-17T05:02:07.320694+0000","last_undegraded":"2024-09-17T05:02:07.320694+0000","last_fullsized":"2024-09-17T05:02:07.320694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:35.430354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350145+0000","last_change":"2024-09-17T05:02:06.332912+0000","last_active":"2024-09-17T05:02:07.350145+0000","last_peered":"2024-09-17T05:02:07.350145+0000","last_clean":"2024-09-17T05:02:07.350145+0000","last_became_active":"2024-09-17T05:02:06.332354+0000","last_became_peered":"2024-09-17T05:02:06.332354+0000","last_unstale":"2024-09-17T05:02:07.350145+0000","last_undegraded":"2024-09-17T05:02:07.350145+0000","last_fullsized":"2024-09-17T05:02:07.350145+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:22.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420872+0000","last_change":"2024-09-17T05:02:06.335589+0000","last_active":"2024-09-17T05:02:07.420872+0000","last_peered":"2024-09-17T05:02:07.420872+0000","last_clean":"2024-09-17T05:02:07.420872+0000","last_became_active":"2024-09-17T05:02:06.335120+0000","last_became_peered":"2024-09-17T05:02:06.335120+0000","last_unstale":"2024-09-17T05:02:07.420872+0000","last_undegraded":"2024-09-17T05:02:07.420872+0000","last_fullsized":"2024-09-17T05:02:07.420872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:06.399066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314944+0000","last_change":"2024-09-17T05:02:06.331500+0000","last_active":"2024-09-17T05:02:07.314944+0000","last_peered":"2024-09-17T05:02:07.314944+0000","last_clean":"2024-09-17T05:02:07.314944+0000","last_became_active":"2024-09-17T05:02:06.331184+0000","last_became_peered":"2024-09-17T05:02:06.331184+0000","last_unstale":"2024-09-17T05:02:07.314944+0000","last_undegraded":"2024-09-17T05:02:07.314944+0000","last_fullsized":"2024-09-17T05:02:07.314944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:48.958710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314593+0000","last_change":"2024-09-17T05:02:06.330526+0000","last_active":"2024-09-17T05:02:07.314593+0000","last_peered":"2024-09-17T05:02:07.314593+0000","last_clean":"2024-09-17T05:02:07.314593+0000","last_became_active":"2024-09-17T05:02:06.330156+0000","last_became_peered":"2024-09-17T05:02:06.330156+0000","last_unstale":"2024-09-17T05:02:07.314593+0000","last_undegraded":"2024-09-17T05:02:07.314593+0000","last_fullsized":"2024-09-17T05:02:07.314593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420681+0000","last_change":"2024-09-17T05:02:06.356417+0000","last_active":"2024-09-17T05:02:07.420681+0000","last_peered":"2024-09-17T05:02:07.420681+0000","last_clean":"2024-09-17T05:02:07.420681+0000","last_became_active":"2024-09-17T05:02:06.342863+0000","last_became_peered":"2024-09-17T05:02:06.342863+0000","last_unstale":"2024-09-17T05:02:07.420681+0000","last_undegraded":"2024-09-17T05:02:07.420681+0000","last_fullsized":"2024-09-17T05:02:07.420681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:46.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320315+0000","last_change":"2024-09-17T05:02:06.337797+0000","last_active":"2024-09-17T05:02:07.320315+0000","last_peered":"2024-09-17T05:02:07.320315+0000","last_clean":"2024-09-17T05:02:07.320315+0000","last_became_active":"2024-09-17T05:02:06.337396+0000","last_became_peered":"2024-09-17T05:02:06.337396+0000","last_unstale":"2024-09-17T05:02:07.320315+0000","last_undegraded":"2024-09-17T05:02:07.320315+0000","last_fullsized":"2024-09-17T05:02:07.320315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:42.616845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315319+0000","last_change":"2024-09-17T05:02:06.348634+0000","last_active":"2024-09-17T05:02:07.315319+0000","last_peered":"2024-09-17T05:02:07.315319+0000","last_clean":"2024-09-17T05:02:07.315319+0000","last_became_active":"2024-09-17T05:02:06.348284+0000","last_became_peered":"2024-09-17T05:02:06.348284+0000","last_unstale":"2024-09-17T05:02:07.315319+0000","last_undegraded":"2024-09-17T05:02:07.315319+0000","last_fullsized":"2024-09-17T05:02:07.315319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:41.455558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420652+0000","last_change":"2024-09-17T05:02:05.319632+0000","last_active":"2024-09-17T05:02:07.420652+0000","last_peered":"2024-09-17T05:02:07.420652+0000","last_clean":"2024-09-17T05:02:07.420652+0000","last_became_active":"2024-09-17T05:02:05.319321+0000","last_became_peered":"2024-09-17T05:02:05.319321+0000","last_unstale":"2024-09-17T05:02:07.420652+0000","last_undegraded":"2024-09-17T05:02:07.420652+0000","last_fullsized":"2024-09-17T05:02:07.420652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:46.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420445+0000","last_change":"2024-09-17T05:02:06.334112+0000","last_active":"2024-09-17T05:02:07.420445+0000","last_peered":"2024-09-17T05:02:07.420445+0000","last_clean":"2024-09-17T05:02:07.420445+0000","last_became_active":"2024-09-17T05:02:06.333727+0000","last_became_peered":"2024-09-17T05:02:06.333727+0000","last_unstale":"2024-09-17T05:02:07.420445+0000","last_undegraded":"2024-09-17T05:02:07.420445+0000","last_fullsized":"2024-09-17T05:02:07.420445+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:48.515835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420440+0000","last_change":"2024-09-17T05:02:05.330941+0000","last_active":"2024-09-17T05:02:07.420440+0000","last_peered":"2024-09-17T05:02:07.420440+0000","last_clean":"2024-09-17T05:02:07.420440+0000","last_became_active":"2024-09-17T05:02:05.330610+0000","last_became_peered":"2024-09-17T05:02:05.330610+0000","last_unstale":"2024-09-17T05:02:07.420440+0000","last_undegraded":"2024-09-17T05:02:07.420440+0000","last_fullsized":"2024-09-17T05:02:07.420440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:35.977474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315276+0000","last_change":"2024-09-17T05:02:06.367617+0000","last_active":"2024-09-17T05:02:07.315276+0000","last_peered":"2024-09-17T05:02:07.315276+0000","last_clean":"2024-09-17T05:02:07.315276+0000","last_became_active":"2024-09-17T05:02:06.367102+0000","last_became_peered":"2024-09-17T05:02:06.367102+0000","last_unstale":"2024-09-17T05:02:07.315276+0000","last_undegraded":"2024-09-17T05:02:07.315276+0000","last_fullsized":"2024-09-17T05:02:07.315276+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.094882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321076+0000","last_change":"2024-09-17T05:02:05.321020+0000","last_active":"2024-09-17T05:02:07.321076+0000","last_peered":"2024-09-17T05:02:07.321076+0000","last_clean":"2024-09-17T05:02:07.321076+0000","last_became_active":"2024-09-17T05:02:05.320171+0000","last_became_peered":"2024-09-17T05:02:05.320171+0000","last_unstale":"2024-09-17T05:02:07.321076+0000","last_undegraded":"2024-09-17T05:02:07.321076+0000","last_fullsized":"2024-09-17T05:02:07.321076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:00.418844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420627+0000","last_change":"2024-09-17T05:02:06.348099+0000","last_active":"2024-09-17T05:02:07.420627+0000","last_peered":"2024-09-17T05:02:07.420627+0000","last_clean":"2024-09-17T05:02:07.420627+0000","last_became_active":"2024-09-17T05:02:06.347767+0000","last_became_peered":"2024-09-17T05:02:06.347767+0000","last_unstale":"2024-09-17T05:02:07.420627+0000","last_undegraded":"2024-09-17T05:02:07.420627+0000","last_fullsized":"2024-09-17T05:02:07.420627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:39.458764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420697+0000","last_change":"2024-09-17T05:02:05.323652+0000","last_active":"2024-09-17T05:02:07.420697+0000","last_peered":"2024-09-17T05:02:07.420697+0000","last_clean":"2024-09-17T05:02:07.420697+0000","last_became_active":"2024-09-17T05:02:05.323250+0000","last_became_peered":"2024-09-17T05:02:05.323250+0000","last_unstale":"2024-09-17T05:02:07.420697+0000","last_undegraded":"2024-09-17T05:02:07.420697+0000","last_fullsized":"2024-09-17T05:02:07.420697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:46.948808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320590+0000","last_change":"2024-09-17T05:02:06.332617+0000","last_active":"2024-09-17T05:02:07.320590+0000","last_peered":"2024-09-17T05:02:07.320590+0000","last_clean":"2024-09-17T05:02:07.320590+0000","last_became_active":"2024-09-17T05:02:06.330747+0000","last_became_peered":"2024-09-17T05:02:06.330747+0000","last_unstale":"2024-09-17T05:02:07.320590+0000","last_undegraded":"2024-09-17T05:02:07.320590+0000","last_fullsized":"2024-09-17T05:02:07.320590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:51.850349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420085+0000","last_change":"2024-09-17T05:02:05.313259+0000","last_active":"2024-09-17T05:02:07.420085+0000","last_peered":"2024-09-17T05:02:07.420085+0000","last_clean":"2024-09-17T05:02:07.420085+0000","last_became_active":"2024-09-17T05:02:05.312893+0000","last_became_peered":"2024-09-17T05:02:05.312893+0000","last_unstale":"2024-09-17T05:02:07.420085+0000","last_undegraded":"2024-09-17T05:02:07.420085+0000","last_fullsized":"2024-09-17T05:02:07.420085+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350174+0000","last_change":"2024-09-17T05:02:06.333044+0000","last_active":"2024-09-17T05:02:07.350174+0000","last_peered":"2024-09-17T05:02:07.350174+0000","last_clean":"2024-09-17T05:02:07.350174+0000","last_became_active":"2024-09-17T05:02:06.332672+0000","last_became_peered":"2024-09-17T05:02:06.332672+0000","last_unstale":"2024-09-17T05:02:07.350174+0000","last_undegraded":"2024-09-17T05:02:07.350174+0000","last_fullsized":"2024-09-17T05:02:07.350174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:28.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419821+0000","last_change":"2024-09-17T05:02:05.331066+0000","last_active":"2024-09-17T05:02:07.419821+0000","last_peered":"2024-09-17T05:02:07.419821+0000","last_clean":"2024-09-17T05:02:07.419821+0000","last_became_active":"2024-09-17T05:02:05.330727+0000","last_became_peered":"2024-09-17T05:02:05.330727+0000","last_unstale":"2024-09-17T05:02:07.419821+0000","last_undegraded":"2024-09-17T05:02:07.419821+0000","last_fullsized":"2024-09-17T05:02:07.419821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314674+0000","last_change":"2024-09-17T05:02:06.350254+0000","last_active":"2024-09-17T05:02:07.314674+0000","last_peered":"2024-09-17T05:02:07.314674+0000","last_clean":"2024-09-17T05:02:07.314674+0000","last_became_active":"2024-09-17T05:02:06.349926+0000","last_became_peered":"2024-09-17T05:02:06.349926+0000","last_unstale":"2024-09-17T05:02:07.314674+0000","last_undegraded":"2024-09-17T05:02:07.314674+0000","last_fullsized":"2024-09-17T05:02:07.314674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320007+0000","last_change":"2024-09-17T05:02:05.330770+0000","last_active":"2024-09-17T05:02:07.320007+0000","last_peered":"2024-09-17T05:02:07.320007+0000","last_clean":"2024-09-17T05:02:07.320007+0000","last_became_active":"2024-09-17T05:02:05.330452+0000","last_became_peered":"2024-09-17T05:02:05.330452+0000","last_unstale":"2024-09-17T05:02:07.320007+0000","last_undegraded":"2024-09-17T05:02:07.320007+0000","last_fullsized":"2024-09-17T05:02:07.320007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420577+0000","last_change":"2024-09-17T05:02:06.339321+0000","last_active":"2024-09-17T05:02:07.420577+0000","last_peered":"2024-09-17T05:02:07.420577+0000","last_clean":"2024-09-17T05:02:07.420577+0000","last_became_active":"2024-09-17T05:02:06.339039+0000","last_became_peered":"2024-09-17T05:02:06.339039+0000","last_unstale":"2024-09-17T05:02:07.420577+0000","last_undegraded":"2024-09-17T05:02:07.420577+0000","last_fullsized":"2024-09-17T05:02:07.420577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420407+0000","last_change":"2024-09-17T05:02:05.334416+0000","last_active":"2024-09-17T05:02:07.420407+0000","last_peered":"2024-09-17T05:02:07.420407+0000","last_clean":"2024-09-17T05:02:07.420407+0000","last_became_active":"2024-09-17T05:02:05.334066+0000","last_became_peered":"2024-09-17T05:02:05.334066+0000","last_unstale":"2024-09-17T05:02:07.420407+0000","last_undegraded":"2024-09-17T05:02:07.420407+0000","last_fullsized":"2024-09-17T05:02:07.420407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:44.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321356+0000","last_change":"2024-09-17T05:02:06.348674+0000","last_active":"2024-09-17T05:02:07.321356+0000","last_peered":"2024-09-17T05:02:07.321356+0000","last_clean":"2024-09-17T05:02:07.321356+0000","last_became_active":"2024-09-17T05:02:06.348346+0000","last_became_peered":"2024-09-17T05:02:06.348346+0000","last_unstale":"2024-09-17T05:02:07.321356+0000","last_undegraded":"2024-09-17T05:02:07.321356+0000","last_fullsized":"2024-09-17T05:02:07.321356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:39.881805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320775+0000","last_change":"2024-09-17T05:02:05.323997+0000","last_active":"2024-09-17T05:02:07.320775+0000","last_peered":"2024-09-17T05:02:07.320775+0000","last_clean":"2024-09-17T05:02:07.320775+0000","last_became_active":"2024-09-17T05:02:05.323410+0000","last_became_peered":"2024-09-17T05:02:05.323410+0000","last_unstale":"2024-09-17T05:02:07.320775+0000","last_undegraded":"2024-09-17T05:02:07.320775+0000","last_fullsized":"2024-09-17T05:02:07.320775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:32.508695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420275+0000","last_change":"2024-09-17T05:02:06.338660+0000","last_active":"2024-09-17T05:02:07.420275+0000","last_peered":"2024-09-17T05:02:07.420275+0000","last_clean":"2024-09-17T05:02:07.420275+0000","last_became_active":"2024-09-17T05:02:06.338256+0000","last_became_peered":"2024-09-17T05:02:06.338256+0000","last_unstale":"2024-09-17T05:02:07.420275+0000","last_undegraded":"2024-09-17T05:02:07.420275+0000","last_fullsized":"2024-09-17T05:02:07.420275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.216527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315076+0000","last_change":"2024-09-17T05:02:05.326255+0000","last_active":"2024-09-17T05:02:07.315076+0000","last_peered":"2024-09-17T05:02:07.315076+0000","last_clean":"2024-09-17T05:02:07.315076+0000","last_became_active":"2024-09-17T05:02:05.325945+0000","last_became_peered":"2024-09-17T05:02:05.325945+0000","last_unstale":"2024-09-17T05:02:07.315076+0000","last_undegraded":"2024-09-17T05:02:07.315076+0000","last_fullsized":"2024-09-17T05:02:07.315076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:21.224964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320746+0000","last_change":"2024-09-17T05:02:06.335521+0000","last_active":"2024-09-17T05:02:07.320746+0000","last_peered":"2024-09-17T05:02:07.320746+0000","last_clean":"2024-09-17T05:02:07.320746+0000","last_became_active":"2024-09-17T05:02:06.335166+0000","last_became_peered":"2024-09-17T05:02:06.335166+0000","last_unstale":"2024-09-17T05:02:07.320746+0000","last_undegraded":"2024-09-17T05:02:07.320746+0000","last_fullsized":"2024-09-17T05:02:07.320746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:02.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314664+0000","last_change":"2024-09-17T05:02:05.336301+0000","last_active":"2024-09-17T05:02:07.314664+0000","last_peered":"2024-09-17T05:02:07.314664+0000","last_clean":"2024-09-17T05:02:07.314664+0000","last_became_active":"2024-09-17T05:02:05.331553+0000","last_became_peered":"2024-09-17T05:02:05.331553+0000","last_unstale":"2024-09-17T05:02:07.314664+0000","last_undegraded":"2024-09-17T05:02:07.314664+0000","last_fullsized":"2024-09-17T05:02:07.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:23.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420226+0000","last_change":"2024-09-17T05:02:06.342104+0000","last_active":"2024-09-17T05:02:07.420226+0000","last_peered":"2024-09-17T05:02:07.420226+0000","last_clean":"2024-09-17T05:02:07.420226+0000","last_became_active":"2024-09-17T05:02:06.341775+0000","last_became_peered":"2024-09-17T05:02:06.341775+0000","last_unstale":"2024-09-17T05:02:07.420226+0000","last_undegraded":"2024-09-17T05:02:07.420226+0000","last_fullsized":"2024-09-17T05:02:07.420226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:24.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321552+0000","last_change":"2024-09-17T05:02:05.323758+0000","last_active":"2024-09-17T05:02:07.321552+0000","last_peered":"2024-09-17T05:02:07.321552+0000","last_clean":"2024-09-17T05:02:07.321552+0000","last_became_active":"2024-09-17T05:02:05.323426+0000","last_became_peered":"2024-09-17T05:02:05.323426+0000","last_unstale":"2024-09-17T05:02:07.321552+0000","last_undegraded":"2024-09-17T05:02:07.321552+0000","last_fullsized":"2024-09-17T05:02:07.321552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:29.210315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320848+0000","last_change":"2024-09-17T05:02:06.345826+0000","last_active":"2024-09-17T05:02:07.320848+0000","last_peered":"2024-09-17T05:02:07.320848+0000","last_clean":"2024-09-17T05:02:07.320848+0000","last_became_active":"2024-09-17T05:02:06.345343+0000","last_became_peered":"2024-09-17T05:02:06.345343+0000","last_unstale":"2024-09-17T05:02:07.320848+0000","last_undegraded":"2024-09-17T05:02:07.320848+0000","last_fullsized":"2024-09-17T05:02:07.320848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:09.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320977+0000","last_change":"2024-09-17T05:02:05.323105+0000","last_active":"2024-09-17T05:02:07.320977+0000","last_peered":"2024-09-17T05:02:07.320977+0000","last_clean":"2024-09-17T05:02:07.320977+0000","last_became_active":"2024-09-17T05:02:05.320585+0000","last_became_peered":"2024-09-17T05:02:05.320585+0000","last_unstale":"2024-09-17T05:02:07.320977+0000","last_undegraded":"2024-09-17T05:02:07.320977+0000","last_fullsized":"2024-09-17T05:02:07.320977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420267+0000","last_change":"2024-09-17T05:02:06.359915+0000","last_active":"2024-09-17T05:02:07.420267+0000","last_peered":"2024-09-17T05:02:07.420267+0000","last_clean":"2024-09-17T05:02:07.420267+0000","last_became_active":"2024-09-17T05:02:06.359544+0000","last_became_peered":"2024-09-17T05:02:06.359544+0000","last_unstale":"2024-09-17T05:02:07.420267+0000","last_undegraded":"2024-09-17T05:02:07.420267+0000","last_fullsized":"2024-09-17T05:02:07.420267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420061+0000","last_change":"2024-09-17T05:02:05.320829+0000","last_active":"2024-09-17T05:02:07.420061+0000","last_peered":"2024-09-17T05:02:07.420061+0000","last_clean":"2024-09-17T05:02:07.420061+0000","last_became_active":"2024-09-17T05:02:05.320509+0000","last_became_peered":"2024-09-17T05:02:05.320509+0000","last_unstale":"2024-09-17T05:02:07.420061+0000","last_undegraded":"2024-09-17T05:02:07.420061+0000","last_fullsized":"2024-09-17T05:02:07.420061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:36.381090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349878+0000","last_change":"2024-09-17T05:02:06.342348+0000","last_active":"2024-09-17T05:02:07.349878+0000","last_peered":"2024-09-17T05:02:07.349878+0000","last_clean":"2024-09-17T05:02:07.349878+0000","last_became_active":"2024-09-17T05:02:06.342077+0000","last_became_peered":"2024-09-17T05:02:06.342077+0000","last_unstale":"2024-09-17T05:02:07.349878+0000","last_undegraded":"2024-09-17T05:02:07.349878+0000","last_fullsized":"2024-09-17T05:02:07.349878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:24.585972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314795+0000","last_change":"2024-09-17T05:02:05.331453+0000","last_active":"2024-09-17T05:02:07.314795+0000","last_peered":"2024-09-17T05:02:07.314795+0000","last_clean":"2024-09-17T05:02:07.314795+0000","last_became_active":"2024-09-17T05:02:05.330843+0000","last_became_peered":"2024-09-17T05:02:05.330843+0000","last_unstale":"2024-09-17T05:02:07.314795+0000","last_undegraded":"2024-09-17T05:02:07.314795+0000","last_fullsized":"2024-09-17T05:02:07.314795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:53.874382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346719+0000","last_change":"2024-09-17T05:02:06.338645+0000","last_active":"2024-09-17T05:02:07.346719+0000","last_peered":"2024-09-17T05:02:07.346719+0000","last_clean":"2024-09-17T05:02:07.346719+0000","last_became_active":"2024-09-17T05:02:06.338343+0000","last_became_peered":"2024-09-17T05:02:06.338343+0000","last_unstale":"2024-09-17T05:02:07.346719+0000","last_undegraded":"2024-09-17T05:02:07.346719+0000","last_fullsized":"2024-09-17T05:02:07.346719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419537+0000","last_change":"2024-09-17T05:02:05.318024+0000","last_active":"2024-09-17T05:02:07.419537+0000","last_peered":"2024-09-17T05:02:07.419537+0000","last_clean":"2024-09-17T05:02:07.419537+0000","last_became_active":"2024-09-17T05:02:05.317528+0000","last_became_peered":"2024-09-17T05:02:05.317528+0000","last_unstale":"2024-09-17T05:02:07.419537+0000","last_undegraded":"2024-09-17T05:02:07.419537+0000","last_fullsized":"2024-09-17T05:02:07.419537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:22.090770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420452+0000","last_change":"2024-09-17T05:02:06.336675+0000","last_active":"2024-09-17T05:02:07.420452+0000","last_peered":"2024-09-17T05:02:07.420452+0000","last_clean":"2024-09-17T05:02:07.420452+0000","last_became_active":"2024-09-17T05:02:06.336377+0000","last_became_peered":"2024-09-17T05:02:06.336377+0000","last_unstale":"2024-09-17T05:02:07.420452+0000","last_undegraded":"2024-09-17T05:02:07.420452+0000","last_fullsized":"2024-09-17T05:02:07.420452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.158532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420724+0000","last_change":"2024-09-17T05:02:05.334942+0000","last_active":"2024-09-17T05:02:07.420724+0000","last_peered":"2024-09-17T05:02:07.420724+0000","last_clean":"2024-09-17T05:02:07.420724+0000","last_became_active":"2024-09-17T05:02:05.334624+0000","last_became_peered":"2024-09-17T05:02:05.334624+0000","last_unstale":"2024-09-17T05:02:07.420724+0000","last_undegraded":"2024-09-17T05:02:07.420724+0000","last_fullsized":"2024-09-17T05:02:07.420724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:14.855709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420380+0000","last_change":"2024-09-17T05:02:06.334939+0000","last_active":"2024-09-17T05:02:07.420380+0000","last_peered":"2024-09-17T05:02:07.420380+0000","last_clean":"2024-09-17T05:02:07.420380+0000","last_became_active":"2024-09-17T05:02:06.334587+0000","last_became_peered":"2024-09-17T05:02:06.334587+0000","last_unstale":"2024-09-17T05:02:07.420380+0000","last_undegraded":"2024-09-17T05:02:07.420380+0000","last_fullsized":"2024-09-17T05:02:07.420380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:37.387302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315252+0000","last_change":"2024-09-17T05:02:05.326248+0000","last_active":"2024-09-17T05:02:07.315252+0000","last_peered":"2024-09-17T05:02:07.315252+0000","last_clean":"2024-09-17T05:02:07.315252+0000","last_became_active":"2024-09-17T05:02:05.325621+0000","last_became_peered":"2024-09-17T05:02:05.325621+0000","last_unstale":"2024-09-17T05:02:07.315252+0000","last_undegraded":"2024-09-17T05:02:07.315252+0000","last_fullsized":"2024-09-17T05:02:07.315252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:35.920973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420572+0000","last_change":"2024-09-17T05:02:06.332684+0000","last_active":"2024-09-17T05:02:07.420572+0000","last_peered":"2024-09-17T05:02:07.420572+0000","last_clean":"2024-09-17T05:02:07.420572+0000","last_became_active":"2024-09-17T05:02:06.332268+0000","last_became_peered":"2024-09-17T05:02:06.332268+0000","last_unstale":"2024-09-17T05:02:07.420572+0000","last_undegraded":"2024-09-17T05:02:07.420572+0000","last_fullsized":"2024-09-17T05:02:07.420572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:36.983687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346854+0000","last_change":"2024-09-17T05:02:05.329129+0000","last_active":"2024-09-17T05:02:07.346854+0000","last_peered":"2024-09-17T05:02:07.346854+0000","last_clean":"2024-09-17T05:02:07.346854+0000","last_became_active":"2024-09-17T05:02:05.328783+0000","last_became_peered":"2024-09-17T05:02:05.328783+0000","last_unstale":"2024-09-17T05:02:07.346854+0000","last_undegraded":"2024-09-17T05:02:07.346854+0000","last_fullsized":"2024-09-17T05:02:07.346854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:01.114967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.315373+0000","last_change":"2024-09-17T05:02:05.315373+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.315373+0000","last_undegraded":"2024-09-17T05:02:05.315373+0000","last_fullsized":"2024-09-17T05:02:05.315373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T05:02:05.325258+0000","last_change":"2024-09-17T05:02:05.324272+0000","last_active":"2024-09-17T05:02:05.325258+0000","last_peered":"2024-09-17T05:02:05.325258+0000","last_clean":"2024-09-17T05:02:05.325258+0000","last_became_active":"2024-09-17T05:02:05.323884+0000","last_became_peered":"2024-09-17T05:02:05.323884+0000","last_unstale":"2024-09-17T05:02:05.325258+0000","last_undegraded":"2024-09-17T05:02:05.325258+0000","last_fullsized":"2024-09-17T05:02:05.325258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:51.335778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314537+0000","last_change":"2024-09-17T05:02:06.321153+0000","last_active":"2024-09-17T05:02:07.314537+0000","last_peered":"2024-09-17T05:02:07.314537+0000","last_clean":"2024-09-17T05:02:07.314537+0000","last_became_active":"2024-09-17T05:02:06.320793+0000","last_became_peered":"2024-09-17T05:02:06.320793+0000","last_unstale":"2024-09-17T05:02:07.314537+0000","last_undegraded":"2024-09-17T05:02:07.314537+0000","last_fullsized":"2024-09-17T05:02:07.314537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:49.889726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T05:02:05.324925+0000","last_change":"2024-09-17T05:02:05.321831+0000","last_active":"2024-09-17T05:02:05.324925+0000","last_peered":"2024-09-17T05:02:05.324925+0000","last_clean":"2024-09-17T05:02:05.324925+0000","last_became_active":"2024-09-17T05:02:05.321507+0000","last_became_peered":"2024-09-17T05:02:05.321507+0000","last_unstale":"2024-09-17T05:02:05.324925+0000","last_undegraded":"2024-09-17T05:02:05.324925+0000","last_fullsized":"2024-09-17T05:02:05.324925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:10.699379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319927+0000","last_change":"2024-09-17T05:02:06.362961+0000","last_active":"2024-09-17T05:02:07.319927+0000","last_peered":"2024-09-17T05:02:07.319927+0000","last_clean":"2024-09-17T05:02:07.319927+0000","last_became_active":"2024-09-17T05:02:06.362589+0000","last_became_peered":"2024-09-17T05:02:06.362589+0000","last_unstale":"2024-09-17T05:02:07.319927+0000","last_undegraded":"2024-09-17T05:02:07.319927+0000","last_fullsized":"2024-09-17T05:02:07.319927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:43.235010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420209+0000","last_change":"2024-09-17T05:02:05.334796+0000","last_active":"2024-09-17T05:02:07.420209+0000","last_peered":"2024-09-17T05:02:07.420209+0000","last_clean":"2024-09-17T05:02:07.420209+0000","last_became_active":"2024-09-17T05:02:05.334457+0000","last_became_peered":"2024-09-17T05:02:05.334457+0000","last_unstale":"2024-09-17T05:02:07.420209+0000","last_undegraded":"2024-09-17T05:02:07.420209+0000","last_fullsized":"2024-09-17T05:02:07.420209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:50.024608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349461+0000","last_change":"2024-09-17T05:02:06.360661+0000","last_active":"2024-09-17T05:02:07.349461+0000","last_peered":"2024-09-17T05:02:07.349461+0000","last_clean":"2024-09-17T05:02:07.349461+0000","last_became_active":"2024-09-17T05:02:06.360293+0000","last_became_peered":"2024-09-17T05:02:06.360293+0000","last_unstale":"2024-09-17T05:02:07.349461+0000","last_undegraded":"2024-09-17T05:02:07.349461+0000","last_fullsized":"2024-09-17T05:02:07.349461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420548+0000","last_change":"2024-09-17T05:02:05.335089+0000","last_active":"2024-09-17T05:02:07.420548+0000","last_peered":"2024-09-17T05:02:07.420548+0000","last_clean":"2024-09-17T05:02:07.420548+0000","last_became_active":"2024-09-17T05:02:05.334761+0000","last_became_peered":"2024-09-17T05:02:05.334761+0000","last_unstale":"2024-09-17T05:02:07.420548+0000","last_undegraded":"2024-09-17T05:02:07.420548+0000","last_fullsized":"2024-09-17T05:02:07.420548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319697+0000","last_change":"2024-09-17T05:02:06.337072+0000","last_active":"2024-09-17T05:02:07.319697+0000","last_peered":"2024-09-17T05:02:07.319697+0000","last_clean":"2024-09-17T05:02:07.319697+0000","last_became_active":"2024-09-17T05:02:06.335541+0000","last_became_peered":"2024-09-17T05:02:06.335541+0000","last_unstale":"2024-09-17T05:02:07.319697+0000","last_undegraded":"2024-09-17T05:02:07.319697+0000","last_fullsized":"2024-09-17T05:02:07.319697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:33.058079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419849+0000","last_change":"2024-09-17T05:02:05.317067+0000","last_active":"2024-09-17T05:02:07.419849+0000","last_peered":"2024-09-17T05:02:07.419849+0000","last_clean":"2024-09-17T05:02:07.419849+0000","last_became_active":"2024-09-17T05:02:05.316750+0000","last_became_peered":"2024-09-17T05:02:05.316750+0000","last_unstale":"2024-09-17T05:02:07.419849+0000","last_undegraded":"2024-09-17T05:02:07.419849+0000","last_fullsized":"2024-09-17T05:02:07.419849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:11.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346952+0000","last_change":"2024-09-17T05:02:06.342012+0000","last_active":"2024-09-17T05:02:07.346952+0000","last_peered":"2024-09-17T05:02:07.346952+0000","last_clean":"2024-09-17T05:02:07.346952+0000","last_became_active":"2024-09-17T05:02:06.341627+0000","last_became_peered":"2024-09-17T05:02:06.341627+0000","last_unstale":"2024-09-17T05:02:07.346952+0000","last_undegraded":"2024-09-17T05:02:07.346952+0000","last_fullsized":"2024-09-17T05:02:07.346952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:04.620408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315375+0000","last_change":"2024-09-17T05:02:05.331974+0000","last_active":"2024-09-17T05:02:07.315375+0000","last_peered":"2024-09-17T05:02:07.315375+0000","last_clean":"2024-09-17T05:02:07.315375+0000","last_became_active":"2024-09-17T05:02:05.331277+0000","last_became_peered":"2024-09-17T05:02:05.331277+0000","last_unstale":"2024-09-17T05:02:07.315375+0000","last_undegraded":"2024-09-17T05:02:07.315375+0000","last_fullsized":"2024-09-17T05:02:07.315375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:14.008073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350015+0000","last_change":"2024-09-17T05:02:06.361595+0000","last_active":"2024-09-17T05:02:07.350015+0000","last_peered":"2024-09-17T05:02:07.350015+0000","last_clean":"2024-09-17T05:02:07.350015+0000","last_became_active":"2024-09-17T05:02:06.361219+0000","last_became_peered":"2024-09-17T05:02:06.361219+0000","last_unstale":"2024-09-17T05:02:07.350015+0000","last_undegraded":"2024-09-17T05:02:07.350015+0000","last_fullsized":"2024-09-17T05:02:07.350015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:47.878683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.348883+0000","last_change":"2024-09-17T05:02:05.334870+0000","last_active":"2024-09-17T05:02:07.348883+0000","last_peered":"2024-09-17T05:02:07.348883+0000","last_clean":"2024-09-17T05:02:07.348883+0000","last_became_active":"2024-09-17T05:02:05.334512+0000","last_became_peered":"2024-09-17T05:02:05.334512+0000","last_unstale":"2024-09-17T05:02:07.348883+0000","last_undegraded":"2024-09-17T05:02:07.348883+0000","last_fullsized":"2024-09-17T05:02:07.348883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.315369+0000","last_change":"2024-09-17T05:02:05.315369+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.315369+0000","last_undegraded":"2024-09-17T05:02:05.315369+0000","last_fullsized":"2024-09-17T05:02:05.315369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346976+0000","last_change":"2024-09-17T05:02:05.325366+0000","last_active":"2024-09-17T05:02:07.346976+0000","last_peered":"2024-09-17T05:02:07.346976+0000","last_clean":"2024-09-17T05:02:07.346976+0000","last_became_active":"2024-09-17T05:02:05.325037+0000","last_became_peered":"2024-09-17T05:02:05.325037+0000","last_unstale":"2024-09-17T05:02:07.346976+0000","last_undegraded":"2024-09-17T05:02:07.346976+0000","last_fullsized":"2024-09-17T05:02:07.346976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346633+0000","last_change":"2024-09-17T05:02:06.336842+0000","last_active":"2024-09-17T05:02:07.346633+0000","last_peered":"2024-09-17T05:02:07.346633+0000","last_clean":"2024-09-17T05:02:07.346633+0000","last_became_active":"2024-09-17T05:02:06.334045+0000","last_became_peered":"2024-09-17T05:02:06.334045+0000","last_unstale":"2024-09-17T05:02:07.346633+0000","last_undegraded":"2024-09-17T05:02:07.346633+0000","last_fullsized":"2024-09-17T05:02:07.346633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:30.924123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320355+0000","last_change":"2024-09-17T05:02:05.316008+0000","last_active":"2024-09-17T05:02:07.320355+0000","last_peered":"2024-09-17T05:02:07.320355+0000","last_clean":"2024-09-17T05:02:07.320355+0000","last_became_active":"2024-09-17T05:02:05.315680+0000","last_became_peered":"2024-09-17T05:02:05.315680+0000","last_unstale":"2024-09-17T05:02:07.320355+0000","last_undegraded":"2024-09-17T05:02:07.320355+0000","last_fullsized":"2024-09-17T05:02:07.320355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:25.314134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420182+0000","last_change":"2024-09-17T05:02:06.319976+0000","last_active":"2024-09-17T05:02:07.420182+0000","last_peered":"2024-09-17T05:02:07.420182+0000","last_clean":"2024-09-17T05:02:07.420182+0000","last_became_active":"2024-09-17T05:02:06.319609+0000","last_became_peered":"2024-09-17T05:02:06.319609+0000","last_unstale":"2024-09-17T05:02:07.420182+0000","last_undegraded":"2024-09-17T05:02:07.420182+0000","last_fullsized":"2024-09-17T05:02:07.420182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349774+0000","last_change":"2024-09-17T05:02:05.331789+0000","last_active":"2024-09-17T05:02:07.349774+0000","last_peered":"2024-09-17T05:02:07.349774+0000","last_clean":"2024-09-17T05:02:07.349774+0000","last_became_active":"2024-09-17T05:02:05.327820+0000","last_became_peered":"2024-09-17T05:02:05.327820+0000","last_unstale":"2024-09-17T05:02:07.349774+0000","last_undegraded":"2024-09-17T05:02:07.349774+0000","last_fullsized":"2024-09-17T05:02:07.349774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314859+0000","last_change":"2024-09-17T05:02:06.333560+0000","last_active":"2024-09-17T05:02:07.314859+0000","last_peered":"2024-09-17T05:02:07.314859+0000","last_clean":"2024-09-17T05:02:07.314859+0000","last_became_active":"2024-09-17T05:02:06.333210+0000","last_became_peered":"2024-09-17T05:02:06.333210+0000","last_unstale":"2024-09-17T05:02:07.314859+0000","last_undegraded":"2024-09-17T05:02:07.314859+0000","last_fullsized":"2024-09-17T05:02:07.314859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.884069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314950+0000","last_change":"2024-09-17T05:02:05.325774+0000","last_active":"2024-09-17T05:02:07.314950+0000","last_peered":"2024-09-17T05:02:07.314950+0000","last_clean":"2024-09-17T05:02:07.314950+0000","last_became_active":"2024-09-17T05:02:05.325459+0000","last_became_peered":"2024-09-17T05:02:05.325459+0000","last_unstale":"2024-09-17T05:02:07.314950+0000","last_undegraded":"2024-09-17T05:02:07.314950+0000","last_fullsized":"2024-09-17T05:02:07.314950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:55.257416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349680+0000","last_change":"2024-09-17T05:02:06.340513+0000","last_active":"2024-09-17T05:02:07.349680+0000","last_peered":"2024-09-17T05:02:07.349680+0000","last_clean":"2024-09-17T05:02:07.349680+0000","last_became_active":"2024-09-17T05:02:06.340154+0000","last_became_peered":"2024-09-17T05:02:06.340154+0000","last_unstale":"2024-09-17T05:02:07.349680+0000","last_undegraded":"2024-09-17T05:02:07.349680+0000","last_fullsized":"2024-09-17T05:02:07.349680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320990+0000","last_change":"2024-09-17T05:02:05.325138+0000","last_active":"2024-09-17T05:02:07.320990+0000","last_peered":"2024-09-17T05:02:07.320990+0000","last_clean":"2024-09-17T05:02:07.320990+0000","last_became_active":"2024-09-17T05:02:05.324679+0000","last_became_peered":"2024-09-17T05:02:05.324679+0000","last_unstale":"2024-09-17T05:02:07.320990+0000","last_undegraded":"2024-09-17T05:02:07.320990+0000","last_fullsized":"2024-09-17T05:02:07.320990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:39.915152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315287+0000","last_change":"2024-09-17T05:02:06.331069+0000","last_active":"2024-09-17T05:02:07.315287+0000","last_peered":"2024-09-17T05:02:07.315287+0000","last_clean":"2024-09-17T05:02:07.315287+0000","last_became_active":"2024-09-17T05:02:06.330731+0000","last_became_peered":"2024-09-17T05:02:06.330731+0000","last_unstale":"2024-09-17T05:02:07.315287+0000","last_undegraded":"2024-09-17T05:02:07.315287+0000","last_fullsized":"2024-09-17T05:02:07.315287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:38.520351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349709+0000","last_change":"2024-09-17T05:02:05.335344+0000","last_active":"2024-09-17T05:02:07.349709+0000","last_peered":"2024-09-17T05:02:07.349709+0000","last_clean":"2024-09-17T05:02:07.349709+0000","last_became_active":"2024-09-17T05:02:05.334313+0000","last_became_peered":"2024-09-17T05:02:05.334313+0000","last_unstale":"2024-09-17T05:02:07.349709+0000","last_undegraded":"2024-09-17T05:02:07.349709+0000","last_fullsized":"2024-09-17T05:02:07.349709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420491+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:02:07.420491+0000","last_peered":"2024-09-17T05:02:07.420491+0000","last_clean":"2024-09-17T05:02:07.420491+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:02:07.420491+0000","last_undegraded":"2024-09-17T05:02:07.420491+0000","last_fullsized":"2024-09-17T05:02:07.420491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":458752,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":462848,"data_stored":190415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":188769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":190873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":779069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":450560,"data_stored":188771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1024000,"data_stored":771880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:02:10.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd247200640 1 -- 172.21.15.16:0/3520636742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2240041d0 msgr2=0x7fd224004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd247200640 1 --2- 172.21.15.16:0/3520636742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2240041d0 0x7fd224004810 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd228006f80 tx=0x7fd22801cee0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd245200640 1 -- 172.21.15.16:0/3520636742 reap_dead start 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd247200640 1 -- 172.21.15.16:0/3520636742 shutdown_connections 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd247200640 1 -- 172.21.15.16:0/3520636742 >> 172.21.15.16:0/3520636742 conn(0x7fd24006f900 msgr2=0x7fd2400b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd247200640 1 -- 172.21.15.16:0/3520636742 shutdown_connections 2024-09-17T05:02:10.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.718+0000 7fd247200640 1 -- 172.21.15.16:0/3520636742 wait complete. 2024-09-17T05:02:10.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 Processor -- start 2024-09-17T05:02:10.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 -- start start 2024-09-17T05:02:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2240041d0 0x7fd240155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 0x7fd2401fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd24015dd10 0x7fd24015e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd240167670 con 0x7fd2401f9ed0 2024-09-17T05:02:10.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd2401674f0 con 0x7fd24015dd10 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd2401677f0 con 0x7fd2240041d0 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 0x7fd2401fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 0x7fd2401fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47990/0 (socket says 172.21.15.16:47990) 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 -- 172.21.15.16:0/2658046375 learned_addr learned my addr 172.21.15.16:0/2658046375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2240041d0 0x7fd240155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd245200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd24015dd10 0x7fd24015e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 -- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2240041d0 msgr2=0x7fd240155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2240041d0 0x7fd240155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 -- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd24015dd10 msgr2=0x7fd24015e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd24015dd10 0x7fd24015e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 -- 172.21.15.16:0/2658046375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd240155bf0 con 0x7fd2401f9ed0 2024-09-17T05:02:10.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd237400640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 0x7fd2401fa2e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd228000c10 tx=0x7fd2280060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228001090 con 0x7fd2401f9ed0 2024-09-17T05:02:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd228003600 con 0x7fd2401f9ed0 2024-09-17T05:02:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228003a50 con 0x7fd2401f9ed0 2024-09-17T05:02:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24015e660 con 0x7fd2401f9ed0 2024-09-17T05:02:10.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.722+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2401f6720 con 0x7fd2401f9ed0 2024-09-17T05:02:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.726+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24014d630 con 0x7fd2401f9ed0 2024-09-17T05:02:10.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.726+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd2280c0060 con 0x7fd2401f9ed0 2024-09-17T05:02:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.726+0000 7fd23cc00640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd21c073200 0x7fd21c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.726+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd22812e7e0 con 0x7fd2401f9ed0 2024-09-17T05:02:10.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.726+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd21c073200 0x7fd21c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:10.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.730+0000 7fd23fe00640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd21c073200 0x7fd21c075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd230006fd0 tx=0x7fd230006ea0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.730+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd24015e660 con 0x7fd2401f9ed0 2024-09-17T05:02:10.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.902+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd24014d350 con 0x7fd21c073200 2024-09-17T05:02:10.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.906+0000 7fd23cc00640 1 -- 172.21.15.16:0/2658046375 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251694 (secure 0 0 0) 0x7fd24014d350 con 0x7fd21c073200 2024-09-17T05:02:10.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:02:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd21c073200 msgr2=0x7fd21c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd21c073200 0x7fd21c075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd230006fd0 tx=0x7fd230006ea0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 msgr2=0x7fd2401fa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 0x7fd2401fa2e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd228000c10 tx=0x7fd2280060c0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 shutdown_connections 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd21c073200 0x7fd21c075710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd24015dd10 0x7fd24015e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2401f9ed0 0x7fd2401fa2e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 --2- 172.21.15.16:0/2658046375 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd2240041d0 0x7fd240155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 >> 172.21.15.16:0/2658046375 conn(0x7fd24006f900 msgr2=0x7fd240073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 shutdown_connections 2024-09-17T05:02:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:10.910+0000 7fd247200640 1 -- 172.21.15.16:0/2658046375 wait complete. 2024-09-17T05:02:10.934 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-17T05:02:09.476186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237588,"kb_used_data":4676,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737132,"statfs":{"total":773094113280,"available":772850823168,"internally_reserved":0,"allocated":4788224,"data_stored":2693931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181300"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321351+0000","last_change":"2024-09-17T05:02:06.338525+0000","last_active":"2024-09-17T05:02:07.321351+0000","last_peered":"2024-09-17T05:02:07.321351+0000","last_clean":"2024-09-17T05:02:07.321351+0000","last_became_active":"2024-09-17T05:02:06.338232+0000","last_became_peered":"2024-09-17T05:02:06.338232+0000","last_unstale":"2024-09-17T05:02:07.321351+0000","last_undegraded":"2024-09-17T05:02:07.321351+0000","last_fullsized":"2024-09-17T05:02:07.321351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:56.646622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420620+0000","last_change":"2024-09-17T05:02:06.332250+0000","last_active":"2024-09-17T05:02:07.420620+0000","last_peered":"2024-09-17T05:02:07.420620+0000","last_clean":"2024-09-17T05:02:07.420620+0000","last_became_active":"2024-09-17T05:02:06.331927+0000","last_became_peered":"2024-09-17T05:02:06.331927+0000","last_unstale":"2024-09-17T05:02:07.420620+0000","last_undegraded":"2024-09-17T05:02:07.420620+0000","last_fullsized":"2024-09-17T05:02:07.420620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320879+0000","last_change":"2024-09-17T05:02:06.336648+0000","last_active":"2024-09-17T05:02:07.320879+0000","last_peered":"2024-09-17T05:02:07.320879+0000","last_clean":"2024-09-17T05:02:07.320879+0000","last_became_active":"2024-09-17T05:02:06.336337+0000","last_became_peered":"2024-09-17T05:02:06.336337+0000","last_unstale":"2024-09-17T05:02:07.320879+0000","last_undegraded":"2024-09-17T05:02:07.320879+0000","last_fullsized":"2024-09-17T05:02:07.320879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:44.810456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321127+0000","last_change":"2024-09-17T05:02:06.348672+0000","last_active":"2024-09-17T05:02:07.321127+0000","last_peered":"2024-09-17T05:02:07.321127+0000","last_clean":"2024-09-17T05:02:07.321127+0000","last_became_active":"2024-09-17T05:02:06.348392+0000","last_became_peered":"2024-09-17T05:02:06.348392+0000","last_unstale":"2024-09-17T05:02:07.321127+0000","last_undegraded":"2024-09-17T05:02:07.321127+0000","last_fullsized":"2024-09-17T05:02:07.321127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:06.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346718+0000","last_change":"2024-09-17T05:02:06.343632+0000","last_active":"2024-09-17T05:02:07.346718+0000","last_peered":"2024-09-17T05:02:07.346718+0000","last_clean":"2024-09-17T05:02:07.346718+0000","last_became_active":"2024-09-17T05:02:06.343257+0000","last_became_peered":"2024-09-17T05:02:06.343257+0000","last_unstale":"2024-09-17T05:02:07.346718+0000","last_undegraded":"2024-09-17T05:02:07.346718+0000","last_fullsized":"2024-09-17T05:02:07.346718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:00.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420107+0000","last_change":"2024-09-17T05:02:06.330126+0000","last_active":"2024-09-17T05:02:07.420107+0000","last_peered":"2024-09-17T05:02:07.420107+0000","last_clean":"2024-09-17T05:02:07.420107+0000","last_became_active":"2024-09-17T05:02:06.329825+0000","last_became_peered":"2024-09-17T05:02:06.329825+0000","last_unstale":"2024-09-17T05:02:07.420107+0000","last_undegraded":"2024-09-17T05:02:07.420107+0000","last_fullsized":"2024-09-17T05:02:07.420107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:46.368942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420406+0000","last_change":"2024-09-17T05:02:06.343766+0000","last_active":"2024-09-17T05:02:07.420406+0000","last_peered":"2024-09-17T05:02:07.420406+0000","last_clean":"2024-09-17T05:02:07.420406+0000","last_became_active":"2024-09-17T05:02:06.343207+0000","last_became_peered":"2024-09-17T05:02:06.343207+0000","last_unstale":"2024-09-17T05:02:07.420406+0000","last_undegraded":"2024-09-17T05:02:07.420406+0000","last_fullsized":"2024-09-17T05:02:07.420406+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:51.030820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320027+0000","last_change":"2024-09-17T05:02:06.338244+0000","last_active":"2024-09-17T05:02:07.320027+0000","last_peered":"2024-09-17T05:02:07.320027+0000","last_clean":"2024-09-17T05:02:07.320027+0000","last_became_active":"2024-09-17T05:02:06.337859+0000","last_became_peered":"2024-09-17T05:02:06.337859+0000","last_unstale":"2024-09-17T05:02:07.320027+0000","last_undegraded":"2024-09-17T05:02:07.320027+0000","last_fullsized":"2024-09-17T05:02:07.320027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:36.957053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.347110+0000","last_change":"2024-09-17T05:02:06.336573+0000","last_active":"2024-09-17T05:02:07.347110+0000","last_peered":"2024-09-17T05:02:07.347110+0000","last_clean":"2024-09-17T05:02:07.347110+0000","last_became_active":"2024-09-17T05:02:06.336253+0000","last_became_peered":"2024-09-17T05:02:06.336253+0000","last_unstale":"2024-09-17T05:02:07.347110+0000","last_undegraded":"2024-09-17T05:02:07.347110+0000","last_fullsized":"2024-09-17T05:02:07.347110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:47.955020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420328+0000","last_change":"2024-09-17T05:02:06.362653+0000","last_active":"2024-09-17T05:02:07.420328+0000","last_peered":"2024-09-17T05:02:07.420328+0000","last_clean":"2024-09-17T05:02:07.420328+0000","last_became_active":"2024-09-17T05:02:06.362345+0000","last_became_peered":"2024-09-17T05:02:06.362345+0000","last_unstale":"2024-09-17T05:02:07.420328+0000","last_undegraded":"2024-09-17T05:02:07.420328+0000","last_fullsized":"2024-09-17T05:02:07.420328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:12.091714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350026+0000","last_change":"2024-09-17T05:02:06.341127+0000","last_active":"2024-09-17T05:02:07.350026+0000","last_peered":"2024-09-17T05:02:07.350026+0000","last_clean":"2024-09-17T05:02:07.350026+0000","last_became_active":"2024-09-17T05:02:06.340796+0000","last_became_peered":"2024-09-17T05:02:06.340796+0000","last_unstale":"2024-09-17T05:02:07.350026+0000","last_undegraded":"2024-09-17T05:02:07.350026+0000","last_fullsized":"2024-09-17T05:02:07.350026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:54.880594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346735+0000","last_change":"2024-09-17T05:02:06.336217+0000","last_active":"2024-09-17T05:02:07.346735+0000","last_peered":"2024-09-17T05:02:07.346735+0000","last_clean":"2024-09-17T05:02:07.346735+0000","last_became_active":"2024-09-17T05:02:06.333449+0000","last_became_peered":"2024-09-17T05:02:06.333449+0000","last_unstale":"2024-09-17T05:02:07.346735+0000","last_undegraded":"2024-09-17T05:02:07.346735+0000","last_fullsized":"2024-09-17T05:02:07.346735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:23.096982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420312+0000","last_change":"2024-09-17T05:02:06.330216+0000","last_active":"2024-09-17T05:02:07.420312+0000","last_peered":"2024-09-17T05:02:07.420312+0000","last_clean":"2024-09-17T05:02:07.420312+0000","last_became_active":"2024-09-17T05:02:06.329869+0000","last_became_peered":"2024-09-17T05:02:06.329869+0000","last_unstale":"2024-09-17T05:02:07.420312+0000","last_undegraded":"2024-09-17T05:02:07.420312+0000","last_fullsized":"2024-09-17T05:02:07.420312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419668+0000","last_change":"2024-09-17T05:02:06.337107+0000","last_active":"2024-09-17T05:02:07.419668+0000","last_peered":"2024-09-17T05:02:07.419668+0000","last_clean":"2024-09-17T05:02:07.419668+0000","last_became_active":"2024-09-17T05:02:06.336774+0000","last_became_peered":"2024-09-17T05:02:06.336774+0000","last_unstale":"2024-09-17T05:02:07.419668+0000","last_undegraded":"2024-09-17T05:02:07.419668+0000","last_fullsized":"2024-09-17T05:02:07.419668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:26.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314512+0000","last_change":"2024-09-17T05:02:06.321133+0000","last_active":"2024-09-17T05:02:07.314512+0000","last_peered":"2024-09-17T05:02:07.314512+0000","last_clean":"2024-09-17T05:02:07.314512+0000","last_became_active":"2024-09-17T05:02:06.320714+0000","last_became_peered":"2024-09-17T05:02:06.320714+0000","last_unstale":"2024-09-17T05:02:07.314512+0000","last_undegraded":"2024-09-17T05:02:07.314512+0000","last_fullsized":"2024-09-17T05:02:07.314512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:18.256704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420854+0000","last_change":"2024-09-17T05:02:06.335127+0000","last_active":"2024-09-17T05:02:07.420854+0000","last_peered":"2024-09-17T05:02:07.420854+0000","last_clean":"2024-09-17T05:02:07.420854+0000","last_became_active":"2024-09-17T05:02:06.334693+0000","last_became_peered":"2024-09-17T05:02:06.334693+0000","last_unstale":"2024-09-17T05:02:07.420854+0000","last_undegraded":"2024-09-17T05:02:07.420854+0000","last_fullsized":"2024-09-17T05:02:07.420854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:20.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315318+0000","last_change":"2024-09-17T05:02:06.352349+0000","last_active":"2024-09-17T05:02:07.315318+0000","last_peered":"2024-09-17T05:02:07.315318+0000","last_clean":"2024-09-17T05:02:07.315318+0000","last_became_active":"2024-09-17T05:02:06.352010+0000","last_became_peered":"2024-09-17T05:02:06.352010+0000","last_unstale":"2024-09-17T05:02:07.315318+0000","last_undegraded":"2024-09-17T05:02:07.315318+0000","last_fullsized":"2024-09-17T05:02:07.315318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.149726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314938+0000","last_change":"2024-09-17T05:02:06.336278+0000","last_active":"2024-09-17T05:02:07.314938+0000","last_peered":"2024-09-17T05:02:07.314938+0000","last_clean":"2024-09-17T05:02:07.314938+0000","last_became_active":"2024-09-17T05:02:06.334917+0000","last_became_peered":"2024-09-17T05:02:06.334917+0000","last_unstale":"2024-09-17T05:02:07.314938+0000","last_undegraded":"2024-09-17T05:02:07.314938+0000","last_fullsized":"2024-09-17T05:02:07.314938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:51.224167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.314200+0000","last_change":"2024-09-17T05:02:05.314200+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.314200+0000","last_undegraded":"2024-09-17T05:02:05.314200+0000","last_fullsized":"2024-09-17T05:02:05.314200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349734+0000","last_change":"2024-09-17T05:02:06.353747+0000","last_active":"2024-09-17T05:02:07.349734+0000","last_peered":"2024-09-17T05:02:07.349734+0000","last_clean":"2024-09-17T05:02:07.349734+0000","last_became_active":"2024-09-17T05:02:06.353157+0000","last_became_peered":"2024-09-17T05:02:06.353157+0000","last_unstale":"2024-09-17T05:02:07.349734+0000","last_undegraded":"2024-09-17T05:02:07.349734+0000","last_fullsized":"2024-09-17T05:02:07.349734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349652+0000","last_change":"2024-09-17T05:02:06.352255+0000","last_active":"2024-09-17T05:02:07.349652+0000","last_peered":"2024-09-17T05:02:07.349652+0000","last_clean":"2024-09-17T05:02:07.349652+0000","last_became_active":"2024-09-17T05:02:06.351865+0000","last_became_peered":"2024-09-17T05:02:06.351865+0000","last_unstale":"2024-09-17T05:02:07.349652+0000","last_undegraded":"2024-09-17T05:02:07.349652+0000","last_fullsized":"2024-09-17T05:02:07.349652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:00.377367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349479+0000","last_change":"2024-09-17T05:02:06.347103+0000","last_active":"2024-09-17T05:02:07.349479+0000","last_peered":"2024-09-17T05:02:07.349479+0000","last_clean":"2024-09-17T05:02:07.349479+0000","last_became_active":"2024-09-17T05:02:06.346592+0000","last_became_peered":"2024-09-17T05:02:06.346592+0000","last_unstale":"2024-09-17T05:02:07.349479+0000","last_undegraded":"2024-09-17T05:02:07.349479+0000","last_fullsized":"2024-09-17T05:02:07.349479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314660+0000","last_change":"2024-09-17T05:02:06.334073+0000","last_active":"2024-09-17T05:02:07.314660+0000","last_peered":"2024-09-17T05:02:07.314660+0000","last_clean":"2024-09-17T05:02:07.314660+0000","last_became_active":"2024-09-17T05:02:06.333712+0000","last_became_peered":"2024-09-17T05:02:06.333712+0000","last_unstale":"2024-09-17T05:02:07.314660+0000","last_undegraded":"2024-09-17T05:02:07.314660+0000","last_fullsized":"2024-09-17T05:02:07.314660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:51.877414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321024+0000","last_change":"2024-09-17T05:02:06.345627+0000","last_active":"2024-09-17T05:02:07.321024+0000","last_peered":"2024-09-17T05:02:07.321024+0000","last_clean":"2024-09-17T05:02:07.321024+0000","last_became_active":"2024-09-17T05:02:06.345225+0000","last_became_peered":"2024-09-17T05:02:06.345225+0000","last_unstale":"2024-09-17T05:02:07.321024+0000","last_undegraded":"2024-09-17T05:02:07.321024+0000","last_fullsized":"2024-09-17T05:02:07.321024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:59.047797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420455+0000","last_change":"2024-09-17T05:02:06.338993+0000","last_active":"2024-09-17T05:02:07.420455+0000","last_peered":"2024-09-17T05:02:07.420455+0000","last_clean":"2024-09-17T05:02:07.420455+0000","last_became_active":"2024-09-17T05:02:06.337798+0000","last_became_peered":"2024-09-17T05:02:06.337798+0000","last_unstale":"2024-09-17T05:02:07.420455+0000","last_undegraded":"2024-09-17T05:02:07.420455+0000","last_fullsized":"2024-09-17T05:02:07.420455+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:01.425057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320694+0000","last_change":"2024-09-17T05:02:06.350077+0000","last_active":"2024-09-17T05:02:07.320694+0000","last_peered":"2024-09-17T05:02:07.320694+0000","last_clean":"2024-09-17T05:02:07.320694+0000","last_became_active":"2024-09-17T05:02:06.349773+0000","last_became_peered":"2024-09-17T05:02:06.349773+0000","last_unstale":"2024-09-17T05:02:07.320694+0000","last_undegraded":"2024-09-17T05:02:07.320694+0000","last_fullsized":"2024-09-17T05:02:07.320694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:35.430354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350145+0000","last_change":"2024-09-17T05:02:06.332912+0000","last_active":"2024-09-17T05:02:07.350145+0000","last_peered":"2024-09-17T05:02:07.350145+0000","last_clean":"2024-09-17T05:02:07.350145+0000","last_became_active":"2024-09-17T05:02:06.332354+0000","last_became_peered":"2024-09-17T05:02:06.332354+0000","last_unstale":"2024-09-17T05:02:07.350145+0000","last_undegraded":"2024-09-17T05:02:07.350145+0000","last_fullsized":"2024-09-17T05:02:07.350145+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:22.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420872+0000","last_change":"2024-09-17T05:02:06.335589+0000","last_active":"2024-09-17T05:02:07.420872+0000","last_peered":"2024-09-17T05:02:07.420872+0000","last_clean":"2024-09-17T05:02:07.420872+0000","last_became_active":"2024-09-17T05:02:06.335120+0000","last_became_peered":"2024-09-17T05:02:06.335120+0000","last_unstale":"2024-09-17T05:02:07.420872+0000","last_undegraded":"2024-09-17T05:02:07.420872+0000","last_fullsized":"2024-09-17T05:02:07.420872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:06.399066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314944+0000","last_change":"2024-09-17T05:02:06.331500+0000","last_active":"2024-09-17T05:02:07.314944+0000","last_peered":"2024-09-17T05:02:07.314944+0000","last_clean":"2024-09-17T05:02:07.314944+0000","last_became_active":"2024-09-17T05:02:06.331184+0000","last_became_peered":"2024-09-17T05:02:06.331184+0000","last_unstale":"2024-09-17T05:02:07.314944+0000","last_undegraded":"2024-09-17T05:02:07.314944+0000","last_fullsized":"2024-09-17T05:02:07.314944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:48.958710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314593+0000","last_change":"2024-09-17T05:02:06.330526+0000","last_active":"2024-09-17T05:02:07.314593+0000","last_peered":"2024-09-17T05:02:07.314593+0000","last_clean":"2024-09-17T05:02:07.314593+0000","last_became_active":"2024-09-17T05:02:06.330156+0000","last_became_peered":"2024-09-17T05:02:06.330156+0000","last_unstale":"2024-09-17T05:02:07.314593+0000","last_undegraded":"2024-09-17T05:02:07.314593+0000","last_fullsized":"2024-09-17T05:02:07.314593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420681+0000","last_change":"2024-09-17T05:02:06.356417+0000","last_active":"2024-09-17T05:02:07.420681+0000","last_peered":"2024-09-17T05:02:07.420681+0000","last_clean":"2024-09-17T05:02:07.420681+0000","last_became_active":"2024-09-17T05:02:06.342863+0000","last_became_peered":"2024-09-17T05:02:06.342863+0000","last_unstale":"2024-09-17T05:02:07.420681+0000","last_undegraded":"2024-09-17T05:02:07.420681+0000","last_fullsized":"2024-09-17T05:02:07.420681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:46.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320315+0000","last_change":"2024-09-17T05:02:06.337797+0000","last_active":"2024-09-17T05:02:07.320315+0000","last_peered":"2024-09-17T05:02:07.320315+0000","last_clean":"2024-09-17T05:02:07.320315+0000","last_became_active":"2024-09-17T05:02:06.337396+0000","last_became_peered":"2024-09-17T05:02:06.337396+0000","last_unstale":"2024-09-17T05:02:07.320315+0000","last_undegraded":"2024-09-17T05:02:07.320315+0000","last_fullsized":"2024-09-17T05:02:07.320315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:42.616845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315319+0000","last_change":"2024-09-17T05:02:06.348634+0000","last_active":"2024-09-17T05:02:07.315319+0000","last_peered":"2024-09-17T05:02:07.315319+0000","last_clean":"2024-09-17T05:02:07.315319+0000","last_became_active":"2024-09-17T05:02:06.348284+0000","last_became_peered":"2024-09-17T05:02:06.348284+0000","last_unstale":"2024-09-17T05:02:07.315319+0000","last_undegraded":"2024-09-17T05:02:07.315319+0000","last_fullsized":"2024-09-17T05:02:07.315319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:41.455558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420652+0000","last_change":"2024-09-17T05:02:05.319632+0000","last_active":"2024-09-17T05:02:07.420652+0000","last_peered":"2024-09-17T05:02:07.420652+0000","last_clean":"2024-09-17T05:02:07.420652+0000","last_became_active":"2024-09-17T05:02:05.319321+0000","last_became_peered":"2024-09-17T05:02:05.319321+0000","last_unstale":"2024-09-17T05:02:07.420652+0000","last_undegraded":"2024-09-17T05:02:07.420652+0000","last_fullsized":"2024-09-17T05:02:07.420652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:46.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420445+0000","last_change":"2024-09-17T05:02:06.334112+0000","last_active":"2024-09-17T05:02:07.420445+0000","last_peered":"2024-09-17T05:02:07.420445+0000","last_clean":"2024-09-17T05:02:07.420445+0000","last_became_active":"2024-09-17T05:02:06.333727+0000","last_became_peered":"2024-09-17T05:02:06.333727+0000","last_unstale":"2024-09-17T05:02:07.420445+0000","last_undegraded":"2024-09-17T05:02:07.420445+0000","last_fullsized":"2024-09-17T05:02:07.420445+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:48.515835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420440+0000","last_change":"2024-09-17T05:02:05.330941+0000","last_active":"2024-09-17T05:02:07.420440+0000","last_peered":"2024-09-17T05:02:07.420440+0000","last_clean":"2024-09-17T05:02:07.420440+0000","last_became_active":"2024-09-17T05:02:05.330610+0000","last_became_peered":"2024-09-17T05:02:05.330610+0000","last_unstale":"2024-09-17T05:02:07.420440+0000","last_undegraded":"2024-09-17T05:02:07.420440+0000","last_fullsized":"2024-09-17T05:02:07.420440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:35.977474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315276+0000","last_change":"2024-09-17T05:02:06.367617+0000","last_active":"2024-09-17T05:02:07.315276+0000","last_peered":"2024-09-17T05:02:07.315276+0000","last_clean":"2024-09-17T05:02:07.315276+0000","last_became_active":"2024-09-17T05:02:06.367102+0000","last_became_peered":"2024-09-17T05:02:06.367102+0000","last_unstale":"2024-09-17T05:02:07.315276+0000","last_undegraded":"2024-09-17T05:02:07.315276+0000","last_fullsized":"2024-09-17T05:02:07.315276+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.094882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321076+0000","last_change":"2024-09-17T05:02:05.321020+0000","last_active":"2024-09-17T05:02:07.321076+0000","last_peered":"2024-09-17T05:02:07.321076+0000","last_clean":"2024-09-17T05:02:07.321076+0000","last_became_active":"2024-09-17T05:02:05.320171+0000","last_became_peered":"2024-09-17T05:02:05.320171+0000","last_unstale":"2024-09-17T05:02:07.321076+0000","last_undegraded":"2024-09-17T05:02:07.321076+0000","last_fullsized":"2024-09-17T05:02:07.321076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:00.418844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420627+0000","last_change":"2024-09-17T05:02:06.348099+0000","last_active":"2024-09-17T05:02:07.420627+0000","last_peered":"2024-09-17T05:02:07.420627+0000","last_clean":"2024-09-17T05:02:07.420627+0000","last_became_active":"2024-09-17T05:02:06.347767+0000","last_became_peered":"2024-09-17T05:02:06.347767+0000","last_unstale":"2024-09-17T05:02:07.420627+0000","last_undegraded":"2024-09-17T05:02:07.420627+0000","last_fullsized":"2024-09-17T05:02:07.420627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:39.458764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420697+0000","last_change":"2024-09-17T05:02:05.323652+0000","last_active":"2024-09-17T05:02:07.420697+0000","last_peered":"2024-09-17T05:02:07.420697+0000","last_clean":"2024-09-17T05:02:07.420697+0000","last_became_active":"2024-09-17T05:02:05.323250+0000","last_became_peered":"2024-09-17T05:02:05.323250+0000","last_unstale":"2024-09-17T05:02:07.420697+0000","last_undegraded":"2024-09-17T05:02:07.420697+0000","last_fullsized":"2024-09-17T05:02:07.420697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:46.948808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320590+0000","last_change":"2024-09-17T05:02:06.332617+0000","last_active":"2024-09-17T05:02:07.320590+0000","last_peered":"2024-09-17T05:02:07.320590+0000","last_clean":"2024-09-17T05:02:07.320590+0000","last_became_active":"2024-09-17T05:02:06.330747+0000","last_became_peered":"2024-09-17T05:02:06.330747+0000","last_unstale":"2024-09-17T05:02:07.320590+0000","last_undegraded":"2024-09-17T05:02:07.320590+0000","last_fullsized":"2024-09-17T05:02:07.320590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:51.850349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420085+0000","last_change":"2024-09-17T05:02:05.313259+0000","last_active":"2024-09-17T05:02:07.420085+0000","last_peered":"2024-09-17T05:02:07.420085+0000","last_clean":"2024-09-17T05:02:07.420085+0000","last_became_active":"2024-09-17T05:02:05.312893+0000","last_became_peered":"2024-09-17T05:02:05.312893+0000","last_unstale":"2024-09-17T05:02:07.420085+0000","last_undegraded":"2024-09-17T05:02:07.420085+0000","last_fullsized":"2024-09-17T05:02:07.420085+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350174+0000","last_change":"2024-09-17T05:02:06.333044+0000","last_active":"2024-09-17T05:02:07.350174+0000","last_peered":"2024-09-17T05:02:07.350174+0000","last_clean":"2024-09-17T05:02:07.350174+0000","last_became_active":"2024-09-17T05:02:06.332672+0000","last_became_peered":"2024-09-17T05:02:06.332672+0000","last_unstale":"2024-09-17T05:02:07.350174+0000","last_undegraded":"2024-09-17T05:02:07.350174+0000","last_fullsized":"2024-09-17T05:02:07.350174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:28.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419821+0000","last_change":"2024-09-17T05:02:05.331066+0000","last_active":"2024-09-17T05:02:07.419821+0000","last_peered":"2024-09-17T05:02:07.419821+0000","last_clean":"2024-09-17T05:02:07.419821+0000","last_became_active":"2024-09-17T05:02:05.330727+0000","last_became_peered":"2024-09-17T05:02:05.330727+0000","last_unstale":"2024-09-17T05:02:07.419821+0000","last_undegraded":"2024-09-17T05:02:07.419821+0000","last_fullsized":"2024-09-17T05:02:07.419821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314674+0000","last_change":"2024-09-17T05:02:06.350254+0000","last_active":"2024-09-17T05:02:07.314674+0000","last_peered":"2024-09-17T05:02:07.314674+0000","last_clean":"2024-09-17T05:02:07.314674+0000","last_became_active":"2024-09-17T05:02:06.349926+0000","last_became_peered":"2024-09-17T05:02:06.349926+0000","last_unstale":"2024-09-17T05:02:07.314674+0000","last_undegraded":"2024-09-17T05:02:07.314674+0000","last_fullsized":"2024-09-17T05:02:07.314674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320007+0000","last_change":"2024-09-17T05:02:05.330770+0000","last_active":"2024-09-17T05:02:07.320007+0000","last_peered":"2024-09-17T05:02:07.320007+0000","last_clean":"2024-09-17T05:02:07.320007+0000","last_became_active":"2024-09-17T05:02:05.330452+0000","last_became_peered":"2024-09-17T05:02:05.330452+0000","last_unstale":"2024-09-17T05:02:07.320007+0000","last_undegraded":"2024-09-17T05:02:07.320007+0000","last_fullsized":"2024-09-17T05:02:07.320007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420577+0000","last_change":"2024-09-17T05:02:06.339321+0000","last_active":"2024-09-17T05:02:07.420577+0000","last_peered":"2024-09-17T05:02:07.420577+0000","last_clean":"2024-09-17T05:02:07.420577+0000","last_became_active":"2024-09-17T05:02:06.339039+0000","last_became_peered":"2024-09-17T05:02:06.339039+0000","last_unstale":"2024-09-17T05:02:07.420577+0000","last_undegraded":"2024-09-17T05:02:07.420577+0000","last_fullsized":"2024-09-17T05:02:07.420577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420407+0000","last_change":"2024-09-17T05:02:05.334416+0000","last_active":"2024-09-17T05:02:07.420407+0000","last_peered":"2024-09-17T05:02:07.420407+0000","last_clean":"2024-09-17T05:02:07.420407+0000","last_became_active":"2024-09-17T05:02:05.334066+0000","last_became_peered":"2024-09-17T05:02:05.334066+0000","last_unstale":"2024-09-17T05:02:07.420407+0000","last_undegraded":"2024-09-17T05:02:07.420407+0000","last_fullsized":"2024-09-17T05:02:07.420407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:44.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321356+0000","last_change":"2024-09-17T05:02:06.348674+0000","last_active":"2024-09-17T05:02:07.321356+0000","last_peered":"2024-09-17T05:02:07.321356+0000","last_clean":"2024-09-17T05:02:07.321356+0000","last_became_active":"2024-09-17T05:02:06.348346+0000","last_became_peered":"2024-09-17T05:02:06.348346+0000","last_unstale":"2024-09-17T05:02:07.321356+0000","last_undegraded":"2024-09-17T05:02:07.321356+0000","last_fullsized":"2024-09-17T05:02:07.321356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:39.881805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320775+0000","last_change":"2024-09-17T05:02:05.323997+0000","last_active":"2024-09-17T05:02:07.320775+0000","last_peered":"2024-09-17T05:02:07.320775+0000","last_clean":"2024-09-17T05:02:07.320775+0000","last_became_active":"2024-09-17T05:02:05.323410+0000","last_became_peered":"2024-09-17T05:02:05.323410+0000","last_unstale":"2024-09-17T05:02:07.320775+0000","last_undegraded":"2024-09-17T05:02:07.320775+0000","last_fullsized":"2024-09-17T05:02:07.320775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:32.508695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420275+0000","last_change":"2024-09-17T05:02:06.338660+0000","last_active":"2024-09-17T05:02:07.420275+0000","last_peered":"2024-09-17T05:02:07.420275+0000","last_clean":"2024-09-17T05:02:07.420275+0000","last_became_active":"2024-09-17T05:02:06.338256+0000","last_became_peered":"2024-09-17T05:02:06.338256+0000","last_unstale":"2024-09-17T05:02:07.420275+0000","last_undegraded":"2024-09-17T05:02:07.420275+0000","last_fullsized":"2024-09-17T05:02:07.420275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.216527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315076+0000","last_change":"2024-09-17T05:02:05.326255+0000","last_active":"2024-09-17T05:02:07.315076+0000","last_peered":"2024-09-17T05:02:07.315076+0000","last_clean":"2024-09-17T05:02:07.315076+0000","last_became_active":"2024-09-17T05:02:05.325945+0000","last_became_peered":"2024-09-17T05:02:05.325945+0000","last_unstale":"2024-09-17T05:02:07.315076+0000","last_undegraded":"2024-09-17T05:02:07.315076+0000","last_fullsized":"2024-09-17T05:02:07.315076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:21.224964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320746+0000","last_change":"2024-09-17T05:02:06.335521+0000","last_active":"2024-09-17T05:02:07.320746+0000","last_peered":"2024-09-17T05:02:07.320746+0000","last_clean":"2024-09-17T05:02:07.320746+0000","last_became_active":"2024-09-17T05:02:06.335166+0000","last_became_peered":"2024-09-17T05:02:06.335166+0000","last_unstale":"2024-09-17T05:02:07.320746+0000","last_undegraded":"2024-09-17T05:02:07.320746+0000","last_fullsized":"2024-09-17T05:02:07.320746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:02.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314664+0000","last_change":"2024-09-17T05:02:05.336301+0000","last_active":"2024-09-17T05:02:07.314664+0000","last_peered":"2024-09-17T05:02:07.314664+0000","last_clean":"2024-09-17T05:02:07.314664+0000","last_became_active":"2024-09-17T05:02:05.331553+0000","last_became_peered":"2024-09-17T05:02:05.331553+0000","last_unstale":"2024-09-17T05:02:07.314664+0000","last_undegraded":"2024-09-17T05:02:07.314664+0000","last_fullsized":"2024-09-17T05:02:07.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:23.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420226+0000","last_change":"2024-09-17T05:02:06.342104+0000","last_active":"2024-09-17T05:02:07.420226+0000","last_peered":"2024-09-17T05:02:07.420226+0000","last_clean":"2024-09-17T05:02:07.420226+0000","last_became_active":"2024-09-17T05:02:06.341775+0000","last_became_peered":"2024-09-17T05:02:06.341775+0000","last_unstale":"2024-09-17T05:02:07.420226+0000","last_undegraded":"2024-09-17T05:02:07.420226+0000","last_fullsized":"2024-09-17T05:02:07.420226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:24.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321552+0000","last_change":"2024-09-17T05:02:05.323758+0000","last_active":"2024-09-17T05:02:07.321552+0000","last_peered":"2024-09-17T05:02:07.321552+0000","last_clean":"2024-09-17T05:02:07.321552+0000","last_became_active":"2024-09-17T05:02:05.323426+0000","last_became_peered":"2024-09-17T05:02:05.323426+0000","last_unstale":"2024-09-17T05:02:07.321552+0000","last_undegraded":"2024-09-17T05:02:07.321552+0000","last_fullsized":"2024-09-17T05:02:07.321552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:29.210315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320848+0000","last_change":"2024-09-17T05:02:06.345826+0000","last_active":"2024-09-17T05:02:07.320848+0000","last_peered":"2024-09-17T05:02:07.320848+0000","last_clean":"2024-09-17T05:02:07.320848+0000","last_became_active":"2024-09-17T05:02:06.345343+0000","last_became_peered":"2024-09-17T05:02:06.345343+0000","last_unstale":"2024-09-17T05:02:07.320848+0000","last_undegraded":"2024-09-17T05:02:07.320848+0000","last_fullsized":"2024-09-17T05:02:07.320848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:09.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320977+0000","last_change":"2024-09-17T05:02:05.323105+0000","last_active":"2024-09-17T05:02:07.320977+0000","last_peered":"2024-09-17T05:02:07.320977+0000","last_clean":"2024-09-17T05:02:07.320977+0000","last_became_active":"2024-09-17T05:02:05.320585+0000","last_became_peered":"2024-09-17T05:02:05.320585+0000","last_unstale":"2024-09-17T05:02:07.320977+0000","last_undegraded":"2024-09-17T05:02:07.320977+0000","last_fullsized":"2024-09-17T05:02:07.320977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420267+0000","last_change":"2024-09-17T05:02:06.359915+0000","last_active":"2024-09-17T05:02:07.420267+0000","last_peered":"2024-09-17T05:02:07.420267+0000","last_clean":"2024-09-17T05:02:07.420267+0000","last_became_active":"2024-09-17T05:02:06.359544+0000","last_became_peered":"2024-09-17T05:02:06.359544+0000","last_unstale":"2024-09-17T05:02:07.420267+0000","last_undegraded":"2024-09-17T05:02:07.420267+0000","last_fullsized":"2024-09-17T05:02:07.420267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420061+0000","last_change":"2024-09-17T05:02:05.320829+0000","last_active":"2024-09-17T05:02:07.420061+0000","last_peered":"2024-09-17T05:02:07.420061+0000","last_clean":"2024-09-17T05:02:07.420061+0000","last_became_active":"2024-09-17T05:02:05.320509+0000","last_became_peered":"2024-09-17T05:02:05.320509+0000","last_unstale":"2024-09-17T05:02:07.420061+0000","last_undegraded":"2024-09-17T05:02:07.420061+0000","last_fullsized":"2024-09-17T05:02:07.420061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:36.381090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349878+0000","last_change":"2024-09-17T05:02:06.342348+0000","last_active":"2024-09-17T05:02:07.349878+0000","last_peered":"2024-09-17T05:02:07.349878+0000","last_clean":"2024-09-17T05:02:07.349878+0000","last_became_active":"2024-09-17T05:02:06.342077+0000","last_became_peered":"2024-09-17T05:02:06.342077+0000","last_unstale":"2024-09-17T05:02:07.349878+0000","last_undegraded":"2024-09-17T05:02:07.349878+0000","last_fullsized":"2024-09-17T05:02:07.349878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:24.585972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314795+0000","last_change":"2024-09-17T05:02:05.331453+0000","last_active":"2024-09-17T05:02:07.314795+0000","last_peered":"2024-09-17T05:02:07.314795+0000","last_clean":"2024-09-17T05:02:07.314795+0000","last_became_active":"2024-09-17T05:02:05.330843+0000","last_became_peered":"2024-09-17T05:02:05.330843+0000","last_unstale":"2024-09-17T05:02:07.314795+0000","last_undegraded":"2024-09-17T05:02:07.314795+0000","last_fullsized":"2024-09-17T05:02:07.314795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:53.874382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346719+0000","last_change":"2024-09-17T05:02:06.338645+0000","last_active":"2024-09-17T05:02:07.346719+0000","last_peered":"2024-09-17T05:02:07.346719+0000","last_clean":"2024-09-17T05:02:07.346719+0000","last_became_active":"2024-09-17T05:02:06.338343+0000","last_became_peered":"2024-09-17T05:02:06.338343+0000","last_unstale":"2024-09-17T05:02:07.346719+0000","last_undegraded":"2024-09-17T05:02:07.346719+0000","last_fullsized":"2024-09-17T05:02:07.346719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419537+0000","last_change":"2024-09-17T05:02:05.318024+0000","last_active":"2024-09-17T05:02:07.419537+0000","last_peered":"2024-09-17T05:02:07.419537+0000","last_clean":"2024-09-17T05:02:07.419537+0000","last_became_active":"2024-09-17T05:02:05.317528+0000","last_became_peered":"2024-09-17T05:02:05.317528+0000","last_unstale":"2024-09-17T05:02:07.419537+0000","last_undegraded":"2024-09-17T05:02:07.419537+0000","last_fullsized":"2024-09-17T05:02:07.419537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:22.090770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420452+0000","last_change":"2024-09-17T05:02:06.336675+0000","last_active":"2024-09-17T05:02:07.420452+0000","last_peered":"2024-09-17T05:02:07.420452+0000","last_clean":"2024-09-17T05:02:07.420452+0000","last_became_active":"2024-09-17T05:02:06.336377+0000","last_became_peered":"2024-09-17T05:02:06.336377+0000","last_unstale":"2024-09-17T05:02:07.420452+0000","last_undegraded":"2024-09-17T05:02:07.420452+0000","last_fullsized":"2024-09-17T05:02:07.420452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.158532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420724+0000","last_change":"2024-09-17T05:02:05.334942+0000","last_active":"2024-09-17T05:02:07.420724+0000","last_peered":"2024-09-17T05:02:07.420724+0000","last_clean":"2024-09-17T05:02:07.420724+0000","last_became_active":"2024-09-17T05:02:05.334624+0000","last_became_peered":"2024-09-17T05:02:05.334624+0000","last_unstale":"2024-09-17T05:02:07.420724+0000","last_undegraded":"2024-09-17T05:02:07.420724+0000","last_fullsized":"2024-09-17T05:02:07.420724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:14.855709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420380+0000","last_change":"2024-09-17T05:02:06.334939+0000","last_active":"2024-09-17T05:02:07.420380+0000","last_peered":"2024-09-17T05:02:07.420380+0000","last_clean":"2024-09-17T05:02:07.420380+0000","last_became_active":"2024-09-17T05:02:06.334587+0000","last_became_peered":"2024-09-17T05:02:06.334587+0000","last_unstale":"2024-09-17T05:02:07.420380+0000","last_undegraded":"2024-09-17T05:02:07.420380+0000","last_fullsized":"2024-09-17T05:02:07.420380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:37.387302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315252+0000","last_change":"2024-09-17T05:02:05.326248+0000","last_active":"2024-09-17T05:02:07.315252+0000","last_peered":"2024-09-17T05:02:07.315252+0000","last_clean":"2024-09-17T05:02:07.315252+0000","last_became_active":"2024-09-17T05:02:05.325621+0000","last_became_peered":"2024-09-17T05:02:05.325621+0000","last_unstale":"2024-09-17T05:02:07.315252+0000","last_undegraded":"2024-09-17T05:02:07.315252+0000","last_fullsized":"2024-09-17T05:02:07.315252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:35.920973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420572+0000","last_change":"2024-09-17T05:02:06.332684+0000","last_active":"2024-09-17T05:02:07.420572+0000","last_peered":"2024-09-17T05:02:07.420572+0000","last_clean":"2024-09-17T05:02:07.420572+0000","last_became_active":"2024-09-17T05:02:06.332268+0000","last_became_peered":"2024-09-17T05:02:06.332268+0000","last_unstale":"2024-09-17T05:02:07.420572+0000","last_undegraded":"2024-09-17T05:02:07.420572+0000","last_fullsized":"2024-09-17T05:02:07.420572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:36.983687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346854+0000","last_change":"2024-09-17T05:02:05.329129+0000","last_active":"2024-09-17T05:02:07.346854+0000","last_peered":"2024-09-17T05:02:07.346854+0000","last_clean":"2024-09-17T05:02:07.346854+0000","last_became_active":"2024-09-17T05:02:05.328783+0000","last_became_peered":"2024-09-17T05:02:05.328783+0000","last_unstale":"2024-09-17T05:02:07.346854+0000","last_undegraded":"2024-09-17T05:02:07.346854+0000","last_fullsized":"2024-09-17T05:02:07.346854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:01.114967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.315373+0000","last_change":"2024-09-17T05:02:05.315373+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.315373+0000","last_undegraded":"2024-09-17T05:02:05.315373+0000","last_fullsized":"2024-09-17T05:02:05.315373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T05:02:05.325258+0000","last_change":"2024-09-17T05:02:05.324272+0000","last_active":"2024-09-17T05:02:05.325258+0000","last_peered":"2024-09-17T05:02:05.325258+0000","last_clean":"2024-09-17T05:02:05.325258+0000","last_became_active":"2024-09-17T05:02:05.323884+0000","last_became_peered":"2024-09-17T05:02:05.323884+0000","last_unstale":"2024-09-17T05:02:05.325258+0000","last_undegraded":"2024-09-17T05:02:05.325258+0000","last_fullsized":"2024-09-17T05:02:05.325258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:51.335778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314537+0000","last_change":"2024-09-17T05:02:06.321153+0000","last_active":"2024-09-17T05:02:07.314537+0000","last_peered":"2024-09-17T05:02:07.314537+0000","last_clean":"2024-09-17T05:02:07.314537+0000","last_became_active":"2024-09-17T05:02:06.320793+0000","last_became_peered":"2024-09-17T05:02:06.320793+0000","last_unstale":"2024-09-17T05:02:07.314537+0000","last_undegraded":"2024-09-17T05:02:07.314537+0000","last_fullsized":"2024-09-17T05:02:07.314537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:49.889726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T05:02:05.324925+0000","last_change":"2024-09-17T05:02:05.321831+0000","last_active":"2024-09-17T05:02:05.324925+0000","last_peered":"2024-09-17T05:02:05.324925+0000","last_clean":"2024-09-17T05:02:05.324925+0000","last_became_active":"2024-09-17T05:02:05.321507+0000","last_became_peered":"2024-09-17T05:02:05.321507+0000","last_unstale":"2024-09-17T05:02:05.324925+0000","last_undegraded":"2024-09-17T05:02:05.324925+0000","last_fullsized":"2024-09-17T05:02:05.324925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:10.699379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319927+0000","last_change":"2024-09-17T05:02:06.362961+0000","last_active":"2024-09-17T05:02:07.319927+0000","last_peered":"2024-09-17T05:02:07.319927+0000","last_clean":"2024-09-17T05:02:07.319927+0000","last_became_active":"2024-09-17T05:02:06.362589+0000","last_became_peered":"2024-09-17T05:02:06.362589+0000","last_unstale":"2024-09-17T05:02:07.319927+0000","last_undegraded":"2024-09-17T05:02:07.319927+0000","last_fullsized":"2024-09-17T05:02:07.319927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:43.235010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420209+0000","last_change":"2024-09-17T05:02:05.334796+0000","last_active":"2024-09-17T05:02:07.420209+0000","last_peered":"2024-09-17T05:02:07.420209+0000","last_clean":"2024-09-17T05:02:07.420209+0000","last_became_active":"2024-09-17T05:02:05.334457+0000","last_became_peered":"2024-09-17T05:02:05.334457+0000","last_unstale":"2024-09-17T05:02:07.420209+0000","last_undegraded":"2024-09-17T05:02:07.420209+0000","last_fullsized":"2024-09-17T05:02:07.420209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:50.024608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349461+0000","last_change":"2024-09-17T05:02:06.360661+0000","last_active":"2024-09-17T05:02:07.349461+0000","last_peered":"2024-09-17T05:02:07.349461+0000","last_clean":"2024-09-17T05:02:07.349461+0000","last_became_active":"2024-09-17T05:02:06.360293+0000","last_became_peered":"2024-09-17T05:02:06.360293+0000","last_unstale":"2024-09-17T05:02:07.349461+0000","last_undegraded":"2024-09-17T05:02:07.349461+0000","last_fullsized":"2024-09-17T05:02:07.349461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420548+0000","last_change":"2024-09-17T05:02:05.335089+0000","last_active":"2024-09-17T05:02:07.420548+0000","last_peered":"2024-09-17T05:02:07.420548+0000","last_clean":"2024-09-17T05:02:07.420548+0000","last_became_active":"2024-09-17T05:02:05.334761+0000","last_became_peered":"2024-09-17T05:02:05.334761+0000","last_unstale":"2024-09-17T05:02:07.420548+0000","last_undegraded":"2024-09-17T05:02:07.420548+0000","last_fullsized":"2024-09-17T05:02:07.420548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319697+0000","last_change":"2024-09-17T05:02:06.337072+0000","last_active":"2024-09-17T05:02:07.319697+0000","last_peered":"2024-09-17T05:02:07.319697+0000","last_clean":"2024-09-17T05:02:07.319697+0000","last_became_active":"2024-09-17T05:02:06.335541+0000","last_became_peered":"2024-09-17T05:02:06.335541+0000","last_unstale":"2024-09-17T05:02:07.319697+0000","last_undegraded":"2024-09-17T05:02:07.319697+0000","last_fullsized":"2024-09-17T05:02:07.319697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:33.058079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419849+0000","last_change":"2024-09-17T05:02:05.317067+0000","last_active":"2024-09-17T05:02:07.419849+0000","last_peered":"2024-09-17T05:02:07.419849+0000","last_clean":"2024-09-17T05:02:07.419849+0000","last_became_active":"2024-09-17T05:02:05.316750+0000","last_became_peered":"2024-09-17T05:02:05.316750+0000","last_unstale":"2024-09-17T05:02:07.419849+0000","last_undegraded":"2024-09-17T05:02:07.419849+0000","last_fullsized":"2024-09-17T05:02:07.419849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:11.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346952+0000","last_change":"2024-09-17T05:02:06.342012+0000","last_active":"2024-09-17T05:02:07.346952+0000","last_peered":"2024-09-17T05:02:07.346952+0000","last_clean":"2024-09-17T05:02:07.346952+0000","last_became_active":"2024-09-17T05:02:06.341627+0000","last_became_peered":"2024-09-17T05:02:06.341627+0000","last_unstale":"2024-09-17T05:02:07.346952+0000","last_undegraded":"2024-09-17T05:02:07.346952+0000","last_fullsized":"2024-09-17T05:02:07.346952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:04.620408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315375+0000","last_change":"2024-09-17T05:02:05.331974+0000","last_active":"2024-09-17T05:02:07.315375+0000","last_peered":"2024-09-17T05:02:07.315375+0000","last_clean":"2024-09-17T05:02:07.315375+0000","last_became_active":"2024-09-17T05:02:05.331277+0000","last_became_peered":"2024-09-17T05:02:05.331277+0000","last_unstale":"2024-09-17T05:02:07.315375+0000","last_undegraded":"2024-09-17T05:02:07.315375+0000","last_fullsized":"2024-09-17T05:02:07.315375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:14.008073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350015+0000","last_change":"2024-09-17T05:02:06.361595+0000","last_active":"2024-09-17T05:02:07.350015+0000","last_peered":"2024-09-17T05:02:07.350015+0000","last_clean":"2024-09-17T05:02:07.350015+0000","last_became_active":"2024-09-17T05:02:06.361219+0000","last_became_peered":"2024-09-17T05:02:06.361219+0000","last_unstale":"2024-09-17T05:02:07.350015+0000","last_undegraded":"2024-09-17T05:02:07.350015+0000","last_fullsized":"2024-09-17T05:02:07.350015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:47.878683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.348883+0000","last_change":"2024-09-17T05:02:05.334870+0000","last_active":"2024-09-17T05:02:07.348883+0000","last_peered":"2024-09-17T05:02:07.348883+0000","last_clean":"2024-09-17T05:02:07.348883+0000","last_became_active":"2024-09-17T05:02:05.334512+0000","last_became_peered":"2024-09-17T05:02:05.334512+0000","last_unstale":"2024-09-17T05:02:07.348883+0000","last_undegraded":"2024-09-17T05:02:07.348883+0000","last_fullsized":"2024-09-17T05:02:07.348883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.315369+0000","last_change":"2024-09-17T05:02:05.315369+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.315369+0000","last_undegraded":"2024-09-17T05:02:05.315369+0000","last_fullsized":"2024-09-17T05:02:05.315369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346976+0000","last_change":"2024-09-17T05:02:05.325366+0000","last_active":"2024-09-17T05:02:07.346976+0000","last_peered":"2024-09-17T05:02:07.346976+0000","last_clean":"2024-09-17T05:02:07.346976+0000","last_became_active":"2024-09-17T05:02:05.325037+0000","last_became_peered":"2024-09-17T05:02:05.325037+0000","last_unstale":"2024-09-17T05:02:07.346976+0000","last_undegraded":"2024-09-17T05:02:07.346976+0000","last_fullsized":"2024-09-17T05:02:07.346976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346633+0000","last_change":"2024-09-17T05:02:06.336842+0000","last_active":"2024-09-17T05:02:07.346633+0000","last_peered":"2024-09-17T05:02:07.346633+0000","last_clean":"2024-09-17T05:02:07.346633+0000","last_became_active":"2024-09-17T05:02:06.334045+0000","last_became_peered":"2024-09-17T05:02:06.334045+0000","last_unstale":"2024-09-17T05:02:07.346633+0000","last_undegraded":"2024-09-17T05:02:07.346633+0000","last_fullsized":"2024-09-17T05:02:07.346633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:30.924123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320355+0000","last_change":"2024-09-17T05:02:05.316008+0000","last_active":"2024-09-17T05:02:07.320355+0000","last_peered":"2024-09-17T05:02:07.320355+0000","last_clean":"2024-09-17T05:02:07.320355+0000","last_became_active":"2024-09-17T05:02:05.315680+0000","last_became_peered":"2024-09-17T05:02:05.315680+0000","last_unstale":"2024-09-17T05:02:07.320355+0000","last_undegraded":"2024-09-17T05:02:07.320355+0000","last_fullsized":"2024-09-17T05:02:07.320355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:25.314134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420182+0000","last_change":"2024-09-17T05:02:06.319976+0000","last_active":"2024-09-17T05:02:07.420182+0000","last_peered":"2024-09-17T05:02:07.420182+0000","last_clean":"2024-09-17T05:02:07.420182+0000","last_became_active":"2024-09-17T05:02:06.319609+0000","last_became_peered":"2024-09-17T05:02:06.319609+0000","last_unstale":"2024-09-17T05:02:07.420182+0000","last_undegraded":"2024-09-17T05:02:07.420182+0000","last_fullsized":"2024-09-17T05:02:07.420182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349774+0000","last_change":"2024-09-17T05:02:05.331789+0000","last_active":"2024-09-17T05:02:07.349774+0000","last_peered":"2024-09-17T05:02:07.349774+0000","last_clean":"2024-09-17T05:02:07.349774+0000","last_became_active":"2024-09-17T05:02:05.327820+0000","last_became_peered":"2024-09-17T05:02:05.327820+0000","last_unstale":"2024-09-17T05:02:07.349774+0000","last_undegraded":"2024-09-17T05:02:07.349774+0000","last_fullsized":"2024-09-17T05:02:07.349774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314859+0000","last_change":"2024-09-17T05:02:06.333560+0000","last_active":"2024-09-17T05:02:07.314859+0000","last_peered":"2024-09-17T05:02:07.314859+0000","last_clean":"2024-09-17T05:02:07.314859+0000","last_became_active":"2024-09-17T05:02:06.333210+0000","last_became_peered":"2024-09-17T05:02:06.333210+0000","last_unstale":"2024-09-17T05:02:07.314859+0000","last_undegraded":"2024-09-17T05:02:07.314859+0000","last_fullsized":"2024-09-17T05:02:07.314859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.884069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314950+0000","last_change":"2024-09-17T05:02:05.325774+0000","last_active":"2024-09-17T05:02:07.314950+0000","last_peered":"2024-09-17T05:02:07.314950+0000","last_clean":"2024-09-17T05:02:07.314950+0000","last_became_active":"2024-09-17T05:02:05.325459+0000","last_became_peered":"2024-09-17T05:02:05.325459+0000","last_unstale":"2024-09-17T05:02:07.314950+0000","last_undegraded":"2024-09-17T05:02:07.314950+0000","last_fullsized":"2024-09-17T05:02:07.314950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:55.257416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349680+0000","last_change":"2024-09-17T05:02:06.340513+0000","last_active":"2024-09-17T05:02:07.349680+0000","last_peered":"2024-09-17T05:02:07.349680+0000","last_clean":"2024-09-17T05:02:07.349680+0000","last_became_active":"2024-09-17T05:02:06.340154+0000","last_became_peered":"2024-09-17T05:02:06.340154+0000","last_unstale":"2024-09-17T05:02:07.349680+0000","last_undegraded":"2024-09-17T05:02:07.349680+0000","last_fullsized":"2024-09-17T05:02:07.349680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320990+0000","last_change":"2024-09-17T05:02:05.325138+0000","last_active":"2024-09-17T05:02:07.320990+0000","last_peered":"2024-09-17T05:02:07.320990+0000","last_clean":"2024-09-17T05:02:07.320990+0000","last_became_active":"2024-09-17T05:02:05.324679+0000","last_became_peered":"2024-09-17T05:02:05.324679+0000","last_unstale":"2024-09-17T05:02:07.320990+0000","last_undegraded":"2024-09-17T05:02:07.320990+0000","last_fullsized":"2024-09-17T05:02:07.320990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:39.915152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315287+0000","last_change":"2024-09-17T05:02:06.331069+0000","last_active":"2024-09-17T05:02:07.315287+0000","last_peered":"2024-09-17T05:02:07.315287+0000","last_clean":"2024-09-17T05:02:07.315287+0000","last_became_active":"2024-09-17T05:02:06.330731+0000","last_became_peered":"2024-09-17T05:02:06.330731+0000","last_unstale":"2024-09-17T05:02:07.315287+0000","last_undegraded":"2024-09-17T05:02:07.315287+0000","last_fullsized":"2024-09-17T05:02:07.315287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:38.520351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349709+0000","last_change":"2024-09-17T05:02:05.335344+0000","last_active":"2024-09-17T05:02:07.349709+0000","last_peered":"2024-09-17T05:02:07.349709+0000","last_clean":"2024-09-17T05:02:07.349709+0000","last_became_active":"2024-09-17T05:02:05.334313+0000","last_became_peered":"2024-09-17T05:02:05.334313+0000","last_unstale":"2024-09-17T05:02:07.349709+0000","last_undegraded":"2024-09-17T05:02:07.349709+0000","last_fullsized":"2024-09-17T05:02:07.349709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420491+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:02:07.420491+0000","last_peered":"2024-09-17T05:02:07.420491+0000","last_clean":"2024-09-17T05:02:07.420491+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:02:07.420491+0000","last_undegraded":"2024-09-17T05:02:07.420491+0000","last_fullsized":"2024-09-17T05:02:07.420491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":458752,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":462848,"data_stored":190415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":188769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":190873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":779069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":450560,"data_stored":188771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1024000,"data_stored":771880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:02:10.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 -- 172.21.15.16:0/3686901698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 msgr2=0x7facc415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 --2- 172.21.15.16:0/3686901698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc415c680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7facac009bc0 tx=0x7facac01cb10 comp rx=0 tx=0).stop 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 -- 172.21.15.16:0/3686901698 shutdown_connections 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 --2- 172.21.15.16:0/3686901698 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7facc4073240 0x7facc4073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 --2- 172.21.15.16:0/3686901698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc415c680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 -- 172.21.15.16:0/3686901698 >> 172.21.15.16:0/3686901698 conn(0x7facc406f900 msgr2=0x7facc40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 -- 172.21.15.16:0/3686901698 shutdown_connections 2024-09-17T05:02:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.134+0000 7facca200640 1 -- 172.21.15.16:0/3686901698 wait complete. 2024-09-17T05:02:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 Processor -- start 2024-09-17T05:02:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 -- start start 2024-09-17T05:02:11.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7facc4073240 0x7facc41f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7facc40b2f50 0x7facc40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc40cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7facc4168d60 con 0x7facc415c270 2024-09-17T05:02:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7facc4168be0 con 0x7facc4073240 2024-09-17T05:02:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7facc4168ee0 con 0x7facc40b2f50 2024-09-17T05:02:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc40cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc40cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48014/0 (socket says 172.21.15.16:48014) 2024-09-17T05:02:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 -- 172.21.15.16:0/1988620221 learned_addr learned my addr 172.21.15.16:0/1988620221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facbaa00640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7facc40b2f50 0x7facc40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3400640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7facc4073240 0x7facc41f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 -- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7facc40b2f50 msgr2=0x7facc40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7facc40b2f50 0x7facc40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 -- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7facc4073240 msgr2=0x7facc41f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7facc4073240 0x7facc41f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 -- 172.21.15.16:0/1988620221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc41fd210 con 0x7facc415c270 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facc3e00640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc40cf4c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7facbc00d640 tx=0x7facbc00db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc017070 con 0x7facc415c270 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facbc005310 con 0x7facc415c270 2024-09-17T05:02:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0047b0 con 0x7facc415c270 2024-09-17T05:02:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc41f8b50 con 0x7facc415c270 2024-09-17T05:02:11.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.138+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41fc970 con 0x7facc415c270 2024-09-17T05:02:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.142+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc4073650 con 0x7facc415c270 2024-09-17T05:02:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.142+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7facbc004950 con 0x7facc415c270 2024-09-17T05:02:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.142+0000 7facbbe00640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faca0073230 0x7faca0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.142+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7facbc094400 con 0x7facc415c270 2024-09-17T05:02:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.142+0000 7facc3400640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faca0073230 0x7faca0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.142+0000 7facc3400640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faca0073230 0x7faca0075740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7facb40015f0 tx=0x7facb4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:11.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.146+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7facbc05f280 con 0x7facc415c270 2024-09-17T05:02:11.264 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-17T05:02:11.262+0000 7fa8eb000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:02:11.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.318+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facc40b42a0 con 0x7faca0073230 2024-09-17T05:02:11.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.322+0000 7facbbe00640 1 -- 172.21.15.16:0/1988620221 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251694 (secure 0 0 0) 0x7facc40b42a0 con 0x7faca0073230 2024-09-17T05:02:11.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:11.330 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:02:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faca0073230 msgr2=0x7faca0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faca0073230 0x7faca0075740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7facb40015f0 tx=0x7facb4009040 comp rx=0 tx=0).stop 2024-09-17T05:02:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 msgr2=0x7facc40cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc40cf4c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7facbc00d640 tx=0x7facbc00db00 comp rx=0 tx=0).stop 2024-09-17T05:02:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 shutdown_connections 2024-09-17T05:02:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faca0073230 0x7faca0075740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc415c270 0x7facc40cf4c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7facc40b2f50 0x7facc40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 --2- 172.21.15.16:0/1988620221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7facc4073240 0x7facc41f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 >> 172.21.15.16:0/1988620221 conn(0x7facc406f900 msgr2=0x7facc4151a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 shutdown_connections 2024-09-17T05:02:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.326+0000 7facca200640 1 -- 172.21.15.16:0/1988620221 wait complete. 2024-09-17T05:02:11.350 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-17T05:02:09.476186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237588,"kb_used_data":4676,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737132,"statfs":{"total":773094113280,"available":772850823168,"internally_reserved":0,"allocated":4788224,"data_stored":2693931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.181300"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321351+0000","last_change":"2024-09-17T05:02:06.338525+0000","last_active":"2024-09-17T05:02:07.321351+0000","last_peered":"2024-09-17T05:02:07.321351+0000","last_clean":"2024-09-17T05:02:07.321351+0000","last_became_active":"2024-09-17T05:02:06.338232+0000","last_became_peered":"2024-09-17T05:02:06.338232+0000","last_unstale":"2024-09-17T05:02:07.321351+0000","last_undegraded":"2024-09-17T05:02:07.321351+0000","last_fullsized":"2024-09-17T05:02:07.321351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:56.646622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420620+0000","last_change":"2024-09-17T05:02:06.332250+0000","last_active":"2024-09-17T05:02:07.420620+0000","last_peered":"2024-09-17T05:02:07.420620+0000","last_clean":"2024-09-17T05:02:07.420620+0000","last_became_active":"2024-09-17T05:02:06.331927+0000","last_became_peered":"2024-09-17T05:02:06.331927+0000","last_unstale":"2024-09-17T05:02:07.420620+0000","last_undegraded":"2024-09-17T05:02:07.420620+0000","last_fullsized":"2024-09-17T05:02:07.420620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320879+0000","last_change":"2024-09-17T05:02:06.336648+0000","last_active":"2024-09-17T05:02:07.320879+0000","last_peered":"2024-09-17T05:02:07.320879+0000","last_clean":"2024-09-17T05:02:07.320879+0000","last_became_active":"2024-09-17T05:02:06.336337+0000","last_became_peered":"2024-09-17T05:02:06.336337+0000","last_unstale":"2024-09-17T05:02:07.320879+0000","last_undegraded":"2024-09-17T05:02:07.320879+0000","last_fullsized":"2024-09-17T05:02:07.320879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:44.810456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321127+0000","last_change":"2024-09-17T05:02:06.348672+0000","last_active":"2024-09-17T05:02:07.321127+0000","last_peered":"2024-09-17T05:02:07.321127+0000","last_clean":"2024-09-17T05:02:07.321127+0000","last_became_active":"2024-09-17T05:02:06.348392+0000","last_became_peered":"2024-09-17T05:02:06.348392+0000","last_unstale":"2024-09-17T05:02:07.321127+0000","last_undegraded":"2024-09-17T05:02:07.321127+0000","last_fullsized":"2024-09-17T05:02:07.321127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:06.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346718+0000","last_change":"2024-09-17T05:02:06.343632+0000","last_active":"2024-09-17T05:02:07.346718+0000","last_peered":"2024-09-17T05:02:07.346718+0000","last_clean":"2024-09-17T05:02:07.346718+0000","last_became_active":"2024-09-17T05:02:06.343257+0000","last_became_peered":"2024-09-17T05:02:06.343257+0000","last_unstale":"2024-09-17T05:02:07.346718+0000","last_undegraded":"2024-09-17T05:02:07.346718+0000","last_fullsized":"2024-09-17T05:02:07.346718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:00.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420107+0000","last_change":"2024-09-17T05:02:06.330126+0000","last_active":"2024-09-17T05:02:07.420107+0000","last_peered":"2024-09-17T05:02:07.420107+0000","last_clean":"2024-09-17T05:02:07.420107+0000","last_became_active":"2024-09-17T05:02:06.329825+0000","last_became_peered":"2024-09-17T05:02:06.329825+0000","last_unstale":"2024-09-17T05:02:07.420107+0000","last_undegraded":"2024-09-17T05:02:07.420107+0000","last_fullsized":"2024-09-17T05:02:07.420107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:46.368942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420406+0000","last_change":"2024-09-17T05:02:06.343766+0000","last_active":"2024-09-17T05:02:07.420406+0000","last_peered":"2024-09-17T05:02:07.420406+0000","last_clean":"2024-09-17T05:02:07.420406+0000","last_became_active":"2024-09-17T05:02:06.343207+0000","last_became_peered":"2024-09-17T05:02:06.343207+0000","last_unstale":"2024-09-17T05:02:07.420406+0000","last_undegraded":"2024-09-17T05:02:07.420406+0000","last_fullsized":"2024-09-17T05:02:07.420406+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:51.030820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320027+0000","last_change":"2024-09-17T05:02:06.338244+0000","last_active":"2024-09-17T05:02:07.320027+0000","last_peered":"2024-09-17T05:02:07.320027+0000","last_clean":"2024-09-17T05:02:07.320027+0000","last_became_active":"2024-09-17T05:02:06.337859+0000","last_became_peered":"2024-09-17T05:02:06.337859+0000","last_unstale":"2024-09-17T05:02:07.320027+0000","last_undegraded":"2024-09-17T05:02:07.320027+0000","last_fullsized":"2024-09-17T05:02:07.320027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:36.957053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.347110+0000","last_change":"2024-09-17T05:02:06.336573+0000","last_active":"2024-09-17T05:02:07.347110+0000","last_peered":"2024-09-17T05:02:07.347110+0000","last_clean":"2024-09-17T05:02:07.347110+0000","last_became_active":"2024-09-17T05:02:06.336253+0000","last_became_peered":"2024-09-17T05:02:06.336253+0000","last_unstale":"2024-09-17T05:02:07.347110+0000","last_undegraded":"2024-09-17T05:02:07.347110+0000","last_fullsized":"2024-09-17T05:02:07.347110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:47.955020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420328+0000","last_change":"2024-09-17T05:02:06.362653+0000","last_active":"2024-09-17T05:02:07.420328+0000","last_peered":"2024-09-17T05:02:07.420328+0000","last_clean":"2024-09-17T05:02:07.420328+0000","last_became_active":"2024-09-17T05:02:06.362345+0000","last_became_peered":"2024-09-17T05:02:06.362345+0000","last_unstale":"2024-09-17T05:02:07.420328+0000","last_undegraded":"2024-09-17T05:02:07.420328+0000","last_fullsized":"2024-09-17T05:02:07.420328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:12.091714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350026+0000","last_change":"2024-09-17T05:02:06.341127+0000","last_active":"2024-09-17T05:02:07.350026+0000","last_peered":"2024-09-17T05:02:07.350026+0000","last_clean":"2024-09-17T05:02:07.350026+0000","last_became_active":"2024-09-17T05:02:06.340796+0000","last_became_peered":"2024-09-17T05:02:06.340796+0000","last_unstale":"2024-09-17T05:02:07.350026+0000","last_undegraded":"2024-09-17T05:02:07.350026+0000","last_fullsized":"2024-09-17T05:02:07.350026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:54.880594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346735+0000","last_change":"2024-09-17T05:02:06.336217+0000","last_active":"2024-09-17T05:02:07.346735+0000","last_peered":"2024-09-17T05:02:07.346735+0000","last_clean":"2024-09-17T05:02:07.346735+0000","last_became_active":"2024-09-17T05:02:06.333449+0000","last_became_peered":"2024-09-17T05:02:06.333449+0000","last_unstale":"2024-09-17T05:02:07.346735+0000","last_undegraded":"2024-09-17T05:02:07.346735+0000","last_fullsized":"2024-09-17T05:02:07.346735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:23.096982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420312+0000","last_change":"2024-09-17T05:02:06.330216+0000","last_active":"2024-09-17T05:02:07.420312+0000","last_peered":"2024-09-17T05:02:07.420312+0000","last_clean":"2024-09-17T05:02:07.420312+0000","last_became_active":"2024-09-17T05:02:06.329869+0000","last_became_peered":"2024-09-17T05:02:06.329869+0000","last_unstale":"2024-09-17T05:02:07.420312+0000","last_undegraded":"2024-09-17T05:02:07.420312+0000","last_fullsized":"2024-09-17T05:02:07.420312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419668+0000","last_change":"2024-09-17T05:02:06.337107+0000","last_active":"2024-09-17T05:02:07.419668+0000","last_peered":"2024-09-17T05:02:07.419668+0000","last_clean":"2024-09-17T05:02:07.419668+0000","last_became_active":"2024-09-17T05:02:06.336774+0000","last_became_peered":"2024-09-17T05:02:06.336774+0000","last_unstale":"2024-09-17T05:02:07.419668+0000","last_undegraded":"2024-09-17T05:02:07.419668+0000","last_fullsized":"2024-09-17T05:02:07.419668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:26.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314512+0000","last_change":"2024-09-17T05:02:06.321133+0000","last_active":"2024-09-17T05:02:07.314512+0000","last_peered":"2024-09-17T05:02:07.314512+0000","last_clean":"2024-09-17T05:02:07.314512+0000","last_became_active":"2024-09-17T05:02:06.320714+0000","last_became_peered":"2024-09-17T05:02:06.320714+0000","last_unstale":"2024-09-17T05:02:07.314512+0000","last_undegraded":"2024-09-17T05:02:07.314512+0000","last_fullsized":"2024-09-17T05:02:07.314512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:18.256704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420854+0000","last_change":"2024-09-17T05:02:06.335127+0000","last_active":"2024-09-17T05:02:07.420854+0000","last_peered":"2024-09-17T05:02:07.420854+0000","last_clean":"2024-09-17T05:02:07.420854+0000","last_became_active":"2024-09-17T05:02:06.334693+0000","last_became_peered":"2024-09-17T05:02:06.334693+0000","last_unstale":"2024-09-17T05:02:07.420854+0000","last_undegraded":"2024-09-17T05:02:07.420854+0000","last_fullsized":"2024-09-17T05:02:07.420854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:20.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315318+0000","last_change":"2024-09-17T05:02:06.352349+0000","last_active":"2024-09-17T05:02:07.315318+0000","last_peered":"2024-09-17T05:02:07.315318+0000","last_clean":"2024-09-17T05:02:07.315318+0000","last_became_active":"2024-09-17T05:02:06.352010+0000","last_became_peered":"2024-09-17T05:02:06.352010+0000","last_unstale":"2024-09-17T05:02:07.315318+0000","last_undegraded":"2024-09-17T05:02:07.315318+0000","last_fullsized":"2024-09-17T05:02:07.315318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.149726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314938+0000","last_change":"2024-09-17T05:02:06.336278+0000","last_active":"2024-09-17T05:02:07.314938+0000","last_peered":"2024-09-17T05:02:07.314938+0000","last_clean":"2024-09-17T05:02:07.314938+0000","last_became_active":"2024-09-17T05:02:06.334917+0000","last_became_peered":"2024-09-17T05:02:06.334917+0000","last_unstale":"2024-09-17T05:02:07.314938+0000","last_undegraded":"2024-09-17T05:02:07.314938+0000","last_fullsized":"2024-09-17T05:02:07.314938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:51.224167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.314200+0000","last_change":"2024-09-17T05:02:05.314200+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.314200+0000","last_undegraded":"2024-09-17T05:02:05.314200+0000","last_fullsized":"2024-09-17T05:02:05.314200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349734+0000","last_change":"2024-09-17T05:02:06.353747+0000","last_active":"2024-09-17T05:02:07.349734+0000","last_peered":"2024-09-17T05:02:07.349734+0000","last_clean":"2024-09-17T05:02:07.349734+0000","last_became_active":"2024-09-17T05:02:06.353157+0000","last_became_peered":"2024-09-17T05:02:06.353157+0000","last_unstale":"2024-09-17T05:02:07.349734+0000","last_undegraded":"2024-09-17T05:02:07.349734+0000","last_fullsized":"2024-09-17T05:02:07.349734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349652+0000","last_change":"2024-09-17T05:02:06.352255+0000","last_active":"2024-09-17T05:02:07.349652+0000","last_peered":"2024-09-17T05:02:07.349652+0000","last_clean":"2024-09-17T05:02:07.349652+0000","last_became_active":"2024-09-17T05:02:06.351865+0000","last_became_peered":"2024-09-17T05:02:06.351865+0000","last_unstale":"2024-09-17T05:02:07.349652+0000","last_undegraded":"2024-09-17T05:02:07.349652+0000","last_fullsized":"2024-09-17T05:02:07.349652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:00.377367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349479+0000","last_change":"2024-09-17T05:02:06.347103+0000","last_active":"2024-09-17T05:02:07.349479+0000","last_peered":"2024-09-17T05:02:07.349479+0000","last_clean":"2024-09-17T05:02:07.349479+0000","last_became_active":"2024-09-17T05:02:06.346592+0000","last_became_peered":"2024-09-17T05:02:06.346592+0000","last_unstale":"2024-09-17T05:02:07.349479+0000","last_undegraded":"2024-09-17T05:02:07.349479+0000","last_fullsized":"2024-09-17T05:02:07.349479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314660+0000","last_change":"2024-09-17T05:02:06.334073+0000","last_active":"2024-09-17T05:02:07.314660+0000","last_peered":"2024-09-17T05:02:07.314660+0000","last_clean":"2024-09-17T05:02:07.314660+0000","last_became_active":"2024-09-17T05:02:06.333712+0000","last_became_peered":"2024-09-17T05:02:06.333712+0000","last_unstale":"2024-09-17T05:02:07.314660+0000","last_undegraded":"2024-09-17T05:02:07.314660+0000","last_fullsized":"2024-09-17T05:02:07.314660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:51.877414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321024+0000","last_change":"2024-09-17T05:02:06.345627+0000","last_active":"2024-09-17T05:02:07.321024+0000","last_peered":"2024-09-17T05:02:07.321024+0000","last_clean":"2024-09-17T05:02:07.321024+0000","last_became_active":"2024-09-17T05:02:06.345225+0000","last_became_peered":"2024-09-17T05:02:06.345225+0000","last_unstale":"2024-09-17T05:02:07.321024+0000","last_undegraded":"2024-09-17T05:02:07.321024+0000","last_fullsized":"2024-09-17T05:02:07.321024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:59.047797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420455+0000","last_change":"2024-09-17T05:02:06.338993+0000","last_active":"2024-09-17T05:02:07.420455+0000","last_peered":"2024-09-17T05:02:07.420455+0000","last_clean":"2024-09-17T05:02:07.420455+0000","last_became_active":"2024-09-17T05:02:06.337798+0000","last_became_peered":"2024-09-17T05:02:06.337798+0000","last_unstale":"2024-09-17T05:02:07.420455+0000","last_undegraded":"2024-09-17T05:02:07.420455+0000","last_fullsized":"2024-09-17T05:02:07.420455+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:01.425057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320694+0000","last_change":"2024-09-17T05:02:06.350077+0000","last_active":"2024-09-17T05:02:07.320694+0000","last_peered":"2024-09-17T05:02:07.320694+0000","last_clean":"2024-09-17T05:02:07.320694+0000","last_became_active":"2024-09-17T05:02:06.349773+0000","last_became_peered":"2024-09-17T05:02:06.349773+0000","last_unstale":"2024-09-17T05:02:07.320694+0000","last_undegraded":"2024-09-17T05:02:07.320694+0000","last_fullsized":"2024-09-17T05:02:07.320694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:35.430354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350145+0000","last_change":"2024-09-17T05:02:06.332912+0000","last_active":"2024-09-17T05:02:07.350145+0000","last_peered":"2024-09-17T05:02:07.350145+0000","last_clean":"2024-09-17T05:02:07.350145+0000","last_became_active":"2024-09-17T05:02:06.332354+0000","last_became_peered":"2024-09-17T05:02:06.332354+0000","last_unstale":"2024-09-17T05:02:07.350145+0000","last_undegraded":"2024-09-17T05:02:07.350145+0000","last_fullsized":"2024-09-17T05:02:07.350145+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:22.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420872+0000","last_change":"2024-09-17T05:02:06.335589+0000","last_active":"2024-09-17T05:02:07.420872+0000","last_peered":"2024-09-17T05:02:07.420872+0000","last_clean":"2024-09-17T05:02:07.420872+0000","last_became_active":"2024-09-17T05:02:06.335120+0000","last_became_peered":"2024-09-17T05:02:06.335120+0000","last_unstale":"2024-09-17T05:02:07.420872+0000","last_undegraded":"2024-09-17T05:02:07.420872+0000","last_fullsized":"2024-09-17T05:02:07.420872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:06.399066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314944+0000","last_change":"2024-09-17T05:02:06.331500+0000","last_active":"2024-09-17T05:02:07.314944+0000","last_peered":"2024-09-17T05:02:07.314944+0000","last_clean":"2024-09-17T05:02:07.314944+0000","last_became_active":"2024-09-17T05:02:06.331184+0000","last_became_peered":"2024-09-17T05:02:06.331184+0000","last_unstale":"2024-09-17T05:02:07.314944+0000","last_undegraded":"2024-09-17T05:02:07.314944+0000","last_fullsized":"2024-09-17T05:02:07.314944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:48.958710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314593+0000","last_change":"2024-09-17T05:02:06.330526+0000","last_active":"2024-09-17T05:02:07.314593+0000","last_peered":"2024-09-17T05:02:07.314593+0000","last_clean":"2024-09-17T05:02:07.314593+0000","last_became_active":"2024-09-17T05:02:06.330156+0000","last_became_peered":"2024-09-17T05:02:06.330156+0000","last_unstale":"2024-09-17T05:02:07.314593+0000","last_undegraded":"2024-09-17T05:02:07.314593+0000","last_fullsized":"2024-09-17T05:02:07.314593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420681+0000","last_change":"2024-09-17T05:02:06.356417+0000","last_active":"2024-09-17T05:02:07.420681+0000","last_peered":"2024-09-17T05:02:07.420681+0000","last_clean":"2024-09-17T05:02:07.420681+0000","last_became_active":"2024-09-17T05:02:06.342863+0000","last_became_peered":"2024-09-17T05:02:06.342863+0000","last_unstale":"2024-09-17T05:02:07.420681+0000","last_undegraded":"2024-09-17T05:02:07.420681+0000","last_fullsized":"2024-09-17T05:02:07.420681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:46.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320315+0000","last_change":"2024-09-17T05:02:06.337797+0000","last_active":"2024-09-17T05:02:07.320315+0000","last_peered":"2024-09-17T05:02:07.320315+0000","last_clean":"2024-09-17T05:02:07.320315+0000","last_became_active":"2024-09-17T05:02:06.337396+0000","last_became_peered":"2024-09-17T05:02:06.337396+0000","last_unstale":"2024-09-17T05:02:07.320315+0000","last_undegraded":"2024-09-17T05:02:07.320315+0000","last_fullsized":"2024-09-17T05:02:07.320315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:42.616845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315319+0000","last_change":"2024-09-17T05:02:06.348634+0000","last_active":"2024-09-17T05:02:07.315319+0000","last_peered":"2024-09-17T05:02:07.315319+0000","last_clean":"2024-09-17T05:02:07.315319+0000","last_became_active":"2024-09-17T05:02:06.348284+0000","last_became_peered":"2024-09-17T05:02:06.348284+0000","last_unstale":"2024-09-17T05:02:07.315319+0000","last_undegraded":"2024-09-17T05:02:07.315319+0000","last_fullsized":"2024-09-17T05:02:07.315319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:41.455558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420652+0000","last_change":"2024-09-17T05:02:05.319632+0000","last_active":"2024-09-17T05:02:07.420652+0000","last_peered":"2024-09-17T05:02:07.420652+0000","last_clean":"2024-09-17T05:02:07.420652+0000","last_became_active":"2024-09-17T05:02:05.319321+0000","last_became_peered":"2024-09-17T05:02:05.319321+0000","last_unstale":"2024-09-17T05:02:07.420652+0000","last_undegraded":"2024-09-17T05:02:07.420652+0000","last_fullsized":"2024-09-17T05:02:07.420652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:46.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420445+0000","last_change":"2024-09-17T05:02:06.334112+0000","last_active":"2024-09-17T05:02:07.420445+0000","last_peered":"2024-09-17T05:02:07.420445+0000","last_clean":"2024-09-17T05:02:07.420445+0000","last_became_active":"2024-09-17T05:02:06.333727+0000","last_became_peered":"2024-09-17T05:02:06.333727+0000","last_unstale":"2024-09-17T05:02:07.420445+0000","last_undegraded":"2024-09-17T05:02:07.420445+0000","last_fullsized":"2024-09-17T05:02:07.420445+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:48.515835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420440+0000","last_change":"2024-09-17T05:02:05.330941+0000","last_active":"2024-09-17T05:02:07.420440+0000","last_peered":"2024-09-17T05:02:07.420440+0000","last_clean":"2024-09-17T05:02:07.420440+0000","last_became_active":"2024-09-17T05:02:05.330610+0000","last_became_peered":"2024-09-17T05:02:05.330610+0000","last_unstale":"2024-09-17T05:02:07.420440+0000","last_undegraded":"2024-09-17T05:02:07.420440+0000","last_fullsized":"2024-09-17T05:02:07.420440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:35.977474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315276+0000","last_change":"2024-09-17T05:02:06.367617+0000","last_active":"2024-09-17T05:02:07.315276+0000","last_peered":"2024-09-17T05:02:07.315276+0000","last_clean":"2024-09-17T05:02:07.315276+0000","last_became_active":"2024-09-17T05:02:06.367102+0000","last_became_peered":"2024-09-17T05:02:06.367102+0000","last_unstale":"2024-09-17T05:02:07.315276+0000","last_undegraded":"2024-09-17T05:02:07.315276+0000","last_fullsized":"2024-09-17T05:02:07.315276+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.094882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321076+0000","last_change":"2024-09-17T05:02:05.321020+0000","last_active":"2024-09-17T05:02:07.321076+0000","last_peered":"2024-09-17T05:02:07.321076+0000","last_clean":"2024-09-17T05:02:07.321076+0000","last_became_active":"2024-09-17T05:02:05.320171+0000","last_became_peered":"2024-09-17T05:02:05.320171+0000","last_unstale":"2024-09-17T05:02:07.321076+0000","last_undegraded":"2024-09-17T05:02:07.321076+0000","last_fullsized":"2024-09-17T05:02:07.321076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:00.418844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420627+0000","last_change":"2024-09-17T05:02:06.348099+0000","last_active":"2024-09-17T05:02:07.420627+0000","last_peered":"2024-09-17T05:02:07.420627+0000","last_clean":"2024-09-17T05:02:07.420627+0000","last_became_active":"2024-09-17T05:02:06.347767+0000","last_became_peered":"2024-09-17T05:02:06.347767+0000","last_unstale":"2024-09-17T05:02:07.420627+0000","last_undegraded":"2024-09-17T05:02:07.420627+0000","last_fullsized":"2024-09-17T05:02:07.420627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:39.458764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420697+0000","last_change":"2024-09-17T05:02:05.323652+0000","last_active":"2024-09-17T05:02:07.420697+0000","last_peered":"2024-09-17T05:02:07.420697+0000","last_clean":"2024-09-17T05:02:07.420697+0000","last_became_active":"2024-09-17T05:02:05.323250+0000","last_became_peered":"2024-09-17T05:02:05.323250+0000","last_unstale":"2024-09-17T05:02:07.420697+0000","last_undegraded":"2024-09-17T05:02:07.420697+0000","last_fullsized":"2024-09-17T05:02:07.420697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:46.948808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320590+0000","last_change":"2024-09-17T05:02:06.332617+0000","last_active":"2024-09-17T05:02:07.320590+0000","last_peered":"2024-09-17T05:02:07.320590+0000","last_clean":"2024-09-17T05:02:07.320590+0000","last_became_active":"2024-09-17T05:02:06.330747+0000","last_became_peered":"2024-09-17T05:02:06.330747+0000","last_unstale":"2024-09-17T05:02:07.320590+0000","last_undegraded":"2024-09-17T05:02:07.320590+0000","last_fullsized":"2024-09-17T05:02:07.320590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:51.850349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420085+0000","last_change":"2024-09-17T05:02:05.313259+0000","last_active":"2024-09-17T05:02:07.420085+0000","last_peered":"2024-09-17T05:02:07.420085+0000","last_clean":"2024-09-17T05:02:07.420085+0000","last_became_active":"2024-09-17T05:02:05.312893+0000","last_became_peered":"2024-09-17T05:02:05.312893+0000","last_unstale":"2024-09-17T05:02:07.420085+0000","last_undegraded":"2024-09-17T05:02:07.420085+0000","last_fullsized":"2024-09-17T05:02:07.420085+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350174+0000","last_change":"2024-09-17T05:02:06.333044+0000","last_active":"2024-09-17T05:02:07.350174+0000","last_peered":"2024-09-17T05:02:07.350174+0000","last_clean":"2024-09-17T05:02:07.350174+0000","last_became_active":"2024-09-17T05:02:06.332672+0000","last_became_peered":"2024-09-17T05:02:06.332672+0000","last_unstale":"2024-09-17T05:02:07.350174+0000","last_undegraded":"2024-09-17T05:02:07.350174+0000","last_fullsized":"2024-09-17T05:02:07.350174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:28.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419821+0000","last_change":"2024-09-17T05:02:05.331066+0000","last_active":"2024-09-17T05:02:07.419821+0000","last_peered":"2024-09-17T05:02:07.419821+0000","last_clean":"2024-09-17T05:02:07.419821+0000","last_became_active":"2024-09-17T05:02:05.330727+0000","last_became_peered":"2024-09-17T05:02:05.330727+0000","last_unstale":"2024-09-17T05:02:07.419821+0000","last_undegraded":"2024-09-17T05:02:07.419821+0000","last_fullsized":"2024-09-17T05:02:07.419821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314674+0000","last_change":"2024-09-17T05:02:06.350254+0000","last_active":"2024-09-17T05:02:07.314674+0000","last_peered":"2024-09-17T05:02:07.314674+0000","last_clean":"2024-09-17T05:02:07.314674+0000","last_became_active":"2024-09-17T05:02:06.349926+0000","last_became_peered":"2024-09-17T05:02:06.349926+0000","last_unstale":"2024-09-17T05:02:07.314674+0000","last_undegraded":"2024-09-17T05:02:07.314674+0000","last_fullsized":"2024-09-17T05:02:07.314674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320007+0000","last_change":"2024-09-17T05:02:05.330770+0000","last_active":"2024-09-17T05:02:07.320007+0000","last_peered":"2024-09-17T05:02:07.320007+0000","last_clean":"2024-09-17T05:02:07.320007+0000","last_became_active":"2024-09-17T05:02:05.330452+0000","last_became_peered":"2024-09-17T05:02:05.330452+0000","last_unstale":"2024-09-17T05:02:07.320007+0000","last_undegraded":"2024-09-17T05:02:07.320007+0000","last_fullsized":"2024-09-17T05:02:07.320007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420577+0000","last_change":"2024-09-17T05:02:06.339321+0000","last_active":"2024-09-17T05:02:07.420577+0000","last_peered":"2024-09-17T05:02:07.420577+0000","last_clean":"2024-09-17T05:02:07.420577+0000","last_became_active":"2024-09-17T05:02:06.339039+0000","last_became_peered":"2024-09-17T05:02:06.339039+0000","last_unstale":"2024-09-17T05:02:07.420577+0000","last_undegraded":"2024-09-17T05:02:07.420577+0000","last_fullsized":"2024-09-17T05:02:07.420577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420407+0000","last_change":"2024-09-17T05:02:05.334416+0000","last_active":"2024-09-17T05:02:07.420407+0000","last_peered":"2024-09-17T05:02:07.420407+0000","last_clean":"2024-09-17T05:02:07.420407+0000","last_became_active":"2024-09-17T05:02:05.334066+0000","last_became_peered":"2024-09-17T05:02:05.334066+0000","last_unstale":"2024-09-17T05:02:07.420407+0000","last_undegraded":"2024-09-17T05:02:07.420407+0000","last_fullsized":"2024-09-17T05:02:07.420407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:44.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321356+0000","last_change":"2024-09-17T05:02:06.348674+0000","last_active":"2024-09-17T05:02:07.321356+0000","last_peered":"2024-09-17T05:02:07.321356+0000","last_clean":"2024-09-17T05:02:07.321356+0000","last_became_active":"2024-09-17T05:02:06.348346+0000","last_became_peered":"2024-09-17T05:02:06.348346+0000","last_unstale":"2024-09-17T05:02:07.321356+0000","last_undegraded":"2024-09-17T05:02:07.321356+0000","last_fullsized":"2024-09-17T05:02:07.321356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:39.881805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320775+0000","last_change":"2024-09-17T05:02:05.323997+0000","last_active":"2024-09-17T05:02:07.320775+0000","last_peered":"2024-09-17T05:02:07.320775+0000","last_clean":"2024-09-17T05:02:07.320775+0000","last_became_active":"2024-09-17T05:02:05.323410+0000","last_became_peered":"2024-09-17T05:02:05.323410+0000","last_unstale":"2024-09-17T05:02:07.320775+0000","last_undegraded":"2024-09-17T05:02:07.320775+0000","last_fullsized":"2024-09-17T05:02:07.320775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:32.508695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420275+0000","last_change":"2024-09-17T05:02:06.338660+0000","last_active":"2024-09-17T05:02:07.420275+0000","last_peered":"2024-09-17T05:02:07.420275+0000","last_clean":"2024-09-17T05:02:07.420275+0000","last_became_active":"2024-09-17T05:02:06.338256+0000","last_became_peered":"2024-09-17T05:02:06.338256+0000","last_unstale":"2024-09-17T05:02:07.420275+0000","last_undegraded":"2024-09-17T05:02:07.420275+0000","last_fullsized":"2024-09-17T05:02:07.420275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.216527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315076+0000","last_change":"2024-09-17T05:02:05.326255+0000","last_active":"2024-09-17T05:02:07.315076+0000","last_peered":"2024-09-17T05:02:07.315076+0000","last_clean":"2024-09-17T05:02:07.315076+0000","last_became_active":"2024-09-17T05:02:05.325945+0000","last_became_peered":"2024-09-17T05:02:05.325945+0000","last_unstale":"2024-09-17T05:02:07.315076+0000","last_undegraded":"2024-09-17T05:02:07.315076+0000","last_fullsized":"2024-09-17T05:02:07.315076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:21.224964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320746+0000","last_change":"2024-09-17T05:02:06.335521+0000","last_active":"2024-09-17T05:02:07.320746+0000","last_peered":"2024-09-17T05:02:07.320746+0000","last_clean":"2024-09-17T05:02:07.320746+0000","last_became_active":"2024-09-17T05:02:06.335166+0000","last_became_peered":"2024-09-17T05:02:06.335166+0000","last_unstale":"2024-09-17T05:02:07.320746+0000","last_undegraded":"2024-09-17T05:02:07.320746+0000","last_fullsized":"2024-09-17T05:02:07.320746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:02.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314664+0000","last_change":"2024-09-17T05:02:05.336301+0000","last_active":"2024-09-17T05:02:07.314664+0000","last_peered":"2024-09-17T05:02:07.314664+0000","last_clean":"2024-09-17T05:02:07.314664+0000","last_became_active":"2024-09-17T05:02:05.331553+0000","last_became_peered":"2024-09-17T05:02:05.331553+0000","last_unstale":"2024-09-17T05:02:07.314664+0000","last_undegraded":"2024-09-17T05:02:07.314664+0000","last_fullsized":"2024-09-17T05:02:07.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:23.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420226+0000","last_change":"2024-09-17T05:02:06.342104+0000","last_active":"2024-09-17T05:02:07.420226+0000","last_peered":"2024-09-17T05:02:07.420226+0000","last_clean":"2024-09-17T05:02:07.420226+0000","last_became_active":"2024-09-17T05:02:06.341775+0000","last_became_peered":"2024-09-17T05:02:06.341775+0000","last_unstale":"2024-09-17T05:02:07.420226+0000","last_undegraded":"2024-09-17T05:02:07.420226+0000","last_fullsized":"2024-09-17T05:02:07.420226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:24.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321552+0000","last_change":"2024-09-17T05:02:05.323758+0000","last_active":"2024-09-17T05:02:07.321552+0000","last_peered":"2024-09-17T05:02:07.321552+0000","last_clean":"2024-09-17T05:02:07.321552+0000","last_became_active":"2024-09-17T05:02:05.323426+0000","last_became_peered":"2024-09-17T05:02:05.323426+0000","last_unstale":"2024-09-17T05:02:07.321552+0000","last_undegraded":"2024-09-17T05:02:07.321552+0000","last_fullsized":"2024-09-17T05:02:07.321552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:29.210315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320848+0000","last_change":"2024-09-17T05:02:06.345826+0000","last_active":"2024-09-17T05:02:07.320848+0000","last_peered":"2024-09-17T05:02:07.320848+0000","last_clean":"2024-09-17T05:02:07.320848+0000","last_became_active":"2024-09-17T05:02:06.345343+0000","last_became_peered":"2024-09-17T05:02:06.345343+0000","last_unstale":"2024-09-17T05:02:07.320848+0000","last_undegraded":"2024-09-17T05:02:07.320848+0000","last_fullsized":"2024-09-17T05:02:07.320848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:09.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320977+0000","last_change":"2024-09-17T05:02:05.323105+0000","last_active":"2024-09-17T05:02:07.320977+0000","last_peered":"2024-09-17T05:02:07.320977+0000","last_clean":"2024-09-17T05:02:07.320977+0000","last_became_active":"2024-09-17T05:02:05.320585+0000","last_became_peered":"2024-09-17T05:02:05.320585+0000","last_unstale":"2024-09-17T05:02:07.320977+0000","last_undegraded":"2024-09-17T05:02:07.320977+0000","last_fullsized":"2024-09-17T05:02:07.320977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420267+0000","last_change":"2024-09-17T05:02:06.359915+0000","last_active":"2024-09-17T05:02:07.420267+0000","last_peered":"2024-09-17T05:02:07.420267+0000","last_clean":"2024-09-17T05:02:07.420267+0000","last_became_active":"2024-09-17T05:02:06.359544+0000","last_became_peered":"2024-09-17T05:02:06.359544+0000","last_unstale":"2024-09-17T05:02:07.420267+0000","last_undegraded":"2024-09-17T05:02:07.420267+0000","last_fullsized":"2024-09-17T05:02:07.420267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420061+0000","last_change":"2024-09-17T05:02:05.320829+0000","last_active":"2024-09-17T05:02:07.420061+0000","last_peered":"2024-09-17T05:02:07.420061+0000","last_clean":"2024-09-17T05:02:07.420061+0000","last_became_active":"2024-09-17T05:02:05.320509+0000","last_became_peered":"2024-09-17T05:02:05.320509+0000","last_unstale":"2024-09-17T05:02:07.420061+0000","last_undegraded":"2024-09-17T05:02:07.420061+0000","last_fullsized":"2024-09-17T05:02:07.420061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:36.381090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349878+0000","last_change":"2024-09-17T05:02:06.342348+0000","last_active":"2024-09-17T05:02:07.349878+0000","last_peered":"2024-09-17T05:02:07.349878+0000","last_clean":"2024-09-17T05:02:07.349878+0000","last_became_active":"2024-09-17T05:02:06.342077+0000","last_became_peered":"2024-09-17T05:02:06.342077+0000","last_unstale":"2024-09-17T05:02:07.349878+0000","last_undegraded":"2024-09-17T05:02:07.349878+0000","last_fullsized":"2024-09-17T05:02:07.349878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:24.585972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314795+0000","last_change":"2024-09-17T05:02:05.331453+0000","last_active":"2024-09-17T05:02:07.314795+0000","last_peered":"2024-09-17T05:02:07.314795+0000","last_clean":"2024-09-17T05:02:07.314795+0000","last_became_active":"2024-09-17T05:02:05.330843+0000","last_became_peered":"2024-09-17T05:02:05.330843+0000","last_unstale":"2024-09-17T05:02:07.314795+0000","last_undegraded":"2024-09-17T05:02:07.314795+0000","last_fullsized":"2024-09-17T05:02:07.314795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:53.874382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346719+0000","last_change":"2024-09-17T05:02:06.338645+0000","last_active":"2024-09-17T05:02:07.346719+0000","last_peered":"2024-09-17T05:02:07.346719+0000","last_clean":"2024-09-17T05:02:07.346719+0000","last_became_active":"2024-09-17T05:02:06.338343+0000","last_became_peered":"2024-09-17T05:02:06.338343+0000","last_unstale":"2024-09-17T05:02:07.346719+0000","last_undegraded":"2024-09-17T05:02:07.346719+0000","last_fullsized":"2024-09-17T05:02:07.346719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419537+0000","last_change":"2024-09-17T05:02:05.318024+0000","last_active":"2024-09-17T05:02:07.419537+0000","last_peered":"2024-09-17T05:02:07.419537+0000","last_clean":"2024-09-17T05:02:07.419537+0000","last_became_active":"2024-09-17T05:02:05.317528+0000","last_became_peered":"2024-09-17T05:02:05.317528+0000","last_unstale":"2024-09-17T05:02:07.419537+0000","last_undegraded":"2024-09-17T05:02:07.419537+0000","last_fullsized":"2024-09-17T05:02:07.419537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:22.090770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420452+0000","last_change":"2024-09-17T05:02:06.336675+0000","last_active":"2024-09-17T05:02:07.420452+0000","last_peered":"2024-09-17T05:02:07.420452+0000","last_clean":"2024-09-17T05:02:07.420452+0000","last_became_active":"2024-09-17T05:02:06.336377+0000","last_became_peered":"2024-09-17T05:02:06.336377+0000","last_unstale":"2024-09-17T05:02:07.420452+0000","last_undegraded":"2024-09-17T05:02:07.420452+0000","last_fullsized":"2024-09-17T05:02:07.420452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.158532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420724+0000","last_change":"2024-09-17T05:02:05.334942+0000","last_active":"2024-09-17T05:02:07.420724+0000","last_peered":"2024-09-17T05:02:07.420724+0000","last_clean":"2024-09-17T05:02:07.420724+0000","last_became_active":"2024-09-17T05:02:05.334624+0000","last_became_peered":"2024-09-17T05:02:05.334624+0000","last_unstale":"2024-09-17T05:02:07.420724+0000","last_undegraded":"2024-09-17T05:02:07.420724+0000","last_fullsized":"2024-09-17T05:02:07.420724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:14.855709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420380+0000","last_change":"2024-09-17T05:02:06.334939+0000","last_active":"2024-09-17T05:02:07.420380+0000","last_peered":"2024-09-17T05:02:07.420380+0000","last_clean":"2024-09-17T05:02:07.420380+0000","last_became_active":"2024-09-17T05:02:06.334587+0000","last_became_peered":"2024-09-17T05:02:06.334587+0000","last_unstale":"2024-09-17T05:02:07.420380+0000","last_undegraded":"2024-09-17T05:02:07.420380+0000","last_fullsized":"2024-09-17T05:02:07.420380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:37.387302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315252+0000","last_change":"2024-09-17T05:02:05.326248+0000","last_active":"2024-09-17T05:02:07.315252+0000","last_peered":"2024-09-17T05:02:07.315252+0000","last_clean":"2024-09-17T05:02:07.315252+0000","last_became_active":"2024-09-17T05:02:05.325621+0000","last_became_peered":"2024-09-17T05:02:05.325621+0000","last_unstale":"2024-09-17T05:02:07.315252+0000","last_undegraded":"2024-09-17T05:02:07.315252+0000","last_fullsized":"2024-09-17T05:02:07.315252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:35.920973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420572+0000","last_change":"2024-09-17T05:02:06.332684+0000","last_active":"2024-09-17T05:02:07.420572+0000","last_peered":"2024-09-17T05:02:07.420572+0000","last_clean":"2024-09-17T05:02:07.420572+0000","last_became_active":"2024-09-17T05:02:06.332268+0000","last_became_peered":"2024-09-17T05:02:06.332268+0000","last_unstale":"2024-09-17T05:02:07.420572+0000","last_undegraded":"2024-09-17T05:02:07.420572+0000","last_fullsized":"2024-09-17T05:02:07.420572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:36.983687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346854+0000","last_change":"2024-09-17T05:02:05.329129+0000","last_active":"2024-09-17T05:02:07.346854+0000","last_peered":"2024-09-17T05:02:07.346854+0000","last_clean":"2024-09-17T05:02:07.346854+0000","last_became_active":"2024-09-17T05:02:05.328783+0000","last_became_peered":"2024-09-17T05:02:05.328783+0000","last_unstale":"2024-09-17T05:02:07.346854+0000","last_undegraded":"2024-09-17T05:02:07.346854+0000","last_fullsized":"2024-09-17T05:02:07.346854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:01.114967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.315373+0000","last_change":"2024-09-17T05:02:05.315373+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.315373+0000","last_undegraded":"2024-09-17T05:02:05.315373+0000","last_fullsized":"2024-09-17T05:02:05.315373+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T05:02:05.325258+0000","last_change":"2024-09-17T05:02:05.324272+0000","last_active":"2024-09-17T05:02:05.325258+0000","last_peered":"2024-09-17T05:02:05.325258+0000","last_clean":"2024-09-17T05:02:05.325258+0000","last_became_active":"2024-09-17T05:02:05.323884+0000","last_became_peered":"2024-09-17T05:02:05.323884+0000","last_unstale":"2024-09-17T05:02:05.325258+0000","last_undegraded":"2024-09-17T05:02:05.325258+0000","last_fullsized":"2024-09-17T05:02:05.325258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:51.335778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314537+0000","last_change":"2024-09-17T05:02:06.321153+0000","last_active":"2024-09-17T05:02:07.314537+0000","last_peered":"2024-09-17T05:02:07.314537+0000","last_clean":"2024-09-17T05:02:07.314537+0000","last_became_active":"2024-09-17T05:02:06.320793+0000","last_became_peered":"2024-09-17T05:02:06.320793+0000","last_unstale":"2024-09-17T05:02:07.314537+0000","last_undegraded":"2024-09-17T05:02:07.314537+0000","last_fullsized":"2024-09-17T05:02:07.314537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:49.889726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T05:02:05.324925+0000","last_change":"2024-09-17T05:02:05.321831+0000","last_active":"2024-09-17T05:02:05.324925+0000","last_peered":"2024-09-17T05:02:05.324925+0000","last_clean":"2024-09-17T05:02:05.324925+0000","last_became_active":"2024-09-17T05:02:05.321507+0000","last_became_peered":"2024-09-17T05:02:05.321507+0000","last_unstale":"2024-09-17T05:02:05.324925+0000","last_undegraded":"2024-09-17T05:02:05.324925+0000","last_fullsized":"2024-09-17T05:02:05.324925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:10.699379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319927+0000","last_change":"2024-09-17T05:02:06.362961+0000","last_active":"2024-09-17T05:02:07.319927+0000","last_peered":"2024-09-17T05:02:07.319927+0000","last_clean":"2024-09-17T05:02:07.319927+0000","last_became_active":"2024-09-17T05:02:06.362589+0000","last_became_peered":"2024-09-17T05:02:06.362589+0000","last_unstale":"2024-09-17T05:02:07.319927+0000","last_undegraded":"2024-09-17T05:02:07.319927+0000","last_fullsized":"2024-09-17T05:02:07.319927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:43.235010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420209+0000","last_change":"2024-09-17T05:02:05.334796+0000","last_active":"2024-09-17T05:02:07.420209+0000","last_peered":"2024-09-17T05:02:07.420209+0000","last_clean":"2024-09-17T05:02:07.420209+0000","last_became_active":"2024-09-17T05:02:05.334457+0000","last_became_peered":"2024-09-17T05:02:05.334457+0000","last_unstale":"2024-09-17T05:02:07.420209+0000","last_undegraded":"2024-09-17T05:02:07.420209+0000","last_fullsized":"2024-09-17T05:02:07.420209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:50.024608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349461+0000","last_change":"2024-09-17T05:02:06.360661+0000","last_active":"2024-09-17T05:02:07.349461+0000","last_peered":"2024-09-17T05:02:07.349461+0000","last_clean":"2024-09-17T05:02:07.349461+0000","last_became_active":"2024-09-17T05:02:06.360293+0000","last_became_peered":"2024-09-17T05:02:06.360293+0000","last_unstale":"2024-09-17T05:02:07.349461+0000","last_undegraded":"2024-09-17T05:02:07.349461+0000","last_fullsized":"2024-09-17T05:02:07.349461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420548+0000","last_change":"2024-09-17T05:02:05.335089+0000","last_active":"2024-09-17T05:02:07.420548+0000","last_peered":"2024-09-17T05:02:07.420548+0000","last_clean":"2024-09-17T05:02:07.420548+0000","last_became_active":"2024-09-17T05:02:05.334761+0000","last_became_peered":"2024-09-17T05:02:05.334761+0000","last_unstale":"2024-09-17T05:02:07.420548+0000","last_undegraded":"2024-09-17T05:02:07.420548+0000","last_fullsized":"2024-09-17T05:02:07.420548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319697+0000","last_change":"2024-09-17T05:02:06.337072+0000","last_active":"2024-09-17T05:02:07.319697+0000","last_peered":"2024-09-17T05:02:07.319697+0000","last_clean":"2024-09-17T05:02:07.319697+0000","last_became_active":"2024-09-17T05:02:06.335541+0000","last_became_peered":"2024-09-17T05:02:06.335541+0000","last_unstale":"2024-09-17T05:02:07.319697+0000","last_undegraded":"2024-09-17T05:02:07.319697+0000","last_fullsized":"2024-09-17T05:02:07.319697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:33.058079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419849+0000","last_change":"2024-09-17T05:02:05.317067+0000","last_active":"2024-09-17T05:02:07.419849+0000","last_peered":"2024-09-17T05:02:07.419849+0000","last_clean":"2024-09-17T05:02:07.419849+0000","last_became_active":"2024-09-17T05:02:05.316750+0000","last_became_peered":"2024-09-17T05:02:05.316750+0000","last_unstale":"2024-09-17T05:02:07.419849+0000","last_undegraded":"2024-09-17T05:02:07.419849+0000","last_fullsized":"2024-09-17T05:02:07.419849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:11.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346952+0000","last_change":"2024-09-17T05:02:06.342012+0000","last_active":"2024-09-17T05:02:07.346952+0000","last_peered":"2024-09-17T05:02:07.346952+0000","last_clean":"2024-09-17T05:02:07.346952+0000","last_became_active":"2024-09-17T05:02:06.341627+0000","last_became_peered":"2024-09-17T05:02:06.341627+0000","last_unstale":"2024-09-17T05:02:07.346952+0000","last_undegraded":"2024-09-17T05:02:07.346952+0000","last_fullsized":"2024-09-17T05:02:07.346952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:04.620408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315375+0000","last_change":"2024-09-17T05:02:05.331974+0000","last_active":"2024-09-17T05:02:07.315375+0000","last_peered":"2024-09-17T05:02:07.315375+0000","last_clean":"2024-09-17T05:02:07.315375+0000","last_became_active":"2024-09-17T05:02:05.331277+0000","last_became_peered":"2024-09-17T05:02:05.331277+0000","last_unstale":"2024-09-17T05:02:07.315375+0000","last_undegraded":"2024-09-17T05:02:07.315375+0000","last_fullsized":"2024-09-17T05:02:07.315375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:14.008073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350015+0000","last_change":"2024-09-17T05:02:06.361595+0000","last_active":"2024-09-17T05:02:07.350015+0000","last_peered":"2024-09-17T05:02:07.350015+0000","last_clean":"2024-09-17T05:02:07.350015+0000","last_became_active":"2024-09-17T05:02:06.361219+0000","last_became_peered":"2024-09-17T05:02:06.361219+0000","last_unstale":"2024-09-17T05:02:07.350015+0000","last_undegraded":"2024-09-17T05:02:07.350015+0000","last_fullsized":"2024-09-17T05:02:07.350015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:47.878683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.348883+0000","last_change":"2024-09-17T05:02:05.334870+0000","last_active":"2024-09-17T05:02:07.348883+0000","last_peered":"2024-09-17T05:02:07.348883+0000","last_clean":"2024-09-17T05:02:07.348883+0000","last_became_active":"2024-09-17T05:02:05.334512+0000","last_became_peered":"2024-09-17T05:02:05.334512+0000","last_unstale":"2024-09-17T05:02:07.348883+0000","last_undegraded":"2024-09-17T05:02:07.348883+0000","last_fullsized":"2024-09-17T05:02:07.348883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T05:02:05.315369+0000","last_change":"2024-09-17T05:02:05.315369+0000","last_active":"2024-09-17T05:02:05.294183+0000","last_peered":"2024-09-17T05:02:05.294183+0000","last_clean":"2024-09-17T05:02:05.294183+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:02:05.315369+0000","last_undegraded":"2024-09-17T05:02:05.315369+0000","last_fullsized":"2024-09-17T05:02:05.315369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346976+0000","last_change":"2024-09-17T05:02:05.325366+0000","last_active":"2024-09-17T05:02:07.346976+0000","last_peered":"2024-09-17T05:02:07.346976+0000","last_clean":"2024-09-17T05:02:07.346976+0000","last_became_active":"2024-09-17T05:02:05.325037+0000","last_became_peered":"2024-09-17T05:02:05.325037+0000","last_unstale":"2024-09-17T05:02:07.346976+0000","last_undegraded":"2024-09-17T05:02:07.346976+0000","last_fullsized":"2024-09-17T05:02:07.346976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346633+0000","last_change":"2024-09-17T05:02:06.336842+0000","last_active":"2024-09-17T05:02:07.346633+0000","last_peered":"2024-09-17T05:02:07.346633+0000","last_clean":"2024-09-17T05:02:07.346633+0000","last_became_active":"2024-09-17T05:02:06.334045+0000","last_became_peered":"2024-09-17T05:02:06.334045+0000","last_unstale":"2024-09-17T05:02:07.346633+0000","last_undegraded":"2024-09-17T05:02:07.346633+0000","last_fullsized":"2024-09-17T05:02:07.346633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:30.924123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320355+0000","last_change":"2024-09-17T05:02:05.316008+0000","last_active":"2024-09-17T05:02:07.320355+0000","last_peered":"2024-09-17T05:02:07.320355+0000","last_clean":"2024-09-17T05:02:07.320355+0000","last_became_active":"2024-09-17T05:02:05.315680+0000","last_became_peered":"2024-09-17T05:02:05.315680+0000","last_unstale":"2024-09-17T05:02:07.320355+0000","last_undegraded":"2024-09-17T05:02:07.320355+0000","last_fullsized":"2024-09-17T05:02:07.320355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:25.314134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420182+0000","last_change":"2024-09-17T05:02:06.319976+0000","last_active":"2024-09-17T05:02:07.420182+0000","last_peered":"2024-09-17T05:02:07.420182+0000","last_clean":"2024-09-17T05:02:07.420182+0000","last_became_active":"2024-09-17T05:02:06.319609+0000","last_became_peered":"2024-09-17T05:02:06.319609+0000","last_unstale":"2024-09-17T05:02:07.420182+0000","last_undegraded":"2024-09-17T05:02:07.420182+0000","last_fullsized":"2024-09-17T05:02:07.420182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349774+0000","last_change":"2024-09-17T05:02:05.331789+0000","last_active":"2024-09-17T05:02:07.349774+0000","last_peered":"2024-09-17T05:02:07.349774+0000","last_clean":"2024-09-17T05:02:07.349774+0000","last_became_active":"2024-09-17T05:02:05.327820+0000","last_became_peered":"2024-09-17T05:02:05.327820+0000","last_unstale":"2024-09-17T05:02:07.349774+0000","last_undegraded":"2024-09-17T05:02:07.349774+0000","last_fullsized":"2024-09-17T05:02:07.349774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314859+0000","last_change":"2024-09-17T05:02:06.333560+0000","last_active":"2024-09-17T05:02:07.314859+0000","last_peered":"2024-09-17T05:02:07.314859+0000","last_clean":"2024-09-17T05:02:07.314859+0000","last_became_active":"2024-09-17T05:02:06.333210+0000","last_became_peered":"2024-09-17T05:02:06.333210+0000","last_unstale":"2024-09-17T05:02:07.314859+0000","last_undegraded":"2024-09-17T05:02:07.314859+0000","last_fullsized":"2024-09-17T05:02:07.314859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.884069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314950+0000","last_change":"2024-09-17T05:02:05.325774+0000","last_active":"2024-09-17T05:02:07.314950+0000","last_peered":"2024-09-17T05:02:07.314950+0000","last_clean":"2024-09-17T05:02:07.314950+0000","last_became_active":"2024-09-17T05:02:05.325459+0000","last_became_peered":"2024-09-17T05:02:05.325459+0000","last_unstale":"2024-09-17T05:02:07.314950+0000","last_undegraded":"2024-09-17T05:02:07.314950+0000","last_fullsized":"2024-09-17T05:02:07.314950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:55.257416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349680+0000","last_change":"2024-09-17T05:02:06.340513+0000","last_active":"2024-09-17T05:02:07.349680+0000","last_peered":"2024-09-17T05:02:07.349680+0000","last_clean":"2024-09-17T05:02:07.349680+0000","last_became_active":"2024-09-17T05:02:06.340154+0000","last_became_peered":"2024-09-17T05:02:06.340154+0000","last_unstale":"2024-09-17T05:02:07.349680+0000","last_undegraded":"2024-09-17T05:02:07.349680+0000","last_fullsized":"2024-09-17T05:02:07.349680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320990+0000","last_change":"2024-09-17T05:02:05.325138+0000","last_active":"2024-09-17T05:02:07.320990+0000","last_peered":"2024-09-17T05:02:07.320990+0000","last_clean":"2024-09-17T05:02:07.320990+0000","last_became_active":"2024-09-17T05:02:05.324679+0000","last_became_peered":"2024-09-17T05:02:05.324679+0000","last_unstale":"2024-09-17T05:02:07.320990+0000","last_undegraded":"2024-09-17T05:02:07.320990+0000","last_fullsized":"2024-09-17T05:02:07.320990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:39.915152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315287+0000","last_change":"2024-09-17T05:02:06.331069+0000","last_active":"2024-09-17T05:02:07.315287+0000","last_peered":"2024-09-17T05:02:07.315287+0000","last_clean":"2024-09-17T05:02:07.315287+0000","last_became_active":"2024-09-17T05:02:06.330731+0000","last_became_peered":"2024-09-17T05:02:06.330731+0000","last_unstale":"2024-09-17T05:02:07.315287+0000","last_undegraded":"2024-09-17T05:02:07.315287+0000","last_fullsized":"2024-09-17T05:02:07.315287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:38.520351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349709+0000","last_change":"2024-09-17T05:02:05.335344+0000","last_active":"2024-09-17T05:02:07.349709+0000","last_peered":"2024-09-17T05:02:07.349709+0000","last_clean":"2024-09-17T05:02:07.349709+0000","last_became_active":"2024-09-17T05:02:05.334313+0000","last_became_peered":"2024-09-17T05:02:05.334313+0000","last_unstale":"2024-09-17T05:02:07.349709+0000","last_undegraded":"2024-09-17T05:02:07.349709+0000","last_fullsized":"2024-09-17T05:02:07.349709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420491+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:02:07.420491+0000","last_peered":"2024-09-17T05:02:07.420491+0000","last_clean":"2024-09-17T05:02:07.420491+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:02:07.420491+0000","last_undegraded":"2024-09-17T05:02:07.420491+0000","last_fullsized":"2024-09-17T05:02:07.420491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":458752,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":462848,"data_stored":190415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":188769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":190873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":779069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":450560,"data_stored":188771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1024000,"data_stored":771880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:02:11.355 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-09-17T05:02:11.355 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:02:05.314200+0000', 'last_change': '2024-09-17T05:02:05.314200+0000', 'last_active': '2024-09-17T05:02:05.294183+0000', 'last_peered': '2024-09-17T05:02:05.294183+0000', 'last_clean': '2024-09-17T05:02:05.294183+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:02:05.314200+0000', 'last_undegraded': '2024-09-17T05:02:05.314200+0000', 'last_fullsized': '2024-09-17T05:02:05.314200+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'last_clean_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:02:11.355 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-17T05:02:11.356 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:02:05.315373+0000', 'last_change': '2024-09-17T05:02:05.315373+0000', 'last_active': '2024-09-17T05:02:05.294183+0000', 'last_peered': '2024-09-17T05:02:05.294183+0000', 'last_clean': '2024-09-17T05:02:05.294183+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:02:05.315373+0000', 'last_undegraded': '2024-09-17T05:02:05.315373+0000', 'last_fullsized': '2024-09-17T05:02:05.315373+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'last_clean_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:02:11.356 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-17T05:02:11.356 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:02:05.315369+0000', 'last_change': '2024-09-17T05:02:05.315369+0000', 'last_active': '2024-09-17T05:02:05.294183+0000', 'last_peered': '2024-09-17T05:02:05.294183+0000', 'last_clean': '2024-09-17T05:02:05.294183+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:02:05.315369+0000', 'last_undegraded': '2024-09-17T05:02:05.315369+0000', 'last_fullsized': '2024-09-17T05:02:05.315369+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'last_clean_scrub_stamp': '2024-09-17T05:02:05.294183+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T05:02:11.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:02:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5e00640 1 -- 172.21.15.16:0/1556394467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 msgr2=0x7f3b84004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/1556394467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3b84004c20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3b88001860 tx=0x7f3b8801f050 comp rx=0 tx=0).stop 2024-09-17T05:02:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5400640 1 -- 172.21.15.16:0/1556394467 reap_dead start 2024-09-17T05:02:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5e00640 1 -- 172.21.15.16:0/1556394467 shutdown_connections 2024-09-17T05:02:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5e00640 1 -- 172.21.15.16:0/1556394467 >> 172.21.15.16:0/1556394467 conn(0x7f3ba0073cf0 msgr2=0x7f3ba0074100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5e00640 1 -- 172.21.15.16:0/1556394467 shutdown_connections 2024-09-17T05:02:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.550+0000 7f3ba5e00640 1 -- 172.21.15.16:0/1556394467 wait complete. 2024-09-17T05:02:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 Processor -- start 2024-09-17T05:02:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 -- start start 2024-09-17T05:02:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3ba013c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3ba015bb50 0x7f3ba0071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ba0071850 0x7f3ba013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00d0520 con 0x7f3b84004810 2024-09-17T05:02:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00707e0 con 0x7f3ba0071850 2024-09-17T05:02:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba00d06a0 con 0x7f3ba015bb50 2024-09-17T05:02:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3ba013c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3ba013c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48026/0 (socket says 172.21.15.16:48026) 2024-09-17T05:02:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 -- 172.21.15.16:0/309342430 learned_addr learned my addr 172.21.15.16:0/309342430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3b97e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3ba015bb50 0x7f3ba0071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5400640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ba0071850 0x7f3ba013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 -- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3ba015bb50 msgr2=0x7f3ba0071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3ba015bb50 0x7f3ba0071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 -- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ba0071850 msgr2=0x7f3ba013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ba0071850 0x7f3ba013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba4a00640 1 -- 172.21.15.16:0/309342430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba0071c60 con 0x7f3b84004810 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.554+0000 7f3ba5400640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ba0071850 0x7f3ba013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3ba4a00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3ba013c9b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000e590 tx=0x7f3b9000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90015070 con 0x7f3b84004810 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b90010980 con 0x7f3b84004810 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba013e140 con 0x7f3b84004810 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90010dd0 con 0x7f3b84004810 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba013d660 con 0x7f3b84004810 2024-09-17T05:02:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba00d0c30 con 0x7f3b84004810 2024-09-17T05:02:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3b90018070 con 0x7f3b84004810 2024-09-17T05:02:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3b9d600640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b7c073230 0x7f3b7c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.558+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3b90094850 con 0x7f3b84004810 2024-09-17T05:02:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.562+0000 7f3b97e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b7c073230 0x7f3b7c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.562+0000 7f3b97e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b7c073230 0x7f3b7c075740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3b88001830 tx=0x7f3b8801be90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.566+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ba0071c60 con 0x7f3b84004810 2024-09-17T05:02:11.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3ba0160500 con 0x7f3b84004810 2024-09-17T05:02:11.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3b9d600640 1 -- 172.21.15.16:0/309342430 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1689 (secure 0 0 0) 0x7f3b9005f640 con 0x7f3b84004810 2024-09-17T05:02:11.803 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:11.803 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":89,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1726549257,"num_in_osds":8,"osd_in_since":1726549246,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":94},{"state_name":"creating+peering","count":3}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":243290112,"bytes_avail":772850823168,"bytes_total":773094113280,"inactive_pgs_ratio":0.030927835032343864,"write_bytes_sec":2766,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5031}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T05:01:59.474381+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:02:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b7c073230 msgr2=0x7f3b7c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b7c073230 0x7f3b7c075740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3b88001830 tx=0x7f3b8801be90 comp rx=0 tx=0).stop 2024-09-17T05:02:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 msgr2=0x7f3ba013c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3ba013c9b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000e590 tx=0x7f3b9000ea50 comp rx=0 tx=0).stop 2024-09-17T05:02:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 shutdown_connections 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b7c073230 0x7f3b7c075740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ba0071850 0x7f3ba013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3ba015bb50 0x7f3ba0071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 --2- 172.21.15.16:0/309342430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b84004810 0x7f3ba013c9b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 >> 172.21.15.16:0/309342430 conn(0x7f3ba0073cf0 msgr2=0x7f3ba00b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 shutdown_connections 2024-09-17T05:02:11.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.802+0000 7f3ba5e00640 1 -- 172.21.15.16:0/309342430 wait complete. 2024-09-17T05:02:11.825 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:02:11.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 -- 172.21.15.16:0/3459584376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 msgr2=0x7f5e84073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 --2- 172.21.15.16:0/3459584376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e84073610 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009900 tx=0x7f5e7801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 -- 172.21.15.16:0/3459584376 shutdown_connections 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 --2- 172.21.15.16:0/3459584376 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e84073b50 0x7f5e8406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 --2- 172.21.15.16:0/3459584376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e84073610 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 -- 172.21.15.16:0/3459584376 >> 172.21.15.16:0/3459584376 conn(0x7f5e840d0860 msgr2=0x7f5e840d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 -- 172.21.15.16:0/3459584376 shutdown_connections 2024-09-17T05:02:11.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 -- 172.21.15.16:0/3459584376 wait complete. 2024-09-17T05:02:11.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 Processor -- start 2024-09-17T05:02:11.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.982+0000 7f5e8c400640 1 -- start start 2024-09-17T05:02:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406d3d0 0x7f5e8414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e84073b50 0x7f5e8414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e8414c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8416a200 con 0x7f5e8415cbb0 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840d0030 con 0x7f5e8406d3d0 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8416a380 con 0x7f5e84073b50 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e8414c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e8414c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48040/0 (socket says 172.21.15.16:48040) 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 -- 172.21.15.16:0/1619853681 learned_addr learned my addr 172.21.15.16:0/1619853681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e89000640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e84073b50 0x7f5e8414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e89a00640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406d3d0 0x7f5e8414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 -- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e84073b50 msgr2=0x7f5e8414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e84073b50 0x7f5e8414f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 -- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406d3d0 msgr2=0x7f5e8414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406d3d0 0x7f5e8414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 -- 172.21.15.16:0/1619853681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8414a680 con 0x7f5e8415cbb0 2024-09-17T05:02:11.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8a400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e8414c0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c760 tx=0x7f5e7c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c003db0 con 0x7f5e8415cbb0 2024-09-17T05:02:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e7c00ce10 con 0x7f5e8415cbb0 2024-09-17T05:02:11.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8414a960 con 0x7f5e8415cbb0 2024-09-17T05:02:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0052d0 con 0x7f5e8415cbb0 2024-09-17T05:02:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8414af20 con 0x7f5e8415cbb0 2024-09-17T05:02:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.986+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8406ce90 con 0x7f5e8415cbb0 2024-09-17T05:02:11.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.990+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e7c005470 con 0x7f5e8415cbb0 2024-09-17T05:02:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.990+0000 7f5e81600640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e64073200 0x7f5e64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.990+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f5e7c093f90 con 0x7f5e8415cbb0 2024-09-17T05:02:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.990+0000 7f5e89a00640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e64073200 0x7f5e64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:11.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.990+0000 7f5e89a00640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e64073200 0x7f5e64075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e780af000 tx=0x7f5e7801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:11.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:11.994+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e84073f60 con 0x7f5e8415cbb0 2024-09-17T05:02:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.170+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e840afc30 con 0x7f5e64073200 2024-09-17T05:02:12.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.174+0000 7f5e81600640 1 -- 172.21.15.16:0/1619853681 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f5e840afc30 con 0x7f5e64073200 2024-09-17T05:02:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:12.180 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e64073200 msgr2=0x7f5e64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e64073200 0x7f5e64075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e780af000 tx=0x7f5e7801b120 comp rx=0 tx=0).stop 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 msgr2=0x7f5e8414c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e8414c0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00c760 tx=0x7f5e7c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 shutdown_connections 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e64073200 0x7f5e64075710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8415cbb0 0x7f5e8414c0e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:12.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e84073b50 0x7f5e8414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 --2- 172.21.15.16:0/1619853681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e8406d3d0 0x7f5e8414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 >> 172.21.15.16:0/1619853681 conn(0x7f5e840d0860 msgr2=0x7f5e84158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 shutdown_connections 2024-09-17T05:02:12.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:12.178+0000 7f5e8c400640 1 -- 172.21.15.16:0/1619853681 wait complete. 2024-09-17T05:02:12.199 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-17T05:02:11.476718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237612,"kb_used_data":4704,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737108,"statfs":{"total":773094113280,"available":772850798592,"internally_reserved":0,"allocated":4816896,"data_stored":2708372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.177701"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321351+0000","last_change":"2024-09-17T05:02:06.338525+0000","last_active":"2024-09-17T05:02:07.321351+0000","last_peered":"2024-09-17T05:02:07.321351+0000","last_clean":"2024-09-17T05:02:07.321351+0000","last_became_active":"2024-09-17T05:02:06.338232+0000","last_became_peered":"2024-09-17T05:02:06.338232+0000","last_unstale":"2024-09-17T05:02:07.321351+0000","last_undegraded":"2024-09-17T05:02:07.321351+0000","last_fullsized":"2024-09-17T05:02:07.321351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:56.646622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420620+0000","last_change":"2024-09-17T05:02:06.332250+0000","last_active":"2024-09-17T05:02:07.420620+0000","last_peered":"2024-09-17T05:02:07.420620+0000","last_clean":"2024-09-17T05:02:07.420620+0000","last_became_active":"2024-09-17T05:02:06.331927+0000","last_became_peered":"2024-09-17T05:02:06.331927+0000","last_unstale":"2024-09-17T05:02:07.420620+0000","last_undegraded":"2024-09-17T05:02:07.420620+0000","last_fullsized":"2024-09-17T05:02:07.420620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320879+0000","last_change":"2024-09-17T05:02:06.336648+0000","last_active":"2024-09-17T05:02:07.320879+0000","last_peered":"2024-09-17T05:02:07.320879+0000","last_clean":"2024-09-17T05:02:07.320879+0000","last_became_active":"2024-09-17T05:02:06.336337+0000","last_became_peered":"2024-09-17T05:02:06.336337+0000","last_unstale":"2024-09-17T05:02:07.320879+0000","last_undegraded":"2024-09-17T05:02:07.320879+0000","last_fullsized":"2024-09-17T05:02:07.320879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:44.810456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321127+0000","last_change":"2024-09-17T05:02:06.348672+0000","last_active":"2024-09-17T05:02:07.321127+0000","last_peered":"2024-09-17T05:02:07.321127+0000","last_clean":"2024-09-17T05:02:07.321127+0000","last_became_active":"2024-09-17T05:02:06.348392+0000","last_became_peered":"2024-09-17T05:02:06.348392+0000","last_unstale":"2024-09-17T05:02:07.321127+0000","last_undegraded":"2024-09-17T05:02:07.321127+0000","last_fullsized":"2024-09-17T05:02:07.321127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:06.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346718+0000","last_change":"2024-09-17T05:02:06.343632+0000","last_active":"2024-09-17T05:02:07.346718+0000","last_peered":"2024-09-17T05:02:07.346718+0000","last_clean":"2024-09-17T05:02:07.346718+0000","last_became_active":"2024-09-17T05:02:06.343257+0000","last_became_peered":"2024-09-17T05:02:06.343257+0000","last_unstale":"2024-09-17T05:02:07.346718+0000","last_undegraded":"2024-09-17T05:02:07.346718+0000","last_fullsized":"2024-09-17T05:02:07.346718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:00.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420107+0000","last_change":"2024-09-17T05:02:06.330126+0000","last_active":"2024-09-17T05:02:07.420107+0000","last_peered":"2024-09-17T05:02:07.420107+0000","last_clean":"2024-09-17T05:02:07.420107+0000","last_became_active":"2024-09-17T05:02:06.329825+0000","last_became_peered":"2024-09-17T05:02:06.329825+0000","last_unstale":"2024-09-17T05:02:07.420107+0000","last_undegraded":"2024-09-17T05:02:07.420107+0000","last_fullsized":"2024-09-17T05:02:07.420107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:46.368942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420406+0000","last_change":"2024-09-17T05:02:06.343766+0000","last_active":"2024-09-17T05:02:07.420406+0000","last_peered":"2024-09-17T05:02:07.420406+0000","last_clean":"2024-09-17T05:02:07.420406+0000","last_became_active":"2024-09-17T05:02:06.343207+0000","last_became_peered":"2024-09-17T05:02:06.343207+0000","last_unstale":"2024-09-17T05:02:07.420406+0000","last_undegraded":"2024-09-17T05:02:07.420406+0000","last_fullsized":"2024-09-17T05:02:07.420406+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:51.030820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320027+0000","last_change":"2024-09-17T05:02:06.338244+0000","last_active":"2024-09-17T05:02:07.320027+0000","last_peered":"2024-09-17T05:02:07.320027+0000","last_clean":"2024-09-17T05:02:07.320027+0000","last_became_active":"2024-09-17T05:02:06.337859+0000","last_became_peered":"2024-09-17T05:02:06.337859+0000","last_unstale":"2024-09-17T05:02:07.320027+0000","last_undegraded":"2024-09-17T05:02:07.320027+0000","last_fullsized":"2024-09-17T05:02:07.320027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:36.957053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.347110+0000","last_change":"2024-09-17T05:02:06.336573+0000","last_active":"2024-09-17T05:02:07.347110+0000","last_peered":"2024-09-17T05:02:07.347110+0000","last_clean":"2024-09-17T05:02:07.347110+0000","last_became_active":"2024-09-17T05:02:06.336253+0000","last_became_peered":"2024-09-17T05:02:06.336253+0000","last_unstale":"2024-09-17T05:02:07.347110+0000","last_undegraded":"2024-09-17T05:02:07.347110+0000","last_fullsized":"2024-09-17T05:02:07.347110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:47.955020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420328+0000","last_change":"2024-09-17T05:02:06.362653+0000","last_active":"2024-09-17T05:02:07.420328+0000","last_peered":"2024-09-17T05:02:07.420328+0000","last_clean":"2024-09-17T05:02:07.420328+0000","last_became_active":"2024-09-17T05:02:06.362345+0000","last_became_peered":"2024-09-17T05:02:06.362345+0000","last_unstale":"2024-09-17T05:02:07.420328+0000","last_undegraded":"2024-09-17T05:02:07.420328+0000","last_fullsized":"2024-09-17T05:02:07.420328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:12.091714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350026+0000","last_change":"2024-09-17T05:02:06.341127+0000","last_active":"2024-09-17T05:02:07.350026+0000","last_peered":"2024-09-17T05:02:07.350026+0000","last_clean":"2024-09-17T05:02:07.350026+0000","last_became_active":"2024-09-17T05:02:06.340796+0000","last_became_peered":"2024-09-17T05:02:06.340796+0000","last_unstale":"2024-09-17T05:02:07.350026+0000","last_undegraded":"2024-09-17T05:02:07.350026+0000","last_fullsized":"2024-09-17T05:02:07.350026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:54.880594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346735+0000","last_change":"2024-09-17T05:02:06.336217+0000","last_active":"2024-09-17T05:02:07.346735+0000","last_peered":"2024-09-17T05:02:07.346735+0000","last_clean":"2024-09-17T05:02:07.346735+0000","last_became_active":"2024-09-17T05:02:06.333449+0000","last_became_peered":"2024-09-17T05:02:06.333449+0000","last_unstale":"2024-09-17T05:02:07.346735+0000","last_undegraded":"2024-09-17T05:02:07.346735+0000","last_fullsized":"2024-09-17T05:02:07.346735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:23.096982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420312+0000","last_change":"2024-09-17T05:02:06.330216+0000","last_active":"2024-09-17T05:02:07.420312+0000","last_peered":"2024-09-17T05:02:07.420312+0000","last_clean":"2024-09-17T05:02:07.420312+0000","last_became_active":"2024-09-17T05:02:06.329869+0000","last_became_peered":"2024-09-17T05:02:06.329869+0000","last_unstale":"2024-09-17T05:02:07.420312+0000","last_undegraded":"2024-09-17T05:02:07.420312+0000","last_fullsized":"2024-09-17T05:02:07.420312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419668+0000","last_change":"2024-09-17T05:02:06.337107+0000","last_active":"2024-09-17T05:02:07.419668+0000","last_peered":"2024-09-17T05:02:07.419668+0000","last_clean":"2024-09-17T05:02:07.419668+0000","last_became_active":"2024-09-17T05:02:06.336774+0000","last_became_peered":"2024-09-17T05:02:06.336774+0000","last_unstale":"2024-09-17T05:02:07.419668+0000","last_undegraded":"2024-09-17T05:02:07.419668+0000","last_fullsized":"2024-09-17T05:02:07.419668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:26.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314512+0000","last_change":"2024-09-17T05:02:06.321133+0000","last_active":"2024-09-17T05:02:07.314512+0000","last_peered":"2024-09-17T05:02:07.314512+0000","last_clean":"2024-09-17T05:02:07.314512+0000","last_became_active":"2024-09-17T05:02:06.320714+0000","last_became_peered":"2024-09-17T05:02:06.320714+0000","last_unstale":"2024-09-17T05:02:07.314512+0000","last_undegraded":"2024-09-17T05:02:07.314512+0000","last_fullsized":"2024-09-17T05:02:07.314512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:18.256704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420854+0000","last_change":"2024-09-17T05:02:06.335127+0000","last_active":"2024-09-17T05:02:07.420854+0000","last_peered":"2024-09-17T05:02:07.420854+0000","last_clean":"2024-09-17T05:02:07.420854+0000","last_became_active":"2024-09-17T05:02:06.334693+0000","last_became_peered":"2024-09-17T05:02:06.334693+0000","last_unstale":"2024-09-17T05:02:07.420854+0000","last_undegraded":"2024-09-17T05:02:07.420854+0000","last_fullsized":"2024-09-17T05:02:07.420854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:20.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315318+0000","last_change":"2024-09-17T05:02:06.352349+0000","last_active":"2024-09-17T05:02:07.315318+0000","last_peered":"2024-09-17T05:02:07.315318+0000","last_clean":"2024-09-17T05:02:07.315318+0000","last_became_active":"2024-09-17T05:02:06.352010+0000","last_became_peered":"2024-09-17T05:02:06.352010+0000","last_unstale":"2024-09-17T05:02:07.315318+0000","last_undegraded":"2024-09-17T05:02:07.315318+0000","last_fullsized":"2024-09-17T05:02:07.315318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.149726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314938+0000","last_change":"2024-09-17T05:02:06.336278+0000","last_active":"2024-09-17T05:02:07.314938+0000","last_peered":"2024-09-17T05:02:07.314938+0000","last_clean":"2024-09-17T05:02:07.314938+0000","last_became_active":"2024-09-17T05:02:06.334917+0000","last_became_peered":"2024-09-17T05:02:06.334917+0000","last_unstale":"2024-09-17T05:02:07.314938+0000","last_undegraded":"2024-09-17T05:02:07.314938+0000","last_fullsized":"2024-09-17T05:02:07.314938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:51.224167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321119+0000","last_change":"2024-09-17T05:02:06.335384+0000","last_active":"2024-09-17T05:02:07.321119+0000","last_peered":"2024-09-17T05:02:07.321119+0000","last_clean":"2024-09-17T05:02:07.321119+0000","last_became_active":"2024-09-17T05:02:06.334178+0000","last_became_peered":"2024-09-17T05:02:06.334178+0000","last_unstale":"2024-09-17T05:02:07.321119+0000","last_undegraded":"2024-09-17T05:02:07.321119+0000","last_fullsized":"2024-09-17T05:02:07.321119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:57.578193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349734+0000","last_change":"2024-09-17T05:02:06.353747+0000","last_active":"2024-09-17T05:02:07.349734+0000","last_peered":"2024-09-17T05:02:07.349734+0000","last_clean":"2024-09-17T05:02:07.349734+0000","last_became_active":"2024-09-17T05:02:06.353157+0000","last_became_peered":"2024-09-17T05:02:06.353157+0000","last_unstale":"2024-09-17T05:02:07.349734+0000","last_undegraded":"2024-09-17T05:02:07.349734+0000","last_fullsized":"2024-09-17T05:02:07.349734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349652+0000","last_change":"2024-09-17T05:02:06.352255+0000","last_active":"2024-09-17T05:02:07.349652+0000","last_peered":"2024-09-17T05:02:07.349652+0000","last_clean":"2024-09-17T05:02:07.349652+0000","last_became_active":"2024-09-17T05:02:06.351865+0000","last_became_peered":"2024-09-17T05:02:06.351865+0000","last_unstale":"2024-09-17T05:02:07.349652+0000","last_undegraded":"2024-09-17T05:02:07.349652+0000","last_fullsized":"2024-09-17T05:02:07.349652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:00.377367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349479+0000","last_change":"2024-09-17T05:02:06.347103+0000","last_active":"2024-09-17T05:02:07.349479+0000","last_peered":"2024-09-17T05:02:07.349479+0000","last_clean":"2024-09-17T05:02:07.349479+0000","last_became_active":"2024-09-17T05:02:06.346592+0000","last_became_peered":"2024-09-17T05:02:06.346592+0000","last_unstale":"2024-09-17T05:02:07.349479+0000","last_undegraded":"2024-09-17T05:02:07.349479+0000","last_fullsized":"2024-09-17T05:02:07.349479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314660+0000","last_change":"2024-09-17T05:02:06.334073+0000","last_active":"2024-09-17T05:02:07.314660+0000","last_peered":"2024-09-17T05:02:07.314660+0000","last_clean":"2024-09-17T05:02:07.314660+0000","last_became_active":"2024-09-17T05:02:06.333712+0000","last_became_peered":"2024-09-17T05:02:06.333712+0000","last_unstale":"2024-09-17T05:02:07.314660+0000","last_undegraded":"2024-09-17T05:02:07.314660+0000","last_fullsized":"2024-09-17T05:02:07.314660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:51.877414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321024+0000","last_change":"2024-09-17T05:02:06.345627+0000","last_active":"2024-09-17T05:02:07.321024+0000","last_peered":"2024-09-17T05:02:07.321024+0000","last_clean":"2024-09-17T05:02:07.321024+0000","last_became_active":"2024-09-17T05:02:06.345225+0000","last_became_peered":"2024-09-17T05:02:06.345225+0000","last_unstale":"2024-09-17T05:02:07.321024+0000","last_undegraded":"2024-09-17T05:02:07.321024+0000","last_fullsized":"2024-09-17T05:02:07.321024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:59.047797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420455+0000","last_change":"2024-09-17T05:02:06.338993+0000","last_active":"2024-09-17T05:02:07.420455+0000","last_peered":"2024-09-17T05:02:07.420455+0000","last_clean":"2024-09-17T05:02:07.420455+0000","last_became_active":"2024-09-17T05:02:06.337798+0000","last_became_peered":"2024-09-17T05:02:06.337798+0000","last_unstale":"2024-09-17T05:02:07.420455+0000","last_undegraded":"2024-09-17T05:02:07.420455+0000","last_fullsized":"2024-09-17T05:02:07.420455+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:01.425057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320694+0000","last_change":"2024-09-17T05:02:06.350077+0000","last_active":"2024-09-17T05:02:07.320694+0000","last_peered":"2024-09-17T05:02:07.320694+0000","last_clean":"2024-09-17T05:02:07.320694+0000","last_became_active":"2024-09-17T05:02:06.349773+0000","last_became_peered":"2024-09-17T05:02:06.349773+0000","last_unstale":"2024-09-17T05:02:07.320694+0000","last_undegraded":"2024-09-17T05:02:07.320694+0000","last_fullsized":"2024-09-17T05:02:07.320694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:35.430354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350145+0000","last_change":"2024-09-17T05:02:06.332912+0000","last_active":"2024-09-17T05:02:07.350145+0000","last_peered":"2024-09-17T05:02:07.350145+0000","last_clean":"2024-09-17T05:02:07.350145+0000","last_became_active":"2024-09-17T05:02:06.332354+0000","last_became_peered":"2024-09-17T05:02:06.332354+0000","last_unstale":"2024-09-17T05:02:07.350145+0000","last_undegraded":"2024-09-17T05:02:07.350145+0000","last_fullsized":"2024-09-17T05:02:07.350145+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:22.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420872+0000","last_change":"2024-09-17T05:02:06.335589+0000","last_active":"2024-09-17T05:02:07.420872+0000","last_peered":"2024-09-17T05:02:07.420872+0000","last_clean":"2024-09-17T05:02:07.420872+0000","last_became_active":"2024-09-17T05:02:06.335120+0000","last_became_peered":"2024-09-17T05:02:06.335120+0000","last_unstale":"2024-09-17T05:02:07.420872+0000","last_undegraded":"2024-09-17T05:02:07.420872+0000","last_fullsized":"2024-09-17T05:02:07.420872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:06.399066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314944+0000","last_change":"2024-09-17T05:02:06.331500+0000","last_active":"2024-09-17T05:02:07.314944+0000","last_peered":"2024-09-17T05:02:07.314944+0000","last_clean":"2024-09-17T05:02:07.314944+0000","last_became_active":"2024-09-17T05:02:06.331184+0000","last_became_peered":"2024-09-17T05:02:06.331184+0000","last_unstale":"2024-09-17T05:02:07.314944+0000","last_undegraded":"2024-09-17T05:02:07.314944+0000","last_fullsized":"2024-09-17T05:02:07.314944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:48.958710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314593+0000","last_change":"2024-09-17T05:02:06.330526+0000","last_active":"2024-09-17T05:02:07.314593+0000","last_peered":"2024-09-17T05:02:07.314593+0000","last_clean":"2024-09-17T05:02:07.314593+0000","last_became_active":"2024-09-17T05:02:06.330156+0000","last_became_peered":"2024-09-17T05:02:06.330156+0000","last_unstale":"2024-09-17T05:02:07.314593+0000","last_undegraded":"2024-09-17T05:02:07.314593+0000","last_fullsized":"2024-09-17T05:02:07.314593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420681+0000","last_change":"2024-09-17T05:02:06.356417+0000","last_active":"2024-09-17T05:02:07.420681+0000","last_peered":"2024-09-17T05:02:07.420681+0000","last_clean":"2024-09-17T05:02:07.420681+0000","last_became_active":"2024-09-17T05:02:06.342863+0000","last_became_peered":"2024-09-17T05:02:06.342863+0000","last_unstale":"2024-09-17T05:02:07.420681+0000","last_undegraded":"2024-09-17T05:02:07.420681+0000","last_fullsized":"2024-09-17T05:02:07.420681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:46.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320315+0000","last_change":"2024-09-17T05:02:06.337797+0000","last_active":"2024-09-17T05:02:07.320315+0000","last_peered":"2024-09-17T05:02:07.320315+0000","last_clean":"2024-09-17T05:02:07.320315+0000","last_became_active":"2024-09-17T05:02:06.337396+0000","last_became_peered":"2024-09-17T05:02:06.337396+0000","last_unstale":"2024-09-17T05:02:07.320315+0000","last_undegraded":"2024-09-17T05:02:07.320315+0000","last_fullsized":"2024-09-17T05:02:07.320315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:42.616845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315319+0000","last_change":"2024-09-17T05:02:06.348634+0000","last_active":"2024-09-17T05:02:07.315319+0000","last_peered":"2024-09-17T05:02:07.315319+0000","last_clean":"2024-09-17T05:02:07.315319+0000","last_became_active":"2024-09-17T05:02:06.348284+0000","last_became_peered":"2024-09-17T05:02:06.348284+0000","last_unstale":"2024-09-17T05:02:07.315319+0000","last_undegraded":"2024-09-17T05:02:07.315319+0000","last_fullsized":"2024-09-17T05:02:07.315319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:41.455558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420652+0000","last_change":"2024-09-17T05:02:05.319632+0000","last_active":"2024-09-17T05:02:07.420652+0000","last_peered":"2024-09-17T05:02:07.420652+0000","last_clean":"2024-09-17T05:02:07.420652+0000","last_became_active":"2024-09-17T05:02:05.319321+0000","last_became_peered":"2024-09-17T05:02:05.319321+0000","last_unstale":"2024-09-17T05:02:07.420652+0000","last_undegraded":"2024-09-17T05:02:07.420652+0000","last_fullsized":"2024-09-17T05:02:07.420652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:46.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420445+0000","last_change":"2024-09-17T05:02:06.334112+0000","last_active":"2024-09-17T05:02:07.420445+0000","last_peered":"2024-09-17T05:02:07.420445+0000","last_clean":"2024-09-17T05:02:07.420445+0000","last_became_active":"2024-09-17T05:02:06.333727+0000","last_became_peered":"2024-09-17T05:02:06.333727+0000","last_unstale":"2024-09-17T05:02:07.420445+0000","last_undegraded":"2024-09-17T05:02:07.420445+0000","last_fullsized":"2024-09-17T05:02:07.420445+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:48.515835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420440+0000","last_change":"2024-09-17T05:02:05.330941+0000","last_active":"2024-09-17T05:02:07.420440+0000","last_peered":"2024-09-17T05:02:07.420440+0000","last_clean":"2024-09-17T05:02:07.420440+0000","last_became_active":"2024-09-17T05:02:05.330610+0000","last_became_peered":"2024-09-17T05:02:05.330610+0000","last_unstale":"2024-09-17T05:02:07.420440+0000","last_undegraded":"2024-09-17T05:02:07.420440+0000","last_fullsized":"2024-09-17T05:02:07.420440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:35.977474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315276+0000","last_change":"2024-09-17T05:02:06.367617+0000","last_active":"2024-09-17T05:02:07.315276+0000","last_peered":"2024-09-17T05:02:07.315276+0000","last_clean":"2024-09-17T05:02:07.315276+0000","last_became_active":"2024-09-17T05:02:06.367102+0000","last_became_peered":"2024-09-17T05:02:06.367102+0000","last_unstale":"2024-09-17T05:02:07.315276+0000","last_undegraded":"2024-09-17T05:02:07.315276+0000","last_fullsized":"2024-09-17T05:02:07.315276+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.094882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321076+0000","last_change":"2024-09-17T05:02:05.321020+0000","last_active":"2024-09-17T05:02:07.321076+0000","last_peered":"2024-09-17T05:02:07.321076+0000","last_clean":"2024-09-17T05:02:07.321076+0000","last_became_active":"2024-09-17T05:02:05.320171+0000","last_became_peered":"2024-09-17T05:02:05.320171+0000","last_unstale":"2024-09-17T05:02:07.321076+0000","last_undegraded":"2024-09-17T05:02:07.321076+0000","last_fullsized":"2024-09-17T05:02:07.321076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:00.418844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420627+0000","last_change":"2024-09-17T05:02:06.348099+0000","last_active":"2024-09-17T05:02:07.420627+0000","last_peered":"2024-09-17T05:02:07.420627+0000","last_clean":"2024-09-17T05:02:07.420627+0000","last_became_active":"2024-09-17T05:02:06.347767+0000","last_became_peered":"2024-09-17T05:02:06.347767+0000","last_unstale":"2024-09-17T05:02:07.420627+0000","last_undegraded":"2024-09-17T05:02:07.420627+0000","last_fullsized":"2024-09-17T05:02:07.420627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:39.458764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420697+0000","last_change":"2024-09-17T05:02:05.323652+0000","last_active":"2024-09-17T05:02:07.420697+0000","last_peered":"2024-09-17T05:02:07.420697+0000","last_clean":"2024-09-17T05:02:07.420697+0000","last_became_active":"2024-09-17T05:02:05.323250+0000","last_became_peered":"2024-09-17T05:02:05.323250+0000","last_unstale":"2024-09-17T05:02:07.420697+0000","last_undegraded":"2024-09-17T05:02:07.420697+0000","last_fullsized":"2024-09-17T05:02:07.420697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:46.948808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320590+0000","last_change":"2024-09-17T05:02:06.332617+0000","last_active":"2024-09-17T05:02:07.320590+0000","last_peered":"2024-09-17T05:02:07.320590+0000","last_clean":"2024-09-17T05:02:07.320590+0000","last_became_active":"2024-09-17T05:02:06.330747+0000","last_became_peered":"2024-09-17T05:02:06.330747+0000","last_unstale":"2024-09-17T05:02:07.320590+0000","last_undegraded":"2024-09-17T05:02:07.320590+0000","last_fullsized":"2024-09-17T05:02:07.320590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:51.850349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420085+0000","last_change":"2024-09-17T05:02:05.313259+0000","last_active":"2024-09-17T05:02:07.420085+0000","last_peered":"2024-09-17T05:02:07.420085+0000","last_clean":"2024-09-17T05:02:07.420085+0000","last_became_active":"2024-09-17T05:02:05.312893+0000","last_became_peered":"2024-09-17T05:02:05.312893+0000","last_unstale":"2024-09-17T05:02:07.420085+0000","last_undegraded":"2024-09-17T05:02:07.420085+0000","last_fullsized":"2024-09-17T05:02:07.420085+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350174+0000","last_change":"2024-09-17T05:02:06.333044+0000","last_active":"2024-09-17T05:02:07.350174+0000","last_peered":"2024-09-17T05:02:07.350174+0000","last_clean":"2024-09-17T05:02:07.350174+0000","last_became_active":"2024-09-17T05:02:06.332672+0000","last_became_peered":"2024-09-17T05:02:06.332672+0000","last_unstale":"2024-09-17T05:02:07.350174+0000","last_undegraded":"2024-09-17T05:02:07.350174+0000","last_fullsized":"2024-09-17T05:02:07.350174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:28.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419821+0000","last_change":"2024-09-17T05:02:05.331066+0000","last_active":"2024-09-17T05:02:07.419821+0000","last_peered":"2024-09-17T05:02:07.419821+0000","last_clean":"2024-09-17T05:02:07.419821+0000","last_became_active":"2024-09-17T05:02:05.330727+0000","last_became_peered":"2024-09-17T05:02:05.330727+0000","last_unstale":"2024-09-17T05:02:07.419821+0000","last_undegraded":"2024-09-17T05:02:07.419821+0000","last_fullsized":"2024-09-17T05:02:07.419821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314674+0000","last_change":"2024-09-17T05:02:06.350254+0000","last_active":"2024-09-17T05:02:07.314674+0000","last_peered":"2024-09-17T05:02:07.314674+0000","last_clean":"2024-09-17T05:02:07.314674+0000","last_became_active":"2024-09-17T05:02:06.349926+0000","last_became_peered":"2024-09-17T05:02:06.349926+0000","last_unstale":"2024-09-17T05:02:07.314674+0000","last_undegraded":"2024-09-17T05:02:07.314674+0000","last_fullsized":"2024-09-17T05:02:07.314674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320007+0000","last_change":"2024-09-17T05:02:05.330770+0000","last_active":"2024-09-17T05:02:07.320007+0000","last_peered":"2024-09-17T05:02:07.320007+0000","last_clean":"2024-09-17T05:02:07.320007+0000","last_became_active":"2024-09-17T05:02:05.330452+0000","last_became_peered":"2024-09-17T05:02:05.330452+0000","last_unstale":"2024-09-17T05:02:07.320007+0000","last_undegraded":"2024-09-17T05:02:07.320007+0000","last_fullsized":"2024-09-17T05:02:07.320007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420577+0000","last_change":"2024-09-17T05:02:06.339321+0000","last_active":"2024-09-17T05:02:07.420577+0000","last_peered":"2024-09-17T05:02:07.420577+0000","last_clean":"2024-09-17T05:02:07.420577+0000","last_became_active":"2024-09-17T05:02:06.339039+0000","last_became_peered":"2024-09-17T05:02:06.339039+0000","last_unstale":"2024-09-17T05:02:07.420577+0000","last_undegraded":"2024-09-17T05:02:07.420577+0000","last_fullsized":"2024-09-17T05:02:07.420577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420407+0000","last_change":"2024-09-17T05:02:05.334416+0000","last_active":"2024-09-17T05:02:07.420407+0000","last_peered":"2024-09-17T05:02:07.420407+0000","last_clean":"2024-09-17T05:02:07.420407+0000","last_became_active":"2024-09-17T05:02:05.334066+0000","last_became_peered":"2024-09-17T05:02:05.334066+0000","last_unstale":"2024-09-17T05:02:07.420407+0000","last_undegraded":"2024-09-17T05:02:07.420407+0000","last_fullsized":"2024-09-17T05:02:07.420407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:44.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321356+0000","last_change":"2024-09-17T05:02:06.348674+0000","last_active":"2024-09-17T05:02:07.321356+0000","last_peered":"2024-09-17T05:02:07.321356+0000","last_clean":"2024-09-17T05:02:07.321356+0000","last_became_active":"2024-09-17T05:02:06.348346+0000","last_became_peered":"2024-09-17T05:02:06.348346+0000","last_unstale":"2024-09-17T05:02:07.321356+0000","last_undegraded":"2024-09-17T05:02:07.321356+0000","last_fullsized":"2024-09-17T05:02:07.321356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:39.881805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320775+0000","last_change":"2024-09-17T05:02:05.323997+0000","last_active":"2024-09-17T05:02:07.320775+0000","last_peered":"2024-09-17T05:02:07.320775+0000","last_clean":"2024-09-17T05:02:07.320775+0000","last_became_active":"2024-09-17T05:02:05.323410+0000","last_became_peered":"2024-09-17T05:02:05.323410+0000","last_unstale":"2024-09-17T05:02:07.320775+0000","last_undegraded":"2024-09-17T05:02:07.320775+0000","last_fullsized":"2024-09-17T05:02:07.320775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:32.508695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420275+0000","last_change":"2024-09-17T05:02:06.338660+0000","last_active":"2024-09-17T05:02:07.420275+0000","last_peered":"2024-09-17T05:02:07.420275+0000","last_clean":"2024-09-17T05:02:07.420275+0000","last_became_active":"2024-09-17T05:02:06.338256+0000","last_became_peered":"2024-09-17T05:02:06.338256+0000","last_unstale":"2024-09-17T05:02:07.420275+0000","last_undegraded":"2024-09-17T05:02:07.420275+0000","last_fullsized":"2024-09-17T05:02:07.420275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.216527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315076+0000","last_change":"2024-09-17T05:02:05.326255+0000","last_active":"2024-09-17T05:02:07.315076+0000","last_peered":"2024-09-17T05:02:07.315076+0000","last_clean":"2024-09-17T05:02:07.315076+0000","last_became_active":"2024-09-17T05:02:05.325945+0000","last_became_peered":"2024-09-17T05:02:05.325945+0000","last_unstale":"2024-09-17T05:02:07.315076+0000","last_undegraded":"2024-09-17T05:02:07.315076+0000","last_fullsized":"2024-09-17T05:02:07.315076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:21.224964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320746+0000","last_change":"2024-09-17T05:02:06.335521+0000","last_active":"2024-09-17T05:02:07.320746+0000","last_peered":"2024-09-17T05:02:07.320746+0000","last_clean":"2024-09-17T05:02:07.320746+0000","last_became_active":"2024-09-17T05:02:06.335166+0000","last_became_peered":"2024-09-17T05:02:06.335166+0000","last_unstale":"2024-09-17T05:02:07.320746+0000","last_undegraded":"2024-09-17T05:02:07.320746+0000","last_fullsized":"2024-09-17T05:02:07.320746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:02.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314664+0000","last_change":"2024-09-17T05:02:05.336301+0000","last_active":"2024-09-17T05:02:07.314664+0000","last_peered":"2024-09-17T05:02:07.314664+0000","last_clean":"2024-09-17T05:02:07.314664+0000","last_became_active":"2024-09-17T05:02:05.331553+0000","last_became_peered":"2024-09-17T05:02:05.331553+0000","last_unstale":"2024-09-17T05:02:07.314664+0000","last_undegraded":"2024-09-17T05:02:07.314664+0000","last_fullsized":"2024-09-17T05:02:07.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:23.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420226+0000","last_change":"2024-09-17T05:02:06.342104+0000","last_active":"2024-09-17T05:02:07.420226+0000","last_peered":"2024-09-17T05:02:07.420226+0000","last_clean":"2024-09-17T05:02:07.420226+0000","last_became_active":"2024-09-17T05:02:06.341775+0000","last_became_peered":"2024-09-17T05:02:06.341775+0000","last_unstale":"2024-09-17T05:02:07.420226+0000","last_undegraded":"2024-09-17T05:02:07.420226+0000","last_fullsized":"2024-09-17T05:02:07.420226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:24.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321552+0000","last_change":"2024-09-17T05:02:05.323758+0000","last_active":"2024-09-17T05:02:07.321552+0000","last_peered":"2024-09-17T05:02:07.321552+0000","last_clean":"2024-09-17T05:02:07.321552+0000","last_became_active":"2024-09-17T05:02:05.323426+0000","last_became_peered":"2024-09-17T05:02:05.323426+0000","last_unstale":"2024-09-17T05:02:07.321552+0000","last_undegraded":"2024-09-17T05:02:07.321552+0000","last_fullsized":"2024-09-17T05:02:07.321552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:29.210315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320848+0000","last_change":"2024-09-17T05:02:06.345826+0000","last_active":"2024-09-17T05:02:07.320848+0000","last_peered":"2024-09-17T05:02:07.320848+0000","last_clean":"2024-09-17T05:02:07.320848+0000","last_became_active":"2024-09-17T05:02:06.345343+0000","last_became_peered":"2024-09-17T05:02:06.345343+0000","last_unstale":"2024-09-17T05:02:07.320848+0000","last_undegraded":"2024-09-17T05:02:07.320848+0000","last_fullsized":"2024-09-17T05:02:07.320848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:09.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320977+0000","last_change":"2024-09-17T05:02:05.323105+0000","last_active":"2024-09-17T05:02:07.320977+0000","last_peered":"2024-09-17T05:02:07.320977+0000","last_clean":"2024-09-17T05:02:07.320977+0000","last_became_active":"2024-09-17T05:02:05.320585+0000","last_became_peered":"2024-09-17T05:02:05.320585+0000","last_unstale":"2024-09-17T05:02:07.320977+0000","last_undegraded":"2024-09-17T05:02:07.320977+0000","last_fullsized":"2024-09-17T05:02:07.320977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420267+0000","last_change":"2024-09-17T05:02:06.359915+0000","last_active":"2024-09-17T05:02:07.420267+0000","last_peered":"2024-09-17T05:02:07.420267+0000","last_clean":"2024-09-17T05:02:07.420267+0000","last_became_active":"2024-09-17T05:02:06.359544+0000","last_became_peered":"2024-09-17T05:02:06.359544+0000","last_unstale":"2024-09-17T05:02:07.420267+0000","last_undegraded":"2024-09-17T05:02:07.420267+0000","last_fullsized":"2024-09-17T05:02:07.420267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420061+0000","last_change":"2024-09-17T05:02:05.320829+0000","last_active":"2024-09-17T05:02:07.420061+0000","last_peered":"2024-09-17T05:02:07.420061+0000","last_clean":"2024-09-17T05:02:07.420061+0000","last_became_active":"2024-09-17T05:02:05.320509+0000","last_became_peered":"2024-09-17T05:02:05.320509+0000","last_unstale":"2024-09-17T05:02:07.420061+0000","last_undegraded":"2024-09-17T05:02:07.420061+0000","last_fullsized":"2024-09-17T05:02:07.420061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:36.381090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349878+0000","last_change":"2024-09-17T05:02:06.342348+0000","last_active":"2024-09-17T05:02:07.349878+0000","last_peered":"2024-09-17T05:02:07.349878+0000","last_clean":"2024-09-17T05:02:07.349878+0000","last_became_active":"2024-09-17T05:02:06.342077+0000","last_became_peered":"2024-09-17T05:02:06.342077+0000","last_unstale":"2024-09-17T05:02:07.349878+0000","last_undegraded":"2024-09-17T05:02:07.349878+0000","last_fullsized":"2024-09-17T05:02:07.349878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:24.585972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314795+0000","last_change":"2024-09-17T05:02:05.331453+0000","last_active":"2024-09-17T05:02:07.314795+0000","last_peered":"2024-09-17T05:02:07.314795+0000","last_clean":"2024-09-17T05:02:07.314795+0000","last_became_active":"2024-09-17T05:02:05.330843+0000","last_became_peered":"2024-09-17T05:02:05.330843+0000","last_unstale":"2024-09-17T05:02:07.314795+0000","last_undegraded":"2024-09-17T05:02:07.314795+0000","last_fullsized":"2024-09-17T05:02:07.314795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:53.874382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346719+0000","last_change":"2024-09-17T05:02:06.338645+0000","last_active":"2024-09-17T05:02:07.346719+0000","last_peered":"2024-09-17T05:02:07.346719+0000","last_clean":"2024-09-17T05:02:07.346719+0000","last_became_active":"2024-09-17T05:02:06.338343+0000","last_became_peered":"2024-09-17T05:02:06.338343+0000","last_unstale":"2024-09-17T05:02:07.346719+0000","last_undegraded":"2024-09-17T05:02:07.346719+0000","last_fullsized":"2024-09-17T05:02:07.346719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419537+0000","last_change":"2024-09-17T05:02:05.318024+0000","last_active":"2024-09-17T05:02:07.419537+0000","last_peered":"2024-09-17T05:02:07.419537+0000","last_clean":"2024-09-17T05:02:07.419537+0000","last_became_active":"2024-09-17T05:02:05.317528+0000","last_became_peered":"2024-09-17T05:02:05.317528+0000","last_unstale":"2024-09-17T05:02:07.419537+0000","last_undegraded":"2024-09-17T05:02:07.419537+0000","last_fullsized":"2024-09-17T05:02:07.419537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:22.090770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420452+0000","last_change":"2024-09-17T05:02:06.336675+0000","last_active":"2024-09-17T05:02:07.420452+0000","last_peered":"2024-09-17T05:02:07.420452+0000","last_clean":"2024-09-17T05:02:07.420452+0000","last_became_active":"2024-09-17T05:02:06.336377+0000","last_became_peered":"2024-09-17T05:02:06.336377+0000","last_unstale":"2024-09-17T05:02:07.420452+0000","last_undegraded":"2024-09-17T05:02:07.420452+0000","last_fullsized":"2024-09-17T05:02:07.420452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.158532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420724+0000","last_change":"2024-09-17T05:02:05.334942+0000","last_active":"2024-09-17T05:02:07.420724+0000","last_peered":"2024-09-17T05:02:07.420724+0000","last_clean":"2024-09-17T05:02:07.420724+0000","last_became_active":"2024-09-17T05:02:05.334624+0000","last_became_peered":"2024-09-17T05:02:05.334624+0000","last_unstale":"2024-09-17T05:02:07.420724+0000","last_undegraded":"2024-09-17T05:02:07.420724+0000","last_fullsized":"2024-09-17T05:02:07.420724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:14.855709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420380+0000","last_change":"2024-09-17T05:02:06.334939+0000","last_active":"2024-09-17T05:02:07.420380+0000","last_peered":"2024-09-17T05:02:07.420380+0000","last_clean":"2024-09-17T05:02:07.420380+0000","last_became_active":"2024-09-17T05:02:06.334587+0000","last_became_peered":"2024-09-17T05:02:06.334587+0000","last_unstale":"2024-09-17T05:02:07.420380+0000","last_undegraded":"2024-09-17T05:02:07.420380+0000","last_fullsized":"2024-09-17T05:02:07.420380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:37.387302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315252+0000","last_change":"2024-09-17T05:02:05.326248+0000","last_active":"2024-09-17T05:02:07.315252+0000","last_peered":"2024-09-17T05:02:07.315252+0000","last_clean":"2024-09-17T05:02:07.315252+0000","last_became_active":"2024-09-17T05:02:05.325621+0000","last_became_peered":"2024-09-17T05:02:05.325621+0000","last_unstale":"2024-09-17T05:02:07.315252+0000","last_undegraded":"2024-09-17T05:02:07.315252+0000","last_fullsized":"2024-09-17T05:02:07.315252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:35.920973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420572+0000","last_change":"2024-09-17T05:02:06.332684+0000","last_active":"2024-09-17T05:02:07.420572+0000","last_peered":"2024-09-17T05:02:07.420572+0000","last_clean":"2024-09-17T05:02:07.420572+0000","last_became_active":"2024-09-17T05:02:06.332268+0000","last_became_peered":"2024-09-17T05:02:06.332268+0000","last_unstale":"2024-09-17T05:02:07.420572+0000","last_undegraded":"2024-09-17T05:02:07.420572+0000","last_fullsized":"2024-09-17T05:02:07.420572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:36.983687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346854+0000","last_change":"2024-09-17T05:02:05.329129+0000","last_active":"2024-09-17T05:02:07.346854+0000","last_peered":"2024-09-17T05:02:07.346854+0000","last_clean":"2024-09-17T05:02:07.346854+0000","last_became_active":"2024-09-17T05:02:05.328783+0000","last_became_peered":"2024-09-17T05:02:05.328783+0000","last_unstale":"2024-09-17T05:02:07.346854+0000","last_undegraded":"2024-09-17T05:02:07.346854+0000","last_fullsized":"2024-09-17T05:02:07.346854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:01.114967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320039+0000","last_change":"2024-09-17T05:02:06.332255+0000","last_active":"2024-09-17T05:02:07.320039+0000","last_peered":"2024-09-17T05:02:07.320039+0000","last_clean":"2024-09-17T05:02:07.320039+0000","last_became_active":"2024-09-17T05:02:06.331914+0000","last_became_peered":"2024-09-17T05:02:06.331914+0000","last_unstale":"2024-09-17T05:02:07.320039+0000","last_undegraded":"2024-09-17T05:02:07.320039+0000","last_fullsized":"2024-09-17T05:02:07.320039+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:08.687459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319356+0000","last_change":"2024-09-17T05:02:05.324272+0000","last_active":"2024-09-17T05:02:07.319356+0000","last_peered":"2024-09-17T05:02:07.319356+0000","last_clean":"2024-09-17T05:02:07.319356+0000","last_became_active":"2024-09-17T05:02:05.323884+0000","last_became_peered":"2024-09-17T05:02:05.323884+0000","last_unstale":"2024-09-17T05:02:07.319356+0000","last_undegraded":"2024-09-17T05:02:07.319356+0000","last_fullsized":"2024-09-17T05:02:07.319356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:39.493914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314537+0000","last_change":"2024-09-17T05:02:06.321153+0000","last_active":"2024-09-17T05:02:07.314537+0000","last_peered":"2024-09-17T05:02:07.314537+0000","last_clean":"2024-09-17T05:02:07.314537+0000","last_became_active":"2024-09-17T05:02:06.320793+0000","last_became_peered":"2024-09-17T05:02:06.320793+0000","last_unstale":"2024-09-17T05:02:07.314537+0000","last_undegraded":"2024-09-17T05:02:07.314537+0000","last_fullsized":"2024-09-17T05:02:07.314537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:49.889726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319666+0000","last_change":"2024-09-17T05:02:05.321831+0000","last_active":"2024-09-17T05:02:07.319666+0000","last_peered":"2024-09-17T05:02:07.319666+0000","last_clean":"2024-09-17T05:02:07.319666+0000","last_became_active":"2024-09-17T05:02:05.321507+0000","last_became_peered":"2024-09-17T05:02:05.321507+0000","last_unstale":"2024-09-17T05:02:07.319666+0000","last_undegraded":"2024-09-17T05:02:07.319666+0000","last_fullsized":"2024-09-17T05:02:07.319666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:31.096320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319927+0000","last_change":"2024-09-17T05:02:06.362961+0000","last_active":"2024-09-17T05:02:07.319927+0000","last_peered":"2024-09-17T05:02:07.319927+0000","last_clean":"2024-09-17T05:02:07.319927+0000","last_became_active":"2024-09-17T05:02:06.362589+0000","last_became_peered":"2024-09-17T05:02:06.362589+0000","last_unstale":"2024-09-17T05:02:07.319927+0000","last_undegraded":"2024-09-17T05:02:07.319927+0000","last_fullsized":"2024-09-17T05:02:07.319927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:43.235010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420209+0000","last_change":"2024-09-17T05:02:05.334796+0000","last_active":"2024-09-17T05:02:07.420209+0000","last_peered":"2024-09-17T05:02:07.420209+0000","last_clean":"2024-09-17T05:02:07.420209+0000","last_became_active":"2024-09-17T05:02:05.334457+0000","last_became_peered":"2024-09-17T05:02:05.334457+0000","last_unstale":"2024-09-17T05:02:07.420209+0000","last_undegraded":"2024-09-17T05:02:07.420209+0000","last_fullsized":"2024-09-17T05:02:07.420209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:50.024608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349461+0000","last_change":"2024-09-17T05:02:06.360661+0000","last_active":"2024-09-17T05:02:07.349461+0000","last_peered":"2024-09-17T05:02:07.349461+0000","last_clean":"2024-09-17T05:02:07.349461+0000","last_became_active":"2024-09-17T05:02:06.360293+0000","last_became_peered":"2024-09-17T05:02:06.360293+0000","last_unstale":"2024-09-17T05:02:07.349461+0000","last_undegraded":"2024-09-17T05:02:07.349461+0000","last_fullsized":"2024-09-17T05:02:07.349461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420548+0000","last_change":"2024-09-17T05:02:05.335089+0000","last_active":"2024-09-17T05:02:07.420548+0000","last_peered":"2024-09-17T05:02:07.420548+0000","last_clean":"2024-09-17T05:02:07.420548+0000","last_became_active":"2024-09-17T05:02:05.334761+0000","last_became_peered":"2024-09-17T05:02:05.334761+0000","last_unstale":"2024-09-17T05:02:07.420548+0000","last_undegraded":"2024-09-17T05:02:07.420548+0000","last_fullsized":"2024-09-17T05:02:07.420548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319697+0000","last_change":"2024-09-17T05:02:06.337072+0000","last_active":"2024-09-17T05:02:07.319697+0000","last_peered":"2024-09-17T05:02:07.319697+0000","last_clean":"2024-09-17T05:02:07.319697+0000","last_became_active":"2024-09-17T05:02:06.335541+0000","last_became_peered":"2024-09-17T05:02:06.335541+0000","last_unstale":"2024-09-17T05:02:07.319697+0000","last_undegraded":"2024-09-17T05:02:07.319697+0000","last_fullsized":"2024-09-17T05:02:07.319697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:33.058079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419849+0000","last_change":"2024-09-17T05:02:05.317067+0000","last_active":"2024-09-17T05:02:07.419849+0000","last_peered":"2024-09-17T05:02:07.419849+0000","last_clean":"2024-09-17T05:02:07.419849+0000","last_became_active":"2024-09-17T05:02:05.316750+0000","last_became_peered":"2024-09-17T05:02:05.316750+0000","last_unstale":"2024-09-17T05:02:07.419849+0000","last_undegraded":"2024-09-17T05:02:07.419849+0000","last_fullsized":"2024-09-17T05:02:07.419849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:11.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346952+0000","last_change":"2024-09-17T05:02:06.342012+0000","last_active":"2024-09-17T05:02:07.346952+0000","last_peered":"2024-09-17T05:02:07.346952+0000","last_clean":"2024-09-17T05:02:07.346952+0000","last_became_active":"2024-09-17T05:02:06.341627+0000","last_became_peered":"2024-09-17T05:02:06.341627+0000","last_unstale":"2024-09-17T05:02:07.346952+0000","last_undegraded":"2024-09-17T05:02:07.346952+0000","last_fullsized":"2024-09-17T05:02:07.346952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:04.620408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315375+0000","last_change":"2024-09-17T05:02:05.331974+0000","last_active":"2024-09-17T05:02:07.315375+0000","last_peered":"2024-09-17T05:02:07.315375+0000","last_clean":"2024-09-17T05:02:07.315375+0000","last_became_active":"2024-09-17T05:02:05.331277+0000","last_became_peered":"2024-09-17T05:02:05.331277+0000","last_unstale":"2024-09-17T05:02:07.315375+0000","last_undegraded":"2024-09-17T05:02:07.315375+0000","last_fullsized":"2024-09-17T05:02:07.315375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:14.008073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350015+0000","last_change":"2024-09-17T05:02:06.361595+0000","last_active":"2024-09-17T05:02:07.350015+0000","last_peered":"2024-09-17T05:02:07.350015+0000","last_clean":"2024-09-17T05:02:07.350015+0000","last_became_active":"2024-09-17T05:02:06.361219+0000","last_became_peered":"2024-09-17T05:02:06.361219+0000","last_unstale":"2024-09-17T05:02:07.350015+0000","last_undegraded":"2024-09-17T05:02:07.350015+0000","last_fullsized":"2024-09-17T05:02:07.350015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:47.878683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.348883+0000","last_change":"2024-09-17T05:02:05.334870+0000","last_active":"2024-09-17T05:02:07.348883+0000","last_peered":"2024-09-17T05:02:07.348883+0000","last_clean":"2024-09-17T05:02:07.348883+0000","last_became_active":"2024-09-17T05:02:05.334512+0000","last_became_peered":"2024-09-17T05:02:05.334512+0000","last_unstale":"2024-09-17T05:02:07.348883+0000","last_undegraded":"2024-09-17T05:02:07.348883+0000","last_fullsized":"2024-09-17T05:02:07.348883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319479+0000","last_change":"2024-09-17T05:02:06.329065+0000","last_active":"2024-09-17T05:02:07.319479+0000","last_peered":"2024-09-17T05:02:07.319479+0000","last_clean":"2024-09-17T05:02:07.319479+0000","last_became_active":"2024-09-17T05:02:06.328707+0000","last_became_peered":"2024-09-17T05:02:06.328707+0000","last_unstale":"2024-09-17T05:02:07.319479+0000","last_undegraded":"2024-09-17T05:02:07.319479+0000","last_fullsized":"2024-09-17T05:02:07.319479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:09.684814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346976+0000","last_change":"2024-09-17T05:02:05.325366+0000","last_active":"2024-09-17T05:02:07.346976+0000","last_peered":"2024-09-17T05:02:07.346976+0000","last_clean":"2024-09-17T05:02:07.346976+0000","last_became_active":"2024-09-17T05:02:05.325037+0000","last_became_peered":"2024-09-17T05:02:05.325037+0000","last_unstale":"2024-09-17T05:02:07.346976+0000","last_undegraded":"2024-09-17T05:02:07.346976+0000","last_fullsized":"2024-09-17T05:02:07.346976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346633+0000","last_change":"2024-09-17T05:02:06.336842+0000","last_active":"2024-09-17T05:02:07.346633+0000","last_peered":"2024-09-17T05:02:07.346633+0000","last_clean":"2024-09-17T05:02:07.346633+0000","last_became_active":"2024-09-17T05:02:06.334045+0000","last_became_peered":"2024-09-17T05:02:06.334045+0000","last_unstale":"2024-09-17T05:02:07.346633+0000","last_undegraded":"2024-09-17T05:02:07.346633+0000","last_fullsized":"2024-09-17T05:02:07.346633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:30.924123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320355+0000","last_change":"2024-09-17T05:02:05.316008+0000","last_active":"2024-09-17T05:02:07.320355+0000","last_peered":"2024-09-17T05:02:07.320355+0000","last_clean":"2024-09-17T05:02:07.320355+0000","last_became_active":"2024-09-17T05:02:05.315680+0000","last_became_peered":"2024-09-17T05:02:05.315680+0000","last_unstale":"2024-09-17T05:02:07.320355+0000","last_undegraded":"2024-09-17T05:02:07.320355+0000","last_fullsized":"2024-09-17T05:02:07.320355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:25.314134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420182+0000","last_change":"2024-09-17T05:02:06.319976+0000","last_active":"2024-09-17T05:02:07.420182+0000","last_peered":"2024-09-17T05:02:07.420182+0000","last_clean":"2024-09-17T05:02:07.420182+0000","last_became_active":"2024-09-17T05:02:06.319609+0000","last_became_peered":"2024-09-17T05:02:06.319609+0000","last_unstale":"2024-09-17T05:02:07.420182+0000","last_undegraded":"2024-09-17T05:02:07.420182+0000","last_fullsized":"2024-09-17T05:02:07.420182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349774+0000","last_change":"2024-09-17T05:02:05.331789+0000","last_active":"2024-09-17T05:02:07.349774+0000","last_peered":"2024-09-17T05:02:07.349774+0000","last_clean":"2024-09-17T05:02:07.349774+0000","last_became_active":"2024-09-17T05:02:05.327820+0000","last_became_peered":"2024-09-17T05:02:05.327820+0000","last_unstale":"2024-09-17T05:02:07.349774+0000","last_undegraded":"2024-09-17T05:02:07.349774+0000","last_fullsized":"2024-09-17T05:02:07.349774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314859+0000","last_change":"2024-09-17T05:02:06.333560+0000","last_active":"2024-09-17T05:02:07.314859+0000","last_peered":"2024-09-17T05:02:07.314859+0000","last_clean":"2024-09-17T05:02:07.314859+0000","last_became_active":"2024-09-17T05:02:06.333210+0000","last_became_peered":"2024-09-17T05:02:06.333210+0000","last_unstale":"2024-09-17T05:02:07.314859+0000","last_undegraded":"2024-09-17T05:02:07.314859+0000","last_fullsized":"2024-09-17T05:02:07.314859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.884069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314950+0000","last_change":"2024-09-17T05:02:05.325774+0000","last_active":"2024-09-17T05:02:07.314950+0000","last_peered":"2024-09-17T05:02:07.314950+0000","last_clean":"2024-09-17T05:02:07.314950+0000","last_became_active":"2024-09-17T05:02:05.325459+0000","last_became_peered":"2024-09-17T05:02:05.325459+0000","last_unstale":"2024-09-17T05:02:07.314950+0000","last_undegraded":"2024-09-17T05:02:07.314950+0000","last_fullsized":"2024-09-17T05:02:07.314950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:55.257416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349680+0000","last_change":"2024-09-17T05:02:06.340513+0000","last_active":"2024-09-17T05:02:07.349680+0000","last_peered":"2024-09-17T05:02:07.349680+0000","last_clean":"2024-09-17T05:02:07.349680+0000","last_became_active":"2024-09-17T05:02:06.340154+0000","last_became_peered":"2024-09-17T05:02:06.340154+0000","last_unstale":"2024-09-17T05:02:07.349680+0000","last_undegraded":"2024-09-17T05:02:07.349680+0000","last_fullsized":"2024-09-17T05:02:07.349680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320990+0000","last_change":"2024-09-17T05:02:05.325138+0000","last_active":"2024-09-17T05:02:07.320990+0000","last_peered":"2024-09-17T05:02:07.320990+0000","last_clean":"2024-09-17T05:02:07.320990+0000","last_became_active":"2024-09-17T05:02:05.324679+0000","last_became_peered":"2024-09-17T05:02:05.324679+0000","last_unstale":"2024-09-17T05:02:07.320990+0000","last_undegraded":"2024-09-17T05:02:07.320990+0000","last_fullsized":"2024-09-17T05:02:07.320990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:39.915152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315287+0000","last_change":"2024-09-17T05:02:06.331069+0000","last_active":"2024-09-17T05:02:07.315287+0000","last_peered":"2024-09-17T05:02:07.315287+0000","last_clean":"2024-09-17T05:02:07.315287+0000","last_became_active":"2024-09-17T05:02:06.330731+0000","last_became_peered":"2024-09-17T05:02:06.330731+0000","last_unstale":"2024-09-17T05:02:07.315287+0000","last_undegraded":"2024-09-17T05:02:07.315287+0000","last_fullsized":"2024-09-17T05:02:07.315287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:38.520351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349709+0000","last_change":"2024-09-17T05:02:05.335344+0000","last_active":"2024-09-17T05:02:07.349709+0000","last_peered":"2024-09-17T05:02:07.349709+0000","last_clean":"2024-09-17T05:02:07.349709+0000","last_became_active":"2024-09-17T05:02:05.334313+0000","last_became_peered":"2024-09-17T05:02:05.334313+0000","last_unstale":"2024-09-17T05:02:07.349709+0000","last_undegraded":"2024-09-17T05:02:07.349709+0000","last_fullsized":"2024-09-17T05:02:07.349709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420491+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:02:07.420491+0000","last_peered":"2024-09-17T05:02:07.420491+0000","last_clean":"2024-09-17T05:02:07.420491+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:02:07.420491+0000","last_undegraded":"2024-09-17T05:02:07.420491+0000","last_fullsized":"2024-09-17T05:02:07.420491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":458752,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":446464,"data_stored":188713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":462848,"data_stored":190415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":446464,"data_stored":188769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":190873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1040384,"data_stored":779069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":450560,"data_stored":188771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":786321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:02:15.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:02:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 -- 172.21.15.16:0/3731004028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4003c20 msgr2=0x7fa1c4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3731004028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1c4003c20 0x7fa1c4004070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c640 tx=0x7fa1d001d260 comp rx=0 tx=0).stop 2024-09-17T05:02:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e6000640 1 -- 172.21.15.16:0/3731004028 reap_dead start 2024-09-17T05:02:15.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 -- 172.21.15.16:0/3731004028 shutdown_connections 2024-09-17T05:02:15.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 -- 172.21.15.16:0/3731004028 >> 172.21.15.16:0/3731004028 conn(0x7fa1e006f900 msgr2=0x7fa1e00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:15.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 -- 172.21.15.16:0/3731004028 shutdown_connections 2024-09-17T05:02:15.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 -- 172.21.15.16:0/3731004028 wait complete. 2024-09-17T05:02:15.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 Processor -- start 2024-09-17T05:02:15.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.374+0000 7fa1e8000640 1 -- start start 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c4003c20 0x7fa1e014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1e00cbd00 0x7fa1e01505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 0x7fa1e0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0167670 con 0x7fa1e01632e0 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e01674f0 con 0x7fa1c4003c20 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e01677f0 con 0x7fa1e00cbd00 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 0x7fa1e0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 0x7fa1e0150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 -- 172.21.15.16:0/3051769278 learned_addr learned my addr 172.21.15.16:0/3051769278 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e5600640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c4003c20 0x7fa1e014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e4c00640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1e00cbd00 0x7fa1e01505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 -- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1e00cbd00 msgr2=0x7fa1e01505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1e00cbd00 0x7fa1e01505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 -- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c4003c20 msgr2=0x7fa1e014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c4003c20 0x7fa1e014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 -- 172.21.15.16:0/3051769278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e01f81c0 con 0x7fa1e01632e0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e6000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 0x7fa1e0150b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa1e015f0b0 tx=0x7fa1d8011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004440 con 0x7fa1e01632e0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1d8016070 con 0x7fa1e01632e0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800fc40 con 0x7fa1e01632e0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e01fc490 con 0x7fa1e01632e0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e0153650 con 0x7fa1e01632e0 2024-09-17T05:02:15.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.378+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e00008e0 con 0x7fa1e01632e0 2024-09-17T05:02:15.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.382+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1d8003600 con 0x7fa1e01632e0 2024-09-17T05:02:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.382+0000 7fa1dd600640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1c0073200 0x7fa1c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.382+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa1d8094f00 con 0x7fa1e01632e0 2024-09-17T05:02:15.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.382+0000 7fa1e5600640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1c0073200 0x7fa1c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.382+0000 7fa1e5600640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1c0073200 0x7fa1c0075710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00ad000 tx=0x7fa1d001a0a0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:15.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.386+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa1d8022020 con 0x7fa1e01632e0 2024-09-17T05:02:15.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.558+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1e0069770 con 0x7fa1c0073200 2024-09-17T05:02:15.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.562+0000 7fa1dd600640 1 -- 172.21.15.16:0/3051769278 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251858 (secure 0 0 0) 0x7fa1e0069770 con 0x7fa1c0073200 2024-09-17T05:02:15.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:02:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1c0073200 msgr2=0x7fa1c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1c0073200 0x7fa1c0075710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00ad000 tx=0x7fa1d001a0a0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 msgr2=0x7fa1e0150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 0x7fa1e0150b00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa1e015f0b0 tx=0x7fa1d8011e10 comp rx=0 tx=0).stop 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 shutdown_connections 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1c0073200 0x7fa1c0075710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e01632e0 0x7fa1e0150b00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1e00cbd00 0x7fa1e01505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 --2- 172.21.15.16:0/3051769278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c4003c20 0x7fa1e014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 >> 172.21.15.16:0/3051769278 conn(0x7fa1e006f900 msgr2=0x7fa1e0155730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 shutdown_connections 2024-09-17T05:02:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.566+0000 7fa1e8000640 1 -- 172.21.15.16:0/3051769278 wait complete. 2024-09-17T05:02:15.589 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-17T05:02:15.477970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237664,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737056,"statfs":{"total":773094113280,"available":772850745344,"internally_reserved":0,"allocated":4890624,"data_stored":2748524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.166783"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321351+0000","last_change":"2024-09-17T05:02:06.338525+0000","last_active":"2024-09-17T05:02:07.321351+0000","last_peered":"2024-09-17T05:02:07.321351+0000","last_clean":"2024-09-17T05:02:07.321351+0000","last_became_active":"2024-09-17T05:02:06.338232+0000","last_became_peered":"2024-09-17T05:02:06.338232+0000","last_unstale":"2024-09-17T05:02:07.321351+0000","last_undegraded":"2024-09-17T05:02:07.321351+0000","last_fullsized":"2024-09-17T05:02:07.321351+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:56.646622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420620+0000","last_change":"2024-09-17T05:02:06.332250+0000","last_active":"2024-09-17T05:02:07.420620+0000","last_peered":"2024-09-17T05:02:07.420620+0000","last_clean":"2024-09-17T05:02:07.420620+0000","last_became_active":"2024-09-17T05:02:06.331927+0000","last_became_peered":"2024-09-17T05:02:06.331927+0000","last_unstale":"2024-09-17T05:02:07.420620+0000","last_undegraded":"2024-09-17T05:02:07.420620+0000","last_fullsized":"2024-09-17T05:02:07.420620+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320879+0000","last_change":"2024-09-17T05:02:06.336648+0000","last_active":"2024-09-17T05:02:07.320879+0000","last_peered":"2024-09-17T05:02:07.320879+0000","last_clean":"2024-09-17T05:02:07.320879+0000","last_became_active":"2024-09-17T05:02:06.336337+0000","last_became_peered":"2024-09-17T05:02:06.336337+0000","last_unstale":"2024-09-17T05:02:07.320879+0000","last_undegraded":"2024-09-17T05:02:07.320879+0000","last_fullsized":"2024-09-17T05:02:07.320879+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:36:44.810456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321127+0000","last_change":"2024-09-17T05:02:06.348672+0000","last_active":"2024-09-17T05:02:07.321127+0000","last_peered":"2024-09-17T05:02:07.321127+0000","last_clean":"2024-09-17T05:02:07.321127+0000","last_became_active":"2024-09-17T05:02:06.348392+0000","last_became_peered":"2024-09-17T05:02:06.348392+0000","last_unstale":"2024-09-17T05:02:07.321127+0000","last_undegraded":"2024-09-17T05:02:07.321127+0000","last_fullsized":"2024-09-17T05:02:07.321127+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:06.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346718+0000","last_change":"2024-09-17T05:02:06.343632+0000","last_active":"2024-09-17T05:02:07.346718+0000","last_peered":"2024-09-17T05:02:07.346718+0000","last_clean":"2024-09-17T05:02:07.346718+0000","last_became_active":"2024-09-17T05:02:06.343257+0000","last_became_peered":"2024-09-17T05:02:06.343257+0000","last_unstale":"2024-09-17T05:02:07.346718+0000","last_undegraded":"2024-09-17T05:02:07.346718+0000","last_fullsized":"2024-09-17T05:02:07.346718+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:33:00.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420107+0000","last_change":"2024-09-17T05:02:06.330126+0000","last_active":"2024-09-17T05:02:07.420107+0000","last_peered":"2024-09-17T05:02:07.420107+0000","last_clean":"2024-09-17T05:02:07.420107+0000","last_became_active":"2024-09-17T05:02:06.329825+0000","last_became_peered":"2024-09-17T05:02:06.329825+0000","last_unstale":"2024-09-17T05:02:07.420107+0000","last_undegraded":"2024-09-17T05:02:07.420107+0000","last_fullsized":"2024-09-17T05:02:07.420107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:46.368942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420406+0000","last_change":"2024-09-17T05:02:06.343766+0000","last_active":"2024-09-17T05:02:07.420406+0000","last_peered":"2024-09-17T05:02:07.420406+0000","last_clean":"2024-09-17T05:02:07.420406+0000","last_became_active":"2024-09-17T05:02:06.343207+0000","last_became_peered":"2024-09-17T05:02:06.343207+0000","last_unstale":"2024-09-17T05:02:07.420406+0000","last_undegraded":"2024-09-17T05:02:07.420406+0000","last_fullsized":"2024-09-17T05:02:07.420406+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:51.030820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320027+0000","last_change":"2024-09-17T05:02:06.338244+0000","last_active":"2024-09-17T05:02:07.320027+0000","last_peered":"2024-09-17T05:02:07.320027+0000","last_clean":"2024-09-17T05:02:07.320027+0000","last_became_active":"2024-09-17T05:02:06.337859+0000","last_became_peered":"2024-09-17T05:02:06.337859+0000","last_unstale":"2024-09-17T05:02:07.320027+0000","last_undegraded":"2024-09-17T05:02:07.320027+0000","last_fullsized":"2024-09-17T05:02:07.320027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:36.957053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.347110+0000","last_change":"2024-09-17T05:02:06.336573+0000","last_active":"2024-09-17T05:02:07.347110+0000","last_peered":"2024-09-17T05:02:07.347110+0000","last_clean":"2024-09-17T05:02:07.347110+0000","last_became_active":"2024-09-17T05:02:06.336253+0000","last_became_peered":"2024-09-17T05:02:06.336253+0000","last_unstale":"2024-09-17T05:02:07.347110+0000","last_undegraded":"2024-09-17T05:02:07.347110+0000","last_fullsized":"2024-09-17T05:02:07.347110+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:47.955020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420328+0000","last_change":"2024-09-17T05:02:06.362653+0000","last_active":"2024-09-17T05:02:07.420328+0000","last_peered":"2024-09-17T05:02:07.420328+0000","last_clean":"2024-09-17T05:02:07.420328+0000","last_became_active":"2024-09-17T05:02:06.362345+0000","last_became_peered":"2024-09-17T05:02:06.362345+0000","last_unstale":"2024-09-17T05:02:07.420328+0000","last_undegraded":"2024-09-17T05:02:07.420328+0000","last_fullsized":"2024-09-17T05:02:07.420328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:12.091714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350026+0000","last_change":"2024-09-17T05:02:06.341127+0000","last_active":"2024-09-17T05:02:07.350026+0000","last_peered":"2024-09-17T05:02:07.350026+0000","last_clean":"2024-09-17T05:02:07.350026+0000","last_became_active":"2024-09-17T05:02:06.340796+0000","last_became_peered":"2024-09-17T05:02:06.340796+0000","last_unstale":"2024-09-17T05:02:07.350026+0000","last_undegraded":"2024-09-17T05:02:07.350026+0000","last_fullsized":"2024-09-17T05:02:07.350026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:54.880594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346735+0000","last_change":"2024-09-17T05:02:06.336217+0000","last_active":"2024-09-17T05:02:07.346735+0000","last_peered":"2024-09-17T05:02:07.346735+0000","last_clean":"2024-09-17T05:02:07.346735+0000","last_became_active":"2024-09-17T05:02:06.333449+0000","last_became_peered":"2024-09-17T05:02:06.333449+0000","last_unstale":"2024-09-17T05:02:07.346735+0000","last_undegraded":"2024-09-17T05:02:07.346735+0000","last_fullsized":"2024-09-17T05:02:07.346735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:23.096982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420312+0000","last_change":"2024-09-17T05:02:06.330216+0000","last_active":"2024-09-17T05:02:07.420312+0000","last_peered":"2024-09-17T05:02:07.420312+0000","last_clean":"2024-09-17T05:02:07.420312+0000","last_became_active":"2024-09-17T05:02:06.329869+0000","last_became_peered":"2024-09-17T05:02:06.329869+0000","last_unstale":"2024-09-17T05:02:07.420312+0000","last_undegraded":"2024-09-17T05:02:07.420312+0000","last_fullsized":"2024-09-17T05:02:07.420312+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419668+0000","last_change":"2024-09-17T05:02:06.337107+0000","last_active":"2024-09-17T05:02:07.419668+0000","last_peered":"2024-09-17T05:02:07.419668+0000","last_clean":"2024-09-17T05:02:07.419668+0000","last_became_active":"2024-09-17T05:02:06.336774+0000","last_became_peered":"2024-09-17T05:02:06.336774+0000","last_unstale":"2024-09-17T05:02:07.419668+0000","last_undegraded":"2024-09-17T05:02:07.419668+0000","last_fullsized":"2024-09-17T05:02:07.419668+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:26.320347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314512+0000","last_change":"2024-09-17T05:02:06.321133+0000","last_active":"2024-09-17T05:02:07.314512+0000","last_peered":"2024-09-17T05:02:07.314512+0000","last_clean":"2024-09-17T05:02:07.314512+0000","last_became_active":"2024-09-17T05:02:06.320714+0000","last_became_peered":"2024-09-17T05:02:06.320714+0000","last_unstale":"2024-09-17T05:02:07.314512+0000","last_undegraded":"2024-09-17T05:02:07.314512+0000","last_fullsized":"2024-09-17T05:02:07.314512+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:18.256704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420854+0000","last_change":"2024-09-17T05:02:06.335127+0000","last_active":"2024-09-17T05:02:07.420854+0000","last_peered":"2024-09-17T05:02:07.420854+0000","last_clean":"2024-09-17T05:02:07.420854+0000","last_became_active":"2024-09-17T05:02:06.334693+0000","last_became_peered":"2024-09-17T05:02:06.334693+0000","last_unstale":"2024-09-17T05:02:07.420854+0000","last_undegraded":"2024-09-17T05:02:07.420854+0000","last_fullsized":"2024-09-17T05:02:07.420854+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:01:20.716033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315318+0000","last_change":"2024-09-17T05:02:06.352349+0000","last_active":"2024-09-17T05:02:07.315318+0000","last_peered":"2024-09-17T05:02:07.315318+0000","last_clean":"2024-09-17T05:02:07.315318+0000","last_became_active":"2024-09-17T05:02:06.352010+0000","last_became_peered":"2024-09-17T05:02:06.352010+0000","last_unstale":"2024-09-17T05:02:07.315318+0000","last_undegraded":"2024-09-17T05:02:07.315318+0000","last_fullsized":"2024-09-17T05:02:07.315318+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:43.149726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314938+0000","last_change":"2024-09-17T05:02:06.336278+0000","last_active":"2024-09-17T05:02:07.314938+0000","last_peered":"2024-09-17T05:02:07.314938+0000","last_clean":"2024-09-17T05:02:07.314938+0000","last_became_active":"2024-09-17T05:02:06.334917+0000","last_became_peered":"2024-09-17T05:02:06.334917+0000","last_unstale":"2024-09-17T05:02:07.314938+0000","last_undegraded":"2024-09-17T05:02:07.314938+0000","last_fullsized":"2024-09-17T05:02:07.314938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:51.224167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321119+0000","last_change":"2024-09-17T05:02:06.335384+0000","last_active":"2024-09-17T05:02:07.321119+0000","last_peered":"2024-09-17T05:02:07.321119+0000","last_clean":"2024-09-17T05:02:07.321119+0000","last_became_active":"2024-09-17T05:02:06.334178+0000","last_became_peered":"2024-09-17T05:02:06.334178+0000","last_unstale":"2024-09-17T05:02:07.321119+0000","last_undegraded":"2024-09-17T05:02:07.321119+0000","last_fullsized":"2024-09-17T05:02:07.321119+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:57.578193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349734+0000","last_change":"2024-09-17T05:02:06.353747+0000","last_active":"2024-09-17T05:02:07.349734+0000","last_peered":"2024-09-17T05:02:07.349734+0000","last_clean":"2024-09-17T05:02:07.349734+0000","last_became_active":"2024-09-17T05:02:06.353157+0000","last_became_peered":"2024-09-17T05:02:06.353157+0000","last_unstale":"2024-09-17T05:02:07.349734+0000","last_undegraded":"2024-09-17T05:02:07.349734+0000","last_fullsized":"2024-09-17T05:02:07.349734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:19.531544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349652+0000","last_change":"2024-09-17T05:02:06.352255+0000","last_active":"2024-09-17T05:02:07.349652+0000","last_peered":"2024-09-17T05:02:07.349652+0000","last_clean":"2024-09-17T05:02:07.349652+0000","last_became_active":"2024-09-17T05:02:06.351865+0000","last_became_peered":"2024-09-17T05:02:06.351865+0000","last_unstale":"2024-09-17T05:02:07.349652+0000","last_undegraded":"2024-09-17T05:02:07.349652+0000","last_fullsized":"2024-09-17T05:02:07.349652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:52:00.377367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349479+0000","last_change":"2024-09-17T05:02:06.347103+0000","last_active":"2024-09-17T05:02:07.349479+0000","last_peered":"2024-09-17T05:02:07.349479+0000","last_clean":"2024-09-17T05:02:07.349479+0000","last_became_active":"2024-09-17T05:02:06.346592+0000","last_became_peered":"2024-09-17T05:02:06.346592+0000","last_unstale":"2024-09-17T05:02:07.349479+0000","last_undegraded":"2024-09-17T05:02:07.349479+0000","last_fullsized":"2024-09-17T05:02:07.349479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314660+0000","last_change":"2024-09-17T05:02:06.334073+0000","last_active":"2024-09-17T05:02:07.314660+0000","last_peered":"2024-09-17T05:02:07.314660+0000","last_clean":"2024-09-17T05:02:07.314660+0000","last_became_active":"2024-09-17T05:02:06.333712+0000","last_became_peered":"2024-09-17T05:02:06.333712+0000","last_unstale":"2024-09-17T05:02:07.314660+0000","last_undegraded":"2024-09-17T05:02:07.314660+0000","last_fullsized":"2024-09-17T05:02:07.314660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:51.877414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321024+0000","last_change":"2024-09-17T05:02:06.345627+0000","last_active":"2024-09-17T05:02:07.321024+0000","last_peered":"2024-09-17T05:02:07.321024+0000","last_clean":"2024-09-17T05:02:07.321024+0000","last_became_active":"2024-09-17T05:02:06.345225+0000","last_became_peered":"2024-09-17T05:02:06.345225+0000","last_unstale":"2024-09-17T05:02:07.321024+0000","last_undegraded":"2024-09-17T05:02:07.321024+0000","last_fullsized":"2024-09-17T05:02:07.321024+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:59.047797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420455+0000","last_change":"2024-09-17T05:02:06.338993+0000","last_active":"2024-09-17T05:02:07.420455+0000","last_peered":"2024-09-17T05:02:07.420455+0000","last_clean":"2024-09-17T05:02:07.420455+0000","last_became_active":"2024-09-17T05:02:06.337798+0000","last_became_peered":"2024-09-17T05:02:06.337798+0000","last_unstale":"2024-09-17T05:02:07.420455+0000","last_undegraded":"2024-09-17T05:02:07.420455+0000","last_fullsized":"2024-09-17T05:02:07.420455+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:01.425057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320694+0000","last_change":"2024-09-17T05:02:06.350077+0000","last_active":"2024-09-17T05:02:07.320694+0000","last_peered":"2024-09-17T05:02:07.320694+0000","last_clean":"2024-09-17T05:02:07.320694+0000","last_became_active":"2024-09-17T05:02:06.349773+0000","last_became_peered":"2024-09-17T05:02:06.349773+0000","last_unstale":"2024-09-17T05:02:07.320694+0000","last_undegraded":"2024-09-17T05:02:07.320694+0000","last_fullsized":"2024-09-17T05:02:07.320694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:35.430354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350145+0000","last_change":"2024-09-17T05:02:06.332912+0000","last_active":"2024-09-17T05:02:07.350145+0000","last_peered":"2024-09-17T05:02:07.350145+0000","last_clean":"2024-09-17T05:02:07.350145+0000","last_became_active":"2024-09-17T05:02:06.332354+0000","last_became_peered":"2024-09-17T05:02:06.332354+0000","last_unstale":"2024-09-17T05:02:07.350145+0000","last_undegraded":"2024-09-17T05:02:07.350145+0000","last_fullsized":"2024-09-17T05:02:07.350145+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:22.231177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420872+0000","last_change":"2024-09-17T05:02:06.335589+0000","last_active":"2024-09-17T05:02:07.420872+0000","last_peered":"2024-09-17T05:02:07.420872+0000","last_clean":"2024-09-17T05:02:07.420872+0000","last_became_active":"2024-09-17T05:02:06.335120+0000","last_became_peered":"2024-09-17T05:02:06.335120+0000","last_unstale":"2024-09-17T05:02:07.420872+0000","last_undegraded":"2024-09-17T05:02:07.420872+0000","last_fullsized":"2024-09-17T05:02:07.420872+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:59:06.399066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314944+0000","last_change":"2024-09-17T05:02:06.331500+0000","last_active":"2024-09-17T05:02:07.314944+0000","last_peered":"2024-09-17T05:02:07.314944+0000","last_clean":"2024-09-17T05:02:07.314944+0000","last_became_active":"2024-09-17T05:02:06.331184+0000","last_became_peered":"2024-09-17T05:02:06.331184+0000","last_unstale":"2024-09-17T05:02:07.314944+0000","last_undegraded":"2024-09-17T05:02:07.314944+0000","last_fullsized":"2024-09-17T05:02:07.314944+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:48.958710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314593+0000","last_change":"2024-09-17T05:02:06.330526+0000","last_active":"2024-09-17T05:02:07.314593+0000","last_peered":"2024-09-17T05:02:07.314593+0000","last_clean":"2024-09-17T05:02:07.314593+0000","last_became_active":"2024-09-17T05:02:06.330156+0000","last_became_peered":"2024-09-17T05:02:06.330156+0000","last_unstale":"2024-09-17T05:02:07.314593+0000","last_undegraded":"2024-09-17T05:02:07.314593+0000","last_fullsized":"2024-09-17T05:02:07.314593+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:17.653602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420681+0000","last_change":"2024-09-17T05:02:06.356417+0000","last_active":"2024-09-17T05:02:07.420681+0000","last_peered":"2024-09-17T05:02:07.420681+0000","last_clean":"2024-09-17T05:02:07.420681+0000","last_became_active":"2024-09-17T05:02:06.342863+0000","last_became_peered":"2024-09-17T05:02:06.342863+0000","last_unstale":"2024-09-17T05:02:07.420681+0000","last_undegraded":"2024-09-17T05:02:07.420681+0000","last_fullsized":"2024-09-17T05:02:07.420681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:46.841971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320315+0000","last_change":"2024-09-17T05:02:06.337797+0000","last_active":"2024-09-17T05:02:07.320315+0000","last_peered":"2024-09-17T05:02:07.320315+0000","last_clean":"2024-09-17T05:02:07.320315+0000","last_became_active":"2024-09-17T05:02:06.337396+0000","last_became_peered":"2024-09-17T05:02:06.337396+0000","last_unstale":"2024-09-17T05:02:07.320315+0000","last_undegraded":"2024-09-17T05:02:07.320315+0000","last_fullsized":"2024-09-17T05:02:07.320315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:17:42.616845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315319+0000","last_change":"2024-09-17T05:02:06.348634+0000","last_active":"2024-09-17T05:02:07.315319+0000","last_peered":"2024-09-17T05:02:07.315319+0000","last_clean":"2024-09-17T05:02:07.315319+0000","last_became_active":"2024-09-17T05:02:06.348284+0000","last_became_peered":"2024-09-17T05:02:06.348284+0000","last_unstale":"2024-09-17T05:02:07.315319+0000","last_undegraded":"2024-09-17T05:02:07.315319+0000","last_fullsized":"2024-09-17T05:02:07.315319+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:41.455558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420652+0000","last_change":"2024-09-17T05:02:05.319632+0000","last_active":"2024-09-17T05:02:07.420652+0000","last_peered":"2024-09-17T05:02:07.420652+0000","last_clean":"2024-09-17T05:02:07.420652+0000","last_became_active":"2024-09-17T05:02:05.319321+0000","last_became_peered":"2024-09-17T05:02:05.319321+0000","last_unstale":"2024-09-17T05:02:07.420652+0000","last_undegraded":"2024-09-17T05:02:07.420652+0000","last_fullsized":"2024-09-17T05:02:07.420652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:46.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420445+0000","last_change":"2024-09-17T05:02:06.334112+0000","last_active":"2024-09-17T05:02:07.420445+0000","last_peered":"2024-09-17T05:02:07.420445+0000","last_clean":"2024-09-17T05:02:07.420445+0000","last_became_active":"2024-09-17T05:02:06.333727+0000","last_became_peered":"2024-09-17T05:02:06.333727+0000","last_unstale":"2024-09-17T05:02:07.420445+0000","last_undegraded":"2024-09-17T05:02:07.420445+0000","last_fullsized":"2024-09-17T05:02:07.420445+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:48.515835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420440+0000","last_change":"2024-09-17T05:02:05.330941+0000","last_active":"2024-09-17T05:02:07.420440+0000","last_peered":"2024-09-17T05:02:07.420440+0000","last_clean":"2024-09-17T05:02:07.420440+0000","last_became_active":"2024-09-17T05:02:05.330610+0000","last_became_peered":"2024-09-17T05:02:05.330610+0000","last_unstale":"2024-09-17T05:02:07.420440+0000","last_undegraded":"2024-09-17T05:02:07.420440+0000","last_fullsized":"2024-09-17T05:02:07.420440+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:35.977474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315276+0000","last_change":"2024-09-17T05:02:06.367617+0000","last_active":"2024-09-17T05:02:07.315276+0000","last_peered":"2024-09-17T05:02:07.315276+0000","last_clean":"2024-09-17T05:02:07.315276+0000","last_became_active":"2024-09-17T05:02:06.367102+0000","last_became_peered":"2024-09-17T05:02:06.367102+0000","last_unstale":"2024-09-17T05:02:07.315276+0000","last_undegraded":"2024-09-17T05:02:07.315276+0000","last_fullsized":"2024-09-17T05:02:07.315276+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:19.094882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321076+0000","last_change":"2024-09-17T05:02:05.321020+0000","last_active":"2024-09-17T05:02:07.321076+0000","last_peered":"2024-09-17T05:02:07.321076+0000","last_clean":"2024-09-17T05:02:07.321076+0000","last_became_active":"2024-09-17T05:02:05.320171+0000","last_became_peered":"2024-09-17T05:02:05.320171+0000","last_unstale":"2024-09-17T05:02:07.321076+0000","last_undegraded":"2024-09-17T05:02:07.321076+0000","last_fullsized":"2024-09-17T05:02:07.321076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:00.418844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420627+0000","last_change":"2024-09-17T05:02:06.348099+0000","last_active":"2024-09-17T05:02:07.420627+0000","last_peered":"2024-09-17T05:02:07.420627+0000","last_clean":"2024-09-17T05:02:07.420627+0000","last_became_active":"2024-09-17T05:02:06.347767+0000","last_became_peered":"2024-09-17T05:02:06.347767+0000","last_unstale":"2024-09-17T05:02:07.420627+0000","last_undegraded":"2024-09-17T05:02:07.420627+0000","last_fullsized":"2024-09-17T05:02:07.420627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:25:39.458764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420697+0000","last_change":"2024-09-17T05:02:05.323652+0000","last_active":"2024-09-17T05:02:07.420697+0000","last_peered":"2024-09-17T05:02:07.420697+0000","last_clean":"2024-09-17T05:02:07.420697+0000","last_became_active":"2024-09-17T05:02:05.323250+0000","last_became_peered":"2024-09-17T05:02:05.323250+0000","last_unstale":"2024-09-17T05:02:07.420697+0000","last_undegraded":"2024-09-17T05:02:07.420697+0000","last_fullsized":"2024-09-17T05:02:07.420697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:46.948808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320590+0000","last_change":"2024-09-17T05:02:06.332617+0000","last_active":"2024-09-17T05:02:07.320590+0000","last_peered":"2024-09-17T05:02:07.320590+0000","last_clean":"2024-09-17T05:02:07.320590+0000","last_became_active":"2024-09-17T05:02:06.330747+0000","last_became_peered":"2024-09-17T05:02:06.330747+0000","last_unstale":"2024-09-17T05:02:07.320590+0000","last_undegraded":"2024-09-17T05:02:07.320590+0000","last_fullsized":"2024-09-17T05:02:07.320590+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:30:51.850349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420085+0000","last_change":"2024-09-17T05:02:05.313259+0000","last_active":"2024-09-17T05:02:07.420085+0000","last_peered":"2024-09-17T05:02:07.420085+0000","last_clean":"2024-09-17T05:02:07.420085+0000","last_became_active":"2024-09-17T05:02:05.312893+0000","last_became_peered":"2024-09-17T05:02:05.312893+0000","last_unstale":"2024-09-17T05:02:07.420085+0000","last_undegraded":"2024-09-17T05:02:07.420085+0000","last_fullsized":"2024-09-17T05:02:07.420085+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350174+0000","last_change":"2024-09-17T05:02:06.333044+0000","last_active":"2024-09-17T05:02:07.350174+0000","last_peered":"2024-09-17T05:02:07.350174+0000","last_clean":"2024-09-17T05:02:07.350174+0000","last_became_active":"2024-09-17T05:02:06.332672+0000","last_became_peered":"2024-09-17T05:02:06.332672+0000","last_unstale":"2024-09-17T05:02:07.350174+0000","last_undegraded":"2024-09-17T05:02:07.350174+0000","last_fullsized":"2024-09-17T05:02:07.350174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:28.284248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419821+0000","last_change":"2024-09-17T05:02:05.331066+0000","last_active":"2024-09-17T05:02:07.419821+0000","last_peered":"2024-09-17T05:02:07.419821+0000","last_clean":"2024-09-17T05:02:07.419821+0000","last_became_active":"2024-09-17T05:02:05.330727+0000","last_became_peered":"2024-09-17T05:02:05.330727+0000","last_unstale":"2024-09-17T05:02:07.419821+0000","last_undegraded":"2024-09-17T05:02:07.419821+0000","last_fullsized":"2024-09-17T05:02:07.419821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314674+0000","last_change":"2024-09-17T05:02:06.350254+0000","last_active":"2024-09-17T05:02:07.314674+0000","last_peered":"2024-09-17T05:02:07.314674+0000","last_clean":"2024-09-17T05:02:07.314674+0000","last_became_active":"2024-09-17T05:02:06.349926+0000","last_became_peered":"2024-09-17T05:02:06.349926+0000","last_unstale":"2024-09-17T05:02:07.314674+0000","last_undegraded":"2024-09-17T05:02:07.314674+0000","last_fullsized":"2024-09-17T05:02:07.314674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:20:39.538304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320007+0000","last_change":"2024-09-17T05:02:05.330770+0000","last_active":"2024-09-17T05:02:07.320007+0000","last_peered":"2024-09-17T05:02:07.320007+0000","last_clean":"2024-09-17T05:02:07.320007+0000","last_became_active":"2024-09-17T05:02:05.330452+0000","last_became_peered":"2024-09-17T05:02:05.330452+0000","last_unstale":"2024-09-17T05:02:07.320007+0000","last_undegraded":"2024-09-17T05:02:07.320007+0000","last_fullsized":"2024-09-17T05:02:07.320007+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420577+0000","last_change":"2024-09-17T05:02:06.339321+0000","last_active":"2024-09-17T05:02:07.420577+0000","last_peered":"2024-09-17T05:02:07.420577+0000","last_clean":"2024-09-17T05:02:07.420577+0000","last_became_active":"2024-09-17T05:02:06.339039+0000","last_became_peered":"2024-09-17T05:02:06.339039+0000","last_unstale":"2024-09-17T05:02:07.420577+0000","last_undegraded":"2024-09-17T05:02:07.420577+0000","last_fullsized":"2024-09-17T05:02:07.420577+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420407+0000","last_change":"2024-09-17T05:02:05.334416+0000","last_active":"2024-09-17T05:02:07.420407+0000","last_peered":"2024-09-17T05:02:07.420407+0000","last_clean":"2024-09-17T05:02:07.420407+0000","last_became_active":"2024-09-17T05:02:05.334066+0000","last_became_peered":"2024-09-17T05:02:05.334066+0000","last_unstale":"2024-09-17T05:02:07.420407+0000","last_undegraded":"2024-09-17T05:02:07.420407+0000","last_fullsized":"2024-09-17T05:02:07.420407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:44.441031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321356+0000","last_change":"2024-09-17T05:02:06.348674+0000","last_active":"2024-09-17T05:02:07.321356+0000","last_peered":"2024-09-17T05:02:07.321356+0000","last_clean":"2024-09-17T05:02:07.321356+0000","last_became_active":"2024-09-17T05:02:06.348346+0000","last_became_peered":"2024-09-17T05:02:06.348346+0000","last_unstale":"2024-09-17T05:02:07.321356+0000","last_undegraded":"2024-09-17T05:02:07.321356+0000","last_fullsized":"2024-09-17T05:02:07.321356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:25:39.881805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320775+0000","last_change":"2024-09-17T05:02:05.323997+0000","last_active":"2024-09-17T05:02:07.320775+0000","last_peered":"2024-09-17T05:02:07.320775+0000","last_clean":"2024-09-17T05:02:07.320775+0000","last_became_active":"2024-09-17T05:02:05.323410+0000","last_became_peered":"2024-09-17T05:02:05.323410+0000","last_unstale":"2024-09-17T05:02:07.320775+0000","last_undegraded":"2024-09-17T05:02:07.320775+0000","last_fullsized":"2024-09-17T05:02:07.320775+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:32.508695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420275+0000","last_change":"2024-09-17T05:02:06.338660+0000","last_active":"2024-09-17T05:02:07.420275+0000","last_peered":"2024-09-17T05:02:07.420275+0000","last_clean":"2024-09-17T05:02:07.420275+0000","last_became_active":"2024-09-17T05:02:06.338256+0000","last_became_peered":"2024-09-17T05:02:06.338256+0000","last_unstale":"2024-09-17T05:02:07.420275+0000","last_undegraded":"2024-09-17T05:02:07.420275+0000","last_fullsized":"2024-09-17T05:02:07.420275+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:30.216527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315076+0000","last_change":"2024-09-17T05:02:05.326255+0000","last_active":"2024-09-17T05:02:07.315076+0000","last_peered":"2024-09-17T05:02:07.315076+0000","last_clean":"2024-09-17T05:02:07.315076+0000","last_became_active":"2024-09-17T05:02:05.325945+0000","last_became_peered":"2024-09-17T05:02:05.325945+0000","last_unstale":"2024-09-17T05:02:07.315076+0000","last_undegraded":"2024-09-17T05:02:07.315076+0000","last_fullsized":"2024-09-17T05:02:07.315076+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:21.224964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320746+0000","last_change":"2024-09-17T05:02:06.335521+0000","last_active":"2024-09-17T05:02:07.320746+0000","last_peered":"2024-09-17T05:02:07.320746+0000","last_clean":"2024-09-17T05:02:07.320746+0000","last_became_active":"2024-09-17T05:02:06.335166+0000","last_became_peered":"2024-09-17T05:02:06.335166+0000","last_unstale":"2024-09-17T05:02:07.320746+0000","last_undegraded":"2024-09-17T05:02:07.320746+0000","last_fullsized":"2024-09-17T05:02:07.320746+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:25:02.262578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314664+0000","last_change":"2024-09-17T05:02:05.336301+0000","last_active":"2024-09-17T05:02:07.314664+0000","last_peered":"2024-09-17T05:02:07.314664+0000","last_clean":"2024-09-17T05:02:07.314664+0000","last_became_active":"2024-09-17T05:02:05.331553+0000","last_became_peered":"2024-09-17T05:02:05.331553+0000","last_unstale":"2024-09-17T05:02:07.314664+0000","last_undegraded":"2024-09-17T05:02:07.314664+0000","last_fullsized":"2024-09-17T05:02:07.314664+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:23.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420226+0000","last_change":"2024-09-17T05:02:06.342104+0000","last_active":"2024-09-17T05:02:07.420226+0000","last_peered":"2024-09-17T05:02:07.420226+0000","last_clean":"2024-09-17T05:02:07.420226+0000","last_became_active":"2024-09-17T05:02:06.341775+0000","last_became_peered":"2024-09-17T05:02:06.341775+0000","last_unstale":"2024-09-17T05:02:07.420226+0000","last_undegraded":"2024-09-17T05:02:07.420226+0000","last_fullsized":"2024-09-17T05:02:07.420226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:24.031897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.321552+0000","last_change":"2024-09-17T05:02:05.323758+0000","last_active":"2024-09-17T05:02:07.321552+0000","last_peered":"2024-09-17T05:02:07.321552+0000","last_clean":"2024-09-17T05:02:07.321552+0000","last_became_active":"2024-09-17T05:02:05.323426+0000","last_became_peered":"2024-09-17T05:02:05.323426+0000","last_unstale":"2024-09-17T05:02:07.321552+0000","last_undegraded":"2024-09-17T05:02:07.321552+0000","last_fullsized":"2024-09-17T05:02:07.321552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:29.210315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320848+0000","last_change":"2024-09-17T05:02:06.345826+0000","last_active":"2024-09-17T05:02:07.320848+0000","last_peered":"2024-09-17T05:02:07.320848+0000","last_clean":"2024-09-17T05:02:07.320848+0000","last_became_active":"2024-09-17T05:02:06.345343+0000","last_became_peered":"2024-09-17T05:02:06.345343+0000","last_unstale":"2024-09-17T05:02:07.320848+0000","last_undegraded":"2024-09-17T05:02:07.320848+0000","last_fullsized":"2024-09-17T05:02:07.320848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:12:09.923578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320977+0000","last_change":"2024-09-17T05:02:05.323105+0000","last_active":"2024-09-17T05:02:07.320977+0000","last_peered":"2024-09-17T05:02:07.320977+0000","last_clean":"2024-09-17T05:02:07.320977+0000","last_became_active":"2024-09-17T05:02:05.320585+0000","last_became_peered":"2024-09-17T05:02:05.320585+0000","last_unstale":"2024-09-17T05:02:07.320977+0000","last_undegraded":"2024-09-17T05:02:07.320977+0000","last_fullsized":"2024-09-17T05:02:07.320977+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420267+0000","last_change":"2024-09-17T05:02:06.359915+0000","last_active":"2024-09-17T05:02:07.420267+0000","last_peered":"2024-09-17T05:02:07.420267+0000","last_clean":"2024-09-17T05:02:07.420267+0000","last_became_active":"2024-09-17T05:02:06.359544+0000","last_became_peered":"2024-09-17T05:02:06.359544+0000","last_unstale":"2024-09-17T05:02:07.420267+0000","last_undegraded":"2024-09-17T05:02:07.420267+0000","last_fullsized":"2024-09-17T05:02:07.420267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:53.355713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420061+0000","last_change":"2024-09-17T05:02:05.320829+0000","last_active":"2024-09-17T05:02:07.420061+0000","last_peered":"2024-09-17T05:02:07.420061+0000","last_clean":"2024-09-17T05:02:07.420061+0000","last_became_active":"2024-09-17T05:02:05.320509+0000","last_became_peered":"2024-09-17T05:02:05.320509+0000","last_unstale":"2024-09-17T05:02:07.420061+0000","last_undegraded":"2024-09-17T05:02:07.420061+0000","last_fullsized":"2024-09-17T05:02:07.420061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:36.381090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349878+0000","last_change":"2024-09-17T05:02:06.342348+0000","last_active":"2024-09-17T05:02:07.349878+0000","last_peered":"2024-09-17T05:02:07.349878+0000","last_clean":"2024-09-17T05:02:07.349878+0000","last_became_active":"2024-09-17T05:02:06.342077+0000","last_became_peered":"2024-09-17T05:02:06.342077+0000","last_unstale":"2024-09-17T05:02:07.349878+0000","last_undegraded":"2024-09-17T05:02:07.349878+0000","last_fullsized":"2024-09-17T05:02:07.349878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:24.585972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314795+0000","last_change":"2024-09-17T05:02:05.331453+0000","last_active":"2024-09-17T05:02:07.314795+0000","last_peered":"2024-09-17T05:02:07.314795+0000","last_clean":"2024-09-17T05:02:07.314795+0000","last_became_active":"2024-09-17T05:02:05.330843+0000","last_became_peered":"2024-09-17T05:02:05.330843+0000","last_unstale":"2024-09-17T05:02:07.314795+0000","last_undegraded":"2024-09-17T05:02:07.314795+0000","last_fullsized":"2024-09-17T05:02:07.314795+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:24:53.874382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346719+0000","last_change":"2024-09-17T05:02:06.338645+0000","last_active":"2024-09-17T05:02:07.346719+0000","last_peered":"2024-09-17T05:02:07.346719+0000","last_clean":"2024-09-17T05:02:07.346719+0000","last_became_active":"2024-09-17T05:02:06.338343+0000","last_became_peered":"2024-09-17T05:02:06.338343+0000","last_unstale":"2024-09-17T05:02:07.346719+0000","last_undegraded":"2024-09-17T05:02:07.346719+0000","last_fullsized":"2024-09-17T05:02:07.346719+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:42.943942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419537+0000","last_change":"2024-09-17T05:02:05.318024+0000","last_active":"2024-09-17T05:02:07.419537+0000","last_peered":"2024-09-17T05:02:07.419537+0000","last_clean":"2024-09-17T05:02:07.419537+0000","last_became_active":"2024-09-17T05:02:05.317528+0000","last_became_peered":"2024-09-17T05:02:05.317528+0000","last_unstale":"2024-09-17T05:02:07.419537+0000","last_undegraded":"2024-09-17T05:02:07.419537+0000","last_fullsized":"2024-09-17T05:02:07.419537+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:22.090770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420452+0000","last_change":"2024-09-17T05:02:06.336675+0000","last_active":"2024-09-17T05:02:07.420452+0000","last_peered":"2024-09-17T05:02:07.420452+0000","last_clean":"2024-09-17T05:02:07.420452+0000","last_became_active":"2024-09-17T05:02:06.336377+0000","last_became_peered":"2024-09-17T05:02:06.336377+0000","last_unstale":"2024-09-17T05:02:07.420452+0000","last_undegraded":"2024-09-17T05:02:07.420452+0000","last_fullsized":"2024-09-17T05:02:07.420452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:11.158532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420724+0000","last_change":"2024-09-17T05:02:05.334942+0000","last_active":"2024-09-17T05:02:07.420724+0000","last_peered":"2024-09-17T05:02:07.420724+0000","last_clean":"2024-09-17T05:02:07.420724+0000","last_became_active":"2024-09-17T05:02:05.334624+0000","last_became_peered":"2024-09-17T05:02:05.334624+0000","last_unstale":"2024-09-17T05:02:07.420724+0000","last_undegraded":"2024-09-17T05:02:07.420724+0000","last_fullsized":"2024-09-17T05:02:07.420724+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:14.855709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420380+0000","last_change":"2024-09-17T05:02:06.334939+0000","last_active":"2024-09-17T05:02:07.420380+0000","last_peered":"2024-09-17T05:02:07.420380+0000","last_clean":"2024-09-17T05:02:07.420380+0000","last_became_active":"2024-09-17T05:02:06.334587+0000","last_became_peered":"2024-09-17T05:02:06.334587+0000","last_unstale":"2024-09-17T05:02:07.420380+0000","last_undegraded":"2024-09-17T05:02:07.420380+0000","last_fullsized":"2024-09-17T05:02:07.420380+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:37.387302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315252+0000","last_change":"2024-09-17T05:02:05.326248+0000","last_active":"2024-09-17T05:02:07.315252+0000","last_peered":"2024-09-17T05:02:07.315252+0000","last_clean":"2024-09-17T05:02:07.315252+0000","last_became_active":"2024-09-17T05:02:05.325621+0000","last_became_peered":"2024-09-17T05:02:05.325621+0000","last_unstale":"2024-09-17T05:02:07.315252+0000","last_undegraded":"2024-09-17T05:02:07.315252+0000","last_fullsized":"2024-09-17T05:02:07.315252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:35.920973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420572+0000","last_change":"2024-09-17T05:02:06.332684+0000","last_active":"2024-09-17T05:02:07.420572+0000","last_peered":"2024-09-17T05:02:07.420572+0000","last_clean":"2024-09-17T05:02:07.420572+0000","last_became_active":"2024-09-17T05:02:06.332268+0000","last_became_peered":"2024-09-17T05:02:06.332268+0000","last_unstale":"2024-09-17T05:02:07.420572+0000","last_undegraded":"2024-09-17T05:02:07.420572+0000","last_fullsized":"2024-09-17T05:02:07.420572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:00:36.983687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346854+0000","last_change":"2024-09-17T05:02:05.329129+0000","last_active":"2024-09-17T05:02:07.346854+0000","last_peered":"2024-09-17T05:02:07.346854+0000","last_clean":"2024-09-17T05:02:07.346854+0000","last_became_active":"2024-09-17T05:02:05.328783+0000","last_became_peered":"2024-09-17T05:02:05.328783+0000","last_unstale":"2024-09-17T05:02:07.346854+0000","last_undegraded":"2024-09-17T05:02:07.346854+0000","last_fullsized":"2024-09-17T05:02:07.346854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:38:01.114967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320039+0000","last_change":"2024-09-17T05:02:06.332255+0000","last_active":"2024-09-17T05:02:07.320039+0000","last_peered":"2024-09-17T05:02:07.320039+0000","last_clean":"2024-09-17T05:02:07.320039+0000","last_became_active":"2024-09-17T05:02:06.331914+0000","last_became_peered":"2024-09-17T05:02:06.331914+0000","last_unstale":"2024-09-17T05:02:07.320039+0000","last_undegraded":"2024-09-17T05:02:07.320039+0000","last_fullsized":"2024-09-17T05:02:07.320039+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:47:08.687459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319356+0000","last_change":"2024-09-17T05:02:05.324272+0000","last_active":"2024-09-17T05:02:07.319356+0000","last_peered":"2024-09-17T05:02:07.319356+0000","last_clean":"2024-09-17T05:02:07.319356+0000","last_became_active":"2024-09-17T05:02:05.323884+0000","last_became_peered":"2024-09-17T05:02:05.323884+0000","last_unstale":"2024-09-17T05:02:07.319356+0000","last_undegraded":"2024-09-17T05:02:07.319356+0000","last_fullsized":"2024-09-17T05:02:07.319356+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:39.493914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314537+0000","last_change":"2024-09-17T05:02:06.321153+0000","last_active":"2024-09-17T05:02:07.314537+0000","last_peered":"2024-09-17T05:02:07.314537+0000","last_clean":"2024-09-17T05:02:07.314537+0000","last_became_active":"2024-09-17T05:02:06.320793+0000","last_became_peered":"2024-09-17T05:02:06.320793+0000","last_unstale":"2024-09-17T05:02:07.314537+0000","last_undegraded":"2024-09-17T05:02:07.314537+0000","last_fullsized":"2024-09-17T05:02:07.314537+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:49.889726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319666+0000","last_change":"2024-09-17T05:02:05.321831+0000","last_active":"2024-09-17T05:02:07.319666+0000","last_peered":"2024-09-17T05:02:07.319666+0000","last_clean":"2024-09-17T05:02:07.319666+0000","last_became_active":"2024-09-17T05:02:05.321507+0000","last_became_peered":"2024-09-17T05:02:05.321507+0000","last_unstale":"2024-09-17T05:02:07.319666+0000","last_undegraded":"2024-09-17T05:02:07.319666+0000","last_fullsized":"2024-09-17T05:02:07.319666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:31.096320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319927+0000","last_change":"2024-09-17T05:02:06.362961+0000","last_active":"2024-09-17T05:02:07.319927+0000","last_peered":"2024-09-17T05:02:07.319927+0000","last_clean":"2024-09-17T05:02:07.319927+0000","last_became_active":"2024-09-17T05:02:06.362589+0000","last_became_peered":"2024-09-17T05:02:06.362589+0000","last_unstale":"2024-09-17T05:02:07.319927+0000","last_undegraded":"2024-09-17T05:02:07.319927+0000","last_fullsized":"2024-09-17T05:02:07.319927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:43.235010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420209+0000","last_change":"2024-09-17T05:02:05.334796+0000","last_active":"2024-09-17T05:02:07.420209+0000","last_peered":"2024-09-17T05:02:07.420209+0000","last_clean":"2024-09-17T05:02:07.420209+0000","last_became_active":"2024-09-17T05:02:05.334457+0000","last_became_peered":"2024-09-17T05:02:05.334457+0000","last_unstale":"2024-09-17T05:02:07.420209+0000","last_undegraded":"2024-09-17T05:02:07.420209+0000","last_fullsized":"2024-09-17T05:02:07.420209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:11:50.024608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349461+0000","last_change":"2024-09-17T05:02:06.360661+0000","last_active":"2024-09-17T05:02:07.349461+0000","last_peered":"2024-09-17T05:02:07.349461+0000","last_clean":"2024-09-17T05:02:07.349461+0000","last_became_active":"2024-09-17T05:02:06.360293+0000","last_became_peered":"2024-09-17T05:02:06.360293+0000","last_unstale":"2024-09-17T05:02:07.349461+0000","last_undegraded":"2024-09-17T05:02:07.349461+0000","last_fullsized":"2024-09-17T05:02:07.349461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420548+0000","last_change":"2024-09-17T05:02:05.335089+0000","last_active":"2024-09-17T05:02:07.420548+0000","last_peered":"2024-09-17T05:02:07.420548+0000","last_clean":"2024-09-17T05:02:07.420548+0000","last_became_active":"2024-09-17T05:02:05.334761+0000","last_became_peered":"2024-09-17T05:02:05.334761+0000","last_unstale":"2024-09-17T05:02:07.420548+0000","last_undegraded":"2024-09-17T05:02:07.420548+0000","last_fullsized":"2024-09-17T05:02:07.420548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319697+0000","last_change":"2024-09-17T05:02:06.337072+0000","last_active":"2024-09-17T05:02:07.319697+0000","last_peered":"2024-09-17T05:02:07.319697+0000","last_clean":"2024-09-17T05:02:07.319697+0000","last_became_active":"2024-09-17T05:02:06.335541+0000","last_became_peered":"2024-09-17T05:02:06.335541+0000","last_unstale":"2024-09-17T05:02:07.319697+0000","last_undegraded":"2024-09-17T05:02:07.319697+0000","last_fullsized":"2024-09-17T05:02:07.319697+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:02:33.058079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.419849+0000","last_change":"2024-09-17T05:02:05.317067+0000","last_active":"2024-09-17T05:02:07.419849+0000","last_peered":"2024-09-17T05:02:07.419849+0000","last_clean":"2024-09-17T05:02:07.419849+0000","last_became_active":"2024-09-17T05:02:05.316750+0000","last_became_peered":"2024-09-17T05:02:05.316750+0000","last_unstale":"2024-09-17T05:02:07.419849+0000","last_undegraded":"2024-09-17T05:02:07.419849+0000","last_fullsized":"2024-09-17T05:02:07.419849+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:11.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346952+0000","last_change":"2024-09-17T05:02:06.342012+0000","last_active":"2024-09-17T05:02:07.346952+0000","last_peered":"2024-09-17T05:02:07.346952+0000","last_clean":"2024-09-17T05:02:07.346952+0000","last_became_active":"2024-09-17T05:02:06.341627+0000","last_became_peered":"2024-09-17T05:02:06.341627+0000","last_unstale":"2024-09-17T05:02:07.346952+0000","last_undegraded":"2024-09-17T05:02:07.346952+0000","last_fullsized":"2024-09-17T05:02:07.346952+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:04.620408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315375+0000","last_change":"2024-09-17T05:02:05.331974+0000","last_active":"2024-09-17T05:02:07.315375+0000","last_peered":"2024-09-17T05:02:07.315375+0000","last_clean":"2024-09-17T05:02:07.315375+0000","last_became_active":"2024-09-17T05:02:05.331277+0000","last_became_peered":"2024-09-17T05:02:05.331277+0000","last_unstale":"2024-09-17T05:02:07.315375+0000","last_undegraded":"2024-09-17T05:02:07.315375+0000","last_fullsized":"2024-09-17T05:02:07.315375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:14.008073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.350015+0000","last_change":"2024-09-17T05:02:06.361595+0000","last_active":"2024-09-17T05:02:07.350015+0000","last_peered":"2024-09-17T05:02:07.350015+0000","last_clean":"2024-09-17T05:02:07.350015+0000","last_became_active":"2024-09-17T05:02:06.361219+0000","last_became_peered":"2024-09-17T05:02:06.361219+0000","last_unstale":"2024-09-17T05:02:07.350015+0000","last_undegraded":"2024-09-17T05:02:07.350015+0000","last_fullsized":"2024-09-17T05:02:07.350015+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:47.878683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.348883+0000","last_change":"2024-09-17T05:02:05.334870+0000","last_active":"2024-09-17T05:02:07.348883+0000","last_peered":"2024-09-17T05:02:07.348883+0000","last_clean":"2024-09-17T05:02:07.348883+0000","last_became_active":"2024-09-17T05:02:05.334512+0000","last_became_peered":"2024-09-17T05:02:05.334512+0000","last_unstale":"2024-09-17T05:02:07.348883+0000","last_undegraded":"2024-09-17T05:02:07.348883+0000","last_fullsized":"2024-09-17T05:02:07.348883+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:46.032913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.319479+0000","last_change":"2024-09-17T05:02:06.329065+0000","last_active":"2024-09-17T05:02:07.319479+0000","last_peered":"2024-09-17T05:02:07.319479+0000","last_clean":"2024-09-17T05:02:07.319479+0000","last_became_active":"2024-09-17T05:02:06.328707+0000","last_became_peered":"2024-09-17T05:02:06.328707+0000","last_unstale":"2024-09-17T05:02:07.319479+0000","last_undegraded":"2024-09-17T05:02:07.319479+0000","last_fullsized":"2024-09-17T05:02:07.319479+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:09.684814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346976+0000","last_change":"2024-09-17T05:02:05.325366+0000","last_active":"2024-09-17T05:02:07.346976+0000","last_peered":"2024-09-17T05:02:07.346976+0000","last_clean":"2024-09-17T05:02:07.346976+0000","last_became_active":"2024-09-17T05:02:05.325037+0000","last_became_peered":"2024-09-17T05:02:05.325037+0000","last_unstale":"2024-09-17T05:02:07.346976+0000","last_undegraded":"2024-09-17T05:02:07.346976+0000","last_fullsized":"2024-09-17T05:02:07.346976+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:44:08.225648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.346633+0000","last_change":"2024-09-17T05:02:06.336842+0000","last_active":"2024-09-17T05:02:07.346633+0000","last_peered":"2024-09-17T05:02:07.346633+0000","last_clean":"2024-09-17T05:02:07.346633+0000","last_became_active":"2024-09-17T05:02:06.334045+0000","last_became_peered":"2024-09-17T05:02:06.334045+0000","last_unstale":"2024-09-17T05:02:07.346633+0000","last_undegraded":"2024-09-17T05:02:07.346633+0000","last_fullsized":"2024-09-17T05:02:07.346633+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:30.924123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320355+0000","last_change":"2024-09-17T05:02:05.316008+0000","last_active":"2024-09-17T05:02:07.320355+0000","last_peered":"2024-09-17T05:02:07.320355+0000","last_clean":"2024-09-17T05:02:07.320355+0000","last_became_active":"2024-09-17T05:02:05.315680+0000","last_became_peered":"2024-09-17T05:02:05.315680+0000","last_unstale":"2024-09-17T05:02:07.320355+0000","last_undegraded":"2024-09-17T05:02:07.320355+0000","last_fullsized":"2024-09-17T05:02:07.320355+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:25.314134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420182+0000","last_change":"2024-09-17T05:02:06.319976+0000","last_active":"2024-09-17T05:02:07.420182+0000","last_peered":"2024-09-17T05:02:07.420182+0000","last_clean":"2024-09-17T05:02:07.420182+0000","last_became_active":"2024-09-17T05:02:06.319609+0000","last_became_peered":"2024-09-17T05:02:06.319609+0000","last_unstale":"2024-09-17T05:02:07.420182+0000","last_undegraded":"2024-09-17T05:02:07.420182+0000","last_fullsized":"2024-09-17T05:02:07.420182+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:48.970719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349774+0000","last_change":"2024-09-17T05:02:05.331789+0000","last_active":"2024-09-17T05:02:07.349774+0000","last_peered":"2024-09-17T05:02:07.349774+0000","last_clean":"2024-09-17T05:02:07.349774+0000","last_became_active":"2024-09-17T05:02:05.327820+0000","last_became_peered":"2024-09-17T05:02:05.327820+0000","last_unstale":"2024-09-17T05:02:07.349774+0000","last_undegraded":"2024-09-17T05:02:07.349774+0000","last_fullsized":"2024-09-17T05:02:07.349774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:34:27.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314859+0000","last_change":"2024-09-17T05:02:06.333560+0000","last_active":"2024-09-17T05:02:07.314859+0000","last_peered":"2024-09-17T05:02:07.314859+0000","last_clean":"2024-09-17T05:02:07.314859+0000","last_became_active":"2024-09-17T05:02:06.333210+0000","last_became_peered":"2024-09-17T05:02:06.333210+0000","last_unstale":"2024-09-17T05:02:07.314859+0000","last_undegraded":"2024-09-17T05:02:07.314859+0000","last_fullsized":"2024-09-17T05:02:07.314859+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.884069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.314950+0000","last_change":"2024-09-17T05:02:05.325774+0000","last_active":"2024-09-17T05:02:07.314950+0000","last_peered":"2024-09-17T05:02:07.314950+0000","last_clean":"2024-09-17T05:02:07.314950+0000","last_became_active":"2024-09-17T05:02:05.325459+0000","last_became_peered":"2024-09-17T05:02:05.325459+0000","last_unstale":"2024-09-17T05:02:07.314950+0000","last_undegraded":"2024-09-17T05:02:07.314950+0000","last_fullsized":"2024-09-17T05:02:07.314950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:55.257416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349680+0000","last_change":"2024-09-17T05:02:06.340513+0000","last_active":"2024-09-17T05:02:07.349680+0000","last_peered":"2024-09-17T05:02:07.349680+0000","last_clean":"2024-09-17T05:02:07.349680+0000","last_became_active":"2024-09-17T05:02:06.340154+0000","last_became_peered":"2024-09-17T05:02:06.340154+0000","last_unstale":"2024-09-17T05:02:07.349680+0000","last_undegraded":"2024-09-17T05:02:07.349680+0000","last_fullsized":"2024-09-17T05:02:07.349680+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.895150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.320990+0000","last_change":"2024-09-17T05:02:05.325138+0000","last_active":"2024-09-17T05:02:07.320990+0000","last_peered":"2024-09-17T05:02:07.320990+0000","last_clean":"2024-09-17T05:02:07.320990+0000","last_became_active":"2024-09-17T05:02:05.324679+0000","last_became_peered":"2024-09-17T05:02:05.324679+0000","last_unstale":"2024-09-17T05:02:07.320990+0000","last_undegraded":"2024-09-17T05:02:07.320990+0000","last_fullsized":"2024-09-17T05:02:07.320990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:39.915152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.315287+0000","last_change":"2024-09-17T05:02:06.331069+0000","last_active":"2024-09-17T05:02:07.315287+0000","last_peered":"2024-09-17T05:02:07.315287+0000","last_clean":"2024-09-17T05:02:07.315287+0000","last_became_active":"2024-09-17T05:02:06.330731+0000","last_became_peered":"2024-09-17T05:02:06.330731+0000","last_unstale":"2024-09-17T05:02:07.315287+0000","last_undegraded":"2024-09-17T05:02:07.315287+0000","last_fullsized":"2024-09-17T05:02:07.315287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:05.294183+0000","last_clean_scrub_stamp":"2024-09-17T05:02:05.294183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:01:38.520351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.349709+0000","last_change":"2024-09-17T05:02:05.335344+0000","last_active":"2024-09-17T05:02:07.349709+0000","last_peered":"2024-09-17T05:02:07.349709+0000","last_clean":"2024-09-17T05:02:07.349709+0000","last_became_active":"2024-09-17T05:02:05.334313+0000","last_became_peered":"2024-09-17T05:02:05.334313+0000","last_unstale":"2024-09-17T05:02:07.349709+0000","last_undegraded":"2024-09-17T05:02:07.349709+0000","last_fullsized":"2024-09-17T05:02:07.349709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:02:04.287970+0000","last_clean_scrub_stamp":"2024-09-17T05:02:04.287970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:03.614196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T05:02:07.420491+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:02:07.420491+0000","last_peered":"2024-09-17T05:02:07.420491+0000","last_clean":"2024-09-17T05:02:07.420491+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:02:07.420491+0000","last_undegraded":"2024-09-17T05:02:07.420491+0000","last_fullsized":"2024-09-17T05:02:07.420491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":458752,"data_stored":195441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":197107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":458752,"data_stored":195461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":471040,"data_stored":197565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":785761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":462848,"data_stored":195463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":786321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:02:15.593 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:02:15.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:02:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 -- 172.21.15.16:0/4275187103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b015f690 msgr2=0x7f82b015faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 --2- 172.21.15.16:0/4275187103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b015f690 0x7f82b015faa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c01c850 comp rx=0 tx=0).stop 2024-09-17T05:02:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 -- 172.21.15.16:0/4275187103 shutdown_connections 2024-09-17T05:02:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 --2- 172.21.15.16:0/4275187103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b015f690 0x7f82b015faa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 -- 172.21.15.16:0/4275187103 >> 172.21.15.16:0/4275187103 conn(0x7f82b006f900 msgr2=0x7f82b00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 -- 172.21.15.16:0/4275187103 shutdown_connections 2024-09-17T05:02:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 -- 172.21.15.16:0/4275187103 wait complete. 2024-09-17T05:02:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 Processor -- start 2024-09-17T05:02:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.794+0000 7f82b8200640 1 -- start start 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82b0073370 0x7f82b014bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 0x7f82b014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82b015f690 0x7f82b01500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0169ce0 con 0x7f82b00b2f50 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0169b60 con 0x7f82b015f690 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f82b0169e60 con 0x7f82b0073370 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 0x7f82b014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 0x7f82b014fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48056/0 (socket says 172.21.15.16:48056) 2024-09-17T05:02:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 -- 172.21.15.16:0/824514355 learned_addr learned my addr 172.21.15.16:0/824514355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b6200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82b015f690 0x7f82b01500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 -- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82b0073370 msgr2=0x7f82b014bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b5800640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82b0073370 0x7f82b014bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82b0073370 0x7f82b014bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 -- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82b015f690 msgr2=0x7f82b01500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82b015f690 0x7f82b01500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 -- 172.21.15.16:0/824514355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b014c020 con 0x7f82b00b2f50 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b6200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82b015f690 0x7f82b01500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b5800640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82b0073370 0x7f82b014bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:15.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b4e00640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 0x7f82b014fb70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f829c009850 tx=0x7f829c0a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0b2040 con 0x7f82b00b2f50 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f829c0bd070 con 0x7f82b00b2f50 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c003860 con 0x7f82b00b2f50 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b0165c80 con 0x7f82b00b2f50 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b0166240 con 0x7f82b00b2f50 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.798+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b015faa0 con 0x7f82b00b2f50 2024-09-17T05:02:15.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.802+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f829c003a00 con 0x7f82b00b2f50 2024-09-17T05:02:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.802+0000 7f82ad600640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8290073200 0x7f8290075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.802+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f829c12ebe0 con 0x7f82b00b2f50 2024-09-17T05:02:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.802+0000 7f82b5800640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8290073200 0x7f8290075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.802+0000 7f82b5800640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8290073200 0x7f8290075710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82a4000f70 tx=0x7f82a4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:15.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:15.806+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f829c0164d0 con 0x7f82b00b2f50 2024-09-17T05:02:16.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.054+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f82b015bc50 con 0x7f82b00b2f50 2024-09-17T05:02:16.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82ad600640 1 -- 172.21.15.16:0/824514355 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7f829c0f99d0 con 0x7f82b00b2f50 2024-09-17T05:02:16.059 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:02:16.059 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:02:16.060 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:02:16.060 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:02:16.060 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:02:16.060 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8290073200 msgr2=0x7f8290075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8290073200 0x7f8290075710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f82a4000f70 tx=0x7f82a4009040 comp rx=0 tx=0).stop 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 msgr2=0x7f82b014fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 0x7f82b014fb70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f829c009850 tx=0x7f829c0a5780 comp rx=0 tx=0).stop 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 shutdown_connections 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8290073200 0x7f8290075710 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82b015f690 0x7f82b01500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b00b2f50 0x7f82b014fb70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 --2- 172.21.15.16:0/824514355 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f82b0073370 0x7f82b014bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 >> 172.21.15.16:0/824514355 conn(0x7f82b006f900 msgr2=0x7f82b015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 shutdown_connections 2024-09-17T05:02:16.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.058+0000 7f82b8200640 1 -- 172.21.15.16:0/824514355 wait complete. 2024-09-17T05:02:16.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 -- 172.21.15.16:0/1460540184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015bc20 msgr2=0x7f0f100cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 --2- 172.21.15.16:0/1460540184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015bc20 0x7f0f100cfb40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009bc0 tx=0x7f0f0801cba0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 -- 172.21.15.16:0/1460540184 shutdown_connections 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 --2- 172.21.15.16:0/1460540184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015bc20 0x7f0f100cfb40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 --2- 172.21.15.16:0/1460540184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f10071920 0x7f0f100cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 -- 172.21.15.16:0/1460540184 >> 172.21.15.16:0/1460540184 conn(0x7f0f100d1030 msgr2=0x7f0f100d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:16.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 -- 172.21.15.16:0/1460540184 shutdown_connections 2024-09-17T05:02:16.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.238+0000 7f0f15200640 1 -- 172.21.15.16:0/1460540184 wait complete. 2024-09-17T05:02:16.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 Processor -- start 2024-09-17T05:02:16.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 -- start start 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 0x7f0f1014b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f1015bc20 0x7f0f101445a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f1015a2e0 0x7f0f1015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10168a80 con 0x7f0f1015b810 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f10168900 con 0x7f0f1015a2e0 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f15200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0f10168c00 con 0x7f0f1015bc20 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 0x7f0f1014b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 0x7f0f1014b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48070/0 (socket says 172.21.15.16:48070) 2024-09-17T05:02:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f0f400640 1 -- 172.21.15.16:0/1639582997 learned_addr learned my addr 172.21.15.16:0/1639582997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.242+0000 7f0f06a00640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f1015bc20 0x7f0f101445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0fe00640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f1015a2e0 0x7f0f1015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0f400640 1 -- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f1015bc20 msgr2=0x7f0f101445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0f400640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f1015bc20 0x7f0f101445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0f400640 1 -- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f1015a2e0 msgr2=0x7f0f1015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0f400640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f1015a2e0 0x7f0f1015a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0f400640 1 -- 172.21.15.16:0/1639582997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1015ac30 con 0x7f0f1015b810 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f0f400640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 0x7f0f1014b920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c460 tx=0x7f0f0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00016020 con 0x7f0f1015b810 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f00012940 con 0x7f0f1015b810 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00012d90 con 0x7f0f1015b810 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1014fa40 con 0x7f0f1015b810 2024-09-17T05:02:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f10150000 con 0x7f0f1015b810 2024-09-17T05:02:16.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f100cf600 con 0x7f0f1015b810 2024-09-17T05:02:16.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.246+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0f00019020 con 0x7f0f1015b810 2024-09-17T05:02:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.250+0000 7f0f07e00640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eec073200 0x7f0eec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.250+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0f00094230 con 0x7f0f1015b810 2024-09-17T05:02:16.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.250+0000 7f0f06a00640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eec073200 0x7f0eec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:16.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.250+0000 7f0f06a00640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eec073200 0x7f0eec075710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f10150c20 tx=0x7f0ef80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:16.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.254+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0f00020020 con 0x7f0f1015b810 2024-09-17T05:02:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.462+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f1006f5e0 con 0x7f0f1015b810 2024-09-17T05:02:16.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.462+0000 7f0f07e00640 1 -- 172.21.15.16:0/1639582997 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f0f1015ac30 con 0x7f0f1015b810 2024-09-17T05:02:16.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:02:16.466 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:02:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eec073200 msgr2=0x7f0eec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eec073200 0x7f0eec075710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f10150c20 tx=0x7f0ef80091e0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 msgr2=0x7f0f1014b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 0x7f0f1014b920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c460 tx=0x7f0f0000c920 comp rx=0 tx=0).stop 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 shutdown_connections 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eec073200 0x7f0eec075710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f1015a2e0 0x7f0f1015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f1015bc20 0x7f0f101445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 --2- 172.21.15.16:0/1639582997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f1015b810 0x7f0f1014b920 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:16.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 >> 172.21.15.16:0/1639582997 conn(0x7f0f100d1030 msgr2=0x7f0f10165780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:16.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 shutdown_connections 2024-09-17T05:02:16.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:16.466+0000 7f0f15200640 1 -- 172.21.15.16:0/1639582997 wait complete. 2024-09-17T05:02:16.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:02:05.738259+0000', 'modified': '2024-09-17T05:02:08.749320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/594213684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 594213684}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 594213684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5031, 'qdb_cluster': [5031]} 2024-09-17T05:02:16.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:02:16.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:02:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 -- 172.21.15.16:0/961359429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6600cf850 msgr2=0x7fd6600b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 --2- 172.21.15.16:0/961359429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6600cf850 0x7fd6600b2100 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd658009cb0 tx=0x7fd65801cce0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 -- 172.21.15.16:0/961359429 shutdown_connections 2024-09-17T05:02:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 --2- 172.21.15.16:0/961359429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6600cf850 0x7fd6600b2100 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 --2- 172.21.15.16:0/961359429 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660159f70 0x7fd6600d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 -- 172.21.15.16:0/961359429 >> 172.21.15.16:0/961359429 conn(0x7fd6600b42d0 msgr2=0x7fd6600b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 -- 172.21.15.16:0/961359429 shutdown_connections 2024-09-17T05:02:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.054+0000 7fd666c00640 1 -- 172.21.15.16:0/961359429 wait complete. 2024-09-17T05:02:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 Processor -- start 2024-09-17T05:02:17.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 -- start start 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6600d02d0 0x7fd66006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 0x7fd660151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660152210 0x7fd6601f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd660169e20 con 0x7fd660159f70 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd660169ca0 con 0x7fd6600d02d0 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd660169fa0 con 0x7fd660152210 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 0x7fd660151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 0x7fd660151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60170/0 (socket says 172.21.15.16:60170) 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 -- 172.21.15.16:0/3925518950 learned_addr learned my addr 172.21.15.16:0/3925518950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6600d02d0 0x7fd66006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd664c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660152210 0x7fd6601f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 -- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660152210 msgr2=0x7fd6601f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660152210 0x7fd6601f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 -- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6600d02d0 msgr2=0x7fd66006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6600d02d0 0x7fd66006e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 -- 172.21.15.16:0/3925518950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6601f8300 con 0x7fd660159f70 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd664c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660152210 0x7fd6601f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6600d02d0 0x7fd66006e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd657e00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 0x7fd660151ac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c3f0 tx=0x7fd64800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd65cc00640 1 -- 172.21.15.16:0/3925518950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648017070 con 0x7fd660159f70 2024-09-17T05:02:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd65cc00640 1 -- 172.21.15.16:0/3925518950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6480053c0 con 0x7fd660159f70 2024-09-17T05:02:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd65cc00640 1 -- 172.21.15.16:0/3925518950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648003860 con 0x7fd660159f70 2024-09-17T05:02:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd660152000 con 0x7fd660159f70 2024-09-17T05:02:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.058+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601fc010 con 0x7fd660159f70 2024-09-17T05:02:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.062+0000 7fd65cc00640 1 -- 172.21.15.16:0/3925518950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd64801a070 con 0x7fd660159f70 2024-09-17T05:02:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.062+0000 7fd65cc00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd63c073200 0x7fd63c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.062+0000 7fd65cc00640 1 -- 172.21.15.16:0/3925518950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd6480937f0 con 0x7fd660159f70 2024-09-17T05:02:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.062+0000 7fd66e40e000 1 Processor -- start 2024-09-17T05:02:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.062+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd63c073200 0x7fd63c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.062+0000 7fd66e40e000 1 -- start start 2024-09-17T05:02:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd63c073200 0x7fd63c075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd650005fd0 tx=0x7fd650005dc0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7357950 0x5609b74a8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7340ca0 0x5609b74a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609b737d410 con 0x5609b6f67460 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5609b72bab40 con 0x5609b7357950 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x5609b72a9f70 con 0x5609b7340ca0 2024-09-17T05:02:17.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60174/0 (socket says 172.21.15.16:60174) 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 -- 172.21.15.16:0/3107030574 learned_addr learned my addr 172.21.15.16:0/3107030574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd657e00640 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7357950 0x5609b74a8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd664c00640 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7340ca0 0x5609b74a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 -- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7340ca0 msgr2=0x5609b74a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7340ca0 0x5609b74a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 -- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7357950 msgr2=0x5609b74a8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7357950 0x5609b74a8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 -- 172.21.15.16:0/3107030574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609b72715b0 con 0x5609b6f67460 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd657e00640 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7357950 0x5609b74a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74a86a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd65000eec0 tx=0x7fd650011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f816e8db6febc25 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd64fe00640 1 -- 172.21.15.16:0/3107030574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65001e640 con 0x5609b6f67460 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd64fe00640 1 -- 172.21.15.16:0/3107030574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd650023070 con 0x5609b6f67460 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd64fe00640 1 -- 172.21.15.16:0/3107030574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65001ead0 con 0x5609b6f67460 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- 172.21.15.16:0/3107030574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 msgr2=0x5609b74a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74a86a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd65000eec0 tx=0x7fd650011000 comp rx=0 tx=0).stop 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- 172.21.15.16:0/3107030574 shutdown_connections 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7340ca0 0x5609b74a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7357950 0x5609b74a8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3107030574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74a86a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- 172.21.15.16:0/3107030574 >> 172.21.15.16:0/3107030574 conn(0x5609b742dd90 msgr2=0x5609b745d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- 172.21.15.16:0/3107030574 shutdown_connections 2024-09-17T05:02:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.066+0000 7fd66e40e000 1 -- 172.21.15.16:0/3107030574 wait complete. 2024-09-17T05:02:17.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 -- start start 2024-09-17T05:02:17.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 Processor -- start 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b74afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b74b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609b72445f0 con 0x5609b6f67460 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5609b72add80 con 0x5609b7340ca0 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x5609b7250520 con 0x5609b7357950 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 10 client.0 _ll_register_callbacks cb 0x5609b7254830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60178/0 (socket says 172.21.15.16:60178) 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 -- 172.21.15.16:0/3586884100 learned_addr learned my addr 172.21.15.16:0/3586884100 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:17.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd664c00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b74b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd657e00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b74afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 -- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 msgr2=0x5609b74b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b74b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 -- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 msgr2=0x5609b74afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b74afe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 -- 172.21.15.16:0/3586884100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b7248190 con 0x5609b6f67460 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd657e00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b74afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd65fe00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74af920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd650013040 tx=0x7fd65001ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aebd0b47424b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd64cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd64cc00640 1 -- 172.21.15.16:0/3586884100 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650032610 con 0x5609b6f67460 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609b7242110 con 0x5609b6f67460 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd64cc00640 1 -- 172.21.15.16:0/3586884100 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd6500327b0 con 0x5609b6f67460 2024-09-17T05:02:17.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd64cc00640 1 -- 172.21.15.16:0/3586884100 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd65003a050 con 0x5609b6f67460 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 10 client.5235 fetch_fsmap learned FSMap version 17 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.070+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5609b732ef10 con 0x5609b6f67460 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd64cc00640 1 -- 172.21.15.16:0/3586884100 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5609b7242110 con 0x5609b6f67460 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd64cc00640 1 -- 172.21.15.16:0/3586884100 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fd65003d020 con 0x5609b6f67460 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 10 client.5235 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 10 client.5235 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 20 client.5235 populate_metadata read hostname 'smithi016' 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f4c7d0 0x5609b74b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 4 client.5235 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 --> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609b72bca90 con 0x5609b6f4c7d0 2024-09-17T05:02:17.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd657e00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f4c7d0 0x5609b74b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd657e00640 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f4c7d0 0x5609b74b9d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25a05f6d7131480c server_cookie=450ead2778da915a in_seq=0 out_seq=0 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd64cc00640 10 client.5235 ms_handle_connect on v2:172.21.15.16:6834/594213684 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd64cc00640 1 -- 172.21.15.16:0/3586884100 <== mds.0 v2:172.21.15.16:6834/594213684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609b72bca90 con 0x5609b6f4c7d0 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd64cc00640 10 client.5235 handle_command_reply: tid=0 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 client.5235 shutdown 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 msgr2=0x5609b74af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74af920 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd650013040 tx=0x7fd65001ea80 comp rx=0 tx=0).stop 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 shutdown_connections 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b74b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f4c7d0 0x5609b74b9d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b74afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 --2- 172.21.15.16:0/3586884100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b74af920 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd64cc00640 0 client.5235 ms_handle_reset on v2:172.21.15.16:6834/594213684 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 >> 172.21.15.16:0/3586884100 conn(0x5609b742dd90 msgr2=0x5609b7464440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 shutdown_connections 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 1 -- 172.21.15.16:0/3586884100 wait complete. 2024-09-17T05:02:17.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.074+0000 7fd66e40e000 20 client.5235 trim_cache size 0 max 16384 2024-09-17T05:02:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 Processor -- start 2024-09-17T05:02:17.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 -- start start 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b751a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b751b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6602059e0 con 0x5609b6f67460 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd660204a20 con 0x5609b7340ca0 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd660203d80 con 0x5609b7357950 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b751a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b751a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60186/0 (socket says 172.21.15.16:60186) 2024-09-17T05:02:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.094+0000 7fd65fe00640 1 -- 172.21.15.16:0/4043983076 learned_addr learned my addr 172.21.15.16:0/4043983076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd657e00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd664c00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b751b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd65fe00640 1 -- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 msgr2=0x5609b751b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd65fe00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b751b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd65fe00640 1 -- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 msgr2=0x5609b751ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd65fe00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd65fe00640 1 -- 172.21.15.16:0/4043983076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609b7305db0 con 0x5609b6f67460 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd657e00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd664c00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b751b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd65fe00640 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b751a840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd650005fa0 tx=0x7fd65001e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc81f85dffadba51 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd64f400640 1 -- 172.21.15.16:0/4043983076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650032d50 con 0x5609b6f67460 2024-09-17T05:02:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd64f400640 1 -- 172.21.15.16:0/4043983076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd650023040 con 0x5609b6f67460 2024-09-17T05:02:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd64f400640 1 -- 172.21.15.16:0/4043983076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650032640 con 0x5609b6f67460 2024-09-17T05:02:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 -- 172.21.15.16:0/4043983076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 msgr2=0x5609b751a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b751a840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd650005fa0 tx=0x7fd65001e8e0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 -- 172.21.15.16:0/4043983076 shutdown_connections 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b7357950 0x5609b751b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 --2- 172.21.15.16:0/4043983076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b6f67460 0x5609b751a840 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 -- 172.21.15.16:0/4043983076 >> 172.21.15.16:0/4043983076 conn(0x5609b742dd90 msgr2=0x5609b745ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 -- 172.21.15.16:0/4043983076 shutdown_connections 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 -- 172.21.15.16:0/4043983076 wait complete. 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 -- start start 2024-09-17T05:02:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.098+0000 7fd66e40e000 1 Processor -- start 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b6f67460 0x5609b751e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 0x5609b751f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609b72ad090 con 0x5609b7357950 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5609b732ea80 con 0x5609b7340ca0 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x5609b7275ee0 con 0x5609b6f67460 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 10 client.0 _ll_register_callbacks cb 0x7fd65000edc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 0x5609b751f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 0x5609b751f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60188/0 (socket says 172.21.15.16:60188) 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 -- 172.21.15.16:0/1724438584 learned_addr learned my addr 172.21.15.16:0/1724438584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd65fe00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b6f67460 0x5609b751e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd657e00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 -- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b6f67460 msgr2=0x5609b751e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b6f67460 0x5609b751e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 -- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 msgr2=0x5609b751eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 -- 172.21.15.16:0/1724438584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b7270ab0 con 0x5609b7357950 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd664c00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 0x5609b751f060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd65801a910 tx=0x7fd6580a5cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea84f917d5ed5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 1 -- 172.21.15.16:0/1724438584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6580be020 con 0x5609b7357950 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 1 -- 172.21.15.16:0/1724438584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd658003b80 con 0x5609b7357950 2024-09-17T05:02:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd650023040 con 0x5609b7357950 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 1 -- 172.21.15.16:0/1724438584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fd6580c8050 con 0x5609b7357950 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 10 client.5241 fetch_fsmap learned FSMap version 17 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5609b724f070 con 0x5609b7357950 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 1 -- 172.21.15.16:0/1724438584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd650023040 con 0x5609b7357950 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 1 -- 172.21.15.16:0/1724438584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fd6580cb020 con 0x5609b7357950 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 10 client.5241 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 10 client.5241 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 20 client.5241 populate_metadata read hostname 'smithi016' 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f3a830 0x5609b74db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 4 client.5241 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 --> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] -- command(tid 0: {"prefix": "status"}) -- 0x5609b72b8540 con 0x5609b6f3a830 2024-09-17T05:02:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd65fe00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f3a830 0x5609b74db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd65fe00640 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f3a830 0x5609b74db020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eadbcbcc5cfb3f1 server_cookie=867e4fc10355264a in_seq=0 out_seq=0 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 10 client.5241 ms_handle_connect on v2:172.21.15.16:6834/594213684 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 1 -- 172.21.15.16:0/1724438584 <== mds.0 v2:172.21.15.16:6834/594213684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x5609b72b8540 con 0x5609b6f3a830 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 10 client.5241 handle_command_reply: tid=0 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 client.5241 shutdown 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 msgr2=0x5609b751f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 0x5609b751f060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd65801a910 tx=0x7fd6580a5cf0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 shutdown_connections 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609b7357950 0x5609b751f060 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x5609b6f3a830 0x5609b74db020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5609b7340ca0 0x5609b751eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 --2- 172.21.15.16:0/1724438584 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x5609b6f67460 0x5609b751e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd64cc00640 0 client.5241 ms_handle_reset on v2:172.21.15.16:6834/594213684 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.102+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 >> 172.21.15.16:0/1724438584 conn(0x5609b742dd90 msgr2=0x5609b745d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 shutdown_connections 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd66e40e000 1 -- 172.21.15.16:0/1724438584 wait complete. 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd66e40e000 20 client.5241 trim_cache size 0 max 16384 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5031, 2024-09-17T05:02:17.110 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.772734857, 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 40, 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 40, 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 17.66705357 2024-09-17T05:02:17.111 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd63c073200 msgr2=0x7fd63c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd63c073200 0x7fd63c075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd650005fd0 tx=0x7fd650005dc0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 msgr2=0x7fd660151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 0x7fd660151ac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c3f0 tx=0x7fd64800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 shutdown_connections 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd63c073200 0x7fd63c075710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd660152210 0x7fd6601f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd660159f70 0x7fd660151ac0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 --2- 172.21.15.16:0/3925518950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6600d02d0 0x7fd66006e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 >> 172.21.15.16:0/3925518950 conn(0x7fd6600b42d0 msgr2=0x7fd66015b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 shutdown_connections 2024-09-17T05:02:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.106+0000 7fd666c00640 1 -- 172.21.15.16:0/3925518950 wait complete. 2024-09-17T05:02:17.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:02:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 -- 172.21.15.16:0/4203714952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 msgr2=0x7f860015dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 --2- 172.21.15.16:0/4203714952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 0x7f860015dd70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009840 tx=0x7f85f401c740 comp rx=0 tx=0).stop 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 -- 172.21.15.16:0/4203714952 shutdown_connections 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 --2- 172.21.15.16:0/4203714952 >> v2:172.21.15.104:3300/0 conn(0x7f860015d630 0x7f86000d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 --2- 172.21.15.16:0/4203714952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 0x7f860015dd70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 -- 172.21.15.16:0/4203714952 >> 172.21.15.16:0/4203714952 conn(0x7f86000cf070 msgr2=0x7f86000cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 -- 172.21.15.16:0/4203714952 shutdown_connections 2024-09-17T05:02:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.282+0000 7f8607e00640 1 -- 172.21.15.16:0/4203714952 wait complete. 2024-09-17T05:02:17.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 Processor -- start 2024-09-17T05:02:17.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 -- start start 2024-09-17T05:02:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86000cca50 0x7f86000caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 0x7f8600156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 0x7f86001f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8600169910 con 0x7f860015d630 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8600169790 con 0x7f86000cca50 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8607e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8600169a90 con 0x7f8600160bb0 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 0x7f8600156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 0x7f8600156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60192/0 (socket says 172.21.15.16:60192) 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 -- 172.21.15.16:0/3076917544 learned_addr learned my addr 172.21.15.16:0/3076917544 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8605e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 0x7f86001f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8605400640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86000cca50 0x7f86000caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 -- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 msgr2=0x7f86001f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 0x7f86001f6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 -- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86000cca50 msgr2=0x7f86000caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86000cca50 0x7f86000caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 -- 172.21.15.16:0/3076917544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86000cb470 con 0x7f860015d630 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8605400640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86000cca50 0x7f86000caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.286+0000 7f8604a00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 0x7f8600156770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c3f0 tx=0x7f85ec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec011790 con 0x7f860015d630 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85ec017070 con 0x7f860015d630 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec011ce0 con 0x7f860015d630 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8600157270 con 0x7f860015d630 2024-09-17T05:02:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8600157830 con 0x7f860015d630 2024-09-17T05:02:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86000d06d0 con 0x7f860015d630 2024-09-17T05:02:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f85ec01c460 con 0x7f860015d630 2024-09-17T05:02:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f85fd600640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85e0073230 0x7f85e0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f85ec094170 con 0x7f860015d630 2024-09-17T05:02:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.290+0000 7f8605400640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85e0073230 0x7f85e0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.294+0000 7f8605400640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85e0073230 0x7f85e0075740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85f40af000 tx=0x7f85f401af80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.294+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f85ec001090 con 0x7f860015d630 2024-09-17T05:02:17.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.502+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8600157000 con 0x7f860015d630 2024-09-17T05:02:17.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f85fd600640 1 -- 172.21.15.16:0/3076917544 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f85ec05eff0 con 0x7f860015d630 2024-09-17T05:02:17.507 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:02:17.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85e0073230 msgr2=0x7f85e0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85e0073230 0x7f85e0075740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f85f40af000 tx=0x7f85f401af80 comp rx=0 tx=0).stop 2024-09-17T05:02:17.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 msgr2=0x7f8600156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 0x7f8600156770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c3f0 tx=0x7f85ec00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 shutdown_connections 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f85e0073230 0x7f85e0075740 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8600160bb0 0x7f86001f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f860015d630 0x7f8600156770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 --2- 172.21.15.16:0/3076917544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86000cca50 0x7f86000caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 >> 172.21.15.16:0/3076917544 conn(0x7f86000cf070 msgr2=0x7f860015b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 shutdown_connections 2024-09-17T05:02:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17.506+0000 7f8607e00640 1 -- 172.21.15.16:0/3076917544 wait complete. 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:02:17.529 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:02:17.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:02:17.530 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:02:17.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:17.530 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:02:17.540 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:02:17.540 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:02:17.541 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42355sec preferred_lft 42355sec 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.542 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:02:17.543 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:02:17.544 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:17.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:17.544 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:02:17.604 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:02:17.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:17.607 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:02:17.663 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:02:17.663 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:02:17.663 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:02:17.664 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:02:17.664 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:02:17.664 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:02:17.664 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:02:17.664 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:02:17.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:02:17.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:02:17.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:02:17.664 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:02:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:02:17.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:17.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:17.805 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:02:17.814 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:02:17.814 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:02:17.815 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:02:17.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:17.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:02:17.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:02:17.882 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:02:17.938 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:02:17.939 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:02:17.939 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:02:17.993 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:02:17.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:17.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:02:18.096 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:02:18.096 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:02:18.096 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:02:18.114 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:02:18.114 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:02:18.115 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:02:18.115 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:02:18.115 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:02:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:02:18.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:02:18.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:18.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:02:18.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:02:18.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:02:18.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:18.262 INFO:teuthology.orchestra.run.smithi016.stdout:client5131 2024-09-17T05:02:18.262 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:02:18.262 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:18.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:02:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 -- 172.21.15.16:0/27861903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0cf560 msgr2=0x7f964c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 --2- 172.21.15.16:0/27861903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0cf560 0x7f964c0b3990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9634009900 tx=0x7f963401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 -- 172.21.15.16:0/27861903 shutdown_connections 2024-09-17T05:02:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 --2- 172.21.15.16:0/27861903 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0b3ed0 0x7f964c0b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 --2- 172.21.15.16:0/27861903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c0cf560 0x7f964c0b3990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 -- 172.21.15.16:0/27861903 >> 172.21.15.16:0/27861903 conn(0x7f964c06f900 msgr2=0x7f964c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 -- 172.21.15.16:0/27861903 shutdown_connections 2024-09-17T05:02:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 -- 172.21.15.16:0/27861903 wait complete. 2024-09-17T05:02:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 Processor -- start 2024-09-17T05:02:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.470+0000 7f9653c00640 1 -- start start 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f964c0b3ed0 0x7f964c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0cf560 0x7f964c1328a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 0x7f964c132de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f964c16a100 con 0x7f964c15d600 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f964c169f80 con 0x7f964c0b3ed0 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f964c16a280 con 0x7f964c0cf560 2024-09-17T05:02:18.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 0x7f964c132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 0x7f964c132de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 -- 172.21.15.16:0/1368437885 learned_addr learned my addr 172.21.15.16:0/1368437885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9643e00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0cf560 0x7f964c1328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651200640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f964c0b3ed0 0x7f964c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 -- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0cf560 msgr2=0x7f964c1328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0cf560 0x7f964c1328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 -- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f964c0b3ed0 msgr2=0x7f964c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f964c0b3ed0 0x7f964c0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 -- 172.21.15.16:0/1368437885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c0a9450 con 0x7f964c15d600 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9643e00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0cf560 0x7f964c1328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9651c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 0x7f964c132de0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9644009fd0 tx=0x7f964400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400ce20 con 0x7f964c15d600 2024-09-17T05:02:18.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9644004b30 con 0x7f964c15d600 2024-09-17T05:02:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644005020 con 0x7f964c15d600 2024-09-17T05:02:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c0af960 con 0x7f964c15d600 2024-09-17T05:02:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c163200 con 0x7f964c15d600 2024-09-17T05:02:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.474+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964c0a9620 con 0x7f964c15d600 2024-09-17T05:02:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.478+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96440051c0 con 0x7f964c15d600 2024-09-17T05:02:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.478+0000 7f9649600640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9628073200 0x7f9628075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.478+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f96440950f0 con 0x7f964c15d600 2024-09-17T05:02:18.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.478+0000 7f9651200640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9628073200 0x7f9628075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:18.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.478+0000 7f9651200640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9628073200 0x7f9628075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f963c000f70 tx=0x7f963c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:18.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.482+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f964c0a9620 con 0x7f964c15d600 2024-09-17T05:02:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f964c0b42e0 con 0x7f964c15d600 2024-09-17T05:02:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9649600640 1 -- 172.21.15.16:0/1368437885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f964405ff70 con 0x7f964c15d600 2024-09-17T05:02:18.695 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:02:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9628073200 msgr2=0x7f9628075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9628073200 0x7f9628075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f963c000f70 tx=0x7f963c009040 comp rx=0 tx=0).stop 2024-09-17T05:02:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 msgr2=0x7f964c132de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 0x7f964c132de0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9644009fd0 tx=0x7f964400ef10 comp rx=0 tx=0).stop 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 shutdown_connections 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9628073200 0x7f9628075710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c15d600 0x7f964c132de0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964c0cf560 0x7f964c1328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 --2- 172.21.15.16:0/1368437885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f964c0b3ed0 0x7f964c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 >> 172.21.15.16:0/1368437885 conn(0x7f964c06f900 msgr2=0x7f964c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 shutdown_connections 2024-09-17T05:02:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:18.694+0000 7f9653c00640 1 -- 172.21.15.16:0/1368437885 wait complete. 2024-09-17T05:02:18.717 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:02:18.717 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:02:18.717 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:02:18.718 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:02:18.718 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:02:18.718 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:02:18.718 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:02:18.718 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:02:18.718 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:02:18.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:18.718 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:02:18.728 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:18.729 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:02:18.730 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42349sec preferred_lft 42349sec 2024-09-17T05:02:18.730 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:02:18.730 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.730 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:02:18.730 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:18.730 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:02:18.731 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:02:18.732 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:02:18.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:18.733 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:02:18.796 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:02:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:18.799 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:02:18.856 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:02:18.857 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:02:18.857 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:02:18.857 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:02:18.857 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:02:18.858 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:02:18.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:02:18.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:02:18.858 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:02:18.858 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:02:18.948 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:02:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:02:18.990 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:02:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:18.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:18.998 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:02:19.007 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:02:19.007 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:02:19.008 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:02:19.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:19.008 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:02:19.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:02:19.068 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:02:19.116 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:02:19.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:02:19.117 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:02:19.168 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:02:19.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:19.168 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:02:19.272 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:02:19.273 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:02:19.273 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:02:19.290 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:02:19.290 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:02:19.290 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:02:19.291 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:02:19.291 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:02:19.302 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:02:19.302 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:02:19.302 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:02:19.302 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:02:19.303 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:02:19.303 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:02:19.303 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:02:19.303 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:02:19.306 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:02:19.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:02:19.307 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:02:19.325 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:02:19.415 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:02:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:02:19.419 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:02:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:19.425 INFO:teuthology.orchestra.run.smithi104.stdout:client5256 2024-09-17T05:02:19.425 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:02:19.425 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:19.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2024-09-17T05:02:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 -- 172.21.15.16:0/1174325261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a4159f70 msgr2=0x7f43a40d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 --2- 172.21.15.16:0/1174325261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a4159f70 0x7f43a40d1b80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f43a40ce030 tx=0x7f439401c6b0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 -- 172.21.15.16:0/1174325261 shutdown_connections 2024-09-17T05:02:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 --2- 172.21.15.16:0/1174325261 >> v2:172.21.15.104:3300/0 conn(0x7f43a40cf850 0x7f43a40b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 --2- 172.21.15.16:0/1174325261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a4159f70 0x7f43a40d1b80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 -- 172.21.15.16:0/1174325261 >> 172.21.15.16:0/1174325261 conn(0x7f43a40b42d0 msgr2=0x7f43a40b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 -- 172.21.15.16:0/1174325261 shutdown_connections 2024-09-17T05:02:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 -- 172.21.15.16:0/1174325261 wait complete. 2024-09-17T05:02:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 Processor -- start 2024-09-17T05:02:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.610+0000 7f43aae00640 1 -- start start 2024-09-17T05:02:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 0x7f43a414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a4159f70 0x7f43a414afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f43a414f020 0x7f43a4157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4169e20 con 0x7f43a40d02d0 2024-09-17T05:02:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f43a4169ca0 con 0x7f43a4159f70 2024-09-17T05:02:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f43a4169fa0 con 0x7f43a414f020 2024-09-17T05:02:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 0x7f43a414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 0x7f43a414ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60222/0 (socket says 172.21.15.16:60222) 2024-09-17T05:02:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 -- 172.21.15.16:0/1382159714 learned_addr learned my addr 172.21.15.16:0/1382159714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a8e00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f43a414f020 0x7f43a4157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f439b400640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a4159f70 0x7f43a414afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 -- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f43a414f020 msgr2=0x7f43a4157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f43a414f020 0x7f43a4157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 -- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a4159f70 msgr2=0x7f43a414afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a4159f70 0x7f43a414afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 -- 172.21.15.16:0/1382159714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a40b68c0 con 0x7f43a40d02d0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a3e00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 0x7f43a414ccd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4394009790 tx=0x7f43940a5570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43940b2040 con 0x7f43a40d02d0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43940042c0 con 0x7f43a40d02d0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a414b790 con 0x7f43a40d02d0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394003600 con 0x7f43a40d02d0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a4159190 con 0x7f43a40d02d0 2024-09-17T05:02:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a4158ca0 con 0x7f43a40d02d0 2024-09-17T05:02:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.614+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f43940037a0 con 0x7f43a40d02d0 2024-09-17T05:02:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.618+0000 7f43a0c00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4380073200 0x7f4380075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.618+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f439412f0e0 con 0x7f43a40d02d0 2024-09-17T05:02:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.618+0000 7f439b400640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4380073200 0x7f4380075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.618+0000 7f439b400640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4380073200 0x7f4380075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f43a41fbab0 tx=0x7f438c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.618+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f43940163f0 con 0x7f43a40d02d0 2024-09-17T05:02:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.794+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f43a40d1db0 con 0x7f43a40d02d0 2024-09-17T05:02:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.798+0000 7f43a0c00640 1 -- 172.21.15.16:0/1382159714 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7f43940fa070 con 0x7f43a40d02d0 2024-09-17T05:02:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4380073200 msgr2=0x7f4380075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4380073200 0x7f4380075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f43a41fbab0 tx=0x7f438c009040 comp rx=0 tx=0).stop 2024-09-17T05:02:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 msgr2=0x7f43a414ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 0x7f43a414ccd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4394009790 tx=0x7f43940a5570 comp rx=0 tx=0).stop 2024-09-17T05:02:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 shutdown_connections 2024-09-17T05:02:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4380073200 0x7f4380075710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f43a414f020 0x7f43a4157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43a4159f70 0x7f43a414afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 --2- 172.21.15.16:0/1382159714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a40d02d0 0x7f43a414ccd0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 >> 172.21.15.16:0/1382159714 conn(0x7f43a40b42d0 msgr2=0x7f43a40b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 shutdown_connections 2024-09-17T05:02:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.802+0000 7f43aae00640 1 -- 172.21.15.16:0/1382159714 wait complete. 2024-09-17T05:02:19.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 -- 172.21.15.16:0/3740185566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840cb1d0 msgr2=0x7fa38415e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 --2- 172.21.15.16:0/3740185566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840cb1d0 0x7fa38415e480 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009bc0 tx=0x7fa37c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 -- 172.21.15.16:0/3740185566 shutdown_connections 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 --2- 172.21.15.16:0/3740185566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840cb1d0 0x7fa38415e480 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 --2- 172.21.15.16:0/3740185566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa384074be0 0x7fa3840cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 -- 172.21.15.16:0/3740185566 >> 172.21.15.16:0/3740185566 conn(0x7fa3840b2250 msgr2=0x7fa3840d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 -- 172.21.15.16:0/3740185566 shutdown_connections 2024-09-17T05:02:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.982+0000 7fa38a000640 1 -- 172.21.15.16:0/3740185566 wait complete. 2024-09-17T05:02:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 Processor -- start 2024-09-17T05:02:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 -- start start 2024-09-17T05:02:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa384074be0 0x7fa384159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 0x7fa3840c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3840cb1d0 0x7fa3840b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa38415f180 con 0x7fa3840ca6b0 2024-09-17T05:02:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa384168ef0 con 0x7fa384074be0 2024-09-17T05:02:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa38a000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa3840b3f80 con 0x7fa3840cb1d0 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa37be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 0x7fa3840c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa37be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 0x7fa3840c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60248/0 (socket says 172.21.15.16:60248) 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa37be00640 1 -- 172.21.15.16:0/3434639926 learned_addr learned my addr 172.21.15.16:0/3434639926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa383400640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa384074be0 0x7fa384159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.986+0000 7fa383e00640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3840cb1d0 0x7fa3840b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37be00640 1 -- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3840cb1d0 msgr2=0x7fa3840b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37be00640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3840cb1d0 0x7fa3840b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37be00640 1 -- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa384074be0 msgr2=0x7fa384159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37be00640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa384074be0 0x7fa384159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37be00640 1 -- 172.21.15.16:0/3434639926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38414fff0 con 0x7fa3840ca6b0 2024-09-17T05:02:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa383e00640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3840cb1d0 0x7fa3840b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:02:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37be00640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 0x7fa3840c9e60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00e7c0 tx=0x7fa36c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c00ccc0 con 0x7fa3840ca6b0 2024-09-17T05:02:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa36c00ce80 con 0x7fa3840ca6b0 2024-09-17T05:02:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c005280 con 0x7fa3840ca6b0 2024-09-17T05:02:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa384158670 con 0x7fa3840ca6b0 2024-09-17T05:02:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa384158c30 con 0x7fa3840ca6b0 2024-09-17T05:02:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3840cac90 con 0x7fa3840ca6b0 2024-09-17T05:02:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.990+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa36c005420 con 0x7fa3840ca6b0 2024-09-17T05:02:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.994+0000 7fa37b400640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa360073200 0x7fa360075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:02:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.994+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa36c095120 con 0x7fa3840ca6b0 2024-09-17T05:02:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.994+0000 7fa383400640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa360073200 0x7fa360075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:02:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.994+0000 7fa383400640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa360073200 0x7fa360075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa374007fd0 tx=0x7fa374009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:02:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:19.994+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa36c05f940 con 0x7fa3840ca6b0 2024-09-17T05:02:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.174+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) -- 0x7fa38414fd80 con 0x7fa3840ca6b0 2024-09-17T05:02:20.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.174+0000 7fa37b400640 1 -- 172.21.15.16:0/3434639926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v34)=0 v34) ==== 135+0+0 (secure 0 0 0) 0x7fa36c064370 con 0x7fa3840ca6b0 2024-09-17T05:02:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa360073200 msgr2=0x7fa360075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa360073200 0x7fa360075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa374007fd0 tx=0x7fa374009040 comp rx=0 tx=0).stop 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 msgr2=0x7fa3840c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 0x7fa3840c9e60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00e7c0 tx=0x7fa36c00ec80 comp rx=0 tx=0).stop 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 shutdown_connections 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa360073200 0x7fa360075710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3840cb1d0 0x7fa3840b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3840ca6b0 0x7fa3840c9e60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 --2- 172.21.15.16:0/3434639926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa384074be0 0x7fa384159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 >> 172.21.15.16:0/3434639926 conn(0x7fa3840b2250 msgr2=0x7fa384151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 shutdown_connections 2024-09-17T05:02:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20.178+0000 7fa38a000640 1 -- 172.21.15.16:0/3434639926 wait complete. 2024-09-17T05:02:20.201 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:02:20.201 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:20.210 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.211 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-17T05:02:20.212 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-17T05:02:20.213 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-17T05:02:20.213 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.213 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:20.213 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-17T05:02:20.213 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-17T05:02:20.213 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:02:20.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:20.488 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:02:20.488 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:20.539 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.540 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-17T05:02:20.541 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-17T05:02:20.542 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-17T05:02:20.542 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.542 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:20.542 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-17T05:02:20.542 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-17T05:02:20.542 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:02:20.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:20.819 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:02:20.819 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-17T05:02:20.875 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.876 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-17T05:02:20.877 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:02:21.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:21.138 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:02:21.138 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:21.197 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:02:21.197 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:02:21.197 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> n = 25 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:02:21.198 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:02:21.199 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:02:21.200 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-17T05:02:21.200 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-17T05:02:21.200 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:02:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:21.468 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:02:21.468 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:02:21.518 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:02:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:02:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:02:21.620 INFO:teuthology.orchestra.run.smithi016.stdout:client5131 2024-09-17T05:02:21.620 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'find . | wc') 2024-09-17T05:02:21.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:02:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:02:30.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:02:30.149 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:03:00.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:03:00.212 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:03:00.693 INFO:teuthology.orchestra.run.smithi016.stdout: 105 105 1490 2024-09-17T05:03:00.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:00.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 -- 172.21.15.16:0/3149630289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e00b2f50 msgr2=0x7ff9e015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/3149630289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e00b2f50 0x7ff9e015bb60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff9e006e960 tx=0x7ff9d001c830 comp rx=0 tx=0).stop 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 -- 172.21.15.16:0/3149630289 shutdown_connections 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/3149630289 >> v2:172.21.15.104:3300/0 conn(0x7ff9e015c0a0 0x7ff9e015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/3149630289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e00b2f50 0x7ff9e015bb60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 -- 172.21.15.16:0/3149630289 >> 172.21.15.16:0/3149630289 conn(0x7ff9e006f900 msgr2=0x7ff9e00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 -- 172.21.15.16:0/3149630289 shutdown_connections 2024-09-17T05:03:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 -- 172.21.15.16:0/3149630289 wait complete. 2024-09-17T05:03:00.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 Processor -- start 2024-09-17T05:03:00.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.861+0000 7ff9e5c00640 1 -- start start 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e00b2f50 0x7ff9e01628c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 0x7ff9e01f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9e015f150 0x7ff9e01f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0169090 con 0x7ff9e015c0a0 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0168f10 con 0x7ff9e00b2f50 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e0169210 con 0x7ff9e015f150 2024-09-17T05:03:00.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 0x7ff9e01f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 0x7ff9e01f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36014/0 (socket says 172.21.15.16:36014) 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 -- 172.21.15.16:0/7348849 learned_addr learned my addr 172.21.15.16:0/7348849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9dea00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e00b2f50 0x7ff9e01628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9df400640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9e015f150 0x7ff9e01f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 -- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9e015f150 msgr2=0x7ff9e01f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9e015f150 0x7ff9e01f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 -- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e00b2f50 msgr2=0x7ff9e01628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e00b2f50 0x7ff9e01628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 -- 172.21.15.16:0/7348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e01f6a90 con 0x7ff9e015c0a0 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9dea00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e00b2f50 0x7ff9e01628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9de000640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 0x7ff9e01f7bc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e7f0 tx=0x7ff9cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00ccc0 con 0x7ff9e015c0a0 2024-09-17T05:03:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9cc00ce80 con 0x7ff9e015c0a0 2024-09-17T05:03:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc005280 con 0x7ff9e015c0a0 2024-09-17T05:03:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e015a1f0 con 0x7ff9e015c0a0 2024-09-17T05:03:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e015a7b0 con 0x7ff9e015c0a0 2024-09-17T05:03:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.865+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e015bb60 con 0x7ff9e015c0a0 2024-09-17T05:03:00.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.869+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff9cc005420 con 0x7ff9e015c0a0 2024-09-17T05:03:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.869+0000 7ff9d6a00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9c0073230 0x7ff9c0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.869+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7ff9cc094df0 con 0x7ff9e015c0a0 2024-09-17T05:03:00.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.869+0000 7ff9dea00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9c0073230 0x7ff9c0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.869+0000 7ff9dea00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9c0073230 0x7ff9c0075740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00af000 tx=0x7ff9d001b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:00.873+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff9e01f6a90 con 0x7ff9e015c0a0 2024-09-17T05:03:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.081+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9e00691b0 con 0x7ff9e015c0a0 2024-09-17T05:03:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9d6a00640 1 -- 172.21.15.16:0/7348849 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7ff9cc05fbb0 con 0x7ff9e015c0a0 2024-09-17T05:03:01.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:01.087 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:03:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:03:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9c0073230 msgr2=0x7ff9c0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9c0073230 0x7ff9c0075740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00af000 tx=0x7ff9d001b070 comp rx=0 tx=0).stop 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 msgr2=0x7ff9e01f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 0x7ff9e01f7bc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00e7f0 tx=0x7ff9cc00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 shutdown_connections 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff9c0073230 0x7ff9c0075740 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9e015f150 0x7ff9e01f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9e015c0a0 0x7ff9e01f7bc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 --2- 172.21.15.16:0/7348849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9e00b2f50 0x7ff9e01628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 >> 172.21.15.16:0/7348849 conn(0x7ff9e006f900 msgr2=0x7ff9e006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 shutdown_connections 2024-09-17T05:03:01.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.085+0000 7ff9e5c00640 1 -- 172.21.15.16:0/7348849 wait complete. 2024-09-17T05:03:01.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-17T05:03:01.281 INFO:teuthology.orchestra.run.smithi016.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":127,"msgr_send_messages":129,"msgr_recv_bytes":31709,"msgr_send_bytes":469846,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.017662029,"msgr_running_send_time":0.005294967,"msgr_running_recv_time":0.005350038,"msgr_running_fast_dispatch_time":0.002914071,"msgr_send_messages_queue_lat":{"avgcount":127,"sum":0.119166955,"avgtime":0.000938322},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"AsyncMessenger::Worker-1":{"msgr_recv_messages":404,"msgr_send_messages":488,"msgr_recv_bytes":108273,"msgr_send_bytes":334970,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.059998879,"msgr_running_send_time":0.036188251,"msgr_running_recv_time":0.026796626,"msgr_running_fast_dispatch_time":0.002217788,"msgr_send_messages_queue_lat":{"avgcount":487,"sum":0.030766261,"avgtime":0.000063175},"msgr_handle_ack_lat":{"avgcount":461,"sum":0.000090504,"avgtime":0.000000196},"msgr_recv_encrypted_bytes":2032,"msgr_send_encrypted_bytes":77152},"AsyncMessenger::Worker-2":{"msgr_recv_messages":139,"msgr_send_messages":95,"msgr_recv_bytes":150617,"msgr_send_bytes":40165,"msgr_created_connections":5,"msgr_active_connections":4,"msgr_running_total_time":0.019993235,"msgr_running_send_time":0.007338479,"msgr_running_recv_time":0.014079861,"msgr_running_fast_dispatch_time":0.002868129,"msgr_send_messages_queue_lat":{"avgcount":95,"sum":0.100292453,"avgtime":0.001055710},"msgr_handle_ack_lat":{"avgcount":15,"sum":0.000006502,"avgtime":0.000000433},"msgr_recv_encrypted_bytes":124704,"msgr_send_encrypted_bytes":17520},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":243,"sum":0.070970643,"avgtime":0.000292060}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000037289,"avgtime":0.000018644}},"mds":{"request":148,"reply":148,"reply_latency":{"avgcount":148,"sum":39.107099786,"avgtime":0.264237160},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":102,"caps":103,"subtrees":2,"traverse":188,"traverse_hit":174,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":4,"ceph_cap_op_grant":112,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":104,"handle_client_caps_dirty":100,"handle_client_cap_release":2,"process_request_cap_release":14},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0,"uninline_started":0,"uninline_succeeded":0,"uninline_write_failed":0},"mds_log":{"evlrg":0,"evadd":219,"evex":0,"evtrm":0,"ev":219,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4615097,"rdpos":0,"jlat":{"avgcount":109,"sum":0.207554228,"avgtime":0.001904167},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":115,"dn-":1,"cap":101,"cap+":106,"cap-":5,"rss":120432,"heap":334258176},"mds_server":{"dispatch_client_request":230,"dispatch_server_request":0,"handle_client_request":148,"handle_client_session":106,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.073948480,"avgtime":0.000739484},"req_getattr_latency":{"avgcount":32,"sum":0.006018370,"avgtime":0.000188074},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":4,"sum":0.000513054,"avgtime":0.000128263},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.008409545,"avgtime":0.002102386},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000244593,"avgtime":0.000122296},"req_readdir_latency":{"avgcount":5,"sum":39.017652246,"avgtime":7.803530449},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000313498,"avgtime":0.000313498},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":93,"average_load":46,"avg_session_uptime":40,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":235409,"buffer_anon_items":91,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5616,"osdmap_items":80,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":396416,"mds_co_items":3209,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":153,"op_send_bytes":438830,"op_resend":31,"op_reply":153,"op_latency":{"avgcount":153,"sum":1.031981319,"avgtime":0.006744975},"op_inflight":0,"oplen_avg":{"avgcount":153,"sum":182},"op":153,"op_r":0,"op_w":153,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":109,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":42,"map_full":0,"map_inc":3,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":23,"omap_rd":0,"omap_del":3},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":666,"get_sum":232791,"get_or_fail_fail":0,"get_or_fail_success":666,"take":0,"take_sum":0,"put":666,"put_sum":232791,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":153,"take_sum":438830,"put":153,"put_sum":438830,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":153,"take_sum":153,"put":153,"put_sum":153,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x55b497ab75f0":{"val":0,"max":3758096384,"get_started":0,"get":219,"get_sum":420793,"get_or_fail_fail":0,"get_or_fail_success":219,"take":0,"take_sum":0,"put":109,"put_sum":420793,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"trackedop":{"slow_ops_count":2}} 2024-09-17T05:03:01.283 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 31709, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 127, "msgr_running_fast_dispatch_time": 0.002914071, "msgr_running_recv_time": 0.005350038, "msgr_running_send_time": 0.005294967, "msgr_running_total_time": 0.017662029, "msgr_send_bytes": 469846, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 129, "msgr_send_messages_queue_lat": { "avgcount": 127, "avgtime": 0.000938322, "sum": 0.119166955 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 461, "avgtime": 1.96e-07, "sum": 9.0504e-05 }, "msgr_recv_bytes": 108273, "msgr_recv_encrypted_bytes": 2032, "msgr_recv_messages": 404, "msgr_running_fast_dispatch_time": 0.002217788, "msgr_running_recv_time": 0.026796626, "msgr_running_send_time": 0.036188251, "msgr_running_total_time": 0.059998879, "msgr_send_bytes": 334970, "msgr_send_encrypted_bytes": 77152, "msgr_send_messages": 488, "msgr_send_messages_queue_lat": { "avgcount": 487, "avgtime": 6.3175e-05, "sum": 0.030766261 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 4, "msgr_created_connections": 5, "msgr_handle_ack_lat": { "avgcount": 15, "avgtime": 4.33e-07, "sum": 6.502e-06 }, "msgr_recv_bytes": 150617, "msgr_recv_encrypted_bytes": 124704, "msgr_recv_messages": 139, "msgr_running_fast_dispatch_time": 0.002868129, "msgr_running_recv_time": 0.014079861, "msgr_running_send_time": 0.007338479, "msgr_running_total_time": 0.019993235, "msgr_send_bytes": 40165, "msgr_send_encrypted_bytes": 17520, "msgr_send_messages": 95, "msgr_send_messages_queue_lat": { "avgcount": 95, "avgtime": 0.00105571, "sum": 0.100292453 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 243, "avgtime": 0.00029206, "sum": 0.070970643 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.8644e-05, "sum": 3.7289e-05 }, "queue_len": 0 }, "mds": { "caps": 103, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 112, "ceph_cap_op_revoke": 4, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 2, "handle_client_caps": 104, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 102, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 14, "q": 0, "reply": 148, "reply_latency": { "avgcount": 148, "avgtime": 0.26423716, "sum": 39.107099786 }, "request": 148, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 188, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 174, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0, "uninline_started": 0, "uninline_succeeded": 0, "uninline_write_failed": 0 }, "mds_log": { "ev": 219, "evadd": 219, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 109, "avgtime": 0.001904167, "sum": 0.207554228 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4615097 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 115, "dn-": 1, "heap": 334258176, "ino": 117, "ino+": 117, "ino-": 0, "rss": 120432 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 230, "dispatch_server_request": 0, "handle_client_request": 148, "handle_client_session": 106, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.000739484, "sum": 0.07394848 }, "req_getattr_latency": { "avgcount": 32, "avgtime": 0.000188074, "sum": 0.00601837 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 4, "avgtime": 0.000128263, "sum": 0.000513054 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.002102386, "sum": 0.008409545 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000122296, "sum": 0.000244593 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.803530449, "sum": 39.017652246 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000313498, "sum": 0.000313498 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 46, "avg_session_uptime": 40, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 93 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 235409, "buffer_anon_items": 91, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 396416, "mds_co_items": 3209, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5616, "osdmap_items": 80, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 42, "map_full": 0, "map_inc": 3, "omap_del": 3, "omap_rd": 0, "omap_wr": 23, "op": 153, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 153, "avgtime": 0.006744975, "sum": 1.031981319 }, "op_pg": 0, "op_r": 0, "op_reply": 153, "op_resend": 31, "op_rmw": 0, "op_send": 153, "op_send_bytes": 438830, "op_w": 153, "oplen_avg": { "avgcount": 153, "sum": 182 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 109, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 666, "get_or_fail_fail": 0, "get_or_fail_success": 666, "get_started": 0, "get_sum": 232791, "max": 104857600, "put": 666, "put_sum": 232791, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 153, "put_sum": 438830, "take": 153, "take_sum": 438830, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 153, "put_sum": 153, "take": 153, "take_sum": 153, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x55b497ab75f0": { "get": 219, "get_or_fail_fail": 0, "get_or_fail_success": 219, "get_started": 0, "get_sum": 420793, "max": 3758096384, "put": 109, "put_sum": 420793, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "trackedop": { "slow_ops_count": 2 } } 2024-09-17T05:03:01.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f5810800640 1 -- 172.21.15.16:0/2849715124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0003fa0 msgr2=0x7f57f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f5810800640 1 --2- 172.21.15.16:0/2849715124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57f0003fa0 0x7f57f0004810 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57f8001770 tx=0x7f57f801c860 comp rx=0 tx=0).stop 2024-09-17T05:03:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f580e800640 1 -- 172.21.15.16:0/2849715124 reap_dead start 2024-09-17T05:03:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f5810800640 1 -- 172.21.15.16:0/2849715124 shutdown_connections 2024-09-17T05:03:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f5810800640 1 -- 172.21.15.16:0/2849715124 >> 172.21.15.16:0/2849715124 conn(0x7f580806f900 msgr2=0x7f58080b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f5810800640 1 -- 172.21.15.16:0/2849715124 shutdown_connections 2024-09-17T05:03:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.441+0000 7f5810800640 1 -- 172.21.15.16:0/2849715124 wait complete. 2024-09-17T05:03:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 Processor -- start 2024-09-17T05:03:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 -- start start 2024-09-17T05:03:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57f0003fa0 0x7f5808140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 0x7f580815de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58081f80d0 0x7f580815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5808167670 con 0x7f58081f7cc0 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58081674f0 con 0x7f58081f80d0 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f5810800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f58081677f0 con 0x7f57f0003fa0 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 0x7f580815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 0x7f580815de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36030/0 (socket says 172.21.15.16:36030) 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 -- 172.21.15.16:0/359194780 learned_addr learned my addr 172.21.15.16:0/359194780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580de00640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57f0003fa0 0x7f5808140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580e800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58081f80d0 0x7f580815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 -- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57f0003fa0 msgr2=0x7f5808140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57f0003fa0 0x7f5808140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 -- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58081f80d0 msgr2=0x7f580815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58081f80d0 0x7f580815e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 -- 172.21.15.16:0/359194780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58081411a0 con 0x7f58081f7cc0 2024-09-17T05:03:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.445+0000 7f580d400640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 0x7f580815de40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57f801c690 tx=0x7f57f8006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a5660 con 0x7f58081f7cc0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57f80a58f0 con 0x7f58081f7cc0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f80a5d40 con 0x7f58081f7cc0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58081fca40 con 0x7f58081f7cc0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58081fc110 con 0x7f58081f7cc0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580815d040 con 0x7f58081f7cc0 2024-09-17T05:03:01.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57f80ae430 con 0x7f58081f7cc0 2024-09-17T05:03:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5802000640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f57ec073200 0x7f57ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f57f812d140 con 0x7f58081f7cc0 2024-09-17T05:03:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.449+0000 7f580de00640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f57ec073200 0x7f57ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.453+0000 7f580de00640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f57ec073200 0x7f57ec075710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f57fc001250 tx=0x7f57fc006730 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.453+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f57f80eadb0 con 0x7f58081f7cc0 2024-09-17T05:03:01.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f580815bae0 con 0x7f58081f7cc0 2024-09-17T05:03:01.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5802000640 1 -- 172.21.15.16:0/359194780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f58081411a0 con 0x7f58081f7cc0 2024-09-17T05:03:01.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:01.672 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:03:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f57ec073200 msgr2=0x7f57ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f57ec073200 0x7f57ec075710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f57fc001250 tx=0x7f57fc006730 comp rx=0 tx=0).stop 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 msgr2=0x7f580815de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 0x7f580815de40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57f801c690 tx=0x7f57f8006620 comp rx=0 tx=0).stop 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 shutdown_connections 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f57ec073200 0x7f57ec075710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58081f80d0 0x7f580815e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58081f7cc0 0x7f580815de40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 --2- 172.21.15.16:0/359194780 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57f0003fa0 0x7f5808140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 >> 172.21.15.16:0/359194780 conn(0x7f580806f900 msgr2=0x7f580815d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 shutdown_connections 2024-09-17T05:03:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:01.669+0000 7f5810800640 1 -- 172.21.15.16:0/359194780 wait complete. 2024-09-17T05:03:01.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T05:03:01.868 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":5256,"entity":{"name":{"type":"client","num":5256},"addr":{"type":"v1","addr":"172.21.15.104:0","nonce":2659414151}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":2,"uptime":42.555508938000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":8.1829008076521976,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5256 v1:172.21.15.104:0/2659414151","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":5131,"entity":{"name":{"type":"client","num":5131},"addr":{"type":"v1","addr":"172.21.15.16:0","nonce":753558643}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":89,"uptime":43.731490491999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":10.927322887042067,"halflife":60},"release_caps":{"value":5.2845496390244566,"halflife":60},"recall_caps_throttle":{"value":1.6110517058652696,"halflife":1.5},"recall_caps_throttle2o":{"value":1.0453661809687547,"halflife":0.5},"session_cache_liveness":{"value":196.66461443070912,"halflife":300},"cap_acquisition":{"value":60.151260906974777,"halflife":30},"last_trim_completed_requests_tid":144,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5131 v1:172.21.15.16:0/753558643","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:03:01.869 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.104:0", "nonce": 2659414151, "type": "v1" }, "name": { "num": 5256, "type": "client" } }, "id": 5256, "inst": "client.5256 v1:172.21.15.104:0/2659414151", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 8.182900807652198 }, "state": "open", "uptime": 42.555508938 }, { "cap_acquisition": { "halflife": 30, "value": 60.15126090697478 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 753558643, "type": "v1" }, "name": { "num": 5131, "type": "client" } }, "id": 5131, "inst": "client.5131 v1:172.21.15.16:0/753558643", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 144, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 10.927322887042067 }, "recall_caps_throttle": { "halflife": 1.5, "value": 1.6110517058652696 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 1.0453661809687547 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 5.284549639024457 }, "request_load_avg": 89, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 196.66461443070912 }, "state": "open", "uptime": 43.731490492 } ] 2024-09-17T05:03:01.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 -- 172.21.15.16:0/1827164652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 msgr2=0x7f5ff0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 --2- 172.21.15.16:0/1827164652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f5ff0005160 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6004009eb0 tx=0x7f600401d5d0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6011a00640 1 -- 172.21.15.16:0/1827164652 reap_dead start 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 -- 172.21.15.16:0/1827164652 shutdown_connections 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 -- 172.21.15.16:0/1827164652 >> 172.21.15.16:0/1827164652 conn(0x7f600c0b2250 msgr2=0x7f600c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 -- 172.21.15.16:0/1827164652 shutdown_connections 2024-09-17T05:03:02.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 -- 172.21.15.16:0/1827164652 wait complete. 2024-09-17T05:03:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 Processor -- start 2024-09-17T05:03:02.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.029+0000 7f6013a00640 1 -- start start 2024-09-17T05:03:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f600c1e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c1e35b0 0x7f600c1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c0b4f60 0x7f600c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0746d0 con 0x7f5ff0004d50 2024-09-17T05:03:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f600c074550 con 0x7f600c0b4f60 2024-09-17T05:03:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f600c074850 con 0x7f600c1e35b0 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f600c1e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f600c1e68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36048/0 (socket says 172.21.15.16:36048) 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 -- 172.21.15.16:0/2815393589 learned_addr learned my addr 172.21.15.16:0/2815393589 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6003e00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c1e35b0 0x7f600c1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 -- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c1e35b0 msgr2=0x7f600c1f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c0b4f60 0x7f600c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c1e35b0 0x7f600c1f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 -- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c0b4f60 msgr2=0x7f600c0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c0b4f60 0x7f600c0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 -- 172.21.15.16:0/2815393589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c0b58b0 con 0x7f5ff0004d50 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6003e00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c1e35b0 0x7f600c1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c0b4f60 0x7f600c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:02.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6011000640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f600c1e68a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc001870 tx=0x7f5ffc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6009600640 1 -- 172.21.15.16:0/2815393589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc015070 con 0x7f5ff0004d50 2024-09-17T05:03:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6009600640 1 -- 172.21.15.16:0/2815393589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ffc011900 con 0x7f5ff0004d50 2024-09-17T05:03:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c1e7280 con 0x7f5ff0004d50 2024-09-17T05:03:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6009600640 1 -- 172.21.15.16:0/2815393589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc011d50 con 0x7f5ff0004d50 2024-09-17T05:03:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.033+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c1e2db0 con 0x7f5ff0004d50 2024-09-17T05:03:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f6009600640 1 -- 172.21.15.16:0/2815393589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ffc010420 con 0x7f5ff0004d50 2024-09-17T05:03:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f6009600640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fe8073200 0x7f5fe8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f6009600640 1 -- 172.21.15.16:0/2815393589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f5ffc095110 con 0x7f5ff0004d50 2024-09-17T05:03:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f6003e00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fe8073200 0x7f5fe8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 Processor -- start 2024-09-17T05:03:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 -- start start 2024-09-17T05:03:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f6003e00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fe8073200 0x7f5fe8075710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f600c1e6110 tx=0x7f5ff40091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f470e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f470e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f470ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5f451fb90 con 0x55e5f459b790 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5f450efc0 con 0x55e5f45bd320 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.037+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55e5f4512dd0 con 0x55e5f45ae7a0 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f470ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f470ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36050/0 (socket says 172.21.15.16:36050) 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 -- 172.21.15.16:0/1342236225 learned_addr learned my addr 172.21.15.16:0/1342236225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6003e00640 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f470e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011000640 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f470e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 -- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 msgr2=0x55e5f470e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f470e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 -- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 msgr2=0x55e5f470e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f470e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 -- 172.21.15.16:0/1342236225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f44b32c0 con 0x55e5f459b790 2024-09-17T05:03:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6011a00640 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f470ead0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6004009e00 tx=0x7f5ff0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1571f14b844411 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6001600640 1 -- 172.21.15.16:0/1342236225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004009200 con 0x55e5f459b790 2024-09-17T05:03:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6001600640 1 -- 172.21.15.16:0/1342236225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6004004a50 con 0x55e5f459b790 2024-09-17T05:03:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f6001600640 1 -- 172.21.15.16:0/1342236225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004004e00 con 0x55e5f459b790 2024-09-17T05:03:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 -- 172.21.15.16:0/1342236225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 msgr2=0x55e5f470ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f470ead0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6004009e00 tx=0x7f5ff0005840 comp rx=0 tx=0).stop 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 -- 172.21.15.16:0/1342236225 shutdown_connections 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f470ead0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f470e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 --2- 172.21.15.16:0/1342236225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f470e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 -- 172.21.15.16:0/1342236225 >> 172.21.15.16:0/1342236225 conn(0x55e5f41cbcd0 msgr2=0x55e5f46f1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 -- 172.21.15.16:0/1342236225 shutdown_connections 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 -- 172.21.15.16:0/1342236225 wait complete. 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 -- start start 2024-09-17T05:03:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.041+0000 7f601afc0000 1 Processor -- start 2024-09-17T05:03:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f46c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f46c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f46c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5f44ba070 con 0x55e5f459b790 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5f44da8b0 con 0x55e5f45bd320 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55e5f451e960 con 0x55e5f45ae7a0 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 10 client.0 _ll_register_callbacks cb 0x55e5f46afc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f46c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f46c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36056/0 (socket says 172.21.15.16:36056) 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 -- 172.21.15.16:0/1496617628 learned_addr learned my addr 172.21.15.16:0/1496617628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6003e00640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f46c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011a00640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f46c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 -- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 msgr2=0x55e5f46c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f46c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 -- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 msgr2=0x55e5f46c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f46c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 -- 172.21.15.16:0/1496617628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f44ad1f0 con 0x55e5f459b790 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011a00640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f46c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6003e00640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f46c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6011000640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f46c5340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc05b870 tx=0x7f5ffc05fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572d81387c48a3da server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f5ffa000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f5ffa000640 1 -- 172.21.15.16:0/1496617628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc020040 con 0x55e5f459b790 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f5ffa000640 1 -- 172.21.15.16:0/1496617628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f5ffc05b1b0 con 0x55e5f459b790 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5f44a7170 con 0x55e5f459b790 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f5ffa000640 1 -- 172.21.15.16:0/1496617628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5ffc05f860 con 0x55e5f459b790 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 10 client.5298 fetch_fsmap learned FSMap version 17 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55e5f45949f0 con 0x55e5f459b790 2024-09-17T05:03:02.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f5ffa000640 1 -- 172.21.15.16:0/1496617628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e5f44a7170 con 0x55e5f459b790 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f5ffa000640 1 -- 172.21.15.16:0/1496617628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5ffc05b3b0 con 0x55e5f459b790 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 10 client.5298 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 10 client.5298 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 20 client.5298 populate_metadata read hostname 'smithi016' 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41b18f0 0x55e5f471f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 4 client.5298 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 --> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f4521ae0 con 0x55e5f41b18f0 2024-09-17T05:03:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.045+0000 7f6003e00640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41b18f0 0x55e5f471f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f5ffa000640 10 client.5298 ms_handle_connect on v2:172.21.15.16:6834/594213684 2024-09-17T05:03:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f6003e00640 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41b18f0 0x55e5f471f610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea5a6fc2056e22c server_cookie=33763fa062be3dfb in_seq=0 out_seq=0 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f5ffa000640 1 -- 172.21.15.16:0/1496617628 <== mds.0 v2:172.21.15.16:6834/594213684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5f4521ae0 con 0x55e5f41b18f0 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f5ffa000640 10 client.5298 handle_command_reply: tid=0 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 client.5298 shutdown 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 msgr2=0x55e5f46c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f46c5340 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc05b870 tx=0x7f5ffc05fe00 comp rx=0 tx=0).stop 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 shutdown_connections 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41b18f0 0x55e5f471f610 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45bd320 0x55e5f46c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f45ae7a0 0x55e5f46c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f5ffa000640 0 client.5298 ms_handle_reset on v2:172.21.15.16:6834/594213684 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 --2- 172.21.15.16:0/1496617628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f459b790 0x55e5f46c5340 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 >> 172.21.15.16:0/1496617628 conn(0x55e5f41cbcd0 msgr2=0x55e5f46f2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 shutdown_connections 2024-09-17T05:03:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 1 -- 172.21.15.16:0/1496617628 wait complete. 2024-09-17T05:03:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.049+0000 7f601afc0000 20 client.5298 trim_cache size 0 max 16384 2024-09-17T05:03:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 Processor -- start 2024-09-17T05:03:02.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 -- start start 2024-09-17T05:03:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f41cbcd0 0x55e5f477f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f477fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 0x55e5f477ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1ff960 con 0x55e5f45ae7a0 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f600c1fed90 con 0x55e5f41cbcd0 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f600c1ff310 con 0x55e5f459b790 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 0x55e5f477ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 0x55e5f477ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36068/0 (socket says 172.21.15.16:36068) 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 -- 172.21.15.16:0/1066216479 learned_addr learned my addr 172.21.15.16:0/1066216479 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6003e00640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f477fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011000640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f41cbcd0 0x55e5f477f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 -- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 msgr2=0x55e5f477fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f477fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 -- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f41cbcd0 msgr2=0x55e5f477f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f41cbcd0 0x55e5f477f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 -- 172.21.15.16:0/1066216479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f456bcb0 con 0x55e5f45ae7a0 2024-09-17T05:03:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6003e00640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f477fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011000640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f41cbcd0 0x55e5f477f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6011a00640 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 0x55e5f477ffd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f600401d040 tx=0x7f600401d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afab1a4255e8601b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6000c00640 1 -- 172.21.15.16:0/1066216479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040c0070 con 0x55e5f45ae7a0 2024-09-17T05:03:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6000c00640 1 -- 172.21.15.16:0/1066216479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6004004d80 con 0x55e5f45ae7a0 2024-09-17T05:03:02.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.065+0000 7f6000c00640 1 -- 172.21.15.16:0/1066216479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004003be0 con 0x55e5f45ae7a0 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- 172.21.15.16:0/1066216479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 msgr2=0x55e5f477ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 0x55e5f477ffd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f600401d040 tx=0x7f600401d5d0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- 172.21.15.16:0/1066216479 shutdown_connections 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f45ae7a0 0x55e5f477ffd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f477fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- 172.21.15.16:0/1066216479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f41cbcd0 0x55e5f477f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- 172.21.15.16:0/1066216479 >> 172.21.15.16:0/1066216479 conn(0x55e5f471aef0 msgr2=0x55e5f46f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- 172.21.15.16:0/1066216479 shutdown_connections 2024-09-17T05:03:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- 172.21.15.16:0/1066216479 wait complete. 2024-09-17T05:03:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- start start 2024-09-17T05:03:02.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 Processor -- start 2024-09-17T05:03:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 0x55e5f47231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f4723730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45ae7a0 0x55e5f473fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5f45120e0 con 0x55e5f41cbcd0 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5f44d16b0 con 0x55e5f45ae7a0 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55e5f4517400 con 0x55e5f459b790 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f601afc0000 10 client.0 _ll_register_callbacks cb 0x7f5ffc05f1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 0x55e5f47231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 0x55e5f47231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36070/0 (socket says 172.21.15.16:36070) 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 -- 172.21.15.16:0/3507352415 learned_addr learned my addr 172.21.15.16:0/3507352415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6003e00640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f4723730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011a00640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45ae7a0 0x55e5f473fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 -- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 msgr2=0x55e5f4723730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f4723730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 -- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45ae7a0 msgr2=0x55e5f473fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45ae7a0 0x55e5f473fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.069+0000 7f6011000640 1 -- 172.21.15.16:0/3507352415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f44d5990 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f6011000640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 0x55e5f47231f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc05fa20 tx=0x7f5ffc05ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0e661bfa6a3686 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 1 -- 172.21.15.16:0/3507352415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc020070 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 1 -- 172.21.15.16:0/3507352415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f5ffc068730 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6004004d80 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 1 -- 172.21.15.16:0/3507352415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5ffc070050 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 10 client.5304 fetch_fsmap learned FSMap version 17 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55e5f44b41e0 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 1 -- 172.21.15.16:0/3507352415 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6004004d80 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 1 -- 172.21.15.16:0/3507352415 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f5ffc073020 con 0x55e5f41cbcd0 2024-09-17T05:03:02.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 10 client.5304 fetch_fsmap finished waiting for FSMap version 17 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 10 client.5304 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 20 client.5304 populate_metadata read hostname 'smithi016' 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41cafa0 0x55e5f4740580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 4 client.5304 mds_command: new command op to 5031 tid=0 multi_id=0 [{"prefix": "dump_historic_ops"}] 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 --> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] -- command(tid 0: {"prefix": "dump_historic_ops"}) -- 0x55e5f451d590 con 0x55e5f41cafa0 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f6003e00640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41cafa0 0x55e5f4740580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f6003e00640 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41cafa0 0x55e5f4740580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef707d324698cfd7 server_cookie=5517a2bbb11da2e in_seq=0 out_seq=0 2024-09-17T05:03:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 10 client.5304 ms_handle_connect on v2:172.21.15.16:6834/594213684 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 1 -- 172.21.15.16:0/3507352415 <== mds.0 v2:172.21.15.16:6834/594213684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+71652 (crc 0 0 0) 0x55e5f451d590 con 0x55e5f41cafa0 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 10 client.5304 handle_command_reply: tid=0 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 client.5304 shutdown 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 msgr2=0x55e5f47231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 0x55e5f47231f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc05fa20 tx=0x7f5ffc05ef00 comp rx=0 tx=0).stop 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 shutdown_connections 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:6834/594213684,v1:172.21.15.16:6835/594213684] conn(0x55e5f41cafa0 0x55e5f4740580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5f45ae7a0 0x55e5f473fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55e5f459b790 0x55e5f4723730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 --2- 172.21.15.16:0/3507352415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5f41cbcd0 0x55e5f47231f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f5ffa000640 0 client.5304 ms_handle_reset on v2:172.21.15.16:6834/594213684 2024-09-17T05:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 >> 172.21.15.16:0/3507352415 conn(0x55e5f471aef0 msgr2=0x55e5f46c0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 shutdown_connections 2024-09-17T05:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 1 -- 172.21.15.16:0/3507352415 wait complete. 2024-09-17T05:03:02.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.073+0000 7f601afc0000 20 client.5304 trim_cache size 0 max 16384 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 20, 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 600, 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "ops": [ 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:2 setattr mode=041777 #0x1 2024-09-17T05:02:18.146272+0000 caller_uid=0, caller_gid=0{0,})", 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:18.150744+0000", 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 43.926715491000003, 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 1.149971963, 2024-09-17T05:03:02.082 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 2 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.083 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.150744+0000", 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.150744+0000", 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.150746+0000", 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.150758+0000", 2024-09-17T05:03:02.084 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.150817+0000", 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.150983+0000", 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.151063+0000", 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:18.151064+0000", 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:19.300551+0000", 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:19.300602+0000", 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.086 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:19.300695+0000", 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:19.300715+0000", 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:19.300716+0000", 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.087 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:27 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_0 2024-09-17T05:02:20.334238+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.336794+0000", 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.740666015000002, 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0099471230000000004, 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.088 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 27 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.336794+0000", 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.089 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.336794+0000", 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.336794+0000", 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.336800+0000", 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.336821+0000", 2024-09-17T05:03:02.090 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.337002+0000", 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.337289+0000", 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.337289+0000", 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.346422+0000", 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.346576+0000", 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.346660+0000", 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.346740+0000", 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.346741+0000", 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:32 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_5 2024-09-17T05:02:20.366237+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.369290+0000", 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.708169828999999, 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0062774759999999997, 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 32 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369290+0000", 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369290+0000", 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369290+0000", 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369296+0000", 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369321+0000", 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369512+0000", 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369866+0000", 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.369867+0000", 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.375255+0000", 2024-09-17T05:03:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.375398+0000", 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.375486+0000", 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.375567+0000", 2024-09-17T05:03:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.375567+0000", 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:47 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_20 2024-09-17T05:02:20.446236+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.449829+0000", 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.62763082, 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0061586139999999998, 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 47 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.449829+0000", 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.449829+0000", 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.449829+0000", 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.449834+0000", 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.449864+0000", 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.450205+0000", 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.450803+0000", 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.450804+0000", 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.455609+0000", 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.103 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.455776+0000", 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.455867+0000", 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.455986+0000", 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.455987+0000", 2024-09-17T05:03:02.104 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:48 create owner_uid=1000, owner_gid=1271 #0x10000000000/file_21 2024-09-17T05:02:20.454236+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.456090+0000", 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.621369549000001, 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0061077079999999999, 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 48 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.456090+0000", 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.106 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.456090+0000", 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.456091+0000", 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.456095+0000", 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.107 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.456125+0000", 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.456528+0000", 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.457130+0000", 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.108 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.457131+0000", 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.461847+0000", 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.461989+0000", 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.462077+0000", 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.462197+0000", 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.462198+0000", 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.110 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:54 mkdir owner_uid=1000, owner_gid=1271 #0x1/dir2 2024-09-17T05:02:20.654233+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.659437+0000", 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.418022034000003, 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0088096800000000003, 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 54 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.659437+0000", 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.659437+0000", 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.659438+0000", 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.659443+0000", 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.659466+0000", 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.659882+0000", 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to wrlock, waiting" 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.660941+0000", 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661459+0000", 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661459+0000", 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.667813+0000", 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668007+0000", 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668176+0000", 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668246+0000", 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668247+0000", 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.116 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:55 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_0 2024-09-17T05:02:20.658233+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.661719+0000", 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.415740896000003, 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0071018360000000003, 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 55 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661719+0000", 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661719+0000", 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.118 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661719+0000", 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661726+0000", 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.661747+0000", 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.119 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.662066+0000", 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.662611+0000", 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.662612+0000", 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668252+0000", 2024-09-17T05:03:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668472+0000", 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668635+0000", 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668819+0000", 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.121 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.668820+0000", 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:60 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_5 2024-09-17T05:02:20.686232+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.691762+0000", 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.385697710000002, 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0062180009999999999, 2024-09-17T05:03:02.122 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 60 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.123 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.691762+0000", 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.691762+0000", 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.691762+0000", 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.124 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.691767+0000", 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.691800+0000", 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.692182+0000", 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.692786+0000", 2024-09-17T05:03:02.125 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.692787+0000", 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.697651+0000", 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.697798+0000", 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.697892+0000", 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.697979+0000", 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.697980+0000", 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.127 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:61 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_6 2024-09-17T05:02:20.694232+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.698093+0000", 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.379366163999997, 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0060854719999999998, 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 61 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.698093+0000", 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.129 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.698093+0000", 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.698094+0000", 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.698098+0000", 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.130 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.698131+0000", 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.698462+0000", 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.699067+0000", 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.699068+0000", 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.703857+0000", 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.703997+0000", 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.132 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704088+0000", 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704178+0000", 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704179+0000", 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:62 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_7 2024-09-17T05:02:20.702232+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.704294+0000", 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.373165683000003, 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0060755289999999997, 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.134 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 62 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704294+0000", 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704294+0000", 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704294+0000", 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704299+0000", 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704333+0000", 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.136 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.704672+0000", 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.705272+0000", 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.705273+0000", 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.137 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710043+0000", 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710187+0000", 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710280+0000", 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.138 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710369+0000", 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710369+0000", 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:63 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_8 2024-09-17T05:02:20.706232+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.139 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.710486+0000", 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.366973406, 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0060546050000000002, 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 63 2024-09-17T05:03:02.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710486+0000", 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710486+0000", 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710487+0000", 2024-09-17T05:03:02.141 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710491+0000", 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710519+0000", 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.710863+0000", 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.711456+0000", 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.711457+0000", 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716209+0000", 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716358+0000", 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716446+0000", 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716540+0000", 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716541+0000", 2024-09-17T05:03:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:64 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_9 2024-09-17T05:02:20.714232+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.716618+0000", 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.360841084999997, 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0064386950000000004, 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 64 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.146 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716618+0000", 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716618+0000", 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716619+0000", 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716624+0000", 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.716652+0000", 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.717037+0000", 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.717639+0000", 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.148 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.717640+0000", 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.722736+0000", 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.722875+0000", 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.722967+0000", 2024-09-17T05:03:02.149 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.723057+0000", 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.723057+0000", 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.150 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:77 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_22 2024-09-17T05:02:20.790231+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.792691+0000", 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.284768440999997, 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.00601054, 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.151 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 77 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.792691+0000", 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.792691+0000", 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.792692+0000", 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.792696+0000", 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.792729+0000", 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.153 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.793070+0000", 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.793674+0000", 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.793675+0000", 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798373+0000", 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798520+0000", 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798611+0000", 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798701+0000", 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798702+0000", 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:78 create owner_uid=1000, owner_gid=1271 #0x1000000020e/file_23 2024-09-17T05:02:20.794231+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.798820+0000", 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.278639433000002, 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0059961550000000004, 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 78 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798820+0000", 2024-09-17T05:03:02.157 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798820+0000", 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798821+0000", 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798825+0000", 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.798858+0000", 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.799201+0000", 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.799857+0000", 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.799858+0000", 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.804477+0000", 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.804621+0000", 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.160 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.804724+0000", 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.804816+0000", 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.804816+0000", 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.161 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:82 mkdir owner_uid=1000, owner_gid=1271 #0x1/dir3 2024-09-17T05:02:20.994227+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:20.999257+0000", 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.078202644000001, 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0070958109999999996, 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.162 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 82 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.999257+0000", 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.999257+0000", 2024-09-17T05:03:02.163 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.999257+0000", 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.999262+0000", 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.999281+0000", 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:20.999497+0000", 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to wrlock, waiting" 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000009+0000", 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000301+0000", 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.165 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000302+0000", 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006116+0000", 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006231+0000", 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006317+0000", 2024-09-17T05:03:02.166 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006352+0000", 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006353+0000", 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:83 create owner_uid=1000, owner_gid=1271 #0x10000000228/file_0 2024-09-17T05:02:20.998227+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:21.000545+0000", 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 41.076914924, 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0061167879999999997, 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.168 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 83 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000545+0000", 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000545+0000", 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000545+0000", 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000550+0000", 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000572+0000", 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.000741+0000", 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.001047+0000", 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.001048+0000", 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.171 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006356+0000", 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006492+0000", 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006572+0000", 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006661+0000", 2024-09-17T05:03:02.172 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.006661+0000", 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:110 mkdir owner_uid=1000, owner_gid=1271 #0x1/dir4 2024-09-17T05:02:21.314223+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:21.319295+0000", 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.758164135999998, 2024-09-17T05:03:02.173 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0071219589999999998, 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 110 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.174 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.319295+0000", 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.319295+0000", 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.319296+0000", 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.319301+0000", 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.319319+0000", 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.319550+0000", 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "failed to wrlock, waiting" 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.320335+0000", 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.320635+0000", 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.320635+0000", 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.177 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326084+0000", 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326300+0000", 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326383+0000", 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326417+0000", 2024-09-17T05:03:02.178 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326417+0000", 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:112 create owner_uid=1000, owner_gid=1271 #0x10000000242/file_1 2024-09-17T05:02:21.322222+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:21.326948+0000", 2024-09-17T05:03:02.179 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.750511316999997, 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 0.0063567490000000001, 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 112 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326948+0000", 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326948+0000", 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326949+0000", 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.181 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326954+0000", 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.326987+0000", 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.327276+0000", 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.327611+0000", 2024-09-17T05:03:02.182 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "early_replied" 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.327612+0000", 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "submit entry: journal_and_reply" 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.332707+0000", 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "journal_committed: " 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.332948+0000", 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.183 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.333114+0000", 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.333303+0000", 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.333305+0000", 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.184 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:143 readdir #0x10000000242 2024-09-17T05:02:21.674217+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:21.677387+0000", 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 40.400072637999997, 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 16.506814894000001, 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.185 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 143 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.677387+0000", 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.186 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.677387+0000", 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.677388+0000", 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.677392+0000", 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.677416+0000", 2024-09-17T05:03:02.187 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:21.677495+0000", 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:22.177723+0000", 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:22.677925+0000", 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:23.178121+0000", 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:23.678291+0000", 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:24.178416+0000", 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.189 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:24.678621+0000", 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:25.178794+0000", 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:25.678976+0000", 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:26.179174+0000", 2024-09-17T05:03:02.190 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:26.679350+0000", 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:27.179489+0000", 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:27.679626+0000", 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:28.179740+0000", 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:28.679862+0000", 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:29.179990+0000", 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.192 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:29.680165+0000", 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:30.180361+0000", 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:30.680565+0000", 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:31.180691+0000", 2024-09-17T05:03:02.193 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:31.680900+0000", 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:32.181103+0000", 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:32.681309+0000", 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:33.181524+0000", 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:33.681714+0000", 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:34.181890+0000", 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.195 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:34.682099+0000", 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:35.182267+0000", 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:35.682475+0000", 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:36.182686+0000", 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:36.682837+0000", 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:37.183051+0000", 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:37.683254+0000", 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.197 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.183518+0000", 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184029+0000", 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184089+0000", 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184199+0000", 2024-09-17T05:03:02.198 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184202+0000", 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "description": "client_request(client.5131:144 readdir #0x10000000228 2024-09-17T05:02:38.181991+0000 caller_uid=1000, caller_gid=1271{6,27,108,1271,10102,})", 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "initiated_at": "2024-09-17T05:02:38.184747+0000", 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 23.892712295999999, 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "duration": 22.509623353999999, 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "continuous": false, 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type_data": { 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "result": 0, 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flag_point": "done", 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "reqid": { 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": { 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5131 2024-09-17T05:03:02.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tid": 144 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "op_type": "client_request", 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "events": [ 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184747+0000", 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "initiated" 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184747+0000", 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "header_read" 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.201 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184748+0000", 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "throttled" 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184756+0000", 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "all_read" 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.184868+0000", 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "dispatched" 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.185010+0000", 2024-09-17T05:03:02.202 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:38.685196+0000", 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:39.185413+0000", 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:39.685560+0000", 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:40.185756+0000", 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:40.685888+0000", 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:41.186058+0000", 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.204 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:41.686242+0000", 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:42.186439+0000", 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:42.686601+0000", 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:43.186815+0000", 2024-09-17T05:03:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:43.687017+0000", 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:44.187196+0000", 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:44.687403+0000", 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.206 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:45.187602+0000", 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:45.687812+0000", 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:46.188013+0000", 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.207 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:46.688193+0000", 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:47.188398+0000", 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:47.688616+0000", 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:48.188823+0000", 2024-09-17T05:03:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:48.689016+0000", 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:49.189239+0000", 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:49.689425+0000", 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:50.189648+0000", 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:50.689842+0000", 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:51.190036+0000", 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:51.690234+0000", 2024-09-17T05:03:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:52.190362+0000", 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:52.690533+0000", 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:53.190714+0000", 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:53.690901+0000", 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:54.191096+0000", 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:54.691313+0000", 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.212 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:55.191527+0000", 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:55.691722+0000", 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:56.191881+0000", 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:56.692071+0000", 2024-09-17T05:03:02.213 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:57.192268+0000", 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:57.692472+0000", 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:58.192679+0000", 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.214 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:58.692841+0000", 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:59.192994+0000", 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:02:59.693200+0000", 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.215 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:03:00.193324+0000", 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cap_acquisition_throttle" 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:03:00.693738+0000", 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "acquired locks" 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:03:00.694214+0000", 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "replying" 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:03:00.694265+0000", 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "finishing request" 2024-09-17T05:03:02.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:03:00.694369+0000", 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "cleaned up request" 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "time": "2024-09-17T05:03:00.694371+0000", 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "event": "done" 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: "locks": null 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:02.217 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:03:02.218 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fe8073200 msgr2=0x7f5fe8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fe8073200 0x7f5fe8075710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f600c1e6110 tx=0x7f5ff40091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 msgr2=0x7f600c1e68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f600c1e68a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc001870 tx=0x7f5ffc0039d0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 shutdown_connections 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5fe8073200 0x7f5fe8075710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f600c0b4f60 0x7f600c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f600c1e35b0 0x7f600c1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 --2- 172.21.15.16:0/2815393589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff0004d50 0x7f600c1e68a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 >> 172.21.15.16:0/2815393589 conn(0x7f600c0b2250 msgr2=0x7f600c1585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 shutdown_connections 2024-09-17T05:03:02.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.077+0000 7f6013a00640 1 -- 172.21.15.16:0/2815393589 wait complete. 2024-09-17T05:03:02.221 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:03:02.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:02.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:02.399 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:03:02.400 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:02.400 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:02.400 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:02.410 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:02.410 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:02.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 -- 172.21.15.16:0/2439522002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 msgr2=0x7f287815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 --2- 172.21.15.16:0/2439522002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f287815b5b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2870009c80 tx=0x7f287001cda0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 -- 172.21.15.16:0/2439522002 shutdown_connections 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 --2- 172.21.15.16:0/2439522002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f287815b5b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 --2- 172.21.15.16:0/2439522002 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 0x7f287815eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 -- 172.21.15.16:0/2439522002 >> 172.21.15.16:0/2439522002 conn(0x7f28780b2250 msgr2=0x7f28780d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 -- 172.21.15.16:0/2439522002 shutdown_connections 2024-09-17T05:03:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.665+0000 7f287dc00640 1 -- 172.21.15.16:0/2439522002 wait complete. 2024-09-17T05:03:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 Processor -- start 2024-09-17T05:03:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 -- start start 2024-09-17T05:03:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f287806e330 0x7f2878163900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f2878162920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 0x7f2878162e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2878167cb0 con 0x7f287815b1a0 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2878167b30 con 0x7f287806e330 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f287dc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2878167e30 con 0x7f287815e790 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f2878162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f2878162920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36086/0 (socket says 172.21.15.16:36086) 2024-09-17T05:03:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 -- 172.21.15.16:0/597515682 learned_addr learned my addr 172.21.15.16:0/597515682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 -- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 msgr2=0x7f2878162e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f2877400640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 0x7f2878162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f2876a00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f287806e330 0x7f2878163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 0x7f2878162e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 -- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f287806e330 msgr2=0x7f2878163900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f287806e330 0x7f2878163900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.669+0000 7f286fe00640 1 -- 172.21.15.16:0/597515682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287814a4d0 con 0x7f287815b1a0 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f2877400640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 0x7f2878162e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f2876a00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f287806e330 0x7f2878163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286fe00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f2878162920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f286000e8b0 tx=0x7f286000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286000cbc0 con 0x7f287815b1a0 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f286000ce50 con 0x7f287815b1a0 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860005260 con 0x7f287815b1a0 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287814a7b0 con 0x7f287815b1a0 2024-09-17T05:03:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287814ad70 con 0x7f287815b1a0 2024-09-17T05:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287815b5b0 con 0x7f287815b1a0 2024-09-17T05:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2860005400 con 0x7f287815b1a0 2024-09-17T05:03:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286ea00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2854073200 0x7f2854075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.673+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f2860094870 con 0x7f287815b1a0 2024-09-17T05:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.677+0000 7f2876a00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2854073200 0x7f2854075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:02.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.677+0000 7f2876a00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2854073200 0x7f2854075710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2868000f70 tx=0x7f2868009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:02.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.681+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f286005f630 con 0x7f287815b1a0 2024-09-17T05:03:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.861+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2878163eb0 con 0x7f287815b1a0 2024-09-17T05:03:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.861+0000 7f286ea00640 1 -- 172.21.15.16:0/597515682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f286005ef30 con 0x7f287815b1a0 2024-09-17T05:03:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:03:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2854073200 msgr2=0x7f2854075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2854073200 0x7f2854075710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2868000f70 tx=0x7f2868009040 comp rx=0 tx=0).stop 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 msgr2=0x7f2878162920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f2878162920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f286000e8b0 tx=0x7f286000ed70 comp rx=0 tx=0).stop 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 shutdown_connections 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2854073200 0x7f2854075710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f287815e790 0x7f2878162e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f287815b1a0 0x7f2878162920 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 --2- 172.21.15.16:0/597515682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f287806e330 0x7f2878163900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 >> 172.21.15.16:0/597515682 conn(0x7f28780b2250 msgr2=0x7f28780cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 shutdown_connections 2024-09-17T05:03:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:02.865+0000 7f287dc00640 1 -- 172.21.15.16:0/597515682 wait complete. 2024-09-17T05:03:02.889 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:03:02.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:02.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:02.998 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:03:02.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:02.999 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:03:03.008 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:03.009 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:03.009 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:03.018 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:03.018 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:03.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:03:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.225+0000 7f30c1800640 1 -- 172.21.15.16:0/2041084149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c130 msgr2=0x7f30bc073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.225+0000 7f30c1800640 1 --2- 172.21.15.16:0/2041084149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c130 0x7f30bc073bc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009840 tx=0x7f30a401c820 comp rx=0 tx=0).stop 2024-09-17T05:03:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 -- 172.21.15.16:0/2041084149 shutdown_connections 2024-09-17T05:03:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 --2- 172.21.15.16:0/2041084149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c130 0x7f30bc073bc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 -- 172.21.15.16:0/2041084149 >> 172.21.15.16:0/2041084149 conn(0x7f30bc0d1170 msgr2=0x7f30bc0734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 -- 172.21.15.16:0/2041084149 shutdown_connections 2024-09-17T05:03:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 -- 172.21.15.16:0/2041084149 wait complete. 2024-09-17T05:03:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 Processor -- start 2024-09-17T05:03:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 -- start start 2024-09-17T05:03:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30bc158e40 0x7f30bc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.229+0000 7f30c1800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30bc15c130 0x7f30bc151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 0x7f30bc13dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc16a560 con 0x7f30bc15c540 2024-09-17T05:03:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc16a3e0 con 0x7f30bc15c130 2024-09-17T05:03:03.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc16a6e0 con 0x7f30bc158e40 2024-09-17T05:03:03.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 0x7f30bc13dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 0x7f30bc13dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36102/0 (socket says 172.21.15.16:36102) 2024-09-17T05:03:03.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 -- 172.21.15.16:0/1022775567 learned_addr learned my addr 172.21.15.16:0/1022775567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30b3400640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30bc15c130 0x7f30bc151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30bbe00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30bc158e40 0x7f30bc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 -- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30bc158e40 msgr2=0x7f30bc158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30bc158e40 0x7f30bc158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 -- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30bc15c130 msgr2=0x7f30bc151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30bc15c130 0x7f30bc151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 -- 172.21.15.16:0/1022775567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc15e9e0 con 0x7f30bc15c540 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30bbe00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30bc158e40 0x7f30bc158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c0e00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 0x7f30bc13dd70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c730 tx=0x7f30b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b4003b80 con 0x7f30bc15c540 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30b4003d20 con 0x7f30bc15c540 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc14cfb0 con 0x7f30bc15c540 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b400aab0 con 0x7f30bc15c540 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc157a50 con 0x7f30bc15c540 2024-09-17T05:03:03.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc073bc0 con 0x7f30bc15c540 2024-09-17T05:03:03.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.233+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30b400ac50 con 0x7f30bc15c540 2024-09-17T05:03:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.237+0000 7f30b8c00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3098073200 0x7f3098075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.237+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f30b4094180 con 0x7f30bc15c540 2024-09-17T05:03:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.237+0000 7f30bbe00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3098073200 0x7f3098075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.237+0000 7f30bbe00640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3098073200 0x7f3098075710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0046c0 tx=0x7f30ac009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:03.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.241+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30b405efe0 con 0x7f30bc15c540 2024-09-17T05:03:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.429+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f30bc0719f0 con 0x7f30bc15c540 2024-09-17T05:03:03.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.429+0000 7f30b8c00640 1 -- 172.21.15.16:0/1022775567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f30b405e8e0 con 0x7f30bc15c540 2024-09-17T05:03:03.433 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3098073200 msgr2=0x7f3098075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3098073200 0x7f3098075710 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0046c0 tx=0x7f30ac009040 comp rx=0 tx=0).stop 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 msgr2=0x7f30bc13dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 0x7f30bc13dd70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f30b400c730 tx=0x7f30b400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 shutdown_connections 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3098073200 0x7f3098075710 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30bc15c540 0x7f30bc13dd70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30bc15c130 0x7f30bc151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 --2- 172.21.15.16:0/1022775567 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30bc158e40 0x7f30bc158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 >> 172.21.15.16:0/1022775567 conn(0x7f30bc0d1170 msgr2=0x7f30bc06fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 shutdown_connections 2024-09-17T05:03:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.433+0000 7f30c1800640 1 -- 172.21.15.16:0/1022775567 wait complete. 2024-09-17T05:03:03.455 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:03:03.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:03.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:03:03.485 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:03:03.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:03.592 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:03:03.601 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:03.602 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:03.602 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:03.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:03:03.611 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:03.611 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:03.611 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:03.652 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:03:03.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f4692800640 1 -- 172.21.15.16:0/4008427632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4670004da0 msgr2=0x7f4670005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f4692800640 1 --2- 172.21.15.16:0/4008427632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4670004da0 0x7f4670005250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f468c0d0390 tx=0x7f468401d5b0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f468be00640 1 -- 172.21.15.16:0/4008427632 reap_dead start 2024-09-17T05:03:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f4692800640 1 -- 172.21.15.16:0/4008427632 shutdown_connections 2024-09-17T05:03:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f4692800640 1 -- 172.21.15.16:0/4008427632 >> 172.21.15.16:0/4008427632 conn(0x7f468c0d0430 msgr2=0x7f468c0d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f4692800640 1 -- 172.21.15.16:0/4008427632 shutdown_connections 2024-09-17T05:03:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.809+0000 7f4692800640 1 -- 172.21.15.16:0/4008427632 wait complete. 2024-09-17T05:03:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 Processor -- start 2024-09-17T05:03:03.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 -- start start 2024-09-17T05:03:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4670004da0 0x7f468c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f468c14c9d0 0x7f468c14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 0x7f468c1f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f468c15d8e0 con 0x7f468c14f1a0 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f468c15d760 con 0x7f4670004da0 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4692800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f468c15da60 con 0x7f468c14c9d0 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 0x7f468c1f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 0x7f468c1f8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36120/0 (socket says 172.21.15.16:36120) 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 -- 172.21.15.16:0/179522818 learned_addr learned my addr 172.21.15.16:0/179522818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4682a00640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f468c14c9d0 0x7f468c14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468b400640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4670004da0 0x7f468c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 -- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f468c14c9d0 msgr2=0x7f468c14ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f468c14c9d0 0x7f468c14ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 -- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4670004da0 msgr2=0x7f468c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4670004da0 0x7f468c0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 -- 172.21.15.16:0/179522818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468c15a4b0 con 0x7f468c14f1a0 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4682a00640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f468c14c9d0 0x7f468c14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468b400640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4670004da0 0x7f468c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f468be00640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 0x7f468c1f8e90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f468401d3e0 tx=0x7f4684009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684009a90 con 0x7f468c14f1a0 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.813+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46840a94f0 con 0x7f468c14f1a0 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46840a9ae0 con 0x7f468c14f1a0 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468c15a790 con 0x7f468c14f1a0 2024-09-17T05:03:03.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468c14ce10 con 0x7f468c14f1a0 2024-09-17T05:03:03.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f468c15ce40 con 0x7f468c14f1a0 2024-09-17T05:03:03.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46840a9c80 con 0x7f468c14f1a0 2024-09-17T05:03:03.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4683e00640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4668073230 0x7f4668075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:03.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f468412f510 con 0x7f468c14f1a0 2024-09-17T05:03:03.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.817+0000 7f468b400640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4668073230 0x7f4668075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:03.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.821+0000 7f468b400640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4668073230 0x7f4668075740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f467c00afd0 tx=0x7f467c000ee0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:03.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:03.821+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f46840c2030 con 0x7f468c14f1a0 2024-09-17T05:03:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f468c14f790 con 0x7f468c14f1a0 2024-09-17T05:03:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4683e00640 1 -- 172.21.15.16:0/179522818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f46840ad070 con 0x7f468c14f1a0 2024-09-17T05:03:04.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:04.036 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:03:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4668073230 msgr2=0x7f4668075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4668073230 0x7f4668075740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f467c00afd0 tx=0x7f467c000ee0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 msgr2=0x7f468c1f8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 0x7f468c1f8e90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f468401d3e0 tx=0x7f4684009760 comp rx=0 tx=0).stop 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 shutdown_connections 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4668073230 0x7f4668075740 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f468c14f1a0 0x7f468c1f8e90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f468c14c9d0 0x7f468c14ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 --2- 172.21.15.16:0/179522818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4670004da0 0x7f468c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 >> 172.21.15.16:0/179522818 conn(0x7f468c0d0430 msgr2=0x7f468c06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 shutdown_connections 2024-09-17T05:03:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.033+0000 7f4692800640 1 -- 172.21.15.16:0/179522818 wait complete. 2024-09-17T05:03:04.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 -- 172.21.15.16:0/2902706168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 msgr2=0x7f5d34073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/2902706168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d34073650 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5d34071ce0 tx=0x7f5d2c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 -- 172.21.15.16:0/2902706168 shutdown_connections 2024-09-17T05:03:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/2902706168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d34073650 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/2902706168 >> v2:172.21.15.104:3300/0 conn(0x7f5d340b2f50 0x7f5d3415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 -- 172.21.15.16:0/2902706168 >> 172.21.15.16:0/2902706168 conn(0x7f5d3406f900 msgr2=0x7f5d340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 -- 172.21.15.16:0/2902706168 shutdown_connections 2024-09-17T05:03:04.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 -- 172.21.15.16:0/2902706168 wait complete. 2024-09-17T05:03:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 Processor -- start 2024-09-17T05:03:04.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.213+0000 7f5d3ae00640 1 -- start start 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d340cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d340b2f50 0x7f5d3413d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3415c0a0 0x7f5d3413fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d34168c70 con 0x7f5d34073240 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d34168af0 con 0x7f5d3415c0a0 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5d34168df0 con 0x7f5d340b2f50 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d340cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d340cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36144/0 (socket says 172.21.15.16:36144) 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 -- 172.21.15.16:0/42762826 learned_addr learned my addr 172.21.15.16:0/42762826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d38e00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3415c0a0 0x7f5d3413fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d2be00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d340b2f50 0x7f5d3413d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 -- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d340b2f50 msgr2=0x7f5d3413d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d340b2f50 0x7f5d3413d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 -- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3415c0a0 msgr2=0x7f5d3413fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3415c0a0 0x7f5d3413fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 -- 172.21.15.16:0/42762826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3413d7c0 con 0x7f5d34073240 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d2be00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d340b2f50 0x7f5d3413d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d33e00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d340cbb40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400c9f0 tx=0x7f5d2400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24007bb0 con 0x7f5d34073240 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d24007e40 con 0x7f5d34073240 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400f690 con 0x7f5d34073240 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d340c6120 con 0x7f5d34073240 2024-09-17T05:03:04.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d340c66e0 con 0x7f5d34073240 2024-09-17T05:03:04.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.217+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d34073650 con 0x7f5d34073240 2024-09-17T05:03:04.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.221+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d2400f830 con 0x7f5d34073240 2024-09-17T05:03:04.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.221+0000 7f5d30c00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d10073200 0x7f5d10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.221+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f5d24094b80 con 0x7f5d34073240 2024-09-17T05:03:04.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.221+0000 7f5d2be00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d10073200 0x7f5d10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.221+0000 7f5d2be00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d10073200 0x7f5d10075710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d34152400 tx=0x7f5d1c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:04.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.225+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5d24014030 con 0x7f5d34073240 2024-09-17T05:03:04.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.433+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d340cc250 con 0x7f5d34073240 2024-09-17T05:03:04.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.433+0000 7f5d30c00640 1 -- 172.21.15.16:0/42762826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f5d2405f9d0 con 0x7f5d34073240 2024-09-17T05:03:04.437 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:04.437 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:03:04.437 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:03:04.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d10073200 msgr2=0x7f5d10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d10073200 0x7f5d10075710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5d34152400 tx=0x7f5d1c009040 comp rx=0 tx=0).stop 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 msgr2=0x7f5d340cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d340cbb40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400c9f0 tx=0x7f5d2400ceb0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 shutdown_connections 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d10073200 0x7f5d10075710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3415c0a0 0x7f5d3413fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d340b2f50 0x7f5d3413d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 --2- 172.21.15.16:0/42762826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d34073240 0x7f5d340cbb40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 >> 172.21.15.16:0/42762826 conn(0x7f5d3406f900 msgr2=0x7f5d340d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 shutdown_connections 2024-09-17T05:03:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.437+0000 7f5d3ae00640 1 -- 172.21.15.16:0/42762826 wait complete. 2024-09-17T05:03:04.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.613+0000 7fef03000640 1 -- 172.21.15.16:0/850599802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0d1c90 msgr2=0x7feefc15cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.613+0000 7fef03000640 1 --2- 172.21.15.16:0/850599802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0d1c90 0x7feefc15cab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feee4009840 tx=0x7feee401c820 comp rx=0 tx=0).stop 2024-09-17T05:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.613+0000 7fef03000640 1 -- 172.21.15.16:0/850599802 shutdown_connections 2024-09-17T05:03:04.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.613+0000 7fef03000640 1 --2- 172.21.15.16:0/850599802 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feefc0c9440 0x7feefc0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.613+0000 7fef03000640 1 --2- 172.21.15.16:0/850599802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0d1c90 0x7feefc15cab0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.613+0000 7fef03000640 1 -- 172.21.15.16:0/850599802 >> 172.21.15.16:0/850599802 conn(0x7feefc0d2270 msgr2=0x7feefc0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 -- 172.21.15.16:0/850599802 shutdown_connections 2024-09-17T05:03:04.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 -- 172.21.15.16:0/850599802 wait complete. 2024-09-17T05:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 Processor -- start 2024-09-17T05:03:04.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 -- start start 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feefc0c9440 0x7feefc155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 0x7feefc159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feefc06f4d0 0x7feefc06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feefc169fd0 con 0x7feefc06f4d0 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feefc169e50 con 0x7feefc0cc8c0 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.617+0000 7fef03000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7feefc16a150 con 0x7feefc0c9440 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7fef01000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feefc06f4d0 0x7feefc06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 0x7feefc159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feefbe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feefc0c9440 0x7feefc155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 0x7feefc159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:60206/0 (socket says 172.21.15.16:60206) 2024-09-17T05:03:04.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 -- 172.21.15.16:0/1846777950 learned_addr learned my addr 172.21.15.16:0/1846777950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:04.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 -- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feefc0c9440 msgr2=0x7feefc155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feefc0c9440 0x7feefc155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 -- 172.21.15.16:0/1846777950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feefc06f4d0 msgr2=0x7feefc06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feefc06f4d0 0x7feefc06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 -- 172.21.15.16:0/1846777950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc06e430 con 0x7feefc0cc8c0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7fef01000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feefc06f4d0 0x7feefc06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef3e00640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 0x7feefc159d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feee401aed0 tx=0x7feee40a5670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40bb070 con 0x7feefc0cc8c0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee4004470 con 0x7feefc0cc8c0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc06e710 con 0x7feefc0cc8c0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40b2040 con 0x7feefc0cc8c0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc06ecd0 con 0x7feefc0cc8c0 2024-09-17T05:03:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.621+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc06e1a0 con 0x7feefc0cc8c0 2024-09-17T05:03:04.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.625+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feee40be070 con 0x7feefc0cc8c0 2024-09-17T05:03:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.625+0000 7feef8c00640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feed8073250 0x7feed8075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.625+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7feee412ddd0 con 0x7feefc0cc8c0 2024-09-17T05:03:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.625+0000 7feefbe00640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feed8073250 0x7feed8075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.625+0000 7feefbe00640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feed8073250 0x7feed8075760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feeec007fd0 tx=0x7feeec007e40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.629+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feee40c5080 con 0x7feefc0cc8c0 2024-09-17T05:03:04.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.801+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feefc0c9850 con 0x7feefc0cc8c0 2024-09-17T05:03:04.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7feef8c00640 1 -- 172.21.15.16:0/1846777950 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17025 (secure 0 0 0) 0x7feee4016410 con 0x7feefc0cc8c0 2024-09-17T05:03:04.807 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:04.807 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:02:49.864913+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:02:04.037604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T05:02:04.686294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.20","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feed8073250 msgr2=0x7feed8075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feed8073250 0x7feed8075760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7feeec007fd0 tx=0x7feeec007e40 comp rx=0 tx=0).stop 2024-09-17T05:03:04.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 msgr2=0x7feefc159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 0x7feefc159d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feee401aed0 tx=0x7feee40a5670 comp rx=0 tx=0).stop 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 shutdown_connections 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feed8073250 0x7feed8075760 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feefc06f4d0 0x7feefc06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feefc0cc8c0 0x7feefc159d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 --2- 172.21.15.16:0/1846777950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feefc0c9440 0x7feefc155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 >> 172.21.15.16:0/1846777950 conn(0x7feefc0d2270 msgr2=0x7feefc0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 shutdown_connections 2024-09-17T05:03:04.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.805+0000 7fef03000640 1 -- 172.21.15.16:0/1846777950 wait complete. 2024-09-17T05:03:04.829 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:03:04.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 -- 172.21.15.16:0/1858950535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880cc980 msgr2=0x7f38880ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 --2- 172.21.15.16:0/1858950535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880cc980 0x7f38880ccd90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f388806dc60 tx=0x7f387001c800 comp rx=0 tx=0).stop 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 -- 172.21.15.16:0/1858950535 shutdown_connections 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 --2- 172.21.15.16:0/1858950535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880cc980 0x7f38880ccd90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 --2- 172.21.15.16:0/1858950535 >> v2:172.21.15.104:3300/0 conn(0x7f38880d1890 0x7f3888160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 -- 172.21.15.16:0/1858950535 >> 172.21.15.16:0/1858950535 conn(0x7f38880ceb80 msgr2=0x7f38880cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 -- 172.21.15.16:0/1858950535 shutdown_connections 2024-09-17T05:03:04.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.985+0000 7f3887200640 1 -- 172.21.15.16:0/1858950535 wait complete. 2024-09-17T05:03:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 Processor -- start 2024-09-17T05:03:04.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 -- start start 2024-09-17T05:03:04.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38880cc980 0x7f388814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38880d01e0 0x7f38880c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 0x7f38880c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f388816a0a0 con 0x7f38880d1890 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3888169f20 con 0x7f38880cc980 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f388816a220 con 0x7f38880d01e0 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 0x7f38880c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 0x7f38880c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36176/0 (socket says 172.21.15.16:36176) 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 -- 172.21.15.16:0/3552082141 learned_addr learned my addr 172.21.15.16:0/3552082141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f387d400640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38880d01e0 0x7f38880c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3885e00640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38880cc980 0x7f388814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 -- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38880d01e0 msgr2=0x7f38880c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38880d01e0 0x7f38880c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 -- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38880cc980 msgr2=0x7f388814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38880cc980 0x7f388814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 -- 172.21.15.16:0/3552082141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388815c270 con 0x7f38880d1890 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3886800640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 0x7f38880c8860 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f388000c3c0 tx=0x7f388000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880017070 con 0x7f38880d1890 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3880003600 con 0x7f38880d1890 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388815c550 con 0x7f38880d1890 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880003a50 con 0x7f38880d1890 2024-09-17T05:03:04.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.989+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38880c6900 con 0x7f38880d1890 2024-09-17T05:03:04.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.993+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388815c790 con 0x7f38880d1890 2024-09-17T05:03:04.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.993+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f388001a070 con 0x7f38880d1890 2024-09-17T05:03:04.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.993+0000 7f387ea00640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3864073200 0x7f3864075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:04.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.993+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f3880093580 con 0x7f38880d1890 2024-09-17T05:03:04.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.993+0000 7f3885e00640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3864073200 0x7f3864075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:04.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.997+0000 7f3885e00640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3864073200 0x7f3864075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3878009e50 tx=0x7f38780095e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:04.997+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f388815c790 con 0x7f38880d1890 2024-09-17T05:03:05.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f388814e690 con 0x7f38880d1890 2024-09-17T05:03:05.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f387ea00640 1 -- 172.21.15.16:0/3552082141 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f388814e690 con 0x7f38880d1890 2024-09-17T05:03:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:05.212 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:03:05.212 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3864073200 msgr2=0x7f3864075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3864073200 0x7f3864075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3878009e50 tx=0x7f38780095e0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 msgr2=0x7f38880c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 0x7f38880c8860 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f388000c3c0 tx=0x7f388000c880 comp rx=0 tx=0).stop 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 shutdown_connections 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3864073200 0x7f3864075710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38880d1890 0x7f38880c8860 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38880d01e0 0x7f38880c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 --2- 172.21.15.16:0/3552082141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38880cc980 0x7f388814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.209+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 >> 172.21.15.16:0/3552082141 conn(0x7f38880ceb80 msgr2=0x7f3888151720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:05.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.213+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 shutdown_connections 2024-09-17T05:03:05.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.213+0000 7f3887200640 1 -- 172.21.15.16:0/3552082141 wait complete. 2024-09-17T05:03:05.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 -- 172.21.15.16:0/2539645187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 msgr2=0x7f62b81608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 --2- 172.21.15.16:0/2539645187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 0x7f62b81608a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009bf0 tx=0x7f62b001cbe0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 -- 172.21.15.16:0/2539645187 shutdown_connections 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 --2- 172.21.15.16:0/2539645187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 0x7f62b81608a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 -- 172.21.15.16:0/2539645187 >> 172.21.15.16:0/2539645187 conn(0x7f62b80cef40 msgr2=0x7f62b80cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 -- 172.21.15.16:0/2539645187 shutdown_connections 2024-09-17T05:03:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.389+0000 7f62bd600640 1 -- 172.21.15.16:0/2539645187 wait complete. 2024-09-17T05:03:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 Processor -- start 2024-09-17T05:03:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 -- start start 2024-09-17T05:03:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b80cc710 0x7f62b81f9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 0x7f62b814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 0x7f62b814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8153610 con 0x7f62b80d1910 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8153490 con 0x7f62b80cc710 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bd600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f62b8153790 con 0x7f62b8160490 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 0x7f62b814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 0x7f62b814d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36190/0 (socket says 172.21.15.16:36190) 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 -- 172.21.15.16:0/2685575369 learned_addr learned my addr 172.21.15.16:0/2685575369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62b7e00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b80cc710 0x7f62b81f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62bcc00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 0x7f62b814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 -- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 msgr2=0x7f62b814dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 0x7f62b814dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 -- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b80cc710 msgr2=0x7f62b81f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b80cc710 0x7f62b81f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 -- 172.21.15.16:0/2685575369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b814ce10 con 0x7f62b80d1910 2024-09-17T05:03:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62afe00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 0x7f62b814d6b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c390 tx=0x7f62a000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0011790 con 0x7f62b80d1910 2024-09-17T05:03:05.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.393+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62a0017070 con 0x7f62b80d1910 2024-09-17T05:03:05.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b80b6d40 con 0x7f62b80d1910 2024-09-17T05:03:05.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a0011cc0 con 0x7f62b80d1910 2024-09-17T05:03:05.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b80b72e0 con 0x7f62b80d1910 2024-09-17T05:03:05.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b81f99c0 con 0x7f62b80d1910 2024-09-17T05:03:05.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f62a001c460 con 0x7f62b80d1910 2024-09-17T05:03:05.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62b4c00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6294073230 0x7f6294075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f62a00941a0 con 0x7f62b80d1910 2024-09-17T05:03:05.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.397+0000 7f62b7e00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6294073230 0x7f6294075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.401+0000 7f62b7e00640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6294073230 0x7f6294075740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62a8001130 tx=0x7f62a8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.401+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f62a005f070 con 0x7f62b80d1910 2024-09-17T05:03:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b81608a0 con 0x7f62b80d1910 2024-09-17T05:03:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62b4c00640 1 -- 172.21.15.16:0/2685575369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5220 (secure 0 0 0) 0x7f62a005e970 con 0x7f62b80d1910 2024-09-17T05:03:05.615 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:05.615 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-17T05:02:09:331584+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:02:05.738259+0000","modified":"2024-09-17T05:02:08.749320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/594213684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":594213684},{"type":"v1","addr":"172.21.15.16:6835","nonce":594213684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5031,"qdb_cluster":[5031]},"id":2}]} 2024-09-17T05:03:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-17T05:03:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6294073230 msgr2=0x7f6294075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6294073230 0x7f6294075740 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62a8001130 tx=0x7f62a8009040 comp rx=0 tx=0).stop 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 msgr2=0x7f62b814d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 0x7f62b814d6b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f62a000c390 tx=0x7f62a000c850 comp rx=0 tx=0).stop 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 shutdown_connections 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6294073230 0x7f6294075740 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62b8160490 0x7f62b814dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62b80d1910 0x7f62b814d6b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 --2- 172.21.15.16:0/2685575369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62b80cc710 0x7f62b81f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 >> 172.21.15.16:0/2685575369 conn(0x7f62b80cef40 msgr2=0x7f62b815f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 shutdown_connections 2024-09-17T05:03:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.613+0000 7f62bd600640 1 -- 172.21.15.16:0/2685575369 wait complete. 2024-09-17T05:03:05.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 -- 172.21.15.16:0/3067356916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c073240 msgr2=0x7f955c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 --2- 172.21.15.16:0/3067356916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c073240 0x7f955c073650 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f955c071ce0 tx=0x7f955401caf0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 -- 172.21.15.16:0/3067356916 shutdown_connections 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 --2- 172.21.15.16:0/3067356916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c073240 0x7f955c073650 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 --2- 172.21.15.16:0/3067356916 >> v2:172.21.15.104:3300/0 conn(0x7f955c0b2f50 0x7f955c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 -- 172.21.15.16:0/3067356916 >> 172.21.15.16:0/3067356916 conn(0x7f955c06f900 msgr2=0x7f955c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 -- 172.21.15.16:0/3067356916 shutdown_connections 2024-09-17T05:03:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.793+0000 7f9561e00640 1 -- 172.21.15.16:0/3067356916 wait complete. 2024-09-17T05:03:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 Processor -- start 2024-09-17T05:03:05.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 -- start start 2024-09-17T05:03:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c073240 0x7f955c0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 0x7f955c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15c0a0 0x7f955c1fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f955c168c70 con 0x7f955c0b2f50 2024-09-17T05:03:05.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f955c168af0 con 0x7f955c15c0a0 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f955c168df0 con 0x7f955c073240 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 0x7f955c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 0x7f955c0c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36216/0 (socket says 172.21.15.16:36216) 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 -- 172.21.15.16:0/2770539148 learned_addr learned my addr 172.21.15.16:0/2770539148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955aa00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c073240 0x7f955c0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955b400640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15c0a0 0x7f955c1fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 -- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c073240 msgr2=0x7f955c0d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c073240 0x7f955c0d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 -- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15c0a0 msgr2=0x7f955c1fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15c0a0 0x7f955c1fbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 -- 172.21.15.16:0/2770539148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c14b190 con 0x7f955c0b2f50 2024-09-17T05:03:05.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f955a000640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 0x7f955c0c4ec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9548001ec0 tx=0x7f954800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:05.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548017070 con 0x7f955c0b2f50 2024-09-17T05:03:05.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9548003600 con 0x7f955c0b2f50 2024-09-17T05:03:05.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548003a50 con 0x7f955c0b2f50 2024-09-17T05:03:05.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c14b470 con 0x7f955c0b2f50 2024-09-17T05:03:05.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.797+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c14ba30 con 0x7f955c0b2f50 2024-09-17T05:03:05.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.801+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c073650 con 0x7f955c0b2f50 2024-09-17T05:03:05.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.801+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f954801a070 con 0x7f955c0b2f50 2024-09-17T05:03:05.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.801+0000 7f9553400640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f953c073200 0x7f953c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:05.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.801+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f9548093720 con 0x7f955c0b2f50 2024-09-17T05:03:05.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.805+0000 7f955aa00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f953c073200 0x7f953c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:05.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.805+0000 7f955aa00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f953c073200 0x7f953c075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f954c00ab40 tx=0x7f954c00a250 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:05.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.809+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f954805de90 con 0x7f955c0b2f50 2024-09-17T05:03:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f955c0b42c0 con 0x7f955c0b2f50 2024-09-17T05:03:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9553400640 1 -- 172.21.15.16:0/2770539148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17025 (secure 0 0 0) 0x7f95480628c0 con 0x7f955c0b2f50 2024-09-17T05:03:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:05.992 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":42,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:02:49.864913+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:02:04.037604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T05:02:04.686294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.0","mappings":[{"from":4,"to":2}]},{"pgid":"6.20","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f953c073200 msgr2=0x7f953c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f953c073200 0x7f953c075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f954c00ab40 tx=0x7f954c00a250 comp rx=0 tx=0).stop 2024-09-17T05:03:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 msgr2=0x7f955c0c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 0x7f955c0c4ec0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9548001ec0 tx=0x7f954800c890 comp rx=0 tx=0).stop 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 shutdown_connections 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f953c073200 0x7f953c075710 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f954c00ab40 tx=0x7f954c00a250 comp rx=0 tx=0).stop 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15c0a0 0x7f955c1fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0b2f50 0x7f955c0c4ec0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 --2- 172.21.15.16:0/2770539148 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c073240 0x7f955c0d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 >> 172.21.15.16:0/2770539148 conn(0x7f955c06f900 msgr2=0x7f955c15d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 shutdown_connections 2024-09-17T05:03:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:05.989+0000 7f9561e00640 1 -- 172.21.15.16:0/2770539148 wait complete. 2024-09-17T05:03:06.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:03:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.169+0000 7f4ad5800640 1 -- 172.21.15.16:0/3412521398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab4003c70 msgr2=0x7f4ab40040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.169+0000 7f4ad5800640 1 --2- 172.21.15.16:0/3412521398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab4003c70 0x7f4ab40040c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000c3f0 tx=0x7f4ac001cf00 comp rx=0 tx=0).stop 2024-09-17T05:03:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 -- 172.21.15.16:0/3412521398 shutdown_connections 2024-09-17T05:03:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 --2- 172.21.15.16:0/3412521398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab4003c70 0x7f4ab40040c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 -- 172.21.15.16:0/3412521398 >> 172.21.15.16:0/3412521398 conn(0x7f4ad00b42d0 msgr2=0x7f4ad00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 -- 172.21.15.16:0/3412521398 shutdown_connections 2024-09-17T05:03:06.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 -- 172.21.15.16:0/3412521398 wait complete. 2024-09-17T05:03:06.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 Processor -- start 2024-09-17T05:03:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 -- start start 2024-09-17T05:03:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ab4003c70 0x7f4ad01eab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.173+0000 7f4ad5800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ad0143e60 0x7f4ad0144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 0x7f4ad01452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00d08d0 con 0x7f4ad01447b0 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad00b2b20 con 0x7f4ad0143e60 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4ad01600f0 con 0x7f4ab4003c70 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 0x7f4ad01452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 0x7f4ad01452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36236/0 (socket says 172.21.15.16:36236) 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 -- 172.21.15.16:0/199722318 learned_addr learned my addr 172.21.15.16:0/199722318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acea00640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ab4003c70 0x7f4ad01eab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ac7e00640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ad0143e60 0x7f4ad0144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 -- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ab4003c70 msgr2=0x7f4ad01eab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ab4003c70 0x7f4ad01eab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 -- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ad0143e60 msgr2=0x7f4ad0144270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ad0143e60 0x7f4ad0144270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 -- 172.21.15.16:0/199722318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad01586a0 con 0x7f4ad01447b0 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4acf400640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 0x7f4ad01452e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ce070 tx=0x7f4ac8002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011900 con 0x7f4ad01447b0 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ac8017070 con 0x7f4ad01447b0 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8011d60 con 0x7f4ad01447b0 2024-09-17T05:03:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad01583c0 con 0x7f4ad01447b0 2024-09-17T05:03:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad0143660 con 0x7f4ad01447b0 2024-09-17T05:03:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ad01605e0 con 0x7f4ad01447b0 2024-09-17T05:03:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.177+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4ac801c460 con 0x7f4ad01447b0 2024-09-17T05:03:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.181+0000 7f4ac6a00640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aac073230 0x7f4aac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.181+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f4ac8094c90 con 0x7f4ad01447b0 2024-09-17T05:03:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.181+0000 7f4acea00640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aac073230 0x7f4aac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.181+0000 7f4acea00640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aac073230 0x7f4aac075740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00ac000 tx=0x7f4ac0005040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.185+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4ac805f550 con 0x7f4ad01447b0 2024-09-17T05:03:06.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.393+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4ad0071a00 con 0x7f4ad01447b0 2024-09-17T05:03:06.398 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:03:06.397+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:03:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.397+0000 7f4ac6a00640 1 -- 172.21.15.16:0/199722318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f4ac8063f80 con 0x7f4ad01447b0 2024-09-17T05:03:06.400 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:03:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aac073230 msgr2=0x7f4aac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aac073230 0x7f4aac075740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00ac000 tx=0x7f4ac0005040 comp rx=0 tx=0).stop 2024-09-17T05:03:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 msgr2=0x7f4ad01452e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 0x7f4ad01452e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ce070 tx=0x7f4ac8002200 comp rx=0 tx=0).stop 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 shutdown_connections 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aac073230 0x7f4aac075740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad01447b0 0x7f4ad01452e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ad0143e60 0x7f4ad0144270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 --2- 172.21.15.16:0/199722318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ab4003c70 0x7f4ad01eab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 >> 172.21.15.16:0/199722318 conn(0x7f4ad00b42d0 msgr2=0x7f4ad00701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 shutdown_connections 2024-09-17T05:03:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.401+0000 7f4ad5800640 1 -- 172.21.15.16:0/199722318 wait complete. 2024-09-17T05:03:06.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:03:06.541 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:03:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.589+0000 7f86e3e00640 1 -- 172.21.15.16:0/2158723852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0d09c0 msgr2=0x7f86dc0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.589+0000 7f86e3e00640 1 --2- 172.21.15.16:0/2158723852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0d09c0 0x7f86dc0cf6e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86c8009840 tx=0x7f86c801c760 comp rx=0 tx=0).stop 2024-09-17T05:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.589+0000 7f86e3e00640 1 -- 172.21.15.16:0/2158723852 shutdown_connections 2024-09-17T05:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.589+0000 7f86e3e00640 1 --2- 172.21.15.16:0/2158723852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0d09c0 0x7f86dc0cf6e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.589+0000 7f86e3e00640 1 -- 172.21.15.16:0/2158723852 >> 172.21.15.16:0/2158723852 conn(0x7f86dc0b42d0 msgr2=0x7f86dc0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 -- 172.21.15.16:0/2158723852 shutdown_connections 2024-09-17T05:03:06.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 -- 172.21.15.16:0/2158723852 wait complete. 2024-09-17T05:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 Processor -- start 2024-09-17T05:03:06.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 -- start start 2024-09-17T05:03:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0ccb40 0x7f86dc140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc0cf2d0 0x7f86dc151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 0x7f86dc15b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc16ce60 con 0x7f86dc1fc090 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc16cce0 con 0x7f86dc0ccb40 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.593+0000 7f86e3e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc16cfe0 con 0x7f86dc0cf2d0 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 0x7f86dc15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 0x7f86dc15b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36242/0 (socket says 172.21.15.16:36242) 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 -- 172.21.15.16:0/1088892609 learned_addr learned my addr 172.21.15.16:0/1088892609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e0a00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc0cf2d0 0x7f86dc151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1400640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0ccb40 0x7f86dc140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 -- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc0cf2d0 msgr2=0x7f86dc151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc0cf2d0 0x7f86dc151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 -- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0ccb40 msgr2=0x7f86dc140170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0ccb40 0x7f86dc140170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:06.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 -- 172.21.15.16:0/1088892609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc140ae0 con 0x7f86dc1fc090 2024-09-17T05:03:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e1e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 0x7f86dc15b920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f86d400caf0 tx=0x7f86d400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4016020 con 0x7f86dc1fc090 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86d400ab60 con 0x7f86dc1fc090 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc140dc0 con 0x7f86dc1fc090 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d400ce30 con 0x7f86dc1fc090 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc1fbb70 con 0x7f86dc1fc090 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc140850 con 0x7f86dc1fc090 2024-09-17T05:03:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.597+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f86d4019020 con 0x7f86dc1fc090 2024-09-17T05:03:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.601+0000 7f86d9600640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86bc073200 0x7f86bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.601+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f86d4094290 con 0x7f86dc1fc090 2024-09-17T05:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.601+0000 7f86e1400640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86bc073200 0x7f86bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:06.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.601+0000 7f86e1400640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86bc073200 0x7f86bc075710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86d0001130 tx=0x7f86d0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:06.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.605+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f86d4020030 con 0x7f86dc1fc090 2024-09-17T05:03:06.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:06.813+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f86dc0cf6e0 con 0x7f86dc1fc090 2024-09-17T05:03:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.397+0000 7f86d9600640 1 -- 172.21.15.16:0/1088892609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f86d405f110 con 0x7f86dc1fc090 2024-09-17T05:03:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86bc073200 msgr2=0x7f86bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86bc073200 0x7f86bc075710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f86d0001130 tx=0x7f86d0009040 comp rx=0 tx=0).stop 2024-09-17T05:03:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 msgr2=0x7f86dc15b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 0x7f86dc15b920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f86d400caf0 tx=0x7f86d400cbe0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 shutdown_connections 2024-09-17T05:03:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f86bc073200 0x7f86bc075710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86dc1fc090 0x7f86dc15b920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f86dc0cf2d0 0x7f86dc151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 --2- 172.21.15.16:0/1088892609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86dc0ccb40 0x7f86dc140170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 >> 172.21.15.16:0/1088892609 conn(0x7f86dc0b42d0 msgr2=0x7f86dc155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 shutdown_connections 2024-09-17T05:03:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.401+0000 7f86e3e00640 1 -- 172.21.15.16:0/1088892609 wait complete. 2024-09-17T05:03:07.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 -- 172.21.15.16:0/2673621873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 msgr2=0x7f6b900cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 --2- 172.21.15.16:0/2673621873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900cf040 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6b900c8e10 tx=0x7f6b7801c890 comp rx=0 tx=0).stop 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 -- 172.21.15.16:0/2673621873 shutdown_connections 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 --2- 172.21.15.16:0/2673621873 >> v2:172.21.15.104:3300/0 conn(0x7f6b9015ccc0 0x7f6b9015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 --2- 172.21.15.16:0/2673621873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900cf040 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 -- 172.21.15.16:0/2673621873 >> 172.21.15.16:0/2673621873 conn(0x7f6b900715e0 msgr2=0x7f6b900719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 -- 172.21.15.16:0/2673621873 shutdown_connections 2024-09-17T05:03:07.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 -- 172.21.15.16:0/2673621873 wait complete. 2024-09-17T05:03:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 Processor -- start 2024-09-17T05:03:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.597+0000 7f6b96200640 1 -- start start 2024-09-17T05:03:07.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:07.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b900cfdf0 0x7f6b9006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:07.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9015ccc0 0x7f6b9006f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:07.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90169dc0 con 0x7f6b900cec30 2024-09-17T05:03:07.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6b90169c40 con 0x7f6b900cfdf0 2024-09-17T05:03:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6b90169f40 con 0x7f6b9015ccc0 2024-09-17T05:03:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38318/0 (socket says 172.21.15.16:38318) 2024-09-17T05:03:07.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 -- 172.21.15.16:0/568315587 learned_addr learned my addr 172.21.15.16:0/568315587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b87e00640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b900cfdf0 0x7f6b9006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8fe00640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9015ccc0 0x7f6b9006f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 -- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9015ccc0 msgr2=0x7f6b9006f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9015ccc0 0x7f6b9006f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 -- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b900cfdf0 msgr2=0x7f6b9006f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b900cfdf0 0x7f6b9006f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 -- 172.21.15.16:0/568315587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9006e810 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b87e00640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b900cfdf0 0x7f6b9006f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b8f400640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900c5000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c3f0 tx=0x7f6b8000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80017070 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b800053c0 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9015f040 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80003860 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9006fb70 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.601+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b900cf040 con 0x7f6b900cec30 2024-09-17T05:03:07.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.605+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b8001a070 con 0x7f6b900cec30 2024-09-17T05:03:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.605+0000 7f6b87400640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b6c073230 0x7f6b6c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.605+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6245+0+0 (secure 0 0 0) 0x7f6b80093730 con 0x7f6b900cec30 2024-09-17T05:03:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.605+0000 7f6b87e00640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b6c073230 0x7f6b6c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.605+0000 7f6b87e00640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b6c073230 0x7f6b6c075740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6b780af000 tx=0x7f6b7801b0d0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.609+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b900d0200 con 0x7f6b900cec30 2024-09-17T05:03:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:07.785+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6b900b3000 con 0x7f6b900cec30 2024-09-17T05:03:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.405+0000 7f6b87400640 1 -- 172.21.15.16:0/568315587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) ==== 181+0+0 (secure 0 0 0) 0x7f6b8005e520 con 0x7f6b900cec30 2024-09-17T05:03:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:03:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b6c073230 msgr2=0x7f6b6c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b6c073230 0x7f6b6c075740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6b780af000 tx=0x7f6b7801b0d0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 msgr2=0x7f6b900c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900c5000 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6b8000c3f0 tx=0x7f6b8000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 shutdown_connections 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b6c073230 0x7f6b6c075740 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b9015ccc0 0x7f6b9006f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b900cfdf0 0x7f6b9006f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 --2- 172.21.15.16:0/568315587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b900cec30 0x7f6b900c5000 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 >> 172.21.15.16:0/568315587 conn(0x7f6b900715e0 msgr2=0x7f6b900b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 shutdown_connections 2024-09-17T05:03:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.409+0000 7f6b96200640 1 -- 172.21.15.16:0/568315587 wait complete. 2024-09-17T05:03:08.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 -- 172.21.15.16:0/2523621946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 msgr2=0x7fb84006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 --2- 172.21.15.16:0/2523621946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84006dfa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b3080 tx=0x7fb83401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 -- 172.21.15.16:0/2523621946 shutdown_connections 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 --2- 172.21.15.16:0/2523621946 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb84006e4e0 0x7fb84015efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 --2- 172.21.15.16:0/2523621946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84006dfa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 -- 172.21.15.16:0/2523621946 >> 172.21.15.16:0/2523621946 conn(0x7fb8400d0050 msgr2=0x7fb8400d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 -- 172.21.15.16:0/2523621946 shutdown_connections 2024-09-17T05:03:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 -- 172.21.15.16:0/2523621946 wait complete. 2024-09-17T05:03:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 Processor -- start 2024-09-17T05:03:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.597+0000 7fb847800640 1 -- start start 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84006e4e0 0x7fb8401f9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84009d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb84015f4e0 0x7fb84015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb840074b00 con 0x7fb8400b4620 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb840074c80 con 0x7fb84006e4e0 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb840074e00 con 0x7fb84015f4e0 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84009d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84009d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38322/0 (socket says 172.21.15.16:38322) 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 -- 172.21.15.16:0/2730969811 learned_addr learned my addr 172.21.15.16:0/2730969811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb844e00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84006e4e0 0x7fb8401f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb845800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb84015f4e0 0x7fb84015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 -- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb84015f4e0 msgr2=0x7fb84015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb84015f4e0 0x7fb84015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 -- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84006e4e0 msgr2=0x7fb8401f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84006e4e0 0x7fb8401f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 -- 172.21.15.16:0/2730969811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb840164910 con 0x7fb8400b4620 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83fe00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84009d0d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0058d0 tx=0x7fb82c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c017070 con 0x7fb8400b4620 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb82c009d20 con 0x7fb8400b4620 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0106c0 con 0x7fb8400b4620 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb840164bf0 con 0x7fb8400b4620 2024-09-17T05:03:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8401651b0 con 0x7fb8400b4620 2024-09-17T05:03:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.601+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84015efa0 con 0x7fb8400b4620 2024-09-17T05:03:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.605+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb82c01a070 con 0x7fb8400b4620 2024-09-17T05:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.605+0000 7fb83cc00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb820073200 0x7fb820075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.605+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5807+0+0 (secure 0 0 0) 0x7fb82c0943c0 con 0x7fb8400b4620 2024-09-17T05:03:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.605+0000 7fb844e00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb820073200 0x7fb820075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.605+0000 7fb844e00640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb820073200 0x7fb820075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8340af000 tx=0x7fb83401b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.609+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb82c05ec90 con 0x7fb8400b4620 2024-09-17T05:03:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:08.785+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb8401510f0 con 0x7fb8400b4620 2024-09-17T05:03:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.409+0000 7fb83cc00640 1 -- 172.21.15.16:0/2730969811 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7fb82c0636c0 con 0x7fb8400b4620 2024-09-17T05:03:09.411 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb820073200 msgr2=0x7fb820075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb820073200 0x7fb820075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb8340af000 tx=0x7fb83401b0f0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 msgr2=0x7fb84009d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84009d0d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0058d0 tx=0x7fb82c00dde0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 shutdown_connections 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb820073200 0x7fb820075710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb84015f4e0 0x7fb84015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8400b4620 0x7fb84009d0d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 --2- 172.21.15.16:0/2730969811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84006e4e0 0x7fb8401f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 >> 172.21.15.16:0/2730969811 conn(0x7fb8400d0050 msgr2=0x7fb8400d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 shutdown_connections 2024-09-17T05:03:09.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.413+0000 7fb847800640 1 -- 172.21.15.16:0/2730969811 wait complete. 2024-09-17T05:03:09.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:03:09.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 -- 172.21.15.16:0/2299114372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15b680 msgr2=0x7f999c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 --2- 172.21.15.16:0/2299114372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15b680 0x7f999c15ba90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f999c15f280 tx=0x7f999401ccd0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 -- 172.21.15.16:0/2299114372 shutdown_connections 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 --2- 172.21.15.16:0/2299114372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15b680 0x7f999c15ba90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 --2- 172.21.15.16:0/2299114372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15ecc0 0x7f999c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 --2- 172.21.15.16:0/2299114372 >> v2:172.21.15.104:3300/0 conn(0x7f999c06e1d0 0x7f999c15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 -- 172.21.15.16:0/2299114372 >> 172.21.15.16:0/2299114372 conn(0x7f999c0b2250 msgr2=0x7f999c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 -- 172.21.15.16:0/2299114372 shutdown_connections 2024-09-17T05:03:09.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.597+0000 7f99a2200640 1 -- 172.21.15.16:0/2299114372 wait complete. 2024-09-17T05:03:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 Processor -- start 2024-09-17T05:03:09.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 -- start start 2024-09-17T05:03:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f999c06e1d0 0x7f999c151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15b680 0x7f999c0c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 0x7f999c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0cab50 con 0x7f999c15ecc0 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0ca9d0 con 0x7f999c06e1d0 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f999c0cacd0 con 0x7f999c15b680 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 0x7f999c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 0x7f999c0b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38350/0 (socket says 172.21.15.16:38350) 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 -- 172.21.15.16:0/1136135305 learned_addr learned my addr 172.21.15.16:0/1136135305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999b400640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f999c06e1d0 0x7f999c151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f9993e00640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15b680 0x7f999c0c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 -- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15b680 msgr2=0x7f999c0c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15b680 0x7f999c0c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 -- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f999c06e1d0 msgr2=0x7f999c151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f999c06e1d0 0x7f999c151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 -- 172.21.15.16:0/1136135305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c1f62e0 con 0x7f999c15ecc0 2024-09-17T05:03:09.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f9993e00640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15b680 0x7f999c0c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999b400640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f999c06e1d0 0x7f999c151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f999be00640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 0x7f999c0b5120 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9994009bd0 tx=0x7f9994005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940b2040 con 0x7f999c15ecc0 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9994004200 con 0x7f999c15ecc0 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c0c60a0 con 0x7f999c15ecc0 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994004610 con 0x7f999c15ecc0 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.601+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c0c45c0 con 0x7f999c15ecc0 2024-09-17T05:03:09.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.605+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c1f64b0 con 0x7f999c15ecc0 2024-09-17T05:03:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.605+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f99940ad440 con 0x7f999c15ecc0 2024-09-17T05:03:09.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.605+0000 7f9993400640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9978073200 0x7f9978075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.605+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f999412e940 con 0x7f999c15ecc0 2024-09-17T05:03:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.605+0000 7f999b400640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9978073200 0x7f9978075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.605+0000 7f999b400640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9978073200 0x7f9978075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f998c005fd0 tx=0x7f998c005e80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:09.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.609+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f999c1f64b0 con 0x7f999c15ecc0 2024-09-17T05:03:09.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.801+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f999c158b40 con 0x7f999c15ecc0 2024-09-17T05:03:09.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.805+0000 7f9993400640 1 -- 172.21.15.16:0/1136135305 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f9994016950 con 0x7f999c15ecc0 2024-09-17T05:03:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9978073200 msgr2=0x7f9978075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9978073200 0x7f9978075710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f998c005fd0 tx=0x7f998c005e80 comp rx=0 tx=0).stop 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 msgr2=0x7f999c0b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 0x7f999c0b5120 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9994009bd0 tx=0x7f9994005b10 comp rx=0 tx=0).stop 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 shutdown_connections 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9978073200 0x7f9978075710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f999c15ecc0 0x7f999c0b5120 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f999c15b680 0x7f999c0c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 --2- 172.21.15.16:0/1136135305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f999c06e1d0 0x7f999c151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 >> 172.21.15.16:0/1136135305 conn(0x7f999c0b2250 msgr2=0x7f999c158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 shutdown_connections 2024-09-17T05:03:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.809+0000 7f99a2200640 1 -- 172.21.15.16:0/1136135305 wait complete. 2024-09-17T05:03:09.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-09-17T05:03:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 -- 172.21.15.16:0/1560490316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d406ce90 msgr2=0x7fa7d406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/1560490316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d406ce90 0x7fa7d406d2a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009bf0 tx=0x7fa7cc01ca80 comp rx=0 tx=0).stop 2024-09-17T05:03:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 -- 172.21.15.16:0/1560490316 shutdown_connections 2024-09-17T05:03:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/1560490316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d406ce90 0x7fa7d406d2a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 -- 172.21.15.16:0/1560490316 >> 172.21.15.16:0/1560490316 conn(0x7fa7d40d10c0 msgr2=0x7fa7d40d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 -- 172.21.15.16:0/1560490316 shutdown_connections 2024-09-17T05:03:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.985+0000 7fa7d9c00640 1 -- 172.21.15.16:0/1560490316 wait complete. 2024-09-17T05:03:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 Processor -- start 2024-09-17T05:03:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 -- start start 2024-09-17T05:03:09.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7d406ce90 0x7fa7d4167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 0x7fa7d40cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d415ca90 0x7fa7d40cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d416b100 con 0x7fa7d4073ec0 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d416af80 con 0x7fa7d415ca90 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d9c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d416b280 con 0x7fa7d406ce90 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 0x7fa7d40cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 0x7fa7d40cabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38374/0 (socket says 172.21.15.16:38374) 2024-09-17T05:03:09.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 -- 172.21.15.16:0/3108173328 learned_addr learned my addr 172.21.15.16:0/3108173328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d3400640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d415ca90 0x7fa7d40cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2a00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7d406ce90 0x7fa7d4167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 -- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7d406ce90 msgr2=0x7fa7d4167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7d406ce90 0x7fa7d4167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 -- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d415ca90 msgr2=0x7fa7d40cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d415ca90 0x7fa7d40cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 -- 172.21.15.16:0/3108173328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d40ccf20 con 0x7fa7d4073ec0 2024-09-17T05:03:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.989+0000 7fa7d2000640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 0x7fa7d40cabb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c3c0 tx=0x7fa7c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0017070 con 0x7fa7d4073ec0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7c00053c0 con 0x7fa7d4073ec0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d41f9850 con 0x7fa7d4073ec0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0003860 con 0x7fa7d4073ec0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d4151960 con 0x7fa7d4073ec0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d4167ff0 con 0x7fa7d4073ec0 2024-09-17T05:03:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7c001a070 con 0x7fa7d4073ec0 2024-09-17T05:03:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7caa00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa7b4073230 0x7fa7b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fa7c0093390 con 0x7fa7d4073ec0 2024-09-17T05:03:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.993+0000 7fa7d2a00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa7b4073230 0x7fa7b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.997+0000 7fa7d2a00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa7b4073230 0x7fa7b4075740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4007d70 tx=0x7fa7c4007d00 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:09.997+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7d4167ff0 con 0x7fa7d4073ec0 2024-09-17T05:03:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.233+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7fa7d406d2a0 con 0x7fa7d4073ec0 2024-09-17T05:03:10.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.413+0000 7fa7caa00640 1 -- 172.21.15.16:0/3108173328 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v118) ==== 165+0+0 (secure 0 0 0) 0x7fa7c005dec0 con 0x7fa7d4073ec0 2024-09-17T05:03:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa7b4073230 msgr2=0x7fa7b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa7b4073230 0x7fa7b4075740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4007d70 tx=0x7fa7c4007d00 comp rx=0 tx=0).stop 2024-09-17T05:03:10.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 msgr2=0x7fa7d40cabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 0x7fa7d40cabb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c3c0 tx=0x7fa7c000c880 comp rx=0 tx=0).stop 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 shutdown_connections 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa7b4073230 0x7fa7b4075740 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7d415ca90 0x7fa7d40cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d4073ec0 0x7fa7d40cabb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 --2- 172.21.15.16:0/3108173328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa7d406ce90 0x7fa7d4167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 >> 172.21.15.16:0/3108173328 conn(0x7fa7d40d10c0 msgr2=0x7fa7d415e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 shutdown_connections 2024-09-17T05:03:10.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.417+0000 7fa7d9c00640 1 -- 172.21.15.16:0/3108173328 wait complete. 2024-09-17T05:03:10.440 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2024-09-17T05:03:10.441 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2024-09-17T05:03:10.441 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:03:10.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 -- 172.21.15.16:0/2685978517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f129006da50 msgr2=0x7f129006ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 --2- 172.21.15.16:0/2685978517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f129006da50 0x7f129006ce90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1280009900 tx=0x7f128001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 -- 172.21.15.16:0/2685978517 shutdown_connections 2024-09-17T05:03:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 --2- 172.21.15.16:0/2685978517 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f129006de60 0x7f12900b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 --2- 172.21.15.16:0/2685978517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f129006da50 0x7f129006ce90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 -- 172.21.15.16:0/2685978517 >> 172.21.15.16:0/2685978517 conn(0x7f129006f900 msgr2=0x7f12900b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 -- 172.21.15.16:0/2685978517 shutdown_connections 2024-09-17T05:03:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.597+0000 7f1294c00640 1 -- 172.21.15.16:0/2685978517 wait complete. 2024-09-17T05:03:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 Processor -- start 2024-09-17T05:03:10.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 -- start start 2024-09-17T05:03:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 0x7f1290143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f129006de60 0x7f12901f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12900b47b0 0x7f12901fa160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1290168980 con 0x7f129006da50 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1290168800 con 0x7f12900b47b0 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1290168b00 con 0x7f129006de60 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 0x7f1290143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 0x7f1290143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38380/0 (socket says 172.21.15.16:38380) 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 -- 172.21.15.16:0/2263279795 learned_addr learned my addr 172.21.15.16:0/2263279795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128e800640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12900b47b0 0x7f12901fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1285400640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f129006de60 0x7f12901f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 -- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f129006de60 msgr2=0x7f12901f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f129006de60 0x7f12901f9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 -- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12900b47b0 msgr2=0x7f12901fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12900b47b0 0x7f12901fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 -- 172.21.15.16:0/2263279795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12900ceb20 con 0x7f129006da50 2024-09-17T05:03:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128e800640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12900b47b0 0x7f12901fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f128de00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 0x7f1290143960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1280009850 tx=0x7f12800a55d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12800b2040 con 0x7f129006da50 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12800043b0 con 0x7f129006da50 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12900c6ce0 con 0x7f129006da50 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280005270 con 0x7f129006da50 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.601+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12900c6720 con 0x7f129006da50 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.605+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129006ce90 con 0x7f129006da50 2024-09-17T05:03:10.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.605+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f12800bd060 con 0x7f129006da50 2024-09-17T05:03:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.605+0000 7f1286a00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f126c073230 0x7f126c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.605+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f128012d8f0 con 0x7f129006da50 2024-09-17T05:03:10.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.605+0000 7f1285400640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f126c073230 0x7f126c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.605+0000 7f1285400640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f126c073230 0x7f126c075740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12901fad20 tx=0x7f12780091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:10.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.609+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12800164d0 con 0x7f129006da50 2024-09-17T05:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.785+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12900af980 con 0x7f129006da50 2024-09-17T05:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.785+0000 7f1286a00640 1 -- 172.21.15.16:0/2263279795 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13291 (secure 0 0 0) 0x7f12800f8b20 con 0x7f129006da50 2024-09-17T05:03:10.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:10.789 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":45,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:09.410236+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/594213684":"2024-09-18T05:03:06.399647+0000","172.21.15.16:6834/594213684":"2024-09-18T05:03:06.399647+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f126c073230 msgr2=0x7f126c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f126c073230 0x7f126c075740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12901fad20 tx=0x7f12780091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 msgr2=0x7f1290143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 0x7f1290143960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1280009850 tx=0x7f12800a55d0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 shutdown_connections 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f126c073230 0x7f126c075740 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12900b47b0 0x7f12901fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f129006de60 0x7f12901f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 --2- 172.21.15.16:0/2263279795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129006da50 0x7f1290143960 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 >> 172.21.15.16:0/2263279795 conn(0x7f129006f900 msgr2=0x7f12900cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 shutdown_connections 2024-09-17T05:03:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.789+0000 7f1294c00640 1 -- 172.21.15.16:0/2263279795 wait complete. 2024-09-17T05:03:10.811 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:03:10.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:03:10.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 -- 172.21.15.16:0/3076251983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074a20 msgr2=0x7f564815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 --2- 172.21.15.16:0/3076251983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074a20 0x7f564815efe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5648072860 tx=0x7f563801c880 comp rx=0 tx=0).stop 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 -- 172.21.15.16:0/3076251983 shutdown_connections 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 --2- 172.21.15.16:0/3076251983 >> v2:172.21.15.104:3300/0 conn(0x7f564815f520 0x7f564815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 --2- 172.21.15.16:0/3076251983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074a20 0x7f564815efe0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 -- 172.21.15.16:0/3076251983 >> 172.21.15.16:0/3076251983 conn(0x7f56480b2250 msgr2=0x7f56480d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 -- 172.21.15.16:0/3076251983 shutdown_connections 2024-09-17T05:03:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.969+0000 7f564d000640 1 -- 172.21.15.16:0/3076251983 wait complete. 2024-09-17T05:03:10.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 Processor -- start 2024-09-17T05:03:10.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 -- start start 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5648074a20 0x7f5648165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 0x7f5648148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f564815f520 0x7f56481fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5648168e80 con 0x7f5648074e30 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5648168d00 con 0x7f5648074a20 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5648169000 con 0x7f564815f520 2024-09-17T05:03:10.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 0x7f5648148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f5645e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5648074a20 0x7f5648165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 0x7f5648148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38392/0 (socket says 172.21.15.16:38392) 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 -- 172.21.15.16:0/1516505166 learned_addr learned my addr 172.21.15.16:0/1516505166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f5645e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5648074a20 0x7f5648165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:60012/0 (socket says 172.21.15.16:60012) 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f5646800640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f564815f520 0x7f56481fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 -- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f564815f520 msgr2=0x7f56481fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f564815f520 0x7f56481fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 -- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5648074a20 msgr2=0x7f5648165970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5648074a20 0x7f5648165970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 -- 172.21.15.16:0/1516505166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5648149360 con 0x7f5648074e30 2024-09-17T05:03:10.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563fe00640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 0x7f5648148e20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f563000b440 tx=0x7f563000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630015020 con 0x7f5648074e30 2024-09-17T05:03:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5630011920 con 0x7f5648074e30 2024-09-17T05:03:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630011d70 con 0x7f5648074e30 2024-09-17T05:03:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5648159f30 con 0x7f5648074e30 2024-09-17T05:03:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.973+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564815b260 con 0x7f5648074e30 2024-09-17T05:03:10.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.977+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564815efe0 con 0x7f5648074e30 2024-09-17T05:03:10.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.977+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5630018020 con 0x7f5648074e30 2024-09-17T05:03:10.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.977+0000 7f563e000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5624073200 0x7f5624075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:10.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.977+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f5630093ba0 con 0x7f5648074e30 2024-09-17T05:03:10.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.977+0000 7f5645e00640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5624073200 0x7f5624075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:10.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.981+0000 7f5645e00640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5624073200 0x7f5624075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56380af000 tx=0x7f56380a8040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:10.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:10.981+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f563001f030 con 0x7f5648074e30 2024-09-17T05:03:11.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.157+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56480cd490 con 0x7f5648074e30 2024-09-17T05:03:11.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.157+0000 7f563e000640 1 -- 172.21.15.16:0/1516505166 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f563005e6d0 con 0x7f5648074e30 2024-09-17T05:03:11.161 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:03:11.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5624073200 msgr2=0x7f5624075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5624073200 0x7f5624075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56380af000 tx=0x7f56380a8040 comp rx=0 tx=0).stop 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 msgr2=0x7f5648148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 0x7f5648148e20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f563000b440 tx=0x7f563000b900 comp rx=0 tx=0).stop 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 shutdown_connections 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5624073200 0x7f5624075710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f564815f520 0x7f56481fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5648074e30 0x7f5648148e20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 --2- 172.21.15.16:0/1516505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5648074a20 0x7f5648165970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 >> 172.21.15.16:0/1516505166 conn(0x7f56480b2250 msgr2=0x7f564815e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 shutdown_connections 2024-09-17T05:03:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.161+0000 7f564d000640 1 -- 172.21.15.16:0/1516505166 wait complete. 2024-09-17T05:03:11.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-09-17T05:03:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 -- 172.21.15.16:0/1783121983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed80744d0 msgr2=0x7f8ed80748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 --2- 172.21.15.16:0/1783121983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed80744d0 0x7f8ed80748e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8073ee0 tx=0x7f8ed001cd80 comp rx=0 tx=0).stop 2024-09-17T05:03:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 -- 172.21.15.16:0/1783121983 shutdown_connections 2024-09-17T05:03:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 --2- 172.21.15.16:0/1783121983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed80744d0 0x7f8ed80748e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 --2- 172.21.15.16:0/1783121983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ed815b7c0 0x7f8ed80afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 --2- 172.21.15.16:0/1783121983 >> v2:172.21.15.104:3300/0 conn(0x7f8ed806e150 0x7f8ed80af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 -- 172.21.15.16:0/1783121983 >> 172.21.15.16:0/1783121983 conn(0x7f8ed80b5300 msgr2=0x7f8ed80b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 -- 172.21.15.16:0/1783121983 shutdown_connections 2024-09-17T05:03:11.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.337+0000 7f8edee00640 1 -- 172.21.15.16:0/1783121983 wait complete. 2024-09-17T05:03:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 Processor -- start 2024-09-17T05:03:11.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 -- start start 2024-09-17T05:03:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 0x7f8ed8158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ed80744d0 0x7f8ed8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ed815b7c0 0x7f8ed80c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed80aead0 con 0x7f8ed806e150 2024-09-17T05:03:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed80ae950 con 0x7f8ed815b7c0 2024-09-17T05:03:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed80aec50 con 0x7f8ed80744d0 2024-09-17T05:03:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8ed7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 0x7f8ed8158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8ed7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 0x7f8ed8158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38420/0 (socket says 172.21.15.16:38420) 2024-09-17T05:03:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8ed7e00640 1 -- 172.21.15.16:0/1931204245 learned_addr learned my addr 172.21.15.16:0/1931204245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8ecfe00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ed80744d0 0x7f8ed8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.341+0000 7f8edce00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ed815b7c0 0x7f8ed80c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed7e00640 1 -- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ed80744d0 msgr2=0x7f8ed8159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed7e00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ed80744d0 0x7f8ed8159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed7e00640 1 -- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ed815b7c0 msgr2=0x7f8ed80c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed7e00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ed815b7c0 0x7f8ed80c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed7e00640 1 -- 172.21.15.16:0/1931204245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed80cc1f0 con 0x7f8ed806e150 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed7e00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 0x7f8ed8158d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d680 tx=0x7f8ec800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004220 con 0x7f8ed806e150 2024-09-17T05:03:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ec8009820 con 0x7f8ed806e150 2024-09-17T05:03:11.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80044b0 con 0x7f8ed806e150 2024-09-17T05:03:11.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed80cc4b0 con 0x7f8ed806e150 2024-09-17T05:03:11.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed80cfb90 con 0x7f8ed806e150 2024-09-17T05:03:11.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed80748e0 con 0x7f8ed806e150 2024-09-17T05:03:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.345+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ec8009c50 con 0x7f8ed806e150 2024-09-17T05:03:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.349+0000 7f8ed4c00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8eb4073230 0x7f8eb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.349+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f8ec80945d0 con 0x7f8ed806e150 2024-09-17T05:03:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.349+0000 7f8ecfe00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8eb4073230 0x7f8eb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.349+0000 7f8ecfe00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8eb4073230 0x7f8eb4075740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ed809a010 tx=0x7f8ec00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.353+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ec8020020 con 0x7f8ed806e150 2024-09-17T05:03:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:11.593+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f8ed80c3ac0 con 0x7f8ed806e150 2024-09-17T05:03:12.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.421+0000 7f8ed4c00640 1 -- 172.21.15.16:0/1931204245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v120) ==== 165+0+0 (secure 0 0 0) 0x7f8ec8066e20 con 0x7f8ed806e150 2024-09-17T05:03:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8eb4073230 msgr2=0x7f8eb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8eb4073230 0x7f8eb4075740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ed809a010 tx=0x7f8ec00091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 msgr2=0x7f8ed8158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 0x7f8ed8158d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d680 tx=0x7f8ec800db40 comp rx=0 tx=0).stop 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 shutdown_connections 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8eb4073230 0x7f8eb4075740 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ed815b7c0 0x7f8ed80c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ed80744d0 0x7f8ed8159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 --2- 172.21.15.16:0/1931204245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ed806e150 0x7f8ed8158d10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 >> 172.21.15.16:0/1931204245 conn(0x7f8ed80b5300 msgr2=0x7f8ed80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 shutdown_connections 2024-09-17T05:03:12.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.425+0000 7f8edee00640 1 -- 172.21.15.16:0/1931204245 wait complete. 2024-09-17T05:03:12.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:03:12.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 -- 172.21.15.16:0/3668876993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f806ce90 msgr2=0x7fa5f80cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 --2- 172.21.15.16:0/3668876993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f806ce90 0x7fa5f80cf320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009840 tx=0x7fa5e801c820 comp rx=0 tx=0).stop 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 -- 172.21.15.16:0/3668876993 shutdown_connections 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 --2- 172.21.15.16:0/3668876993 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f80cf860 0x7fa5f815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 --2- 172.21.15.16:0/3668876993 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f806ce90 0x7fa5f80cf320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 -- 172.21.15.16:0/3668876993 >> 172.21.15.16:0/3668876993 conn(0x7fa5f80d0860 msgr2=0x7fa5f80d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 -- 172.21.15.16:0/3668876993 shutdown_connections 2024-09-17T05:03:12.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.613+0000 7fa5fde00640 1 -- 172.21.15.16:0/3668876993 wait complete. 2024-09-17T05:03:12.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 Processor -- start 2024-09-17T05:03:12.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 -- start start 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f806ce90 0x7fa5f81f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 0x7fa5f8164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f815f580 0x7fa5f8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f81689e0 con 0x7fa5f80cf860 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f80b5a80 con 0x7fa5f815f580 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5fde00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f8168b60 con 0x7fa5f806ce90 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 0x7fa5f8164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 0x7fa5f8164b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38436/0 (socket says 172.21.15.16:38436) 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 -- 172.21.15.16:0/1886562293 learned_addr learned my addr 172.21.15.16:0/1886562293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:12.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6a00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f806ce90 0x7fa5f81f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f7400640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f815f580 0x7fa5f8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 -- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f806ce90 msgr2=0x7fa5f81f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f806ce90 0x7fa5f81f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 -- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f815f580 msgr2=0x7fa5f8157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f815f580 0x7fa5f8157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6000640 1 -- 172.21.15.16:0/1886562293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f8153bf0 con 0x7fa5f80cf860 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.617+0000 7fa5f6a00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f806ce90 0x7fa5f81f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:12.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5f6000640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 0x7fa5f8164b40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e7f0 tx=0x7fa5e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cc60 con 0x7fa5f80cf860 2024-09-17T05:03:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e400ce00 con 0x7fa5f80cf860 2024-09-17T05:03:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f8153ed0 con 0x7fa5f80cf860 2024-09-17T05:03:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005280 con 0x7fa5f80cf860 2024-09-17T05:03:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f8154490 con 0x7fa5f80cf860 2024-09-17T05:03:12.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f80cf320 con 0x7fa5f80cf860 2024-09-17T05:03:12.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.621+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa5e4005420 con 0x7fa5f80cf860 2024-09-17T05:03:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.625+0000 7fa5ef400640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa5d8073200 0x7fa5d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.625+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fa5e4094a70 con 0x7fa5f80cf860 2024-09-17T05:03:12.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.625+0000 7fa5f6a00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa5d8073200 0x7fa5d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.625+0000 7fa5f6a00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa5d8073200 0x7fa5d8075710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80af000 tx=0x7fa5e801b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:12.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.629+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa5e405fbf0 con 0x7fa5f80cf860 2024-09-17T05:03:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa5f80b05b0 con 0x7fa5f80cf860 2024-09-17T05:03:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5ef400640 1 -- 172.21.15.16:0/1886562293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7fa5e4094e20 con 0x7fa5f80cf860 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:03:12.808 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:03:12.809 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:03:12.810 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa5d8073200 msgr2=0x7fa5d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa5d8073200 0x7fa5d8075710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80af000 tx=0x7fa5e801b060 comp rx=0 tx=0).stop 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 msgr2=0x7fa5f8164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 0x7fa5f8164b40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e7f0 tx=0x7fa5e400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 shutdown_connections 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa5d8073200 0x7fa5d8075710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5f815f580 0x7fa5f8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5f80cf860 0x7fa5f8164b40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 --2- 172.21.15.16:0/1886562293 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa5f806ce90 0x7fa5f81f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.805+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 >> 172.21.15.16:0/1886562293 conn(0x7fa5f80d0860 msgr2=0x7fa5f815f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.809+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 shutdown_connections 2024-09-17T05:03:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.809+0000 7fa5fde00640 1 -- 172.21.15.16:0/1886562293 wait complete. 2024-09-17T05:03:12.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:03:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 -- 172.21.15.16:0/243162385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5015f170 msgr2=0x7fbc5015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 --2- 172.21.15.16:0/243162385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5015f170 0x7fbc5015f580 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbc5006e940 tx=0x7fbc4801cd80 comp rx=0 tx=0).stop 2024-09-17T05:03:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 -- 172.21.15.16:0/243162385 shutdown_connections 2024-09-17T05:03:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 --2- 172.21.15.16:0/243162385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5015f170 0x7fbc5015f580 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 --2- 172.21.15.16:0/243162385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc500c5400 0x7fbc500c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 --2- 172.21.15.16:0/243162385 >> v2:172.21.15.104:3300/0 conn(0x7fbc5006ce90 0x7fbc5015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 -- 172.21.15.16:0/243162385 >> 172.21.15.16:0/243162385 conn(0x7fbc500ccfe0 msgr2=0x7fbc500cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 -- 172.21.15.16:0/243162385 shutdown_connections 2024-09-17T05:03:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.989+0000 7fbc54a00640 1 -- 172.21.15.16:0/243162385 wait complete. 2024-09-17T05:03:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 Processor -- start 2024-09-17T05:03:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 -- start start 2024-09-17T05:03:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5006ce90 0x7fbc50141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 0x7fbc500b5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5015f170 0x7fbc50150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5006e2c0 con 0x7fbc500c5400 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5006e140 con 0x7fbc5006ce90 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc54a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5006e440 con 0x7fbc5015f170 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 0x7fbc500b5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 0x7fbc500b5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38456/0 (socket says 172.21.15.16:38456) 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 -- 172.21.15.16:0/3763308483 learned_addr learned my addr 172.21.15.16:0/3763308483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc4e800640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5015f170 0x7fbc50150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc4de00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5006ce90 0x7fbc50141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 -- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5015f170 msgr2=0x7fbc50150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5015f170 0x7fbc50150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 -- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5006ce90 msgr2=0x7fbc50141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5006ce90 0x7fbc50141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 -- 172.21.15.16:0/3763308483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc501416a0 con 0x7fbc500c5400 2024-09-17T05:03:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.993+0000 7fbc47e00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 0x7fbc500b5ea0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d6a0 tx=0x7fbc3800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38017070 con 0x7fbc500c5400 2024-09-17T05:03:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc38005310 con 0x7fbc500c5400 2024-09-17T05:03:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc500b4e00 con 0x7fbc500c5400 2024-09-17T05:03:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38009820 con 0x7fbc500c5400 2024-09-17T05:03:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc500b53c0 con 0x7fbc500c5400 2024-09-17T05:03:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc500c5810 con 0x7fbc500c5400 2024-09-17T05:03:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc380099c0 con 0x7fbc500c5400 2024-09-17T05:03:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc46000640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc2c073200 0x7fbc2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:13.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:12.997+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fbc38094130 con 0x7fbc500c5400 2024-09-17T05:03:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.001+0000 7fbc4de00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc2c073200 0x7fbc2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.001+0000 7fbc4de00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc2c073200 0x7fbc2c075710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004610 tx=0x7fbc4000b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:13.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.001+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc3805ec60 con 0x7fbc500c5400 2024-09-17T05:03:13.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.181+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbc50150b80 con 0x7fbc500c5400 2024-09-17T05:03:13.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc46000640 1 -- 172.21.15.16:0/3763308483 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fbc38063690 con 0x7fbc500c5400 2024-09-17T05:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc2c073200 msgr2=0x7fbc2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc2c073200 0x7fbc2c075710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004610 tx=0x7fbc4000b260 comp rx=0 tx=0).stop 2024-09-17T05:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 msgr2=0x7fbc500b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 0x7fbc500b5ea0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800d6a0 tx=0x7fbc3800db60 comp rx=0 tx=0).stop 2024-09-17T05:03:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 shutdown_connections 2024-09-17T05:03:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc2c073200 0x7fbc2c075710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5015f170 0x7fbc50150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc500c5400 0x7fbc500b5ea0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 --2- 172.21.15.16:0/3763308483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5006ce90 0x7fbc50141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 >> 172.21.15.16:0/3763308483 conn(0x7fbc500ccfe0 msgr2=0x7fbc50068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:13.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 shutdown_connections 2024-09-17T05:03:13.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.185+0000 7fbc54a00640 1 -- 172.21.15.16:0/3763308483 wait complete. 2024-09-17T05:03:13.210 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:03:13.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:13.328 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:13.415 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:03:13.415 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:13.415 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:13.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:03:13.425 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:13.425 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:13.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:03:13.435 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:13.435 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:13.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:03:13.475 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:13.475 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:13.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:03:13.485 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:03:13.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 -- 172.21.15.16:0/1785813425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06e7a0 msgr2=0x7f900c0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 --2- 172.21.15.16:0/1785813425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06e7a0 0x7f900c0b40d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f900c06e490 tx=0x7f8ff401c810 comp rx=0 tx=0).stop 2024-09-17T05:03:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 -- 172.21.15.16:0/1785813425 shutdown_connections 2024-09-17T05:03:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 --2- 172.21.15.16:0/1785813425 >> v2:172.21.15.104:3300/0 conn(0x7f900c06ebb0 0x7f900c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 --2- 172.21.15.16:0/1785813425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06e7a0 0x7f900c0b40d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 -- 172.21.15.16:0/1785813425 >> 172.21.15.16:0/1785813425 conn(0x7f900c0d10c0 msgr2=0x7f900c0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 -- 172.21.15.16:0/1785813425 shutdown_connections 2024-09-17T05:03:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 -- 172.21.15.16:0/1785813425 wait complete. 2024-09-17T05:03:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 Processor -- start 2024-09-17T05:03:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.673+0000 7f9011000640 1 -- start start 2024-09-17T05:03:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f900c06e7a0 0x7f900c143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 0x7f900c160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f900c152540 0x7f900c1605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1692d0 con 0x7f900c06ebb0 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f900c169150 con 0x7f900c06e7a0 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f900c169450 con 0x7f900c152540 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 0x7f900c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 0x7f900c160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38476/0 (socket says 172.21.15.16:38476) 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 -- 172.21.15.16:0/3863035967 learned_addr learned my addr 172.21.15.16:0/3863035967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f900a800640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f900c152540 0x7f900c1605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9009e00640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f900c06e7a0 0x7f900c143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 -- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f900c152540 msgr2=0x7f900c1605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f900c152540 0x7f900c1605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 -- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f900c06e7a0 msgr2=0x7f900c143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f900c06e7a0 0x7f900c143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 -- 172.21.15.16:0/3863035967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c1625e0 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9003e00640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 0x7f900c160070 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40b0040 tx=0x7f8ff40a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40b2040 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ff40042c0 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c1628c0 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40052d0 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c162e80 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.677+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c162350 con 0x7f900c06ebb0 2024-09-17T05:03:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.681+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ff4005470 con 0x7f900c06ebb0 2024-09-17T05:03:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.681+0000 7f9002000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8fe8073200 0x7f8fe8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:13.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.681+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f8ff412d780 con 0x7f900c06ebb0 2024-09-17T05:03:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.681+0000 7f9009e00640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8fe8073200 0x7f8fe8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:13.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.681+0000 7f9009e00640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8fe8073200 0x7f8fe8075710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00b810 tx=0x7f8ffc008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:13.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.685+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ff4016490 con 0x7f900c06ebb0 2024-09-17T05:03:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.893+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f900c074470 con 0x7f900c06ebb0 2024-09-17T05:03:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.893+0000 7f9002000640 1 -- 172.21.15.16:0/3863035967 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3911 (secure 0 0 0) 0x7f8ff40044b0 con 0x7f900c06ebb0 2024-09-17T05:03:13.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:13.897 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"btime":"2024-09-17T05:03:07:401826+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4727,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/702589662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":702589662},{"type":"v1","addr":"172.21.15.104:6837","nonce":702589662}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/945498473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":945498473},{"type":"v1","addr":"172.21.15.104:6836","nonce":945498473}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5025,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3545540808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3545540808},{"type":"v1","addr":"172.21.15.16:6833","nonce":3545540808}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5373,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2676122255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2676122255},{"type":"v1","addr":"172.21.15.16:6835","nonce":2676122255}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-17T05:03:13.897 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 19 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8fe8073200 msgr2=0x7f8fe8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8fe8073200 0x7f8fe8075710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00b810 tx=0x7f8ffc008040 comp rx=0 tx=0).stop 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 msgr2=0x7f900c160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 0x7f900c160070 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40b0040 tx=0x7f8ff40a5620 comp rx=0 tx=0).stop 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 shutdown_connections 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8fe8073200 0x7f8fe8075710 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f900c152540 0x7f900c1605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f900c06ebb0 0x7f900c160070 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 --2- 172.21.15.16:0/3863035967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f900c06e7a0 0x7f900c143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 >> 172.21.15.16:0/3863035967 conn(0x7f900c0d10c0 msgr2=0x7f900c1653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 shutdown_connections 2024-09-17T05:03:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:13.897+0000 7f9011000640 1 -- 172.21.15.16:0/3863035967 wait complete. 2024-09-17T05:03:13.920 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:03:13.920 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:03:13.921 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:03:13.921 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:13.921 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:03:13.921 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:03:13.921 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:03:13.921 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:13.922 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:03:13.922 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:03:13.922 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:03:13.922 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:13.922 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:03:13.922 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:03:13.922 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:03:13.923 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:13.933 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:03:13.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:03:13.937 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:03:13.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:03:13.937 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:03:13.938 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:03:13.942 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:03:13.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:03:13.987 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:03:13.988 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:13.988 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:03:13.989 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:13.989 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:03:13.989 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:13.990 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:03:13.991 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:13.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:03:14.089 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:03:14.091 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:03:14.092 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:03:14.092 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 -- 172.21.15.16:0/3436869830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 msgr2=0x7fbd38004c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3436869830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd38004c60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009b10 tx=0x7fbd3c01d220 comp rx=0 tx=0).stop 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd55e00640 1 -- 172.21.15.16:0/3436869830 reap_dead start 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 -- 172.21.15.16:0/3436869830 shutdown_connections 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 -- 172.21.15.16:0/3436869830 >> 172.21.15.16:0/3436869830 conn(0x7fbd500b42d0 msgr2=0x7fbd500b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 -- 172.21.15.16:0/3436869830 shutdown_connections 2024-09-17T05:03:14.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 -- 172.21.15.16:0/3436869830 wait complete. 2024-09-17T05:03:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 Processor -- start 2024-09-17T05:03:14.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.157+0000 7fbd57e00640 1 -- start start 2024-09-17T05:03:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd500cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd50158600 0x7fbd50162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd50158a10 0x7fbd50162580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50168e20 con 0x7fbd38004810 2024-09-17T05:03:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd50168ca0 con 0x7fbd50158a10 2024-09-17T05:03:14.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbd50168fa0 con 0x7fbd50158600 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd500cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd500cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38536/0 (socket says 172.21.15.16:38536) 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 -- 172.21.15.16:0/3454865681 learned_addr learned my addr 172.21.15.16:0/3454865681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd54a00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd50158600 0x7fbd50162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd50158a10 0x7fbd50162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 -- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd50158600 msgr2=0x7fbd50162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd50158600 0x7fbd50162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 -- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd50158a10 msgr2=0x7fbd50162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd50158a10 0x7fbd50162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 -- 172.21.15.16:0/3454865681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd500c5a80 con 0x7fbd38004810 2024-09-17T05:03:14.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd55400640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd500cb830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd44003f20 tx=0x7fbd440018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44015070 con 0x7fbd38004810 2024-09-17T05:03:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd44011920 con 0x7fbd38004810 2024-09-17T05:03:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44011d70 con 0x7fbd38004810 2024-09-17T05:03:14.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd500c5ef0 con 0x7fbd38004810 2024-09-17T05:03:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd500c64b0 con 0x7fbd38004810 2024-09-17T05:03:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.161+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd500b2100 con 0x7fbd38004810 2024-09-17T05:03:14.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.165+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbd4400f430 con 0x7fbd38004810 2024-09-17T05:03:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.165+0000 7fbd4d600640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd30073230 0x7fbd30075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.165+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fbd44094cb0 con 0x7fbd38004810 2024-09-17T05:03:14.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.165+0000 7fbd54a00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd30073230 0x7fbd30075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.165+0000 7fbd54a00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd30073230 0x7fbd30075740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0ad000 tx=0x7fbd3c019fc0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.169+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd4405f7e0 con 0x7fbd38004810 2024-09-17T05:03:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.353+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbd500c6b70 con 0x7fbd38004810 2024-09-17T05:03:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.433+0000 7fbd4d600640 1 -- 172.21.15.16:0/3454865681 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) ==== 97+0+0 (secure 0 0 0) 0x7fbd44064210 con 0x7fbd38004810 2024-09-17T05:03:14.437 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:03:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd30073230 msgr2=0x7fbd30075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd30073230 0x7fbd30075740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0ad000 tx=0x7fbd3c019fc0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 msgr2=0x7fbd500cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd500cb830 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd44003f20 tx=0x7fbd440018d0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 shutdown_connections 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd30073230 0x7fbd30075740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd50158a10 0x7fbd50162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd50158600 0x7fbd50162040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 --2- 172.21.15.16:0/3454865681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd38004810 0x7fbd500cb830 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 >> 172.21.15.16:0/3454865681 conn(0x7fbd500b42d0 msgr2=0x7fbd50165540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 shutdown_connections 2024-09-17T05:03:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.437+0000 7fbd57e00640 1 -- 172.21.15.16:0/3454865681 wait complete. 2024-09-17T05:03:14.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:03:14.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 -- 172.21.15.16:0/1124853887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf10073240 msgr2=0x7faf10073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 --2- 172.21.15.16:0/1124853887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf10073240 0x7faf10073650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faf08009bc0 tx=0x7faf0801cb30 comp rx=0 tx=0).stop 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 -- 172.21.15.16:0/1124853887 shutdown_connections 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 --2- 172.21.15.16:0/1124853887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf10073240 0x7faf10073650 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 --2- 172.21.15.16:0/1124853887 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf100b2f50 0x7faf1015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 -- 172.21.15.16:0/1124853887 >> 172.21.15.16:0/1124853887 conn(0x7faf1006f900 msgr2=0x7faf100b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 -- 172.21.15.16:0/1124853887 shutdown_connections 2024-09-17T05:03:14.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 -- 172.21.15.16:0/1124853887 wait complete. 2024-09-17T05:03:14.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 Processor -- start 2024-09-17T05:03:14.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.625+0000 7faf17200640 1 -- start start 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 0x7faf101fca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf100b2f50 0x7faf100af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1015c0a0 0x7faf101515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf10168e00 con 0x7faf10073240 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf10168c80 con 0x7faf1015c0a0 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7faf10168f80 con 0x7faf100b2f50 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 0x7faf101fca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 0x7faf101fca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38550/0 (socket says 172.21.15.16:38550) 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 -- 172.21.15.16:0/280238005 learned_addr learned my addr 172.21.15.16:0/280238005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf07400640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf100b2f50 0x7faf100af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf15200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1015c0a0 0x7faf101515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 -- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf100b2f50 msgr2=0x7faf100af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf100b2f50 0x7faf100af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 -- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1015c0a0 msgr2=0x7faf101515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1015c0a0 0x7faf101515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 -- 172.21.15.16:0/280238005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf10162f70 con 0x7faf10073240 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0fe00640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 0x7faf101fca40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf0000e9f0 tx=0x7faf0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0000cd80 con 0x7faf10073240 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf10163250 con 0x7faf10073240 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf00004020 con 0x7faf10073240 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00004470 con 0x7faf10073240 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf101fc520 con 0x7faf10073240 2024-09-17T05:03:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.629+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf10163490 con 0x7faf10073240 2024-09-17T05:03:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.633+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf00003600 con 0x7faf10073240 2024-09-17T05:03:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.633+0000 7faf0cc00640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faeec073200 0x7faeec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.633+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7faf00094b30 con 0x7faf10073240 2024-09-17T05:03:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.633+0000 7faf07400640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faeec073200 0x7faeec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:14.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.633+0000 7faf07400640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faeec073200 0x7faeec075710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf10165540 tx=0x7faef8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.637+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf0005fbf0 con 0x7faf10073240 2024-09-17T05:03:14.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.869+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7faf101fc030 con 0x7faf10073240 2024-09-17T05:03:14.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.869+0000 7faf0cc00640 1 -- 172.21.15.16:0/280238005 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5304 (secure 0 0 0) 0x7faf0005f4f0 con 0x7faf10073240 2024-09-17T05:03:14.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:14.872 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:03:14.872 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:03:14.873 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.874 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.875 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:03:14.876 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.877 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:03:14.878 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.879 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.880 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.882 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.883 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:03:14.883 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:03:14.883 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.883 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.883 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:03:14.884 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:03:14.884 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:03:14.884 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.884 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:03:14.885 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.886 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:03:14.887 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:03:14.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faeec073200 msgr2=0x7faeec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faeec073200 0x7faeec075710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf10165540 tx=0x7faef8009040 comp rx=0 tx=0).stop 2024-09-17T05:03:14.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 msgr2=0x7faf101fca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 0x7faf101fca40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf0000e9f0 tx=0x7faf0000edf0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 shutdown_connections 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faeec073200 0x7faeec075710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf1015c0a0 0x7faf101515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf100b2f50 0x7faf100af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 --2- 172.21.15.16:0/280238005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf10073240 0x7faf101fca40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 >> 172.21.15.16:0/280238005 conn(0x7faf1006f900 msgr2=0x7faf1015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 shutdown_connections 2024-09-17T05:03:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:14.873+0000 7faf17200640 1 -- 172.21.15.16:0/280238005 wait complete. 2024-09-17T05:03:14.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 -- 172.21.15.16:0/1656965538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f9415a050 msgr2=0x7f7f940cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 --2- 172.21.15.16:0/1656965538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f9415a050 0x7f7f940cf420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009900 tx=0x7f7f8401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 -- 172.21.15.16:0/1656965538 shutdown_connections 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 --2- 172.21.15.16:0/1656965538 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f940b1c40 0x7f7f940b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 --2- 172.21.15.16:0/1656965538 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f9415a050 0x7f7f940cf420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 -- 172.21.15.16:0/1656965538 >> 172.21.15.16:0/1656965538 conn(0x7f7f940d2240 msgr2=0x7f7f940d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 -- 172.21.15.16:0/1656965538 shutdown_connections 2024-09-17T05:03:15.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 -- 172.21.15.16:0/1656965538 wait complete. 2024-09-17T05:03:15.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 Processor -- start 2024-09-17T05:03:15.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.053+0000 7f7f99800640 1 -- start start 2024-09-17T05:03:15.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f940b1c40 0x7f7f9413e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 0x7f7f94140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f9415a050 0x7f7f94141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9416a250 con 0x7f7f940b6a90 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f941460f0 con 0x7f7f940b1c40 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7f9416a3d0 con 0x7f7f9415a050 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 0x7f7f94140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 0x7f7f94140ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38552/0 (socket says 172.21.15.16:38552) 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 -- 172.21.15.16:0/3544428965 learned_addr learned my addr 172.21.15.16:0/3544428965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f92a00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f940b1c40 0x7f7f9413e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f93400640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f9415a050 0x7f7f94141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 -- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f9415a050 msgr2=0x7f7f94141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f9415a050 0x7f7f94141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 -- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f940b1c40 msgr2=0x7f7f9413e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f940b1c40 0x7f7f9413e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 -- 172.21.15.16:0/3544428965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f94153460 con 0x7f7f940b6a90 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8be00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 0x7f7f94140ae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00e7a0 tx=0x7f7f7c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c00ccc0 con 0x7f7f940b6a90 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f7c010430 con 0x7f7f940b6a90 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c010860 con 0x7f7f940b6a90 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f94153740 con 0x7f7f940b6a90 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f94153d00 con 0x7f7f940b6a90 2024-09-17T05:03:15.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f940b2050 con 0x7f7f940b6a90 2024-09-17T05:03:15.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.057+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7f7c010a00 con 0x7f7f940b6a90 2024-09-17T05:03:15.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.061+0000 7f7f8aa00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f70073200 0x7f7f70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.061+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f7f7c094a50 con 0x7f7f940b6a90 2024-09-17T05:03:15.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.061+0000 7f7f92a00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f70073200 0x7f7f70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.061+0000 7f7f92a00640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f70073200 0x7f7f70075710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f840af000 tx=0x7f7f8401b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:15.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.065+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f7c05f5d0 con 0x7f7f940b6a90 2024-09-17T05:03:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.241+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7f941428c0 con 0x7f7f940b6a90 2024-09-17T05:03:15.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.241+0000 7f7f8aa00640 1 -- 172.21.15.16:0/3544428965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) ==== 84+0+65 (secure 0 0 0) 0x7f7f7c064000 con 0x7f7f940b6a90 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:15.245 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:03:15.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f70073200 msgr2=0x7f7f70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f70073200 0x7f7f70075710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7f840af000 tx=0x7f7f8401b0f0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 msgr2=0x7f7f94140ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 0x7f7f94140ae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00e7a0 tx=0x7f7f7c00ec60 comp rx=0 tx=0).stop 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 shutdown_connections 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f70073200 0x7f7f70075710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f9415a050 0x7f7f94141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f940b6a90 0x7f7f94140ae0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 --2- 172.21.15.16:0/3544428965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f940b1c40 0x7f7f9413e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 >> 172.21.15.16:0/3544428965 conn(0x7f7f940d2240 msgr2=0x7f7f94159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 shutdown_connections 2024-09-17T05:03:15.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.245+0000 7f7f99800640 1 -- 172.21.15.16:0/3544428965 wait complete. 2024-09-17T05:03:15.267 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:03:15.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:03:15.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 -- 172.21.15.16:0/1586884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 msgr2=0x7fb6ac074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 --2- 172.21.15.16:0/1586884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac074530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac073800 tx=0x7fb6a401cd50 comp rx=0 tx=0).stop 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 -- 172.21.15.16:0/1586884830 shutdown_connections 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 --2- 172.21.15.16:0/1586884830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac074530 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 --2- 172.21.15.16:0/1586884830 >> v2:172.21.15.104:3300/0 conn(0x7fb6ac06e6b0 0x7fb6ac073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 -- 172.21.15.16:0/1586884830 >> 172.21.15.16:0/1586884830 conn(0x7fb6ac0d0050 msgr2=0x7fb6ac0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 -- 172.21.15.16:0/1586884830 shutdown_connections 2024-09-17T05:03:15.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 -- 172.21.15.16:0/1586884830 wait complete. 2024-09-17T05:03:15.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 Processor -- start 2024-09-17T05:03:15.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.429+0000 7fb6b3600640 1 -- start start 2024-09-17T05:03:15.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac13eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b3600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6ac0b4620 0x7fb6ac152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b3600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb6ac14b270 0x7fb6ac152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac168b30 con 0x7fb6ac074120 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b3600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ac152890 con 0x7fb6ac0b4620 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b3600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ac168cb0 con 0x7fb6ac14b270 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac13eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac13eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38570/0 (socket says 172.21.15.16:38570) 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 -- 172.21.15.16:0/3704255955 learned_addr learned my addr 172.21.15.16:0/3704255955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6abe00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6ac0b4620 0x7fb6ac152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b1600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb6ac14b270 0x7fb6ac152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 -- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb6ac14b270 msgr2=0x7fb6ac152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb6ac14b270 0x7fb6ac152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 -- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6ac0b4620 msgr2=0x7fb6ac152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6ac0b4620 0x7fb6ac152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 -- 172.21.15.16:0/3704255955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ac14fc30 con 0x7fb6ac074120 2024-09-17T05:03:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.433+0000 7fb6b0c00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac13eb90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c3d0 tx=0x7fb6a000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.437+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0017070 con 0x7fb6ac074120 2024-09-17T05:03:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.437+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ac14ff10 con 0x7fb6ac074120 2024-09-17T05:03:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.437+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ac1504d0 con 0x7fb6ac074120 2024-09-17T05:03:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.437+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6a00052d0 con 0x7fb6ac074120 2024-09-17T05:03:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.437+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0003860 con 0x7fb6ac074120 2024-09-17T05:03:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.437+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6880052a0 con 0x7fb6ac074120 2024-09-17T05:03:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.441+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb6a001a070 con 0x7fb6ac074120 2024-09-17T05:03:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.441+0000 7fb6a8c00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb68c073230 0x7fb68c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.441+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7fb6a0093300 con 0x7fb6ac074120 2024-09-17T05:03:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.441+0000 7fb6abe00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb68c073230 0x7fb68c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.441+0000 7fb6abe00640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb68c073230 0x7fb68c075740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0b17b0 tx=0x7fb6980091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:15.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.445+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb6a0005480 con 0x7fb6ac074120 2024-09-17T05:03:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:15.625+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb688005590 con 0x7fb6ac074120 2024-09-17T05:03:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.445+0000 7fb6a8c00640 1 -- 172.21.15.16:0/3704255955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) ==== 138+0+0 (secure 0 0 0) 0x7fb6a005de80 con 0x7fb6ac074120 2024-09-17T05:03:16.449 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:03:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb68c073230 msgr2=0x7fb68c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb68c073230 0x7fb68c075740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0b17b0 tx=0x7fb6980091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 msgr2=0x7fb6ac13eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac13eb90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000c3d0 tx=0x7fb6a000c890 comp rx=0 tx=0).stop 2024-09-17T05:03:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 shutdown_connections 2024-09-17T05:03:16.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb68c073230 0x7fb68c075740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb6ac14b270 0x7fb6ac152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6ac0b4620 0x7fb6ac152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 --2- 172.21.15.16:0/3704255955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ac074120 0x7fb6ac13eb90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 >> 172.21.15.16:0/3704255955 conn(0x7fb6ac0d0050 msgr2=0x7fb6ac159970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 shutdown_connections 2024-09-17T05:03:16.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.449+0000 7fb6b3600640 1 -- 172.21.15.16:0/3704255955 wait complete. 2024-09-17T05:03:16.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 -- 172.21.15.16:0/1898999625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815c0a0 msgr2=0x7fabe815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 --2- 172.21.15.16:0/1898999625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815c0a0 0x7fabe815c4b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fabe8071ce0 tx=0x7fabd001c830 comp rx=0 tx=0).stop 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 -- 172.21.15.16:0/1898999625 shutdown_connections 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 --2- 172.21.15.16:0/1898999625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815c0a0 0x7fabe815c4b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 --2- 172.21.15.16:0/1898999625 >> v2:172.21.15.104:3300/0 conn(0x7fabe80b2f50 0x7fabe815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 -- 172.21.15.16:0/1898999625 >> 172.21.15.16:0/1898999625 conn(0x7fabe806f900 msgr2=0x7fabe80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 -- 172.21.15.16:0/1898999625 shutdown_connections 2024-09-17T05:03:16.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.649+0000 7fabef000640 1 -- 172.21.15.16:0/1898999625 wait complete. 2024-09-17T05:03:16.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 Processor -- start 2024-09-17T05:03:16.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 -- start start 2024-09-17T05:03:16.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fabe80b2f50 0x7fabe8148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe815c0a0 0x7fabe8144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 0x7fabe81fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabe81690f0 con 0x7fabe815f150 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8168f70 con 0x7fabe815c0a0 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabef000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fabe8169270 con 0x7fabe80b2f50 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 0x7fabe81fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabe7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fabe80b2f50 0x7fabe8148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabdfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe815c0a0 0x7fabe8144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabe7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fabe80b2f50 0x7fabe8148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:54478/0 (socket says 172.21.15.16:54478) 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 0x7fabe81fa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38590/0 (socket says 172.21.15.16:38590) 2024-09-17T05:03:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabe7e00640 1 -- 172.21.15.16:0/3134668163 learned_addr learned my addr 172.21.15.16:0/3134668163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 -- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fabe80b2f50 msgr2=0x7fabe8148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fabe80b2f50 0x7fabe8148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 -- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe815c0a0 msgr2=0x7fabe8144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe815c0a0 0x7fabe8144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 -- 172.21.15.16:0/3134668163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe81f8120 con 0x7fabe815f150 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabdfe00640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe815c0a0 0x7fabe8144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabed000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 0x7fabe81fa5e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fabe81561f0 tx=0x7fabe000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0003b80 con 0x7fabe815f150 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.653+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabe0003d20 con 0x7fabe815f150 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe814b750 con 0x7fabe815f150 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0005330 con 0x7fabe815f150 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe814bff0 con 0x7fabe815f150 2024-09-17T05:03:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe81f82f0 con 0x7fabe815f150 2024-09-17T05:03:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fabe00054d0 con 0x7fabe815f150 2024-09-17T05:03:16.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabe4c00640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fabc4073230 0x7fabc4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:16.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5619+0+0 (secure 0 0 0) 0x7fabe0094940 con 0x7fabe815f150 2024-09-17T05:03:16.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.657+0000 7fabe7e00640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fabc4073230 0x7fabc4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.661+0000 7fabe7e00640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fabc4073230 0x7fabc4075740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009c40 tx=0x7fabd80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.661+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fabe005f2c0 con 0x7fabe815f150 2024-09-17T05:03:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:16.833+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fabe814b4c0 con 0x7fabe815f150 2024-09-17T05:03:17.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.449+0000 7fabe4c00640 1 -- 172.21.15.16:0/3134668163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) ==== 170+0+0 (secure 0 0 0) 0x7fabe0063cf0 con 0x7fabe815f150 2024-09-17T05:03:17.453 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:03:17.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fabc4073230 msgr2=0x7fabc4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fabc4073230 0x7fabc4075740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009c40 tx=0x7fabd80091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 msgr2=0x7fabe81fa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 0x7fabe81fa5e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fabe81561f0 tx=0x7fabe000cae0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 shutdown_connections 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fabc4073230 0x7fabc4075740 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabe815f150 0x7fabe81fa5e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabe815c0a0 0x7fabe8144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 --2- 172.21.15.16:0/3134668163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fabe80b2f50 0x7fabe8148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 >> 172.21.15.16:0/3134668163 conn(0x7fabe806f900 msgr2=0x7fabe815e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 shutdown_connections 2024-09-17T05:03:17.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.453+0000 7fabef000640 1 -- 172.21.15.16:0/3134668163 wait complete. 2024-09-17T05:03:17.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 -- 172.21.15.16:0/2442140075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 msgr2=0x7efee81657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 --2- 172.21.15.16:0/2442140075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 0x7efee81657c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efee000ad00 tx=0x7efee001dda0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 -- 172.21.15.16:0/2442140075 shutdown_connections 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 --2- 172.21.15.16:0/2442140075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 0x7efee81657c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 -- 172.21.15.16:0/2442140075 >> 172.21.15.16:0/2442140075 conn(0x7efee80b2250 msgr2=0x7efee80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 -- 172.21.15.16:0/2442140075 shutdown_connections 2024-09-17T05:03:17.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.661+0000 7efeeee00640 1 -- 172.21.15.16:0/2442140075 wait complete. 2024-09-17T05:03:17.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 Processor -- start 2024-09-17T05:03:17.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 -- start start 2024-09-17T05:03:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efee8073610 0x7efee8144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 0x7efee8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 0x7efee81f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efee8168a70 con 0x7efee8073a20 2024-09-17T05:03:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efee81688f0 con 0x7efee8073610 2024-09-17T05:03:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeeee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efee8168bf0 con 0x7efee80cbaa0 2024-09-17T05:03:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efedf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 0x7efee8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efedf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 0x7efee8155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45668/0 (socket says 172.21.15.16:45668) 2024-09-17T05:03:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efedf400640 1 -- 172.21.15.16:0/1368597995 learned_addr learned my addr 172.21.15.16:0/1368597995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efeece00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 0x7efee81f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.665+0000 7efee7e00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efee8073610 0x7efee8144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efedf400640 1 -- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 msgr2=0x7efee81f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efedf400640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 0x7efee81f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efedf400640 1 -- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efee8073610 msgr2=0x7efee8144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efedf400640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efee8073610 0x7efee8144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efedf400640 1 -- 172.21.15.16:0/1368597995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee81441d0 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efedf400640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 0x7efee8155bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efed000c4e0 tx=0x7efed000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0016020 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efed0012920 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee81f9d20 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0012d50 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81fa510 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee81f9fa0 con 0x7efee8073a20 2024-09-17T05:03:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.669+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efed0019020 con 0x7efee8073a20 2024-09-17T05:03:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.673+0000 7efee4c00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efec4073230 0x7efec4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.673+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6001+0+0 (secure 0 0 0) 0x7efed0094790 con 0x7efee8073a20 2024-09-17T05:03:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.673+0000 7efee7e00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efec4073230 0x7efec4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.673+0000 7efee7e00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efec4073230 0x7efec4075740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efed8007fd0 tx=0x7efed8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:17.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.673+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efed0020030 con 0x7efee8073a20 2024-09-17T05:03:17.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:17.885+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efee815b140 con 0x7efee8073a20 2024-09-17T05:03:18.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.477+0000 7efee4c00640 1 -- 172.21.15.16:0/1368597995 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7efed005f1b0 con 0x7efee8073a20 2024-09-17T05:03:18.487 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 7 and data pool 8 2024-09-17T05:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efec4073230 msgr2=0x7efec4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efec4073230 0x7efec4075740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efed8007fd0 tx=0x7efed8009040 comp rx=0 tx=0).stop 2024-09-17T05:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 msgr2=0x7efee8155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 0x7efee8155bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7efed000c4e0 tx=0x7efed000c9a0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 shutdown_connections 2024-09-17T05:03:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efec4073230 0x7efec4075740 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efed8007fd0 tx=0x7efed8009040 comp rx=0 tx=0).stop 2024-09-17T05:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efee80cbaa0 0x7efee81f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8073a20 0x7efee8155bc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 --2- 172.21.15.16:0/1368597995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efee8073610 0x7efee8144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.501+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 >> 172.21.15.16:0/1368597995 conn(0x7efee80b2250 msgr2=0x7efee806d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.505+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 shutdown_connections 2024-09-17T05:03:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.505+0000 7efeeee00640 1 -- 172.21.15.16:0/1368597995 wait complete. 2024-09-17T05:03:18.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 -- 172.21.15.16:0/2417719523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f570015edf0 msgr2=0x7f570015f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 --2- 172.21.15.16:0/2417719523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f570015edf0 0x7f570015f200 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009bc0 tx=0x7f56f801cab0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 -- 172.21.15.16:0/2417719523 shutdown_connections 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 --2- 172.21.15.16:0/2417719523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f570015edf0 0x7f570015f200 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 --2- 172.21.15.16:0/2417719523 >> v2:172.21.15.104:3300/0 conn(0x7f57000b3020 0x7f570015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 -- 172.21.15.16:0/2417719523 >> 172.21.15.16:0/2417719523 conn(0x7f57000d2270 msgr2=0x7f57000d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 -- 172.21.15.16:0/2417719523 shutdown_connections 2024-09-17T05:03:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.693+0000 7f5706a00640 1 -- 172.21.15.16:0/2417719523 wait complete. 2024-09-17T05:03:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 Processor -- start 2024-09-17T05:03:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 -- start start 2024-09-17T05:03:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 0x7f570014dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57000b3020 0x7f57000ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:18.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f570015edf0 0x7f57000b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f570016bfa0 con 0x7f570006e170 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f570016be20 con 0x7f570015edf0 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f5706a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f570016c120 con 0x7f57000b3020 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.697+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 0x7f570014dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 0x7f570014dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45670/0 (socket says 172.21.15.16:45670) 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 -- 172.21.15.16:0/1331450848 learned_addr learned my addr 172.21.15.16:0/1331450848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56f7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57000b3020 0x7f57000ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f5704a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f570015edf0 0x7f57000b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 -- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57000b3020 msgr2=0x7f57000ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57000b3020 0x7f57000ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 -- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f570015edf0 msgr2=0x7f57000b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f570015edf0 0x7f57000b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 -- 172.21.15.16:0/1331450848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5700154eb0 con 0x7f570006e170 2024-09-17T05:03:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56ffe00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 0x7f570014dae0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c400 tx=0x7f56f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0017070 con 0x7f570006e170 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56f0003600 con 0x7f570006e170 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57000cd870 con 0x7f570006e170 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0003a50 con 0x7f570006e170 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57000cde30 con 0x7f570006e170 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f570015e8b0 con 0x7f570006e170 2024-09-17T05:03:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.701+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f56f001a070 con 0x7f570006e170 2024-09-17T05:03:18.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.705+0000 7f56fcc00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f56dc073200 0x7f56dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.705+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f56f0093550 con 0x7f570006e170 2024-09-17T05:03:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.705+0000 7f56f7e00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f56dc073200 0x7f56dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.705+0000 7f56f7e00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f56dc073200 0x7f56dc075710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f570015b5f0 tx=0x7f56e8009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.709+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f56f005dd80 con 0x7f570006e170 2024-09-17T05:03:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.885+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5700076230 con 0x7f570006e170 2024-09-17T05:03:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f56fcc00640 1 -- 172.21.15.16:0/1331450848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16943 (secure 0 0 0) 0x7f56f00627b0 con 0x7f570006e170 2024-09-17T05:03:18.890 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:18.890 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:17.889350+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:15.637532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:16.849662+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f56dc073200 msgr2=0x7f56dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f56dc073200 0x7f56dc075710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f570015b5f0 tx=0x7f56e8009260 comp rx=0 tx=0).stop 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 msgr2=0x7f570014dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 0x7f570014dae0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f56f000c400 tx=0x7f56f000c8c0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 shutdown_connections 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f56dc073200 0x7f56dc075710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f570015edf0 0x7f57000b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f57000b3020 0x7f57000ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 --2- 172.21.15.16:0/1331450848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570006e170 0x7f570014dae0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 >> 172.21.15.16:0/1331450848 conn(0x7f57000d2270 msgr2=0x7f5700071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 shutdown_connections 2024-09-17T05:03:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:18.889+0000 7f5706a00640 1 -- 172.21.15.16:0/1331450848 wait complete. 2024-09-17T05:03:18.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 -- 172.21.15.16:0/2667527628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c06e5b0 msgr2=0x7f629c0afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 --2- 172.21.15.16:0/2667527628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c06e5b0 0x7f629c0afb80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6284009870 tx=0x7f628401c780 comp rx=0 tx=0).stop 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 -- 172.21.15.16:0/2667527628 shutdown_connections 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 --2- 172.21.15.16:0/2667527628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c06e5b0 0x7f629c0afb80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 -- 172.21.15.16:0/2667527628 >> 172.21.15.16:0/2667527628 conn(0x7f629c0734b0 msgr2=0x7f629c0738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 -- 172.21.15.16:0/2667527628 shutdown_connections 2024-09-17T05:03:19.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.069+0000 7f62a1200640 1 -- 172.21.15.16:0/2667527628 wait complete. 2024-09-17T05:03:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 Processor -- start 2024-09-17T05:03:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 -- start start 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c073fd0 0x7f629c14c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f629c15c8f0 0x7f629c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 0x7f629c0cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f629c16a680 con 0x7f629c0cc310 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f629c16a500 con 0x7f629c15c8f0 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f62a1200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f629c16a800 con 0x7f629c073fd0 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 0x7f629c0cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 0x7f629c0cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45678/0 (socket says 172.21.15.16:45678) 2024-09-17T05:03:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 -- 172.21.15.16:0/3333488989 learned_addr learned my addr 172.21.15.16:0/3333488989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629b400640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c073fd0 0x7f629c14c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f6292a00640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f629c15c8f0 0x7f629c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 -- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c073fd0 msgr2=0x7f629c14c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c073fd0 0x7f629c14c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 -- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f629c15c8f0 msgr2=0x7f629c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f629c15c8f0 0x7f629c15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 -- 172.21.15.16:0/3333488989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c0ccc60 con 0x7f629c0cc310 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f629be00640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 0x7f629c0cc720 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f629400cb90 tx=0x7f629400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.073+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294016020 con 0x7f629c0cc310 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f629400ab40 con 0x7f629c0cc310 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294012420 con 0x7f629c0cc310 2024-09-17T05:03:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c14c590 con 0x7f629c0cc310 2024-09-17T05:03:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c1fbed0 con 0x7f629c0cc310 2024-09-17T05:03:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c0afb80 con 0x7f629c0cc310 2024-09-17T05:03:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6294019020 con 0x7f629c0cc310 2024-09-17T05:03:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f6293e00640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6278073230 0x7f6278075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f6294094670 con 0x7f629c0cc310 2024-09-17T05:03:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.077+0000 7f629b400640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6278073230 0x7f6278075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.081+0000 7f629b400640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6278073230 0x7f6278075740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f628c007fd0 tx=0x7f628c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.081+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6294020020 con 0x7f629c0cc310 2024-09-17T05:03:19.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.257+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f629c15b1a0 con 0x7f629c0cc310 2024-09-17T05:03:19.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.257+0000 7f6293e00640 1 -- 172.21.15.16:0/3333488989 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16943 (secure 0 0 0) 0x7f629c0ccc60 con 0x7f629c0cc310 2024-09-17T05:03:19.261 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:19.262 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:17.889350+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:15.637532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:16.849662+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6278073230 msgr2=0x7f6278075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6278073230 0x7f6278075740 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f628c007fd0 tx=0x7f628c009040 comp rx=0 tx=0).stop 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 msgr2=0x7f629c0cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 0x7f629c0cc720 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f629400cb90 tx=0x7f629400cc80 comp rx=0 tx=0).stop 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 shutdown_connections 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6278073230 0x7f6278075740 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f629c0cc310 0x7f629c0cc720 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f629c15c8f0 0x7f629c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 --2- 172.21.15.16:0/3333488989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f629c073fd0 0x7f629c14c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 >> 172.21.15.16:0/3333488989 conn(0x7f629c0734b0 msgr2=0x7f629c15b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 shutdown_connections 2024-09-17T05:03:19.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.261+0000 7f62a1200640 1 -- 172.21.15.16:0/3333488989 wait complete. 2024-09-17T05:03:19.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 -- 172.21.15.16:0/535625232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 msgr2=0x7f09ec0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 --2- 172.21.15.16:0/535625232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec0ce5d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0cca90 tx=0x7f09d801c880 comp rx=0 tx=0).stop 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 -- 172.21.15.16:0/535625232 shutdown_connections 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 --2- 172.21.15.16:0/535625232 >> v2:172.21.15.104:3300/0 conn(0x7f09ec15f6f0 0x7f09ec0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 --2- 172.21.15.16:0/535625232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec0ce5d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 -- 172.21.15.16:0/535625232 >> 172.21.15.16:0/535625232 conn(0x7f09ec0b2250 msgr2=0x7f09ec0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 -- 172.21.15.16:0/535625232 shutdown_connections 2024-09-17T05:03:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.441+0000 7f09f4400640 1 -- 172.21.15.16:0/535625232 wait complete. 2024-09-17T05:03:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 Processor -- start 2024-09-17T05:03:19.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 -- start start 2024-09-17T05:03:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09ec074a20 0x7f09ec14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ec15f6f0 0x7f09ec14c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec168e50 con 0x7f09ec074e30 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09ec168cd0 con 0x7f09ec15f6f0 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f4400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f09ec168fd0 con 0x7f09ec074a20 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.445+0000 7f09f1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45706/0 (socket says 172.21.15.16:45706) 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 -- 172.21.15.16:0/955836811 learned_addr learned my addr 172.21.15.16:0/955836811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f2400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ec15f6f0 0x7f09ec14c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1a00640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09ec074a20 0x7f09ec14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 -- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09ec074a20 msgr2=0x7f09ec14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09ec074a20 0x7f09ec14fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 -- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ec15f6f0 msgr2=0x7f09ec14c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ec15f6f0 0x7f09ec14c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 -- 172.21.15.16:0/955836811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ec1f6860 con 0x7f09ec074e30 2024-09-17T05:03:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f1000640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec14c250 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09d80b0040 tx=0x7f09d80a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8004380 con 0x7f09ec074e30 2024-09-17T05:03:19.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09d8005020 con 0x7f09ec074e30 2024-09-17T05:03:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ec15b0d0 con 0x7f09ec074e30 2024-09-17T05:03:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8005470 con 0x7f09ec074e30 2024-09-17T05:03:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ec1f6ae0 con 0x7f09ec074e30 2024-09-17T05:03:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ec15efe0 con 0x7f09ec074e30 2024-09-17T05:03:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.449+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f09d80be060 con 0x7f09ec074e30 2024-09-17T05:03:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.453+0000 7f09e9600640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09cc073200 0x7f09cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.453+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f09d812e460 con 0x7f09ec074e30 2024-09-17T05:03:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.453+0000 7f09f1a00640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09cc073200 0x7f09cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.453+0000 7f09f1a00640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09cc073200 0x7f09cc075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f09e00098e0 tx=0x7f09e0009060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.457+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f09ec1f6860 con 0x7f09ec074e30 2024-09-17T05:03:19.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:19.669+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f09ec154430 con 0x7f09ec074e30 2024-09-17T05:03:20.444 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-17T05:03:20.441+0000 7f60cda00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:03:20.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.493+0000 7f09e9600640 1 -- 172.21.15.16:0/955836811 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f09d8016470 con 0x7f09ec074e30 2024-09-17T05:03:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09cc073200 msgr2=0x7f09cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09cc073200 0x7f09cc075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f09e00098e0 tx=0x7f09e0009060 comp rx=0 tx=0).stop 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 msgr2=0x7f09ec14c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec14c250 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f09d80b0040 tx=0x7f09d80a66d0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 shutdown_connections 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09cc073200 0x7f09cc075710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09ec15f6f0 0x7f09ec14c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09ec074e30 0x7f09ec14c250 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 --2- 172.21.15.16:0/955836811 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09ec074a20 0x7f09ec14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 >> 172.21.15.16:0/955836811 conn(0x7f09ec0b2250 msgr2=0x7f09ec0cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 shutdown_connections 2024-09-17T05:03:20.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.497+0000 7f09f4400640 1 -- 172.21.15.16:0/955836811 wait complete. 2024-09-17T05:03:20.525 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:03:20.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:03:20.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 -- 172.21.15.16:0/3795323839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 msgr2=0x7fdf8815f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 --2- 172.21.15.16:0/3795323839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf8815f520 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdf880cca90 tx=0x7fdf7001c8a0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 -- 172.21.15.16:0/3795323839 shutdown_connections 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 --2- 172.21.15.16:0/3795323839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf8815f520 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 --2- 172.21.15.16:0/3795323839 >> v2:172.21.15.104:3300/0 conn(0x7fdf88074a20 0x7fdf8815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 -- 172.21.15.16:0/3795323839 >> 172.21.15.16:0/3795323839 conn(0x7fdf880b2250 msgr2=0x7fdf880d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 -- 172.21.15.16:0/3795323839 shutdown_connections 2024-09-17T05:03:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 -- 172.21.15.16:0/3795323839 wait complete. 2024-09-17T05:03:20.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 Processor -- start 2024-09-17T05:03:20.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.689+0000 7fdf8e200640 1 -- start start 2024-09-17T05:03:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf88074a20 0x7fdf88146ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf881515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf8815ba50 0x7fdf88154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88168eb0 con 0x7fdf88074e30 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88168d30 con 0x7fdf8815ba50 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdf88169030 con 0x7fdf88074a20 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf881515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf881515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45724/0 (socket says 172.21.15.16:45724) 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 -- 172.21.15.16:0/4059660802 learned_addr learned my addr 172.21.15.16:0/4059660802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf87e00640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf8815ba50 0x7fdf88154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf87400640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf88074a20 0x7fdf88146ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 -- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf88074a20 msgr2=0x7fdf88146ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf88074a20 0x7fdf88146ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 -- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf8815ba50 msgr2=0x7fdf88154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf8815ba50 0x7fdf88154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 -- 172.21.15.16:0/4059660802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf880add70 con 0x7fdf88074e30 2024-09-17T05:03:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf87e00640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf8815ba50 0x7fdf88154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7fe00640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf881515b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b0040 tx=0x7fdf70005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf700b2040 con 0x7fdf88074e30 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf70004330 con 0x7fdf88074e30 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70005290 con 0x7fdf88074e30 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf880ae380 con 0x7fdf88074e30 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881470e0 con 0x7fdf88074e30 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.693+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8815be60 con 0x7fdf88074e30 2024-09-17T05:03:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.697+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf700bd060 con 0x7fdf88074e30 2024-09-17T05:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.697+0000 7fdf7f400640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf64073200 0x7fdf64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.697+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdf7012d7e0 con 0x7fdf88074e30 2024-09-17T05:03:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.697+0000 7fdf87400640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf64073200 0x7fdf64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.697+0000 7fdf87400640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf64073200 0x7fdf64075710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf780058d0 tx=0x7fdf78005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.701+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf70016490 con 0x7fdf88074e30 2024-09-17T05:03:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:20.913+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdf8815d840 con 0x7fdf88074e30 2024-09-17T05:03:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.501+0000 7fdf7f400640 1 -- 172.21.15.16:0/4059660802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fdf700f8660 con 0x7fdf88074e30 2024-09-17T05:03:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf64073200 msgr2=0x7fdf64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf64073200 0x7fdf64075710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdf780058d0 tx=0x7fdf78005860 comp rx=0 tx=0).stop 2024-09-17T05:03:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 msgr2=0x7fdf881515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf881515b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b0040 tx=0x7fdf70005e30 comp rx=0 tx=0).stop 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 shutdown_connections 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf64073200 0x7fdf64075710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf8815ba50 0x7fdf88154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf88074e30 0x7fdf881515b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 --2- 172.21.15.16:0/4059660802 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf88074a20 0x7fdf88146ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 >> 172.21.15.16:0/4059660802 conn(0x7fdf880b2250 msgr2=0x7fdf880cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 shutdown_connections 2024-09-17T05:03:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.505+0000 7fdf8e200640 1 -- 172.21.15.16:0/4059660802 wait complete. 2024-09-17T05:03:21.532 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:03:21.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 -- 172.21.15.16:0/3632331708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 msgr2=0x7f77b015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 --2- 172.21.15.16:0/3632331708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b015bb60 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f77b015cc70 tx=0x7f77a401c830 comp rx=0 tx=0).stop 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 -- 172.21.15.16:0/3632331708 shutdown_connections 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 --2- 172.21.15.16:0/3632331708 >> v2:172.21.15.104:3300/0 conn(0x7f77b0073240 0x7f77b0073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 --2- 172.21.15.16:0/3632331708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b015bb60 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 -- 172.21.15.16:0/3632331708 >> 172.21.15.16:0/3632331708 conn(0x7f77b006f900 msgr2=0x7f77b00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 -- 172.21.15.16:0/3632331708 shutdown_connections 2024-09-17T05:03:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.693+0000 7f77b8400640 1 -- 172.21.15.16:0/3632331708 wait complete. 2024-09-17T05:03:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 Processor -- start 2024-09-17T05:03:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 -- start start 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77b0073240 0x7f77b014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b01fc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f77b015c0a0 0x7f77b01fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0168c70 con 0x7f77b00b2f50 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0168af0 con 0x7f77b0073240 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f77b0168df0 con 0x7f77b015c0a0 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b01fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b01fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45738/0 (socket says 172.21.15.16:45738) 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 -- 172.21.15.16:0/3120057941 learned_addr learned my addr 172.21.15.16:0/3120057941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b6400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f77b015c0a0 0x7f77b01fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5a00640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77b0073240 0x7f77b014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 -- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f77b015c0a0 msgr2=0x7f77b01fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f77b015c0a0 0x7f77b01fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 -- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77b0073240 msgr2=0x7f77b014fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77b0073240 0x7f77b014fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 -- 172.21.15.16:0/3120057941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b0150420 con 0x7f77b00b2f50 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b5000640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b01fc790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c400 tx=0x7f779c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c017070 con 0x7f77b00b2f50 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f779c003600 con 0x7f77b00b2f50 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c003a50 con 0x7f77b00b2f50 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b01fd510 con 0x7f77b00b2f50 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.697+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b0160f30 con 0x7f77b00b2f50 2024-09-17T05:03:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.701+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b0073650 con 0x7f77b00b2f50 2024-09-17T05:03:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.701+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f779c01a070 con 0x7f77b00b2f50 2024-09-17T05:03:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.701+0000 7f77ad600640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7790073200 0x7f7790075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.701+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f779c093660 con 0x7f77b00b2f50 2024-09-17T05:03:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.701+0000 7f77b5a00640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7790073200 0x7f7790075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:21.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.705+0000 7f77b5a00640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7790073200 0x7f7790075710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77a40af000 tx=0x7f77a401b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:21.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.705+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f779c05de90 con 0x7f77b00b2f50 2024-09-17T05:03:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.921+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b015e2d0 con 0x7f77b00b2f50 2024-09-17T05:03:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.921+0000 7f77ad600640 1 -- 172.21.15.16:0/3120057941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f779c0628c0 con 0x7f77b00b2f50 2024-09-17T05:03:21.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:21.924 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:17.889311+0000","modified":"2024-09-17T05:03:20.917921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5460":{"gid":5460,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3845424541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3845424541},{"type":"v1","addr":"172.21.15.16:6834","nonce":3845424541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5460,"qdb_cluster":[5460]},"id":3}]} 2024-09-17T05:03:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7790073200 msgr2=0x7f7790075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7790073200 0x7f7790075710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77a40af000 tx=0x7f77a401b070 comp rx=0 tx=0).stop 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 msgr2=0x7f77b01fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b01fc790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f779c00c400 tx=0x7f779c00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 shutdown_connections 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7790073200 0x7f7790075710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f77b015c0a0 0x7f77b01fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77b00b2f50 0x7f77b01fc790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 --2- 172.21.15.16:0/3120057941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77b0073240 0x7f77b014fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 >> 172.21.15.16:0/3120057941 conn(0x7f77b006f900 msgr2=0x7f77b015cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 shutdown_connections 2024-09-17T05:03:21.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:21.925+0000 7f77b8400640 1 -- 172.21.15.16:0/3120057941 wait complete. 2024-09-17T05:03:21.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.101+0000 7f7d1d400640 1 -- 172.21.15.16:0/3591755862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180cc8c0 msgr2=0x7f7d1815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.101+0000 7f7d1d400640 1 --2- 172.21.15.16:0/3591755862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180cc8c0 0x7f7d1815cab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009840 tx=0x7f7d0801c820 comp rx=0 tx=0).stop 2024-09-17T05:03:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 -- 172.21.15.16:0/3591755862 shutdown_connections 2024-09-17T05:03:22.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 --2- 172.21.15.16:0/3591755862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180cc8c0 0x7f7d1815cab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 -- 172.21.15.16:0/3591755862 >> 172.21.15.16:0/3591755862 conn(0x7f7d180d2270 msgr2=0x7f7d180d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:22.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 -- 172.21.15.16:0/3591755862 shutdown_connections 2024-09-17T05:03:22.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 -- 172.21.15.16:0/3591755862 wait complete. 2024-09-17T05:03:22.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 Processor -- start 2024-09-17T05:03:22.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 -- start start 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 0x7f7d1813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d180cc8c0 0x7f7d1815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180d1580 0x7f7d180b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.105+0000 7f7d1d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1816af60 con 0x7f7d180c9440 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d1d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1816ade0 con 0x7f7d180d1580 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d1d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1816b0e0 con 0x7f7d180cc8c0 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 0x7f7d1813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 0x7f7d1813f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45744/0 (socket says 172.21.15.16:45744) 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 -- 172.21.15.16:0/135660945 learned_addr learned my addr 172.21.15.16:0/135660945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 -- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d180cc8c0 msgr2=0x7f7d1815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d0fe00640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d180cc8c0 0x7f7d1815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d17200640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180d1580 0x7f7d180b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d180cc8c0 0x7f7d1815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 -- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180d1580 msgr2=0x7f7d180b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180d1580 0x7f7d180b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 -- 172.21.15.16:0/135660945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1813fcc0 con 0x7f7d180c9440 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d0fe00640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d180cc8c0 0x7f7d1815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d17200640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180d1580 0x7f7d180b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d16800640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 0x7f7d1813f780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7d0801cd00 tx=0x7f7d08005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080bb070 con 0x7f7d180c9440 2024-09-17T05:03:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d08004330 con 0x7f7d180c9440 2024-09-17T05:03:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080b2040 con 0x7f7d180c9440 2024-09-17T05:03:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1815ed20 con 0x7f7d180c9440 2024-09-17T05:03:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d18151470 con 0x7f7d180c9440 2024-09-17T05:03:22.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1815cab0 con 0x7f7d180c9440 2024-09-17T05:03:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.109+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7d08005230 con 0x7f7d180c9440 2024-09-17T05:03:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.113+0000 7f7d0ea00640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7cf4073200 0x7f7cf4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.113+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f7d0812db80 con 0x7f7d180c9440 2024-09-17T05:03:22.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.113+0000 7f7d0fe00640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7cf4073200 0x7f7cf4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.113+0000 7f7d0fe00640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7cf4073200 0x7f7cf4075710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f70 tx=0x7f7d00009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:22.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.117+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d080f2db0 con 0x7f7d180c9440 2024-09-17T05:03:22.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.293+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d18071a90 con 0x7f7d180c9440 2024-09-17T05:03:22.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.293+0000 7f7d0ea00640 1 -- 172.21.15.16:0/135660945 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16930 (secure 0 0 0) 0x7f7d08016410 con 0x7f7d180c9440 2024-09-17T05:03:22.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:22.297 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:19.480354+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:15.637532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:16.849662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7cf4073200 msgr2=0x7f7cf4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7cf4073200 0x7f7cf4075710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d00000f70 tx=0x7f7d00009040 comp rx=0 tx=0).stop 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 msgr2=0x7f7d1813f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 0x7f7d1813f780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7d0801cd00 tx=0x7f7d08005d10 comp rx=0 tx=0).stop 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 shutdown_connections 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7cf4073200 0x7f7cf4075710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d180d1580 0x7f7d180b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d180cc8c0 0x7f7d1815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 --2- 172.21.15.16:0/135660945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d180c9440 0x7f7d1813f780 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 >> 172.21.15.16:0/135660945 conn(0x7f7d180d2270 msgr2=0x7f7d18070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 shutdown_connections 2024-09-17T05:03:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.297+0000 7f7d1d400640 1 -- 172.21.15.16:0/135660945 wait complete. 2024-09-17T05:03:22.320 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:03:22.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 -- 172.21.15.16:0/1514158523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f815c270 msgr2=0x7f34f815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 --2- 172.21.15.16:0/1514158523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f815c270 0x7f34f815c680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009900 tx=0x7f34e001c850 comp rx=0 tx=0).stop 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 -- 172.21.15.16:0/1514158523 shutdown_connections 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 --2- 172.21.15.16:0/1514158523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f8073240 0x7f34f8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 --2- 172.21.15.16:0/1514158523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f815c270 0x7f34f815c680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 -- 172.21.15.16:0/1514158523 >> 172.21.15.16:0/1514158523 conn(0x7f34f806f900 msgr2=0x7f34f80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 -- 172.21.15.16:0/1514158523 shutdown_connections 2024-09-17T05:03:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.477+0000 7f34fd200640 1 -- 172.21.15.16:0/1514158523 wait complete. 2024-09-17T05:03:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 Processor -- start 2024-09-17T05:03:22.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 -- start start 2024-09-17T05:03:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 0x7f34f81505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b2f50 0x7f34f81632e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f815c270 0x7f34f81fa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8168d60 con 0x7f34f8073240 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8168be0 con 0x7f34f80b2f50 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f34f8168ee0 con 0x7f34f815c270 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 0x7f34f81505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 0x7f34f81505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45760/0 (socket says 172.21.15.16:45760) 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 -- 172.21.15.16:0/3968000067 learned_addr learned my addr 172.21.15.16:0/3968000067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34efe00640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b2f50 0x7f34f81632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f6800640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f815c270 0x7f34f81fa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 -- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f815c270 msgr2=0x7f34f81fa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f815c270 0x7f34f81fa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 -- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b2f50 msgr2=0x7f34f81632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b2f50 0x7f34f81632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 -- 172.21.15.16:0/3968000067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f81f9d10 con 0x7f34f8073240 2024-09-17T05:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f6800640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f815c270 0x7f34f81fa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34f5e00640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 0x7f34f81505e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c3c0 tx=0x7f34e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8017070 con 0x7f34f8073240 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34e80053c0 con 0x7f34f8073240 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8003860 con 0x7f34f8073240 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81654b0 con 0x7f34f8073240 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.481+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f80af130 con 0x7f34f8073240 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.485+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f8073650 con 0x7f34f8073240 2024-09-17T05:03:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.485+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f34e801a070 con 0x7f34f8073240 2024-09-17T05:03:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.485+0000 7f34ee000640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d4073200 0x7f34d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.485+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f34e80936f0 con 0x7f34f8073240 2024-09-17T05:03:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.485+0000 7f34efe00640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d4073200 0x7f34d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.485+0000 7f34efe00640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d4073200 0x7f34d4075710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f34e00af000 tx=0x7f34e001b090 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.489+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34f81654b0 con 0x7f34f8073240 2024-09-17T05:03:22.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.665+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f34f8164a90 con 0x7f34d4073200 2024-09-17T05:03:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34ee000640 1 -- 172.21.15.16:0/3968000067 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247336 (secure 0 0 0) 0x7f34e00f8640 con 0x7f34d4073200 2024-09-17T05:03:22.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:22.679 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:22.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d4073200 msgr2=0x7f34d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d4073200 0x7f34d4075710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f34e00af000 tx=0x7f34e001b090 comp rx=0 tx=0).stop 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 msgr2=0x7f34f81505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 0x7f34f81505e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c3c0 tx=0x7f34e800c880 comp rx=0 tx=0).stop 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 shutdown_connections 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d4073200 0x7f34d4075710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34f815c270 0x7f34f81fa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b2f50 0x7f34f81632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 --2- 172.21.15.16:0/3968000067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f8073240 0x7f34f81505e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 >> 172.21.15.16:0/3968000067 conn(0x7f34f806f900 msgr2=0x7f34f80b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 shutdown_connections 2024-09-17T05:03:22.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.673+0000 7f34fd200640 1 -- 172.21.15.16:0/3968000067 wait complete. 2024-09-17T05:03:22.698 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-17T05:03:21.495567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":111,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242848,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731872,"statfs":{"total":773094113280,"available":772845436928,"internally_reserved":0,"allocated":5939200,"data_stored":3252023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.044611"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.478211+0000","last_change":"2024-09-17T05:03:17.478211+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.478211+0000","last_undegraded":"2024-09-17T05:03:17.478211+0000","last_fullsized":"2024-09-17T05:03:17.478211+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466131+0000","last_change":"2024-09-17T05:03:17.466131+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466131+0000","last_undegraded":"2024-09-17T05:03:17.466131+0000","last_fullsized":"2024-09-17T05:03:17.466131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468087+0000","last_change":"2024-09-17T05:03:17.468087+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468087+0000","last_undegraded":"2024-09-17T05:03:17.468087+0000","last_fullsized":"2024-09-17T05:03:17.468087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.473650+0000","last_change":"2024-09-17T05:03:17.473650+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.473650+0000","last_undegraded":"2024-09-17T05:03:17.473650+0000","last_fullsized":"2024-09-17T05:03:17.473650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466191+0000","last_change":"2024-09-17T05:03:17.466191+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466191+0000","last_undegraded":"2024-09-17T05:03:17.466191+0000","last_fullsized":"2024-09-17T05:03:17.466191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468839+0000","last_change":"2024-09-17T05:03:17.468839+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468839+0000","last_undegraded":"2024-09-17T05:03:17.468839+0000","last_fullsized":"2024-09-17T05:03:17.468839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486528+0000","last_change":"2024-09-17T05:03:18.485411+0000","last_active":"2024-09-17T05:03:19.486528+0000","last_peered":"2024-09-17T05:03:19.486528+0000","last_clean":"2024-09-17T05:03:19.486528+0000","last_became_active":"2024-09-17T05:03:18.485074+0000","last_became_peered":"2024-09-17T05:03:18.485074+0000","last_unstale":"2024-09-17T05:03:19.486528+0000","last_undegraded":"2024-09-17T05:03:19.486528+0000","last_fullsized":"2024-09-17T05:03:19.486528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:07.445182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468046+0000","last_change":"2024-09-17T05:03:17.468046+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468046+0000","last_undegraded":"2024-09-17T05:03:17.468046+0000","last_fullsized":"2024-09-17T05:03:17.468046+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.467189+0000","last_change":"2024-09-17T05:03:17.467189+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.467189+0000","last_undegraded":"2024-09-17T05:03:17.467189+0000","last_fullsized":"2024-09-17T05:03:17.467189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487208+0000","last_change":"2024-09-17T05:03:18.490713+0000","last_active":"2024-09-17T05:03:19.487208+0000","last_peered":"2024-09-17T05:03:19.487208+0000","last_clean":"2024-09-17T05:03:19.487208+0000","last_became_active":"2024-09-17T05:03:18.490369+0000","last_became_peered":"2024-09-17T05:03:18.490369+0000","last_unstale":"2024-09-17T05:03:19.487208+0000","last_undegraded":"2024-09-17T05:03:19.487208+0000","last_fullsized":"2024-09-17T05:03:19.487208+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.356295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474235+0000","last_change":"2024-09-17T05:03:17.474235+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474235+0000","last_undegraded":"2024-09-17T05:03:17.474235+0000","last_fullsized":"2024-09-17T05:03:17.474235+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468399+0000","last_change":"2024-09-17T05:03:17.468399+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468399+0000","last_undegraded":"2024-09-17T05:03:17.468399+0000","last_fullsized":"2024-09-17T05:03:17.468399+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468489+0000","last_change":"2024-09-17T05:03:17.468489+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468489+0000","last_undegraded":"2024-09-17T05:03:17.468489+0000","last_fullsized":"2024-09-17T05:03:17.468489+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.477051+0000","last_change":"2024-09-17T05:03:17.477051+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.477051+0000","last_undegraded":"2024-09-17T05:03:17.477051+0000","last_fullsized":"2024-09-17T05:03:17.477051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474677+0000","last_change":"2024-09-17T05:03:17.474677+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474677+0000","last_undegraded":"2024-09-17T05:03:17.474677+0000","last_fullsized":"2024-09-17T05:03:17.474677+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.467432+0000","last_change":"2024-09-17T05:03:17.467432+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.467432+0000","last_undegraded":"2024-09-17T05:03:17.467432+0000","last_fullsized":"2024-09-17T05:03:17.467432+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472328+0000","last_change":"2024-09-17T05:03:17.472328+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472328+0000","last_undegraded":"2024-09-17T05:03:17.472328+0000","last_fullsized":"2024-09-17T05:03:17.472328+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474185+0000","last_change":"2024-09-17T05:03:17.474185+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474185+0000","last_undegraded":"2024-09-17T05:03:17.474185+0000","last_fullsized":"2024-09-17T05:03:17.474185+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.467619+0000","last_change":"2024-09-17T05:03:17.467619+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.467619+0000","last_undegraded":"2024-09-17T05:03:17.467619+0000","last_fullsized":"2024-09-17T05:03:17.467619+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465808+0000","last_change":"2024-09-17T05:03:17.465808+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465808+0000","last_undegraded":"2024-09-17T05:03:17.465808+0000","last_fullsized":"2024-09-17T05:03:17.465808+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472172+0000","last_change":"2024-09-17T05:03:17.472172+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472172+0000","last_undegraded":"2024-09-17T05:03:17.472172+0000","last_fullsized":"2024-09-17T05:03:17.472172+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465345+0000","last_change":"2024-09-17T05:03:17.465345+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465345+0000","last_undegraded":"2024-09-17T05:03:17.465345+0000","last_fullsized":"2024-09-17T05:03:17.465345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465516+0000","last_change":"2024-09-17T05:03:17.465516+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465516+0000","last_undegraded":"2024-09-17T05:03:17.465516+0000","last_fullsized":"2024-09-17T05:03:17.465516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463065+0000","last_change":"2024-09-17T05:03:17.463065+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463065+0000","last_undegraded":"2024-09-17T05:03:17.463065+0000","last_fullsized":"2024-09-17T05:03:17.463065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.477654+0000","last_change":"2024-09-17T05:03:17.477654+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.477654+0000","last_undegraded":"2024-09-17T05:03:17.477654+0000","last_fullsized":"2024-09-17T05:03:17.477654+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.464012+0000","last_change":"2024-09-17T05:03:17.464012+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.464012+0000","last_undegraded":"2024-09-17T05:03:17.464012+0000","last_fullsized":"2024-09-17T05:03:17.464012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470026+0000","last_change":"2024-09-17T05:03:17.470026+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470026+0000","last_undegraded":"2024-09-17T05:03:17.470026+0000","last_fullsized":"2024-09-17T05:03:17.470026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465257+0000","last_change":"2024-09-17T05:03:17.465257+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465257+0000","last_undegraded":"2024-09-17T05:03:17.465257+0000","last_fullsized":"2024-09-17T05:03:17.465257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.476898+0000","last_change":"2024-09-17T05:03:17.476898+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.476898+0000","last_undegraded":"2024-09-17T05:03:17.476898+0000","last_fullsized":"2024-09-17T05:03:17.476898+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462748+0000","last_change":"2024-09-17T05:03:17.462748+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462748+0000","last_undegraded":"2024-09-17T05:03:17.462748+0000","last_fullsized":"2024-09-17T05:03:17.462748+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463326+0000","last_change":"2024-09-17T05:03:17.463326+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463326+0000","last_undegraded":"2024-09-17T05:03:17.463326+0000","last_fullsized":"2024-09-17T05:03:17.463326+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462335+0000","last_change":"2024-09-17T05:03:17.462335+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462335+0000","last_undegraded":"2024-09-17T05:03:17.462335+0000","last_fullsized":"2024-09-17T05:03:17.462335+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462819+0000","last_change":"2024-09-17T05:03:17.462819+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462819+0000","last_undegraded":"2024-09-17T05:03:17.462819+0000","last_fullsized":"2024-09-17T05:03:17.462819+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.488704+0000","last_change":"2024-09-17T05:03:17.487458+0000","last_active":"2024-09-17T05:03:17.488704+0000","last_peered":"2024-09-17T05:03:17.488704+0000","last_clean":"2024-09-17T05:03:17.488704+0000","last_became_active":"2024-09-17T05:03:17.487174+0000","last_became_peered":"2024-09-17T05:03:17.487174+0000","last_unstale":"2024-09-17T05:03:17.488704+0000","last_undegraded":"2024-09-17T05:03:17.488704+0000","last_fullsized":"2024-09-17T05:03:17.488704+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:18.208628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463888+0000","last_change":"2024-09-17T05:03:17.463888+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463888+0000","last_undegraded":"2024-09-17T05:03:17.463888+0000","last_fullsized":"2024-09-17T05:03:17.463888+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492771+0000","last_change":"2024-09-17T05:03:17.489555+0000","last_active":"2024-09-17T05:03:17.492771+0000","last_peered":"2024-09-17T05:03:17.492771+0000","last_clean":"2024-09-17T05:03:17.492771+0000","last_became_active":"2024-09-17T05:03:17.489182+0000","last_became_peered":"2024-09-17T05:03:17.489182+0000","last_unstale":"2024-09-17T05:03:17.492771+0000","last_undegraded":"2024-09-17T05:03:17.492771+0000","last_fullsized":"2024-09-17T05:03:17.492771+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:30.679941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.464628+0000","last_change":"2024-09-17T05:03:17.464628+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.464628+0000","last_undegraded":"2024-09-17T05:03:17.464628+0000","last_fullsized":"2024-09-17T05:03:17.464628+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487289+0000","last_change":"2024-09-17T05:03:17.473668+0000","last_active":"2024-09-17T05:03:19.487289+0000","last_peered":"2024-09-17T05:03:19.487289+0000","last_clean":"2024-09-17T05:03:19.487289+0000","last_became_active":"2024-09-17T05:03:17.473283+0000","last_became_peered":"2024-09-17T05:03:17.473283+0000","last_unstale":"2024-09-17T05:03:19.487289+0000","last_undegraded":"2024-09-17T05:03:19.487289+0000","last_fullsized":"2024-09-17T05:03:19.487289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:34.241801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487050+0000","last_change":"2024-09-17T05:03:18.495083+0000","last_active":"2024-09-17T05:03:19.487050+0000","last_peered":"2024-09-17T05:03:19.487050+0000","last_clean":"2024-09-17T05:03:19.487050+0000","last_became_active":"2024-09-17T05:03:18.494736+0000","last_became_peered":"2024-09-17T05:03:18.494736+0000","last_unstale":"2024-09-17T05:03:19.487050+0000","last_undegraded":"2024-09-17T05:03:19.487050+0000","last_fullsized":"2024-09-17T05:03:19.487050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:24.301106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.486423+0000","last_change":"2024-09-17T05:03:17.484272+0000","last_active":"2024-09-17T05:03:17.486423+0000","last_peered":"2024-09-17T05:03:17.486423+0000","last_clean":"2024-09-17T05:03:17.486423+0000","last_became_active":"2024-09-17T05:03:17.483583+0000","last_became_peered":"2024-09-17T05:03:17.483583+0000","last_unstale":"2024-09-17T05:03:17.486423+0000","last_undegraded":"2024-09-17T05:03:17.486423+0000","last_fullsized":"2024-09-17T05:03:17.486423+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:26.162683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472066+0000","last_change":"2024-09-17T05:03:17.472066+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472066+0000","last_undegraded":"2024-09-17T05:03:17.472066+0000","last_fullsized":"2024-09-17T05:03:17.472066+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.491356+0000","last_change":"2024-09-17T05:03:17.489922+0000","last_active":"2024-09-17T05:03:17.491356+0000","last_peered":"2024-09-17T05:03:17.491356+0000","last_clean":"2024-09-17T05:03:17.491356+0000","last_became_active":"2024-09-17T05:03:17.489611+0000","last_became_peered":"2024-09-17T05:03:17.489611+0000","last_unstale":"2024-09-17T05:03:17.491356+0000","last_undegraded":"2024-09-17T05:03:17.491356+0000","last_fullsized":"2024-09-17T05:03:17.491356+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:36.881176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465612+0000","last_change":"2024-09-17T05:03:17.465612+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465612+0000","last_undegraded":"2024-09-17T05:03:17.465612+0000","last_fullsized":"2024-09-17T05:03:17.465612+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.478897+0000","last_change":"2024-09-17T05:03:17.477682+0000","last_active":"2024-09-17T05:03:17.478897+0000","last_peered":"2024-09-17T05:03:17.478897+0000","last_clean":"2024-09-17T05:03:17.478897+0000","last_became_active":"2024-09-17T05:03:17.477257+0000","last_became_peered":"2024-09-17T05:03:17.477257+0000","last_unstale":"2024-09-17T05:03:17.478897+0000","last_undegraded":"2024-09-17T05:03:17.478897+0000","last_fullsized":"2024-09-17T05:03:17.478897+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:43.005082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487353+0000","last_change":"2024-09-17T05:03:18.504857+0000","last_active":"2024-09-17T05:03:19.487353+0000","last_peered":"2024-09-17T05:03:19.487353+0000","last_clean":"2024-09-17T05:03:19.487353+0000","last_became_active":"2024-09-17T05:03:18.504185+0000","last_became_peered":"2024-09-17T05:03:18.504185+0000","last_unstale":"2024-09-17T05:03:19.487353+0000","last_undegraded":"2024-09-17T05:03:19.487353+0000","last_fullsized":"2024-09-17T05:03:19.487353+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:51.971141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.489670+0000","last_change":"2024-09-17T05:03:17.488461+0000","last_active":"2024-09-17T05:03:17.489670+0000","last_peered":"2024-09-17T05:03:17.489670+0000","last_clean":"2024-09-17T05:03:17.489670+0000","last_became_active":"2024-09-17T05:03:17.488076+0000","last_became_peered":"2024-09-17T05:03:17.488076+0000","last_unstale":"2024-09-17T05:03:17.489670+0000","last_undegraded":"2024-09-17T05:03:17.489670+0000","last_fullsized":"2024-09-17T05:03:17.489670+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:33.816677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472891+0000","last_change":"2024-09-17T05:03:17.472891+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472891+0000","last_undegraded":"2024-09-17T05:03:17.472891+0000","last_fullsized":"2024-09-17T05:03:17.472891+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.483798+0000","last_change":"2024-09-17T05:03:17.476199+0000","last_active":"2024-09-17T05:03:17.483798+0000","last_peered":"2024-09-17T05:03:17.483798+0000","last_clean":"2024-09-17T05:03:17.483798+0000","last_became_active":"2024-09-17T05:03:17.475834+0000","last_became_peered":"2024-09-17T05:03:17.475834+0000","last_unstale":"2024-09-17T05:03:17.483798+0000","last_undegraded":"2024-09-17T05:03:17.483798+0000","last_fullsized":"2024-09-17T05:03:17.483798+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:22.586431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470481+0000","last_change":"2024-09-17T05:03:17.470481+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470481+0000","last_undegraded":"2024-09-17T05:03:17.470481+0000","last_fullsized":"2024-09-17T05:03:17.470481+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487622+0000","last_change":"2024-09-17T05:03:17.484357+0000","last_active":"2024-09-17T05:03:17.487622+0000","last_peered":"2024-09-17T05:03:17.487622+0000","last_clean":"2024-09-17T05:03:17.487622+0000","last_became_active":"2024-09-17T05:03:17.483648+0000","last_became_peered":"2024-09-17T05:03:17.483648+0000","last_unstale":"2024-09-17T05:03:17.487622+0000","last_undegraded":"2024-09-17T05:03:17.487622+0000","last_fullsized":"2024-09-17T05:03:17.487622+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:06.028991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474136+0000","last_change":"2024-09-17T05:03:17.474136+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474136+0000","last_undegraded":"2024-09-17T05:03:17.474136+0000","last_fullsized":"2024-09-17T05:03:17.474136+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487665+0000","last_change":"2024-09-17T05:03:17.483837+0000","last_active":"2024-09-17T05:03:17.487665+0000","last_peered":"2024-09-17T05:03:17.487665+0000","last_clean":"2024-09-17T05:03:17.487665+0000","last_became_active":"2024-09-17T05:03:17.483474+0000","last_became_peered":"2024-09-17T05:03:17.483474+0000","last_unstale":"2024-09-17T05:03:17.487665+0000","last_undegraded":"2024-09-17T05:03:17.487665+0000","last_fullsized":"2024-09-17T05:03:17.487665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:35.734369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472247+0000","last_change":"2024-09-17T05:03:17.472247+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472247+0000","last_undegraded":"2024-09-17T05:03:17.472247+0000","last_fullsized":"2024-09-17T05:03:17.472247+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.474754+0000","last_change":"2024-09-17T05:03:17.471492+0000","last_active":"2024-09-17T05:03:17.474754+0000","last_peered":"2024-09-17T05:03:17.474754+0000","last_clean":"2024-09-17T05:03:17.474754+0000","last_became_active":"2024-09-17T05:03:17.471119+0000","last_became_peered":"2024-09-17T05:03:17.471119+0000","last_unstale":"2024-09-17T05:03:17.474754+0000","last_undegraded":"2024-09-17T05:03:17.474754+0000","last_fullsized":"2024-09-17T05:03:17.474754+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:33.579393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.461994+0000","last_change":"2024-09-17T05:03:17.461994+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.461994+0000","last_undegraded":"2024-09-17T05:03:17.461994+0000","last_fullsized":"2024-09-17T05:03:17.461994+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487172+0000","last_change":"2024-09-17T05:03:17.480393+0000","last_active":"2024-09-17T05:03:19.487172+0000","last_peered":"2024-09-17T05:03:19.487172+0000","last_clean":"2024-09-17T05:03:19.487172+0000","last_became_active":"2024-09-17T05:03:17.479912+0000","last_became_peered":"2024-09-17T05:03:17.479912+0000","last_unstale":"2024-09-17T05:03:19.487172+0000","last_undegraded":"2024-09-17T05:03:19.487172+0000","last_fullsized":"2024-09-17T05:03:19.487172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:06.645919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463221+0000","last_change":"2024-09-17T05:03:17.463221+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463221+0000","last_undegraded":"2024-09-17T05:03:17.463221+0000","last_fullsized":"2024-09-17T05:03:17.463221+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.488548+0000","last_change":"2024-09-17T05:03:17.482800+0000","last_active":"2024-09-17T05:03:17.488548+0000","last_peered":"2024-09-17T05:03:17.488548+0000","last_clean":"2024-09-17T05:03:17.488548+0000","last_became_active":"2024-09-17T05:03:17.482376+0000","last_became_peered":"2024-09-17T05:03:17.482376+0000","last_unstale":"2024-09-17T05:03:17.488548+0000","last_undegraded":"2024-09-17T05:03:17.488548+0000","last_fullsized":"2024-09-17T05:03:17.488548+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:04.504110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472050+0000","last_change":"2024-09-17T05:03:17.472050+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472050+0000","last_undegraded":"2024-09-17T05:03:17.472050+0000","last_fullsized":"2024-09-17T05:03:17.472050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.480761+0000","last_change":"2024-09-17T05:03:17.477301+0000","last_active":"2024-09-17T05:03:17.480761+0000","last_peered":"2024-09-17T05:03:17.480761+0000","last_clean":"2024-09-17T05:03:17.480761+0000","last_became_active":"2024-09-17T05:03:17.476888+0000","last_became_peered":"2024-09-17T05:03:17.476888+0000","last_unstale":"2024-09-17T05:03:17.480761+0000","last_undegraded":"2024-09-17T05:03:17.480761+0000","last_fullsized":"2024-09-17T05:03:17.480761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:30.679941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.469666+0000","last_change":"2024-09-17T05:03:17.469666+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.469666+0000","last_undegraded":"2024-09-17T05:03:17.469666+0000","last_fullsized":"2024-09-17T05:03:17.469666+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.485409+0000","last_change":"2024-09-17T05:03:17.481847+0000","last_active":"2024-09-17T05:03:17.485409+0000","last_peered":"2024-09-17T05:03:17.485409+0000","last_clean":"2024-09-17T05:03:17.485409+0000","last_became_active":"2024-09-17T05:03:17.481546+0000","last_became_peered":"2024-09-17T05:03:17.481546+0000","last_unstale":"2024-09-17T05:03:17.485409+0000","last_undegraded":"2024-09-17T05:03:17.485409+0000","last_fullsized":"2024-09-17T05:03:17.485409+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:48.075582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466681+0000","last_change":"2024-09-17T05:03:17.466681+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466681+0000","last_undegraded":"2024-09-17T05:03:17.466681+0000","last_fullsized":"2024-09-17T05:03:17.466681+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492755+0000","last_change":"2024-09-17T05:03:17.489475+0000","last_active":"2024-09-17T05:03:17.492755+0000","last_peered":"2024-09-17T05:03:17.492755+0000","last_clean":"2024-09-17T05:03:17.492755+0000","last_became_active":"2024-09-17T05:03:17.489135+0000","last_became_peered":"2024-09-17T05:03:17.489135+0000","last_unstale":"2024-09-17T05:03:17.492755+0000","last_undegraded":"2024-09-17T05:03:17.492755+0000","last_fullsized":"2024-09-17T05:03:17.492755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.043546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486537+0000","last_change":"2024-09-17T05:03:18.487882+0000","last_active":"2024-09-17T05:03:19.486537+0000","last_peered":"2024-09-17T05:03:19.486537+0000","last_clean":"2024-09-17T05:03:19.486537+0000","last_became_active":"2024-09-17T05:03:18.487523+0000","last_became_peered":"2024-09-17T05:03:18.487523+0000","last_unstale":"2024-09-17T05:03:19.486537+0000","last_undegraded":"2024-09-17T05:03:19.486537+0000","last_fullsized":"2024-09-17T05:03:19.486537+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:44.737783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492762+0000","last_change":"2024-09-17T05:03:17.489385+0000","last_active":"2024-09-17T05:03:17.492762+0000","last_peered":"2024-09-17T05:03:17.492762+0000","last_clean":"2024-09-17T05:03:17.492762+0000","last_became_active":"2024-09-17T05:03:17.489069+0000","last_became_peered":"2024-09-17T05:03:17.489069+0000","last_unstale":"2024-09-17T05:03:17.492762+0000","last_undegraded":"2024-09-17T05:03:17.492762+0000","last_fullsized":"2024-09-17T05:03:17.492762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:11.525764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470948+0000","last_change":"2024-09-17T05:03:17.470948+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470948+0000","last_undegraded":"2024-09-17T05:03:17.470948+0000","last_fullsized":"2024-09-17T05:03:17.470948+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.488510+0000","last_change":"2024-09-17T05:03:17.483869+0000","last_active":"2024-09-17T05:03:17.488510+0000","last_peered":"2024-09-17T05:03:17.488510+0000","last_clean":"2024-09-17T05:03:17.488510+0000","last_became_active":"2024-09-17T05:03:17.483565+0000","last_became_peered":"2024-09-17T05:03:17.483565+0000","last_unstale":"2024-09-17T05:03:17.488510+0000","last_undegraded":"2024-09-17T05:03:17.488510+0000","last_fullsized":"2024-09-17T05:03:17.488510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:59.664232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.461989+0000","last_change":"2024-09-17T05:03:17.461989+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.461989+0000","last_undegraded":"2024-09-17T05:03:17.461989+0000","last_fullsized":"2024-09-17T05:03:17.461989+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487642+0000","last_change":"2024-09-17T05:03:17.484821+0000","last_active":"2024-09-17T05:03:17.487642+0000","last_peered":"2024-09-17T05:03:17.487642+0000","last_clean":"2024-09-17T05:03:17.487642+0000","last_became_active":"2024-09-17T05:03:17.484445+0000","last_became_peered":"2024-09-17T05:03:17.484445+0000","last_unstale":"2024-09-17T05:03:17.487642+0000","last_undegraded":"2024-09-17T05:03:17.487642+0000","last_fullsized":"2024-09-17T05:03:17.487642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.469034+0000","last_change":"2024-09-17T05:03:17.469034+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.469034+0000","last_undegraded":"2024-09-17T05:03:17.469034+0000","last_fullsized":"2024-09-17T05:03:17.469034+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.478888+0000","last_change":"2024-09-17T05:03:17.469441+0000","last_active":"2024-09-17T05:03:17.478888+0000","last_peered":"2024-09-17T05:03:17.478888+0000","last_clean":"2024-09-17T05:03:17.478888+0000","last_became_active":"2024-09-17T05:03:17.469019+0000","last_became_peered":"2024-09-17T05:03:17.469019+0000","last_unstale":"2024-09-17T05:03:17.478888+0000","last_undegraded":"2024-09-17T05:03:17.478888+0000","last_fullsized":"2024-09-17T05:03:17.478888+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.464644+0000","last_change":"2024-09-17T05:03:17.464644+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.464644+0000","last_undegraded":"2024-09-17T05:03:17.464644+0000","last_fullsized":"2024-09-17T05:03:17.464644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492232+0000","last_change":"2024-09-17T05:03:17.488710+0000","last_active":"2024-09-17T05:03:17.492232+0000","last_peered":"2024-09-17T05:03:17.492232+0000","last_clean":"2024-09-17T05:03:17.492232+0000","last_became_active":"2024-09-17T05:03:17.488178+0000","last_became_peered":"2024-09-17T05:03:17.488178+0000","last_unstale":"2024-09-17T05:03:17.492232+0000","last_undegraded":"2024-09-17T05:03:17.492232+0000","last_fullsized":"2024-09-17T05:03:17.492232+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:21.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.475056+0000","last_change":"2024-09-17T05:03:17.475056+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.475056+0000","last_undegraded":"2024-09-17T05:03:17.475056+0000","last_fullsized":"2024-09-17T05:03:17.475056+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.489164+0000","last_change":"2024-09-17T05:03:17.487381+0000","last_active":"2024-09-17T05:03:17.489164+0000","last_peered":"2024-09-17T05:03:17.489164+0000","last_clean":"2024-09-17T05:03:17.489164+0000","last_became_active":"2024-09-17T05:03:17.487025+0000","last_became_peered":"2024-09-17T05:03:17.487025+0000","last_unstale":"2024-09-17T05:03:17.489164+0000","last_undegraded":"2024-09-17T05:03:17.489164+0000","last_fullsized":"2024-09-17T05:03:17.489164+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:06.819108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.471657+0000","last_change":"2024-09-17T05:03:17.471657+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.471657+0000","last_undegraded":"2024-09-17T05:03:17.471657+0000","last_fullsized":"2024-09-17T05:03:17.471657+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486706+0000","last_change":"2024-09-17T05:03:17.488730+0000","last_active":"2024-09-17T05:03:19.486706+0000","last_peered":"2024-09-17T05:03:19.486706+0000","last_clean":"2024-09-17T05:03:19.486706+0000","last_became_active":"2024-09-17T05:03:17.487850+0000","last_became_peered":"2024-09-17T05:03:17.487850+0000","last_unstale":"2024-09-17T05:03:19.486706+0000","last_undegraded":"2024-09-17T05:03:19.486706+0000","last_fullsized":"2024-09-17T05:03:19.486706+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:20.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.469271+0000","last_change":"2024-09-17T05:03:17.469271+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.469271+0000","last_undegraded":"2024-09-17T05:03:17.469271+0000","last_fullsized":"2024-09-17T05:03:17.469271+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486411+0000","last_change":"2024-09-17T05:03:17.477112+0000","last_active":"2024-09-17T05:03:19.486411+0000","last_peered":"2024-09-17T05:03:19.486411+0000","last_clean":"2024-09-17T05:03:19.486411+0000","last_became_active":"2024-09-17T05:03:17.476642+0000","last_became_peered":"2024-09-17T05:03:17.476642+0000","last_unstale":"2024-09-17T05:03:19.486411+0000","last_undegraded":"2024-09-17T05:03:19.486411+0000","last_fullsized":"2024-09-17T05:03:19.486411+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:50.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468943+0000","last_change":"2024-09-17T05:03:17.468943+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468943+0000","last_undegraded":"2024-09-17T05:03:17.468943+0000","last_fullsized":"2024-09-17T05:03:17.468943+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.489818+0000","last_change":"2024-09-17T05:03:17.488680+0000","last_active":"2024-09-17T05:03:17.489818+0000","last_peered":"2024-09-17T05:03:17.489818+0000","last_clean":"2024-09-17T05:03:17.489818+0000","last_became_active":"2024-09-17T05:03:17.488050+0000","last_became_peered":"2024-09-17T05:03:17.488050+0000","last_unstale":"2024-09-17T05:03:17.489818+0000","last_undegraded":"2024-09-17T05:03:17.489818+0000","last_fullsized":"2024-09-17T05:03:17.489818+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:15.702498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487087+0000","last_change":"2024-09-17T05:03:18.495602+0000","last_active":"2024-09-17T05:03:19.487087+0000","last_peered":"2024-09-17T05:03:19.487087+0000","last_clean":"2024-09-17T05:03:19.487087+0000","last_became_active":"2024-09-17T05:03:18.495190+0000","last_became_peered":"2024-09-17T05:03:18.495190+0000","last_unstale":"2024-09-17T05:03:19.487087+0000","last_undegraded":"2024-09-17T05:03:19.487087+0000","last_fullsized":"2024-09-17T05:03:19.487087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:49.184314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.474724+0000","last_change":"2024-09-17T05:03:17.471433+0000","last_active":"2024-09-17T05:03:17.474724+0000","last_peered":"2024-09-17T05:03:17.474724+0000","last_clean":"2024-09-17T05:03:17.474724+0000","last_became_active":"2024-09-17T05:03:17.470676+0000","last_became_peered":"2024-09-17T05:03:17.470676+0000","last_unstale":"2024-09-17T05:03:17.474724+0000","last_undegraded":"2024-09-17T05:03:17.474724+0000","last_fullsized":"2024-09-17T05:03:17.474724+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:27.740332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466246+0000","last_change":"2024-09-17T05:03:17.466246+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466246+0000","last_undegraded":"2024-09-17T05:03:17.466246+0000","last_fullsized":"2024-09-17T05:03:17.466246+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.484675+0000","last_change":"2024-09-17T05:03:17.481418+0000","last_active":"2024-09-17T05:03:17.484675+0000","last_peered":"2024-09-17T05:03:17.484675+0000","last_clean":"2024-09-17T05:03:17.484675+0000","last_became_active":"2024-09-17T05:03:17.481090+0000","last_became_peered":"2024-09-17T05:03:17.481090+0000","last_unstale":"2024-09-17T05:03:17.484675+0000","last_undegraded":"2024-09-17T05:03:17.484675+0000","last_fullsized":"2024-09-17T05:03:17.484675+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:39.926235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487212+0000","last_change":"2024-09-17T05:03:18.496866+0000","last_active":"2024-09-17T05:03:19.487212+0000","last_peered":"2024-09-17T05:03:19.487212+0000","last_clean":"2024-09-17T05:03:19.487212+0000","last_became_active":"2024-09-17T05:03:18.496533+0000","last_became_peered":"2024-09-17T05:03:18.496533+0000","last_unstale":"2024-09-17T05:03:19.487212+0000","last_undegraded":"2024-09-17T05:03:19.487212+0000","last_fullsized":"2024-09-17T05:03:19.487212+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:06.312704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487198+0000","last_change":"2024-09-17T05:03:17.470269+0000","last_active":"2024-09-17T05:03:19.487198+0000","last_peered":"2024-09-17T05:03:19.487198+0000","last_clean":"2024-09-17T05:03:19.487198+0000","last_became_active":"2024-09-17T05:03:17.469912+0000","last_became_peered":"2024-09-17T05:03:17.469912+0000","last_unstale":"2024-09-17T05:03:19.487198+0000","last_undegraded":"2024-09-17T05:03:19.487198+0000","last_fullsized":"2024-09-17T05:03:19.487198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:33.570989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462044+0000","last_change":"2024-09-17T05:03:17.462044+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462044+0000","last_undegraded":"2024-09-17T05:03:17.462044+0000","last_fullsized":"2024-09-17T05:03:17.462044+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487835+0000","last_change":"2024-09-17T05:03:17.486561+0000","last_active":"2024-09-17T05:03:17.487835+0000","last_peered":"2024-09-17T05:03:17.487835+0000","last_clean":"2024-09-17T05:03:17.487835+0000","last_became_active":"2024-09-17T05:03:17.486224+0000","last_became_peered":"2024-09-17T05:03:17.486224+0000","last_unstale":"2024-09-17T05:03:17.487835+0000","last_undegraded":"2024-09-17T05:03:17.487835+0000","last_fullsized":"2024-09-17T05:03:17.487835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:51.146579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487417+0000","last_change":"2024-09-17T05:03:18.504505+0000","last_active":"2024-09-17T05:03:19.487417+0000","last_peered":"2024-09-17T05:03:19.487417+0000","last_clean":"2024-09-17T05:03:19.487417+0000","last_became_active":"2024-09-17T05:03:18.503742+0000","last_became_peered":"2024-09-17T05:03:18.503742+0000","last_unstale":"2024-09-17T05:03:19.487417+0000","last_undegraded":"2024-09-17T05:03:19.487417+0000","last_fullsized":"2024-09-17T05:03:19.487417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.483791+0000","last_change":"2024-09-17T05:03:17.479945+0000","last_active":"2024-09-17T05:03:17.483791+0000","last_peered":"2024-09-17T05:03:17.483791+0000","last_clean":"2024-09-17T05:03:17.483791+0000","last_became_active":"2024-09-17T05:03:17.479596+0000","last_became_peered":"2024-09-17T05:03:17.479596+0000","last_unstale":"2024-09-17T05:03:17.483791+0000","last_undegraded":"2024-09-17T05:03:17.483791+0000","last_fullsized":"2024-09-17T05:03:17.483791+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:33.379573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470770+0000","last_change":"2024-09-17T05:03:17.470770+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470770+0000","last_undegraded":"2024-09-17T05:03:17.470770+0000","last_fullsized":"2024-09-17T05:03:17.470770+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.480660+0000","last_change":"2024-09-17T05:03:17.476961+0000","last_active":"2024-09-17T05:03:17.480660+0000","last_peered":"2024-09-17T05:03:17.480660+0000","last_clean":"2024-09-17T05:03:17.480660+0000","last_became_active":"2024-09-17T05:03:17.476427+0000","last_became_peered":"2024-09-17T05:03:17.476427+0000","last_unstale":"2024-09-17T05:03:17.480660+0000","last_undegraded":"2024-09-17T05:03:17.480660+0000","last_fullsized":"2024-09-17T05:03:17.480660+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.043546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.471261+0000","last_change":"2024-09-17T05:03:17.471261+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.471261+0000","last_undegraded":"2024-09-17T05:03:17.471261+0000","last_fullsized":"2024-09-17T05:03:17.471261+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494256+0000","last_change":"2024-09-17T05:03:17.483817+0000","last_active":"2024-09-17T05:03:19.494256+0000","last_peered":"2024-09-17T05:03:19.494256+0000","last_clean":"2024-09-17T05:03:19.494256+0000","last_became_active":"2024-09-17T05:03:17.483478+0000","last_became_peered":"2024-09-17T05:03:17.483478+0000","last_unstale":"2024-09-17T05:03:19.494256+0000","last_undegraded":"2024-09-17T05:03:19.494256+0000","last_fullsized":"2024-09-17T05:03:19.494256+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:27.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.476740+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:17.476740+0000","last_peered":"2024-09-17T05:03:17.476740+0000","last_clean":"2024-09-17T05:03:17.476740+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:17.476740+0000","last_undegraded":"2024-09-17T05:03:17.476740+0000","last_fullsized":"2024-09-17T05:03:17.476740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":845667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":598016,"data_stored":261966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1232896,"data_stored":867895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:22.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 -- 172.21.15.16:0/2983868143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f586815c270 msgr2=0x7f586815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 --2- 172.21.15.16:0/2983868143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f586815c270 0x7f586815c680 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5854009870 tx=0x7f585401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 -- 172.21.15.16:0/2983868143 shutdown_connections 2024-09-17T05:03:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 --2- 172.21.15.16:0/2983868143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f586815c270 0x7f586815c680 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 -- 172.21.15.16:0/2983868143 >> 172.21.15.16:0/2983868143 conn(0x7f586806f900 msgr2=0x7f58680b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 -- 172.21.15.16:0/2983868143 shutdown_connections 2024-09-17T05:03:22.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.897+0000 7f5870000640 1 -- 172.21.15.16:0/2983868143 wait complete. 2024-09-17T05:03:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 Processor -- start 2024-09-17T05:03:22.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 -- start start 2024-09-17T05:03:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5868073240 0x7f586814de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 0x7f58680cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f586815c270 0x7f58680cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5868169e50 con 0x7f58680b2f50 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5868169cd0 con 0x7f586815c270 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f5870000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5868169fd0 con 0x7f5868073240 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 0x7f58680cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 0x7f58680cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45762/0 (socket says 172.21.15.16:45762) 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 -- 172.21.15.16:0/670432621 learned_addr learned my addr 172.21.15.16:0/670432621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586d600640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5868073240 0x7f586814de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586e000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f586815c270 0x7f58680cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 -- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5868073240 msgr2=0x7f586814de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5868073240 0x7f586814de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 -- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f586815c270 msgr2=0x7f58680cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f586815c270 0x7f58680cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586cc00640 1 -- 172.21.15.16:0/670432621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58681fa850 con 0x7f58680b2f50 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.901+0000 7f586e000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f586815c270 0x7f58680cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f586cc00640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 0x7f58680cee80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5854000c10 tx=0x7f5854005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58540bb070 con 0x7f58680b2f50 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5854004330 con 0x7f58680b2f50 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58540b2040 con 0x7f58680b2f50 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58681f9fb0 con 0x7f58680b2f50 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681fa570 con 0x7f58680b2f50 2024-09-17T05:03:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586815c680 con 0x7f58680b2f50 2024-09-17T05:03:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5854003820 con 0x7f58680b2f50 2024-09-17T05:03:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5865600640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5848073200 0x7f5848075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.905+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f585412da90 con 0x7f58680b2f50 2024-09-17T05:03:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.909+0000 7f586d600640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5848073200 0x7f5848075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.909+0000 7f586d600640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5848073200 0x7f5848075710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f585c001130 tx=0x7f585c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:22.909+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f58540f1db0 con 0x7f58680b2f50 2024-09-17T05:03:23.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.081+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58681f9d40 con 0x7f5848073200 2024-09-17T05:03:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5865600640 1 -- 172.21.15.16:0/670432621 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247336 (secure 0 0 0) 0x7f58681f9d40 con 0x7f5848073200 2024-09-17T05:03:23.094 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5848073200 msgr2=0x7f5848075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5848073200 0x7f5848075710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f585c001130 tx=0x7f585c009040 comp rx=0 tx=0).stop 2024-09-17T05:03:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 msgr2=0x7f58680cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 0x7f58680cee80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5854000c10 tx=0x7f5854005d20 comp rx=0 tx=0).stop 2024-09-17T05:03:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 shutdown_connections 2024-09-17T05:03:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5848073200 0x7f5848075710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f586815c270 0x7f58680cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58680b2f50 0x7f58680cee80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 --2- 172.21.15.16:0/670432621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5868073240 0x7f586814de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 >> 172.21.15.16:0/670432621 conn(0x7f586806f900 msgr2=0x7f58680b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:23.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 shutdown_connections 2024-09-17T05:03:23.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.089+0000 7f5870000640 1 -- 172.21.15.16:0/670432621 wait complete. 2024-09-17T05:03:23.114 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-17T05:03:21.495567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":111,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242848,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731872,"statfs":{"total":773094113280,"available":772845436928,"internally_reserved":0,"allocated":5939200,"data_stored":3252023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.044611"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.478211+0000","last_change":"2024-09-17T05:03:17.478211+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.478211+0000","last_undegraded":"2024-09-17T05:03:17.478211+0000","last_fullsized":"2024-09-17T05:03:17.478211+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466131+0000","last_change":"2024-09-17T05:03:17.466131+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466131+0000","last_undegraded":"2024-09-17T05:03:17.466131+0000","last_fullsized":"2024-09-17T05:03:17.466131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468087+0000","last_change":"2024-09-17T05:03:17.468087+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468087+0000","last_undegraded":"2024-09-17T05:03:17.468087+0000","last_fullsized":"2024-09-17T05:03:17.468087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.473650+0000","last_change":"2024-09-17T05:03:17.473650+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.473650+0000","last_undegraded":"2024-09-17T05:03:17.473650+0000","last_fullsized":"2024-09-17T05:03:17.473650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466191+0000","last_change":"2024-09-17T05:03:17.466191+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466191+0000","last_undegraded":"2024-09-17T05:03:17.466191+0000","last_fullsized":"2024-09-17T05:03:17.466191+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468839+0000","last_change":"2024-09-17T05:03:17.468839+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468839+0000","last_undegraded":"2024-09-17T05:03:17.468839+0000","last_fullsized":"2024-09-17T05:03:17.468839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486528+0000","last_change":"2024-09-17T05:03:18.485411+0000","last_active":"2024-09-17T05:03:19.486528+0000","last_peered":"2024-09-17T05:03:19.486528+0000","last_clean":"2024-09-17T05:03:19.486528+0000","last_became_active":"2024-09-17T05:03:18.485074+0000","last_became_peered":"2024-09-17T05:03:18.485074+0000","last_unstale":"2024-09-17T05:03:19.486528+0000","last_undegraded":"2024-09-17T05:03:19.486528+0000","last_fullsized":"2024-09-17T05:03:19.486528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:07.445182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468046+0000","last_change":"2024-09-17T05:03:17.468046+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468046+0000","last_undegraded":"2024-09-17T05:03:17.468046+0000","last_fullsized":"2024-09-17T05:03:17.468046+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.467189+0000","last_change":"2024-09-17T05:03:17.467189+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.467189+0000","last_undegraded":"2024-09-17T05:03:17.467189+0000","last_fullsized":"2024-09-17T05:03:17.467189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487208+0000","last_change":"2024-09-17T05:03:18.490713+0000","last_active":"2024-09-17T05:03:19.487208+0000","last_peered":"2024-09-17T05:03:19.487208+0000","last_clean":"2024-09-17T05:03:19.487208+0000","last_became_active":"2024-09-17T05:03:18.490369+0000","last_became_peered":"2024-09-17T05:03:18.490369+0000","last_unstale":"2024-09-17T05:03:19.487208+0000","last_undegraded":"2024-09-17T05:03:19.487208+0000","last_fullsized":"2024-09-17T05:03:19.487208+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.356295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474235+0000","last_change":"2024-09-17T05:03:17.474235+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474235+0000","last_undegraded":"2024-09-17T05:03:17.474235+0000","last_fullsized":"2024-09-17T05:03:17.474235+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468399+0000","last_change":"2024-09-17T05:03:17.468399+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468399+0000","last_undegraded":"2024-09-17T05:03:17.468399+0000","last_fullsized":"2024-09-17T05:03:17.468399+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468489+0000","last_change":"2024-09-17T05:03:17.468489+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468489+0000","last_undegraded":"2024-09-17T05:03:17.468489+0000","last_fullsized":"2024-09-17T05:03:17.468489+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.477051+0000","last_change":"2024-09-17T05:03:17.477051+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.477051+0000","last_undegraded":"2024-09-17T05:03:17.477051+0000","last_fullsized":"2024-09-17T05:03:17.477051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474677+0000","last_change":"2024-09-17T05:03:17.474677+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474677+0000","last_undegraded":"2024-09-17T05:03:17.474677+0000","last_fullsized":"2024-09-17T05:03:17.474677+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.467432+0000","last_change":"2024-09-17T05:03:17.467432+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.467432+0000","last_undegraded":"2024-09-17T05:03:17.467432+0000","last_fullsized":"2024-09-17T05:03:17.467432+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472328+0000","last_change":"2024-09-17T05:03:17.472328+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472328+0000","last_undegraded":"2024-09-17T05:03:17.472328+0000","last_fullsized":"2024-09-17T05:03:17.472328+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474185+0000","last_change":"2024-09-17T05:03:17.474185+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474185+0000","last_undegraded":"2024-09-17T05:03:17.474185+0000","last_fullsized":"2024-09-17T05:03:17.474185+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.467619+0000","last_change":"2024-09-17T05:03:17.467619+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.467619+0000","last_undegraded":"2024-09-17T05:03:17.467619+0000","last_fullsized":"2024-09-17T05:03:17.467619+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465808+0000","last_change":"2024-09-17T05:03:17.465808+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465808+0000","last_undegraded":"2024-09-17T05:03:17.465808+0000","last_fullsized":"2024-09-17T05:03:17.465808+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472172+0000","last_change":"2024-09-17T05:03:17.472172+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472172+0000","last_undegraded":"2024-09-17T05:03:17.472172+0000","last_fullsized":"2024-09-17T05:03:17.472172+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465345+0000","last_change":"2024-09-17T05:03:17.465345+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465345+0000","last_undegraded":"2024-09-17T05:03:17.465345+0000","last_fullsized":"2024-09-17T05:03:17.465345+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465516+0000","last_change":"2024-09-17T05:03:17.465516+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465516+0000","last_undegraded":"2024-09-17T05:03:17.465516+0000","last_fullsized":"2024-09-17T05:03:17.465516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463065+0000","last_change":"2024-09-17T05:03:17.463065+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463065+0000","last_undegraded":"2024-09-17T05:03:17.463065+0000","last_fullsized":"2024-09-17T05:03:17.463065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.477654+0000","last_change":"2024-09-17T05:03:17.477654+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.477654+0000","last_undegraded":"2024-09-17T05:03:17.477654+0000","last_fullsized":"2024-09-17T05:03:17.477654+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.464012+0000","last_change":"2024-09-17T05:03:17.464012+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.464012+0000","last_undegraded":"2024-09-17T05:03:17.464012+0000","last_fullsized":"2024-09-17T05:03:17.464012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470026+0000","last_change":"2024-09-17T05:03:17.470026+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470026+0000","last_undegraded":"2024-09-17T05:03:17.470026+0000","last_fullsized":"2024-09-17T05:03:17.470026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465257+0000","last_change":"2024-09-17T05:03:17.465257+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465257+0000","last_undegraded":"2024-09-17T05:03:17.465257+0000","last_fullsized":"2024-09-17T05:03:17.465257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.476898+0000","last_change":"2024-09-17T05:03:17.476898+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.476898+0000","last_undegraded":"2024-09-17T05:03:17.476898+0000","last_fullsized":"2024-09-17T05:03:17.476898+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462748+0000","last_change":"2024-09-17T05:03:17.462748+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462748+0000","last_undegraded":"2024-09-17T05:03:17.462748+0000","last_fullsized":"2024-09-17T05:03:17.462748+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463326+0000","last_change":"2024-09-17T05:03:17.463326+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463326+0000","last_undegraded":"2024-09-17T05:03:17.463326+0000","last_fullsized":"2024-09-17T05:03:17.463326+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462335+0000","last_change":"2024-09-17T05:03:17.462335+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462335+0000","last_undegraded":"2024-09-17T05:03:17.462335+0000","last_fullsized":"2024-09-17T05:03:17.462335+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462819+0000","last_change":"2024-09-17T05:03:17.462819+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462819+0000","last_undegraded":"2024-09-17T05:03:17.462819+0000","last_fullsized":"2024-09-17T05:03:17.462819+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.488704+0000","last_change":"2024-09-17T05:03:17.487458+0000","last_active":"2024-09-17T05:03:17.488704+0000","last_peered":"2024-09-17T05:03:17.488704+0000","last_clean":"2024-09-17T05:03:17.488704+0000","last_became_active":"2024-09-17T05:03:17.487174+0000","last_became_peered":"2024-09-17T05:03:17.487174+0000","last_unstale":"2024-09-17T05:03:17.488704+0000","last_undegraded":"2024-09-17T05:03:17.488704+0000","last_fullsized":"2024-09-17T05:03:17.488704+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:18.208628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463888+0000","last_change":"2024-09-17T05:03:17.463888+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463888+0000","last_undegraded":"2024-09-17T05:03:17.463888+0000","last_fullsized":"2024-09-17T05:03:17.463888+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492771+0000","last_change":"2024-09-17T05:03:17.489555+0000","last_active":"2024-09-17T05:03:17.492771+0000","last_peered":"2024-09-17T05:03:17.492771+0000","last_clean":"2024-09-17T05:03:17.492771+0000","last_became_active":"2024-09-17T05:03:17.489182+0000","last_became_peered":"2024-09-17T05:03:17.489182+0000","last_unstale":"2024-09-17T05:03:17.492771+0000","last_undegraded":"2024-09-17T05:03:17.492771+0000","last_fullsized":"2024-09-17T05:03:17.492771+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:30.679941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.464628+0000","last_change":"2024-09-17T05:03:17.464628+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.464628+0000","last_undegraded":"2024-09-17T05:03:17.464628+0000","last_fullsized":"2024-09-17T05:03:17.464628+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487289+0000","last_change":"2024-09-17T05:03:17.473668+0000","last_active":"2024-09-17T05:03:19.487289+0000","last_peered":"2024-09-17T05:03:19.487289+0000","last_clean":"2024-09-17T05:03:19.487289+0000","last_became_active":"2024-09-17T05:03:17.473283+0000","last_became_peered":"2024-09-17T05:03:17.473283+0000","last_unstale":"2024-09-17T05:03:19.487289+0000","last_undegraded":"2024-09-17T05:03:19.487289+0000","last_fullsized":"2024-09-17T05:03:19.487289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:34.241801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487050+0000","last_change":"2024-09-17T05:03:18.495083+0000","last_active":"2024-09-17T05:03:19.487050+0000","last_peered":"2024-09-17T05:03:19.487050+0000","last_clean":"2024-09-17T05:03:19.487050+0000","last_became_active":"2024-09-17T05:03:18.494736+0000","last_became_peered":"2024-09-17T05:03:18.494736+0000","last_unstale":"2024-09-17T05:03:19.487050+0000","last_undegraded":"2024-09-17T05:03:19.487050+0000","last_fullsized":"2024-09-17T05:03:19.487050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:24.301106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.486423+0000","last_change":"2024-09-17T05:03:17.484272+0000","last_active":"2024-09-17T05:03:17.486423+0000","last_peered":"2024-09-17T05:03:17.486423+0000","last_clean":"2024-09-17T05:03:17.486423+0000","last_became_active":"2024-09-17T05:03:17.483583+0000","last_became_peered":"2024-09-17T05:03:17.483583+0000","last_unstale":"2024-09-17T05:03:17.486423+0000","last_undegraded":"2024-09-17T05:03:17.486423+0000","last_fullsized":"2024-09-17T05:03:17.486423+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:26.162683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472066+0000","last_change":"2024-09-17T05:03:17.472066+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472066+0000","last_undegraded":"2024-09-17T05:03:17.472066+0000","last_fullsized":"2024-09-17T05:03:17.472066+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.491356+0000","last_change":"2024-09-17T05:03:17.489922+0000","last_active":"2024-09-17T05:03:17.491356+0000","last_peered":"2024-09-17T05:03:17.491356+0000","last_clean":"2024-09-17T05:03:17.491356+0000","last_became_active":"2024-09-17T05:03:17.489611+0000","last_became_peered":"2024-09-17T05:03:17.489611+0000","last_unstale":"2024-09-17T05:03:17.491356+0000","last_undegraded":"2024-09-17T05:03:17.491356+0000","last_fullsized":"2024-09-17T05:03:17.491356+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:36.881176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.465612+0000","last_change":"2024-09-17T05:03:17.465612+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.465612+0000","last_undegraded":"2024-09-17T05:03:17.465612+0000","last_fullsized":"2024-09-17T05:03:17.465612+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.478897+0000","last_change":"2024-09-17T05:03:17.477682+0000","last_active":"2024-09-17T05:03:17.478897+0000","last_peered":"2024-09-17T05:03:17.478897+0000","last_clean":"2024-09-17T05:03:17.478897+0000","last_became_active":"2024-09-17T05:03:17.477257+0000","last_became_peered":"2024-09-17T05:03:17.477257+0000","last_unstale":"2024-09-17T05:03:17.478897+0000","last_undegraded":"2024-09-17T05:03:17.478897+0000","last_fullsized":"2024-09-17T05:03:17.478897+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:43.005082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487353+0000","last_change":"2024-09-17T05:03:18.504857+0000","last_active":"2024-09-17T05:03:19.487353+0000","last_peered":"2024-09-17T05:03:19.487353+0000","last_clean":"2024-09-17T05:03:19.487353+0000","last_became_active":"2024-09-17T05:03:18.504185+0000","last_became_peered":"2024-09-17T05:03:18.504185+0000","last_unstale":"2024-09-17T05:03:19.487353+0000","last_undegraded":"2024-09-17T05:03:19.487353+0000","last_fullsized":"2024-09-17T05:03:19.487353+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:51.971141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.489670+0000","last_change":"2024-09-17T05:03:17.488461+0000","last_active":"2024-09-17T05:03:17.489670+0000","last_peered":"2024-09-17T05:03:17.489670+0000","last_clean":"2024-09-17T05:03:17.489670+0000","last_became_active":"2024-09-17T05:03:17.488076+0000","last_became_peered":"2024-09-17T05:03:17.488076+0000","last_unstale":"2024-09-17T05:03:17.489670+0000","last_undegraded":"2024-09-17T05:03:17.489670+0000","last_fullsized":"2024-09-17T05:03:17.489670+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:33.816677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472891+0000","last_change":"2024-09-17T05:03:17.472891+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472891+0000","last_undegraded":"2024-09-17T05:03:17.472891+0000","last_fullsized":"2024-09-17T05:03:17.472891+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.483798+0000","last_change":"2024-09-17T05:03:17.476199+0000","last_active":"2024-09-17T05:03:17.483798+0000","last_peered":"2024-09-17T05:03:17.483798+0000","last_clean":"2024-09-17T05:03:17.483798+0000","last_became_active":"2024-09-17T05:03:17.475834+0000","last_became_peered":"2024-09-17T05:03:17.475834+0000","last_unstale":"2024-09-17T05:03:17.483798+0000","last_undegraded":"2024-09-17T05:03:17.483798+0000","last_fullsized":"2024-09-17T05:03:17.483798+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:22.586431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470481+0000","last_change":"2024-09-17T05:03:17.470481+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470481+0000","last_undegraded":"2024-09-17T05:03:17.470481+0000","last_fullsized":"2024-09-17T05:03:17.470481+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487622+0000","last_change":"2024-09-17T05:03:17.484357+0000","last_active":"2024-09-17T05:03:17.487622+0000","last_peered":"2024-09-17T05:03:17.487622+0000","last_clean":"2024-09-17T05:03:17.487622+0000","last_became_active":"2024-09-17T05:03:17.483648+0000","last_became_peered":"2024-09-17T05:03:17.483648+0000","last_unstale":"2024-09-17T05:03:17.487622+0000","last_undegraded":"2024-09-17T05:03:17.487622+0000","last_fullsized":"2024-09-17T05:03:17.487622+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:06.028991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.474136+0000","last_change":"2024-09-17T05:03:17.474136+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.474136+0000","last_undegraded":"2024-09-17T05:03:17.474136+0000","last_fullsized":"2024-09-17T05:03:17.474136+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487665+0000","last_change":"2024-09-17T05:03:17.483837+0000","last_active":"2024-09-17T05:03:17.487665+0000","last_peered":"2024-09-17T05:03:17.487665+0000","last_clean":"2024-09-17T05:03:17.487665+0000","last_became_active":"2024-09-17T05:03:17.483474+0000","last_became_peered":"2024-09-17T05:03:17.483474+0000","last_unstale":"2024-09-17T05:03:17.487665+0000","last_undegraded":"2024-09-17T05:03:17.487665+0000","last_fullsized":"2024-09-17T05:03:17.487665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:35.734369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472247+0000","last_change":"2024-09-17T05:03:17.472247+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472247+0000","last_undegraded":"2024-09-17T05:03:17.472247+0000","last_fullsized":"2024-09-17T05:03:17.472247+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.474754+0000","last_change":"2024-09-17T05:03:17.471492+0000","last_active":"2024-09-17T05:03:17.474754+0000","last_peered":"2024-09-17T05:03:17.474754+0000","last_clean":"2024-09-17T05:03:17.474754+0000","last_became_active":"2024-09-17T05:03:17.471119+0000","last_became_peered":"2024-09-17T05:03:17.471119+0000","last_unstale":"2024-09-17T05:03:17.474754+0000","last_undegraded":"2024-09-17T05:03:17.474754+0000","last_fullsized":"2024-09-17T05:03:17.474754+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:33.579393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.461994+0000","last_change":"2024-09-17T05:03:17.461994+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.461994+0000","last_undegraded":"2024-09-17T05:03:17.461994+0000","last_fullsized":"2024-09-17T05:03:17.461994+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487172+0000","last_change":"2024-09-17T05:03:17.480393+0000","last_active":"2024-09-17T05:03:19.487172+0000","last_peered":"2024-09-17T05:03:19.487172+0000","last_clean":"2024-09-17T05:03:19.487172+0000","last_became_active":"2024-09-17T05:03:17.479912+0000","last_became_peered":"2024-09-17T05:03:17.479912+0000","last_unstale":"2024-09-17T05:03:19.487172+0000","last_undegraded":"2024-09-17T05:03:19.487172+0000","last_fullsized":"2024-09-17T05:03:19.487172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:06.645919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.463221+0000","last_change":"2024-09-17T05:03:17.463221+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.463221+0000","last_undegraded":"2024-09-17T05:03:17.463221+0000","last_fullsized":"2024-09-17T05:03:17.463221+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.488548+0000","last_change":"2024-09-17T05:03:17.482800+0000","last_active":"2024-09-17T05:03:17.488548+0000","last_peered":"2024-09-17T05:03:17.488548+0000","last_clean":"2024-09-17T05:03:17.488548+0000","last_became_active":"2024-09-17T05:03:17.482376+0000","last_became_peered":"2024-09-17T05:03:17.482376+0000","last_unstale":"2024-09-17T05:03:17.488548+0000","last_undegraded":"2024-09-17T05:03:17.488548+0000","last_fullsized":"2024-09-17T05:03:17.488548+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:04.504110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.472050+0000","last_change":"2024-09-17T05:03:17.472050+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.472050+0000","last_undegraded":"2024-09-17T05:03:17.472050+0000","last_fullsized":"2024-09-17T05:03:17.472050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.480761+0000","last_change":"2024-09-17T05:03:17.477301+0000","last_active":"2024-09-17T05:03:17.480761+0000","last_peered":"2024-09-17T05:03:17.480761+0000","last_clean":"2024-09-17T05:03:17.480761+0000","last_became_active":"2024-09-17T05:03:17.476888+0000","last_became_peered":"2024-09-17T05:03:17.476888+0000","last_unstale":"2024-09-17T05:03:17.480761+0000","last_undegraded":"2024-09-17T05:03:17.480761+0000","last_fullsized":"2024-09-17T05:03:17.480761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:30.679941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.469666+0000","last_change":"2024-09-17T05:03:17.469666+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.469666+0000","last_undegraded":"2024-09-17T05:03:17.469666+0000","last_fullsized":"2024-09-17T05:03:17.469666+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.485409+0000","last_change":"2024-09-17T05:03:17.481847+0000","last_active":"2024-09-17T05:03:17.485409+0000","last_peered":"2024-09-17T05:03:17.485409+0000","last_clean":"2024-09-17T05:03:17.485409+0000","last_became_active":"2024-09-17T05:03:17.481546+0000","last_became_peered":"2024-09-17T05:03:17.481546+0000","last_unstale":"2024-09-17T05:03:17.485409+0000","last_undegraded":"2024-09-17T05:03:17.485409+0000","last_fullsized":"2024-09-17T05:03:17.485409+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:48.075582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466681+0000","last_change":"2024-09-17T05:03:17.466681+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466681+0000","last_undegraded":"2024-09-17T05:03:17.466681+0000","last_fullsized":"2024-09-17T05:03:17.466681+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492755+0000","last_change":"2024-09-17T05:03:17.489475+0000","last_active":"2024-09-17T05:03:17.492755+0000","last_peered":"2024-09-17T05:03:17.492755+0000","last_clean":"2024-09-17T05:03:17.492755+0000","last_became_active":"2024-09-17T05:03:17.489135+0000","last_became_peered":"2024-09-17T05:03:17.489135+0000","last_unstale":"2024-09-17T05:03:17.492755+0000","last_undegraded":"2024-09-17T05:03:17.492755+0000","last_fullsized":"2024-09-17T05:03:17.492755+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.043546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486537+0000","last_change":"2024-09-17T05:03:18.487882+0000","last_active":"2024-09-17T05:03:19.486537+0000","last_peered":"2024-09-17T05:03:19.486537+0000","last_clean":"2024-09-17T05:03:19.486537+0000","last_became_active":"2024-09-17T05:03:18.487523+0000","last_became_peered":"2024-09-17T05:03:18.487523+0000","last_unstale":"2024-09-17T05:03:19.486537+0000","last_undegraded":"2024-09-17T05:03:19.486537+0000","last_fullsized":"2024-09-17T05:03:19.486537+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:44.737783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492762+0000","last_change":"2024-09-17T05:03:17.489385+0000","last_active":"2024-09-17T05:03:17.492762+0000","last_peered":"2024-09-17T05:03:17.492762+0000","last_clean":"2024-09-17T05:03:17.492762+0000","last_became_active":"2024-09-17T05:03:17.489069+0000","last_became_peered":"2024-09-17T05:03:17.489069+0000","last_unstale":"2024-09-17T05:03:17.492762+0000","last_undegraded":"2024-09-17T05:03:17.492762+0000","last_fullsized":"2024-09-17T05:03:17.492762+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:53:11.525764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470948+0000","last_change":"2024-09-17T05:03:17.470948+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470948+0000","last_undegraded":"2024-09-17T05:03:17.470948+0000","last_fullsized":"2024-09-17T05:03:17.470948+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.488510+0000","last_change":"2024-09-17T05:03:17.483869+0000","last_active":"2024-09-17T05:03:17.488510+0000","last_peered":"2024-09-17T05:03:17.488510+0000","last_clean":"2024-09-17T05:03:17.488510+0000","last_became_active":"2024-09-17T05:03:17.483565+0000","last_became_peered":"2024-09-17T05:03:17.483565+0000","last_unstale":"2024-09-17T05:03:17.488510+0000","last_undegraded":"2024-09-17T05:03:17.488510+0000","last_fullsized":"2024-09-17T05:03:17.488510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:47:59.664232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.461989+0000","last_change":"2024-09-17T05:03:17.461989+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.461989+0000","last_undegraded":"2024-09-17T05:03:17.461989+0000","last_fullsized":"2024-09-17T05:03:17.461989+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487642+0000","last_change":"2024-09-17T05:03:17.484821+0000","last_active":"2024-09-17T05:03:17.487642+0000","last_peered":"2024-09-17T05:03:17.487642+0000","last_clean":"2024-09-17T05:03:17.487642+0000","last_became_active":"2024-09-17T05:03:17.484445+0000","last_became_peered":"2024-09-17T05:03:17.484445+0000","last_unstale":"2024-09-17T05:03:17.487642+0000","last_undegraded":"2024-09-17T05:03:17.487642+0000","last_fullsized":"2024-09-17T05:03:17.487642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.469034+0000","last_change":"2024-09-17T05:03:17.469034+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.469034+0000","last_undegraded":"2024-09-17T05:03:17.469034+0000","last_fullsized":"2024-09-17T05:03:17.469034+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.478888+0000","last_change":"2024-09-17T05:03:17.469441+0000","last_active":"2024-09-17T05:03:17.478888+0000","last_peered":"2024-09-17T05:03:17.478888+0000","last_clean":"2024-09-17T05:03:17.478888+0000","last_became_active":"2024-09-17T05:03:17.469019+0000","last_became_peered":"2024-09-17T05:03:17.469019+0000","last_unstale":"2024-09-17T05:03:17.478888+0000","last_undegraded":"2024-09-17T05:03:17.478888+0000","last_fullsized":"2024-09-17T05:03:17.478888+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.464644+0000","last_change":"2024-09-17T05:03:17.464644+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.464644+0000","last_undegraded":"2024-09-17T05:03:17.464644+0000","last_fullsized":"2024-09-17T05:03:17.464644+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.492232+0000","last_change":"2024-09-17T05:03:17.488710+0000","last_active":"2024-09-17T05:03:17.492232+0000","last_peered":"2024-09-17T05:03:17.492232+0000","last_clean":"2024-09-17T05:03:17.492232+0000","last_became_active":"2024-09-17T05:03:17.488178+0000","last_became_peered":"2024-09-17T05:03:17.488178+0000","last_unstale":"2024-09-17T05:03:17.492232+0000","last_undegraded":"2024-09-17T05:03:17.492232+0000","last_fullsized":"2024-09-17T05:03:17.492232+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:21.071975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.475056+0000","last_change":"2024-09-17T05:03:17.475056+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.475056+0000","last_undegraded":"2024-09-17T05:03:17.475056+0000","last_fullsized":"2024-09-17T05:03:17.475056+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.489164+0000","last_change":"2024-09-17T05:03:17.487381+0000","last_active":"2024-09-17T05:03:17.489164+0000","last_peered":"2024-09-17T05:03:17.489164+0000","last_clean":"2024-09-17T05:03:17.489164+0000","last_became_active":"2024-09-17T05:03:17.487025+0000","last_became_peered":"2024-09-17T05:03:17.487025+0000","last_unstale":"2024-09-17T05:03:17.489164+0000","last_undegraded":"2024-09-17T05:03:17.489164+0000","last_fullsized":"2024-09-17T05:03:17.489164+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:06.819108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.471657+0000","last_change":"2024-09-17T05:03:17.471657+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.471657+0000","last_undegraded":"2024-09-17T05:03:17.471657+0000","last_fullsized":"2024-09-17T05:03:17.471657+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486706+0000","last_change":"2024-09-17T05:03:17.488730+0000","last_active":"2024-09-17T05:03:19.486706+0000","last_peered":"2024-09-17T05:03:19.486706+0000","last_clean":"2024-09-17T05:03:19.486706+0000","last_became_active":"2024-09-17T05:03:17.487850+0000","last_became_peered":"2024-09-17T05:03:17.487850+0000","last_unstale":"2024-09-17T05:03:19.486706+0000","last_undegraded":"2024-09-17T05:03:19.486706+0000","last_fullsized":"2024-09-17T05:03:19.486706+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:20.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.469271+0000","last_change":"2024-09-17T05:03:17.469271+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.469271+0000","last_undegraded":"2024-09-17T05:03:17.469271+0000","last_fullsized":"2024-09-17T05:03:17.469271+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486411+0000","last_change":"2024-09-17T05:03:17.477112+0000","last_active":"2024-09-17T05:03:19.486411+0000","last_peered":"2024-09-17T05:03:19.486411+0000","last_clean":"2024-09-17T05:03:19.486411+0000","last_became_active":"2024-09-17T05:03:17.476642+0000","last_became_peered":"2024-09-17T05:03:17.476642+0000","last_unstale":"2024-09-17T05:03:19.486411+0000","last_undegraded":"2024-09-17T05:03:19.486411+0000","last_fullsized":"2024-09-17T05:03:19.486411+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:50.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.468943+0000","last_change":"2024-09-17T05:03:17.468943+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.468943+0000","last_undegraded":"2024-09-17T05:03:17.468943+0000","last_fullsized":"2024-09-17T05:03:17.468943+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.489818+0000","last_change":"2024-09-17T05:03:17.488680+0000","last_active":"2024-09-17T05:03:17.489818+0000","last_peered":"2024-09-17T05:03:17.489818+0000","last_clean":"2024-09-17T05:03:17.489818+0000","last_became_active":"2024-09-17T05:03:17.488050+0000","last_became_peered":"2024-09-17T05:03:17.488050+0000","last_unstale":"2024-09-17T05:03:17.489818+0000","last_undegraded":"2024-09-17T05:03:17.489818+0000","last_fullsized":"2024-09-17T05:03:17.489818+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:15.702498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487087+0000","last_change":"2024-09-17T05:03:18.495602+0000","last_active":"2024-09-17T05:03:19.487087+0000","last_peered":"2024-09-17T05:03:19.487087+0000","last_clean":"2024-09-17T05:03:19.487087+0000","last_became_active":"2024-09-17T05:03:18.495190+0000","last_became_peered":"2024-09-17T05:03:18.495190+0000","last_unstale":"2024-09-17T05:03:19.487087+0000","last_undegraded":"2024-09-17T05:03:19.487087+0000","last_fullsized":"2024-09-17T05:03:19.487087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:49.184314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.474724+0000","last_change":"2024-09-17T05:03:17.471433+0000","last_active":"2024-09-17T05:03:17.474724+0000","last_peered":"2024-09-17T05:03:17.474724+0000","last_clean":"2024-09-17T05:03:17.474724+0000","last_became_active":"2024-09-17T05:03:17.470676+0000","last_became_peered":"2024-09-17T05:03:17.470676+0000","last_unstale":"2024-09-17T05:03:17.474724+0000","last_undegraded":"2024-09-17T05:03:17.474724+0000","last_fullsized":"2024-09-17T05:03:17.474724+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:27.740332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.466246+0000","last_change":"2024-09-17T05:03:17.466246+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.466246+0000","last_undegraded":"2024-09-17T05:03:17.466246+0000","last_fullsized":"2024-09-17T05:03:17.466246+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.484675+0000","last_change":"2024-09-17T05:03:17.481418+0000","last_active":"2024-09-17T05:03:17.484675+0000","last_peered":"2024-09-17T05:03:17.484675+0000","last_clean":"2024-09-17T05:03:17.484675+0000","last_became_active":"2024-09-17T05:03:17.481090+0000","last_became_peered":"2024-09-17T05:03:17.481090+0000","last_unstale":"2024-09-17T05:03:17.484675+0000","last_undegraded":"2024-09-17T05:03:17.484675+0000","last_fullsized":"2024-09-17T05:03:17.484675+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:39.926235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487212+0000","last_change":"2024-09-17T05:03:18.496866+0000","last_active":"2024-09-17T05:03:19.487212+0000","last_peered":"2024-09-17T05:03:19.487212+0000","last_clean":"2024-09-17T05:03:19.487212+0000","last_became_active":"2024-09-17T05:03:18.496533+0000","last_became_peered":"2024-09-17T05:03:18.496533+0000","last_unstale":"2024-09-17T05:03:19.487212+0000","last_undegraded":"2024-09-17T05:03:19.487212+0000","last_fullsized":"2024-09-17T05:03:19.487212+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:06.312704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487198+0000","last_change":"2024-09-17T05:03:17.470269+0000","last_active":"2024-09-17T05:03:19.487198+0000","last_peered":"2024-09-17T05:03:19.487198+0000","last_clean":"2024-09-17T05:03:19.487198+0000","last_became_active":"2024-09-17T05:03:17.469912+0000","last_became_peered":"2024-09-17T05:03:17.469912+0000","last_unstale":"2024-09-17T05:03:19.487198+0000","last_undegraded":"2024-09-17T05:03:19.487198+0000","last_fullsized":"2024-09-17T05:03:19.487198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:33.570989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.462044+0000","last_change":"2024-09-17T05:03:17.462044+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.462044+0000","last_undegraded":"2024-09-17T05:03:17.462044+0000","last_fullsized":"2024-09-17T05:03:17.462044+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.487835+0000","last_change":"2024-09-17T05:03:17.486561+0000","last_active":"2024-09-17T05:03:17.487835+0000","last_peered":"2024-09-17T05:03:17.487835+0000","last_clean":"2024-09-17T05:03:17.487835+0000","last_became_active":"2024-09-17T05:03:17.486224+0000","last_became_peered":"2024-09-17T05:03:17.486224+0000","last_unstale":"2024-09-17T05:03:17.487835+0000","last_undegraded":"2024-09-17T05:03:17.487835+0000","last_fullsized":"2024-09-17T05:03:17.487835+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:51.146579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487417+0000","last_change":"2024-09-17T05:03:18.504505+0000","last_active":"2024-09-17T05:03:19.487417+0000","last_peered":"2024-09-17T05:03:19.487417+0000","last_clean":"2024-09-17T05:03:19.487417+0000","last_became_active":"2024-09-17T05:03:18.503742+0000","last_became_peered":"2024-09-17T05:03:18.503742+0000","last_unstale":"2024-09-17T05:03:19.487417+0000","last_undegraded":"2024-09-17T05:03:19.487417+0000","last_fullsized":"2024-09-17T05:03:19.487417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.483791+0000","last_change":"2024-09-17T05:03:17.479945+0000","last_active":"2024-09-17T05:03:17.483791+0000","last_peered":"2024-09-17T05:03:17.483791+0000","last_clean":"2024-09-17T05:03:17.483791+0000","last_became_active":"2024-09-17T05:03:17.479596+0000","last_became_peered":"2024-09-17T05:03:17.479596+0000","last_unstale":"2024-09-17T05:03:17.483791+0000","last_undegraded":"2024-09-17T05:03:17.483791+0000","last_fullsized":"2024-09-17T05:03:17.483791+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:33.379573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.470770+0000","last_change":"2024-09-17T05:03:17.470770+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.470770+0000","last_undegraded":"2024-09-17T05:03:17.470770+0000","last_fullsized":"2024-09-17T05:03:17.470770+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.480660+0000","last_change":"2024-09-17T05:03:17.476961+0000","last_active":"2024-09-17T05:03:17.480660+0000","last_peered":"2024-09-17T05:03:17.480660+0000","last_clean":"2024-09-17T05:03:17.480660+0000","last_became_active":"2024-09-17T05:03:17.476427+0000","last_became_peered":"2024-09-17T05:03:17.476427+0000","last_unstale":"2024-09-17T05:03:17.480660+0000","last_undegraded":"2024-09-17T05:03:17.480660+0000","last_fullsized":"2024-09-17T05:03:17.480660+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:08.043546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-09-17T05:03:17.471261+0000","last_change":"2024-09-17T05:03:17.471261+0000","last_active":"2024-09-17T05:03:17.450231+0000","last_peered":"2024-09-17T05:03:17.450231+0000","last_clean":"2024-09-17T05:03:17.450231+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:17.471261+0000","last_undegraded":"2024-09-17T05:03:17.471261+0000","last_fullsized":"2024-09-17T05:03:17.471261+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494256+0000","last_change":"2024-09-17T05:03:17.483817+0000","last_active":"2024-09-17T05:03:19.494256+0000","last_peered":"2024-09-17T05:03:19.494256+0000","last_clean":"2024-09-17T05:03:19.494256+0000","last_became_active":"2024-09-17T05:03:17.483478+0000","last_became_peered":"2024-09-17T05:03:17.483478+0000","last_unstale":"2024-09-17T05:03:19.494256+0000","last_undegraded":"2024-09-17T05:03:19.494256+0000","last_fullsized":"2024-09-17T05:03:19.494256+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:27.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-17T05:03:17.476740+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:17.476740+0000","last_peered":"2024-09-17T05:03:17.476740+0000","last_clean":"2024-09-17T05:03:17.476740+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:17.476740+0000","last_undegraded":"2024-09-17T05:03:17.476740+0000","last_fullsized":"2024-09-17T05:03:17.476740+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":255299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":845667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509472,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":598016,"data_stored":261966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1232896,"data_stored":867895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:23.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 -- 172.21.15.16:0/3945869842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15c0a0 msgr2=0x7f537c15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 --2- 172.21.15.16:0/3945869842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15c0a0 0x7f537c15c4b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5368009900 tx=0x7f536801c850 comp rx=0 tx=0).stop 2024-09-17T05:03:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 -- 172.21.15.16:0/3945869842 shutdown_connections 2024-09-17T05:03:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 --2- 172.21.15.16:0/3945869842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15c0a0 0x7f537c15c4b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 --2- 172.21.15.16:0/3945869842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c0b2f50 0x7f537c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 -- 172.21.15.16:0/3945869842 >> 172.21.15.16:0/3945869842 conn(0x7f537c06f900 msgr2=0x7f537c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 -- 172.21.15.16:0/3945869842 shutdown_connections 2024-09-17T05:03:23.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.341+0000 7f5383600640 1 -- 172.21.15.16:0/3945869842 wait complete. 2024-09-17T05:03:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 Processor -- start 2024-09-17T05:03:23.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 -- start start 2024-09-17T05:03:23.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c0b2f50 0x7f537c1f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15c0a0 0x7f537c162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 0x7f537c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f537c169150 con 0x7f537c15f150 2024-09-17T05:03:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f537c168fd0 con 0x7f537c0b2f50 2024-09-17T05:03:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f537c1692d0 con 0x7f537c15c0a0 2024-09-17T05:03:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 0x7f537c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 0x7f537c14fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45770/0 (socket says 172.21.15.16:45770) 2024-09-17T05:03:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 -- 172.21.15.16:0/4278681449 learned_addr learned my addr 172.21.15.16:0/4278681449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5380c00640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c0b2f50 0x7f537c1f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f537be00640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15c0a0 0x7f537c162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 -- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15c0a0 msgr2=0x7f537c162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15c0a0 0x7f537c162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 -- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c0b2f50 msgr2=0x7f537c1f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c0b2f50 0x7f537c1f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 -- 172.21.15.16:0/4278681449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c15b250 con 0x7f537c15f150 2024-09-17T05:03:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5381600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 0x7f537c14fed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f537400c730 tx=0x7f537400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5374003b80 con 0x7f537c15f150 2024-09-17T05:03:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5374003d20 con 0x7f537c15f150 2024-09-17T05:03:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c160390 con 0x7f537c15f150 2024-09-17T05:03:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537400aab0 con 0x7f537c15f150 2024-09-17T05:03:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.345+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c160950 con 0x7f537c15f150 2024-09-17T05:03:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.349+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c15bb60 con 0x7f537c15f150 2024-09-17T05:03:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.349+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f537400ac50 con 0x7f537c15f150 2024-09-17T05:03:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.349+0000 7f5378c00640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f535c073200 0x7f535c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.349+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f53740940b0 con 0x7f537c15f150 2024-09-17T05:03:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.349+0000 7f5380c00640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f535c073200 0x7f535c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.353+0000 7f5380c00640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f535c073200 0x7f535c075710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f537000bb50 tx=0x7f537000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.353+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f537405e8e0 con 0x7f537c15f150 2024-09-17T05:03:23.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.525+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f537c15b530 con 0x7f535c073200 2024-09-17T05:03:23.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.533+0000 7f5378c00640 1 -- 172.21.15.16:0/4278681449 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f537c15b530 con 0x7f535c073200 2024-09-17T05:03:23.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:23.539 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f535c073200 msgr2=0x7f535c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f535c073200 0x7f535c075710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f537000bb50 tx=0x7f537000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 msgr2=0x7f537c14fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 0x7f537c14fed0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f537400c730 tx=0x7f537400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 shutdown_connections 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f535c073200 0x7f535c075710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c15f150 0x7f537c14fed0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f537c15c0a0 0x7f537c162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 --2- 172.21.15.16:0/4278681449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c0b2f50 0x7f537c1f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 >> 172.21.15.16:0/4278681449 conn(0x7f537c06f900 msgr2=0x7f537c15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 shutdown_connections 2024-09-17T05:03:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.537+0000 7f5383600640 1 -- 172.21.15.16:0/4278681449 wait complete. 2024-09-17T05:03:23.558 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-17T05:03:23.496192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243112,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731608,"statfs":{"total":773094113280,"available":772845166592,"internally_reserved":0,"allocated":6234112,"data_stored":3392156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.041179"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795632+0000","last_change":"2024-09-17T05:03:18.501500+0000","last_active":"2024-09-17T05:03:21.795632+0000","last_peered":"2024-09-17T05:03:21.795632+0000","last_clean":"2024-09-17T05:03:21.795632+0000","last_became_active":"2024-09-17T05:03:18.501008+0000","last_became_peered":"2024-09-17T05:03:18.501008+0000","last_unstale":"2024-09-17T05:03:21.795632+0000","last_undegraded":"2024-09-17T05:03:21.795632+0000","last_fullsized":"2024-09-17T05:03:21.795632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:37.679190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796248+0000","last_change":"2024-09-17T05:03:18.490954+0000","last_active":"2024-09-17T05:03:21.796248+0000","last_peered":"2024-09-17T05:03:21.796248+0000","last_clean":"2024-09-17T05:03:21.796248+0000","last_became_active":"2024-09-17T05:03:18.490645+0000","last_became_peered":"2024-09-17T05:03:18.490645+0000","last_unstale":"2024-09-17T05:03:21.796248+0000","last_undegraded":"2024-09-17T05:03:21.796248+0000","last_fullsized":"2024-09-17T05:03:21.796248+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:07.251450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485314+0000","last_change":"2024-09-17T05:03:18.505087+0000","last_active":"2024-09-17T05:03:19.485314+0000","last_peered":"2024-09-17T05:03:19.485314+0000","last_clean":"2024-09-17T05:03:19.485314+0000","last_became_active":"2024-09-17T05:03:18.504758+0000","last_became_peered":"2024-09-17T05:03:18.504758+0000","last_unstale":"2024-09-17T05:03:19.485314+0000","last_undegraded":"2024-09-17T05:03:19.485314+0000","last_fullsized":"2024-09-17T05:03:19.485314+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:15.676197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484737+0000","last_change":"2024-09-17T05:03:18.498547+0000","last_active":"2024-09-17T05:03:19.484737+0000","last_peered":"2024-09-17T05:03:19.484737+0000","last_clean":"2024-09-17T05:03:19.484737+0000","last_became_active":"2024-09-17T05:03:18.498140+0000","last_became_peered":"2024-09-17T05:03:18.498140+0000","last_unstale":"2024-09-17T05:03:19.484737+0000","last_undegraded":"2024-09-17T05:03:19.484737+0000","last_fullsized":"2024-09-17T05:03:19.484737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795682+0000","last_change":"2024-09-17T05:03:18.493490+0000","last_active":"2024-09-17T05:03:21.795682+0000","last_peered":"2024-09-17T05:03:21.795682+0000","last_clean":"2024-09-17T05:03:21.795682+0000","last_became_active":"2024-09-17T05:03:18.493086+0000","last_became_peered":"2024-09-17T05:03:18.493086+0000","last_unstale":"2024-09-17T05:03:21.795682+0000","last_undegraded":"2024-09-17T05:03:21.795682+0000","last_fullsized":"2024-09-17T05:03:21.795682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:47.377246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414436+0000","last_change":"2024-09-17T05:03:18.503188+0000","last_active":"2024-09-17T05:03:20.414436+0000","last_peered":"2024-09-17T05:03:20.414436+0000","last_clean":"2024-09-17T05:03:20.414436+0000","last_became_active":"2024-09-17T05:03:18.502531+0000","last_became_peered":"2024-09-17T05:03:18.502531+0000","last_unstale":"2024-09-17T05:03:20.414436+0000","last_undegraded":"2024-09-17T05:03:20.414436+0000","last_fullsized":"2024-09-17T05:03:20.414436+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486528+0000","last_change":"2024-09-17T05:03:18.485411+0000","last_active":"2024-09-17T05:03:19.486528+0000","last_peered":"2024-09-17T05:03:19.486528+0000","last_clean":"2024-09-17T05:03:19.486528+0000","last_became_active":"2024-09-17T05:03:18.485074+0000","last_became_peered":"2024-09-17T05:03:18.485074+0000","last_unstale":"2024-09-17T05:03:19.486528+0000","last_undegraded":"2024-09-17T05:03:19.486528+0000","last_fullsized":"2024-09-17T05:03:19.486528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:07.445182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414394+0000","last_change":"2024-09-17T05:03:18.499467+0000","last_active":"2024-09-17T05:03:20.414394+0000","last_peered":"2024-09-17T05:03:20.414394+0000","last_clean":"2024-09-17T05:03:20.414394+0000","last_became_active":"2024-09-17T05:03:18.499137+0000","last_became_peered":"2024-09-17T05:03:18.499137+0000","last_unstale":"2024-09-17T05:03:20.414394+0000","last_undegraded":"2024-09-17T05:03:20.414394+0000","last_fullsized":"2024-09-17T05:03:20.414394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:18.636910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485862+0000","last_change":"2024-09-17T05:03:18.496016+0000","last_active":"2024-09-17T05:03:19.485862+0000","last_peered":"2024-09-17T05:03:19.485862+0000","last_clean":"2024-09-17T05:03:19.485862+0000","last_became_active":"2024-09-17T05:03:18.495706+0000","last_became_peered":"2024-09-17T05:03:18.495706+0000","last_unstale":"2024-09-17T05:03:19.485862+0000","last_undegraded":"2024-09-17T05:03:19.485862+0000","last_fullsized":"2024-09-17T05:03:19.485862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:03.396864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487208+0000","last_change":"2024-09-17T05:03:18.490713+0000","last_active":"2024-09-17T05:03:19.487208+0000","last_peered":"2024-09-17T05:03:19.487208+0000","last_clean":"2024-09-17T05:03:19.487208+0000","last_became_active":"2024-09-17T05:03:18.490369+0000","last_became_peered":"2024-09-17T05:03:18.490369+0000","last_unstale":"2024-09-17T05:03:19.487208+0000","last_undegraded":"2024-09-17T05:03:19.487208+0000","last_fullsized":"2024-09-17T05:03:19.487208+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.356295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795500+0000","last_change":"2024-09-17T05:03:18.502622+0000","last_active":"2024-09-17T05:03:21.795500+0000","last_peered":"2024-09-17T05:03:21.795500+0000","last_clean":"2024-09-17T05:03:21.795500+0000","last_became_active":"2024-09-17T05:03:18.502143+0000","last_became_peered":"2024-09-17T05:03:18.502143+0000","last_unstale":"2024-09-17T05:03:21.795500+0000","last_undegraded":"2024-09-17T05:03:21.795500+0000","last_fullsized":"2024-09-17T05:03:21.795500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:17.126958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414390+0000","last_change":"2024-09-17T05:03:18.499587+0000","last_active":"2024-09-17T05:03:20.414390+0000","last_peered":"2024-09-17T05:03:20.414390+0000","last_clean":"2024-09-17T05:03:20.414390+0000","last_became_active":"2024-09-17T05:03:18.499252+0000","last_became_peered":"2024-09-17T05:03:18.499252+0000","last_unstale":"2024-09-17T05:03:20.414390+0000","last_undegraded":"2024-09-17T05:03:20.414390+0000","last_fullsized":"2024-09-17T05:03:20.414390+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414386+0000","last_change":"2024-09-17T05:03:18.504641+0000","last_active":"2024-09-17T05:03:20.414386+0000","last_peered":"2024-09-17T05:03:20.414386+0000","last_clean":"2024-09-17T05:03:20.414386+0000","last_became_active":"2024-09-17T05:03:18.504311+0000","last_became_peered":"2024-09-17T05:03:18.504311+0000","last_unstale":"2024-09-17T05:03:20.414386+0000","last_undegraded":"2024-09-17T05:03:20.414386+0000","last_fullsized":"2024-09-17T05:03:20.414386+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:52.037853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795414+0000","last_change":"2024-09-17T05:03:18.499614+0000","last_active":"2024-09-17T05:03:21.795414+0000","last_peered":"2024-09-17T05:03:21.795414+0000","last_clean":"2024-09-17T05:03:21.795414+0000","last_became_active":"2024-09-17T05:03:18.498970+0000","last_became_peered":"2024-09-17T05:03:18.498970+0000","last_unstale":"2024-09-17T05:03:21.795414+0000","last_undegraded":"2024-09-17T05:03:21.795414+0000","last_fullsized":"2024-09-17T05:03:21.795414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:01.114758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484404+0000","last_change":"2024-09-17T05:03:18.486986+0000","last_active":"2024-09-17T05:03:19.484404+0000","last_peered":"2024-09-17T05:03:19.484404+0000","last_clean":"2024-09-17T05:03:19.484404+0000","last_became_active":"2024-09-17T05:03:18.486638+0000","last_became_peered":"2024-09-17T05:03:18.486638+0000","last_unstale":"2024-09-17T05:03:19.484404+0000","last_undegraded":"2024-09-17T05:03:19.484404+0000","last_fullsized":"2024-09-17T05:03:19.484404+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:04.033462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414318+0000","last_change":"2024-09-17T05:03:18.508226+0000","last_active":"2024-09-17T05:03:20.414318+0000","last_peered":"2024-09-17T05:03:20.414318+0000","last_clean":"2024-09-17T05:03:20.414318+0000","last_became_active":"2024-09-17T05:03:18.506993+0000","last_became_peered":"2024-09-17T05:03:18.506993+0000","last_unstale":"2024-09-17T05:03:20.414318+0000","last_undegraded":"2024-09-17T05:03:20.414318+0000","last_fullsized":"2024-09-17T05:03:20.414318+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:01.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796065+0000","last_change":"2024-09-17T05:03:18.501235+0000","last_active":"2024-09-17T05:03:21.796065+0000","last_peered":"2024-09-17T05:03:21.796065+0000","last_clean":"2024-09-17T05:03:21.796065+0000","last_became_active":"2024-09-17T05:03:18.500966+0000","last_became_peered":"2024-09-17T05:03:18.500966+0000","last_unstale":"2024-09-17T05:03:21.796065+0000","last_undegraded":"2024-09-17T05:03:21.796065+0000","last_fullsized":"2024-09-17T05:03:21.796065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:40.933886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485015+0000","last_change":"2024-09-17T05:03:18.499110+0000","last_active":"2024-09-17T05:03:19.485015+0000","last_peered":"2024-09-17T05:03:19.485015+0000","last_clean":"2024-09-17T05:03:19.485015+0000","last_became_active":"2024-09-17T05:03:18.498458+0000","last_became_peered":"2024-09-17T05:03:18.498458+0000","last_unstale":"2024-09-17T05:03:19.485015+0000","last_undegraded":"2024-09-17T05:03:19.485015+0000","last_fullsized":"2024-09-17T05:03:19.485015+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:19.040116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485907+0000","last_change":"2024-09-17T05:03:18.498320+0000","last_active":"2024-09-17T05:03:19.485907+0000","last_peered":"2024-09-17T05:03:19.485907+0000","last_clean":"2024-09-17T05:03:19.485907+0000","last_became_active":"2024-09-17T05:03:18.498041+0000","last_became_peered":"2024-09-17T05:03:18.498041+0000","last_unstale":"2024-09-17T05:03:19.485907+0000","last_undegraded":"2024-09-17T05:03:19.485907+0000","last_fullsized":"2024-09-17T05:03:19.485907+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:49.128082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796031+0000","last_change":"2024-09-17T05:03:18.509422+0000","last_active":"2024-09-17T05:03:21.796031+0000","last_peered":"2024-09-17T05:03:21.796031+0000","last_clean":"2024-09-17T05:03:21.796031+0000","last_became_active":"2024-09-17T05:03:18.506222+0000","last_became_peered":"2024-09-17T05:03:18.506222+0000","last_unstale":"2024-09-17T05:03:21.796031+0000","last_undegraded":"2024-09-17T05:03:21.796031+0000","last_fullsized":"2024-09-17T05:03:21.796031+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:01.114758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484565+0000","last_change":"2024-09-17T05:03:18.488562+0000","last_active":"2024-09-17T05:03:19.484565+0000","last_peered":"2024-09-17T05:03:19.484565+0000","last_clean":"2024-09-17T05:03:19.484565+0000","last_became_active":"2024-09-17T05:03:18.488267+0000","last_became_peered":"2024-09-17T05:03:18.488267+0000","last_unstale":"2024-09-17T05:03:19.484565+0000","last_undegraded":"2024-09-17T05:03:19.484565+0000","last_fullsized":"2024-09-17T05:03:19.484565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485314+0000","last_change":"2024-09-17T05:03:18.491046+0000","last_active":"2024-09-17T05:03:19.485314+0000","last_peered":"2024-09-17T05:03:19.485314+0000","last_clean":"2024-09-17T05:03:19.485314+0000","last_became_active":"2024-09-17T05:03:18.490734+0000","last_became_peered":"2024-09-17T05:03:18.490734+0000","last_unstale":"2024-09-17T05:03:19.485314+0000","last_undegraded":"2024-09-17T05:03:19.485314+0000","last_fullsized":"2024-09-17T05:03:19.485314+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:49.725834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485069+0000","last_change":"2024-09-17T05:03:18.495503+0000","last_active":"2024-09-17T05:03:19.485069+0000","last_peered":"2024-09-17T05:03:19.485069+0000","last_clean":"2024-09-17T05:03:19.485069+0000","last_became_active":"2024-09-17T05:03:18.494911+0000","last_became_peered":"2024-09-17T05:03:18.494911+0000","last_unstale":"2024-09-17T05:03:19.485069+0000","last_undegraded":"2024-09-17T05:03:19.485069+0000","last_fullsized":"2024-09-17T05:03:19.485069+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:02.757954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485067+0000","last_change":"2024-09-17T05:03:18.492326+0000","last_active":"2024-09-17T05:03:19.485067+0000","last_peered":"2024-09-17T05:03:19.485067+0000","last_clean":"2024-09-17T05:03:19.485067+0000","last_became_active":"2024-09-17T05:03:18.492003+0000","last_became_peered":"2024-09-17T05:03:18.492003+0000","last_unstale":"2024-09-17T05:03:19.485067+0000","last_undegraded":"2024-09-17T05:03:19.485067+0000","last_fullsized":"2024-09-17T05:03:19.485067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:47.441012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795905+0000","last_change":"2024-09-17T05:03:18.493346+0000","last_active":"2024-09-17T05:03:21.795905+0000","last_peered":"2024-09-17T05:03:21.795905+0000","last_clean":"2024-09-17T05:03:21.795905+0000","last_became_active":"2024-09-17T05:03:18.490847+0000","last_became_peered":"2024-09-17T05:03:18.490847+0000","last_unstale":"2024-09-17T05:03:21.795905+0000","last_undegraded":"2024-09-17T05:03:21.795905+0000","last_fullsized":"2024-09-17T05:03:21.795905+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:23.594082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485645+0000","last_change":"2024-09-17T05:03:18.492283+0000","last_active":"2024-09-17T05:03:19.485645+0000","last_peered":"2024-09-17T05:03:19.485645+0000","last_clean":"2024-09-17T05:03:19.485645+0000","last_became_active":"2024-09-17T05:03:18.491925+0000","last_became_peered":"2024-09-17T05:03:18.491925+0000","last_unstale":"2024-09-17T05:03:19.485645+0000","last_undegraded":"2024-09-17T05:03:19.485645+0000","last_fullsized":"2024-09-17T05:03:19.485645+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:36.834342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414831+0000","last_change":"2024-09-17T05:03:18.509121+0000","last_active":"2024-09-17T05:03:20.414831+0000","last_peered":"2024-09-17T05:03:20.414831+0000","last_clean":"2024-09-17T05:03:20.414831+0000","last_became_active":"2024-09-17T05:03:18.508808+0000","last_became_peered":"2024-09-17T05:03:18.508808+0000","last_unstale":"2024-09-17T05:03:20.414831+0000","last_undegraded":"2024-09-17T05:03:20.414831+0000","last_fullsized":"2024-09-17T05:03:20.414831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:19.040116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796106+0000","last_change":"2024-09-17T05:03:18.497050+0000","last_active":"2024-09-17T05:03:21.796106+0000","last_peered":"2024-09-17T05:03:21.796106+0000","last_clean":"2024-09-17T05:03:21.796106+0000","last_became_active":"2024-09-17T05:03:18.496820+0000","last_became_peered":"2024-09-17T05:03:18.496820+0000","last_unstale":"2024-09-17T05:03:21.796106+0000","last_undegraded":"2024-09-17T05:03:21.796106+0000","last_fullsized":"2024-09-17T05:03:21.796106+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:17.126958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795843+0000","last_change":"2024-09-17T05:03:18.500297+0000","last_active":"2024-09-17T05:03:21.795843+0000","last_peered":"2024-09-17T05:03:21.795843+0000","last_clean":"2024-09-17T05:03:21.795843+0000","last_became_active":"2024-09-17T05:03:18.499968+0000","last_became_peered":"2024-09-17T05:03:18.499968+0000","last_unstale":"2024-09-17T05:03:21.795843+0000","last_undegraded":"2024-09-17T05:03:21.795843+0000","last_fullsized":"2024-09-17T05:03:21.795843+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:36.618926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796625+0000","last_change":"2024-09-17T05:03:18.484183+0000","last_active":"2024-09-17T05:03:21.796625+0000","last_peered":"2024-09-17T05:03:21.796625+0000","last_clean":"2024-09-17T05:03:21.796625+0000","last_became_active":"2024-09-17T05:03:18.483876+0000","last_became_peered":"2024-09-17T05:03:18.483876+0000","last_unstale":"2024-09-17T05:03:21.796625+0000","last_undegraded":"2024-09-17T05:03:21.796625+0000","last_fullsized":"2024-09-17T05:03:21.796625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:49.128082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485437+0000","last_change":"2024-09-17T05:03:18.495213+0000","last_active":"2024-09-17T05:03:19.485437+0000","last_peered":"2024-09-17T05:03:19.485437+0000","last_clean":"2024-09-17T05:03:19.485437+0000","last_became_active":"2024-09-17T05:03:18.494507+0000","last_became_peered":"2024-09-17T05:03:18.494507+0000","last_unstale":"2024-09-17T05:03:19.485437+0000","last_undegraded":"2024-09-17T05:03:19.485437+0000","last_fullsized":"2024-09-17T05:03:19.485437+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:27.674507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795374+0000","last_change":"2024-09-17T05:03:18.491751+0000","last_active":"2024-09-17T05:03:21.795374+0000","last_peered":"2024-09-17T05:03:21.795374+0000","last_clean":"2024-09-17T05:03:21.795374+0000","last_became_active":"2024-09-17T05:03:18.491440+0000","last_became_peered":"2024-09-17T05:03:18.491440+0000","last_unstale":"2024-09-17T05:03:21.795374+0000","last_undegraded":"2024-09-17T05:03:21.795374+0000","last_fullsized":"2024-09-17T05:03:21.795374+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:02.045773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796105+0000","last_change":"2024-09-17T05:03:18.500083+0000","last_active":"2024-09-17T05:03:21.796105+0000","last_peered":"2024-09-17T05:03:21.796105+0000","last_clean":"2024-09-17T05:03:21.796105+0000","last_became_active":"2024-09-17T05:03:18.499799+0000","last_became_peered":"2024-09-17T05:03:18.499799+0000","last_unstale":"2024-09-17T05:03:21.796105+0000","last_undegraded":"2024-09-17T05:03:21.796105+0000","last_fullsized":"2024-09-17T05:03:21.796105+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:36.834342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.495433+0000","last_change":"2024-09-17T05:03:17.487458+0000","last_active":"2024-09-17T05:03:19.495433+0000","last_peered":"2024-09-17T05:03:19.495433+0000","last_clean":"2024-09-17T05:03:19.495433+0000","last_became_active":"2024-09-17T05:03:17.487174+0000","last_became_peered":"2024-09-17T05:03:17.487174+0000","last_unstale":"2024-09-17T05:03:19.495433+0000","last_undegraded":"2024-09-17T05:03:19.495433+0000","last_fullsized":"2024-09-17T05:03:19.495433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.060756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796394+0000","last_change":"2024-09-17T05:03:18.497880+0000","last_active":"2024-09-17T05:03:21.796394+0000","last_peered":"2024-09-17T05:03:21.796394+0000","last_clean":"2024-09-17T05:03:21.796394+0000","last_became_active":"2024-09-17T05:03:18.497468+0000","last_became_peered":"2024-09-17T05:03:18.497468+0000","last_unstale":"2024-09-17T05:03:21.796394+0000","last_undegraded":"2024-09-17T05:03:21.796394+0000","last_fullsized":"2024-09-17T05:03:21.796394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:03.135305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414697+0000","last_change":"2024-09-17T05:03:17.489555+0000","last_active":"2024-09-17T05:03:20.414697+0000","last_peered":"2024-09-17T05:03:20.414697+0000","last_clean":"2024-09-17T05:03:20.414697+0000","last_became_active":"2024-09-17T05:03:17.489182+0000","last_became_peered":"2024-09-17T05:03:17.489182+0000","last_unstale":"2024-09-17T05:03:20.414697+0000","last_undegraded":"2024-09-17T05:03:20.414697+0000","last_fullsized":"2024-09-17T05:03:20.414697+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796408+0000","last_change":"2024-09-17T05:03:18.500202+0000","last_active":"2024-09-17T05:03:21.796408+0000","last_peered":"2024-09-17T05:03:21.796408+0000","last_clean":"2024-09-17T05:03:21.796408+0000","last_became_active":"2024-09-17T05:03:18.499893+0000","last_became_peered":"2024-09-17T05:03:18.499893+0000","last_unstale":"2024-09-17T05:03:21.796408+0000","last_undegraded":"2024-09-17T05:03:21.796408+0000","last_fullsized":"2024-09-17T05:03:21.796408+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:03.396864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487289+0000","last_change":"2024-09-17T05:03:17.473668+0000","last_active":"2024-09-17T05:03:19.487289+0000","last_peered":"2024-09-17T05:03:19.487289+0000","last_clean":"2024-09-17T05:03:19.487289+0000","last_became_active":"2024-09-17T05:03:17.473283+0000","last_became_peered":"2024-09-17T05:03:17.473283+0000","last_unstale":"2024-09-17T05:03:19.487289+0000","last_undegraded":"2024-09-17T05:03:19.487289+0000","last_fullsized":"2024-09-17T05:03:19.487289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:34.241801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487050+0000","last_change":"2024-09-17T05:03:18.495083+0000","last_active":"2024-09-17T05:03:19.487050+0000","last_peered":"2024-09-17T05:03:19.487050+0000","last_clean":"2024-09-17T05:03:19.487050+0000","last_became_active":"2024-09-17T05:03:18.494736+0000","last_became_peered":"2024-09-17T05:03:18.494736+0000","last_unstale":"2024-09-17T05:03:19.487050+0000","last_undegraded":"2024-09-17T05:03:19.487050+0000","last_fullsized":"2024-09-17T05:03:19.487050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:24.301106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796281+0000","last_change":"2024-09-17T05:03:17.484272+0000","last_active":"2024-09-17T05:03:21.796281+0000","last_peered":"2024-09-17T05:03:21.796281+0000","last_clean":"2024-09-17T05:03:21.796281+0000","last_became_active":"2024-09-17T05:03:17.483583+0000","last_became_peered":"2024-09-17T05:03:17.483583+0000","last_unstale":"2024-09-17T05:03:21.796281+0000","last_undegraded":"2024-09-17T05:03:21.796281+0000","last_fullsized":"2024-09-17T05:03:21.796281+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:53.765242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.495255+0000","last_change":"2024-09-17T05:03:18.514276+0000","last_active":"2024-09-17T05:03:19.495255+0000","last_peered":"2024-09-17T05:03:19.495255+0000","last_clean":"2024-09-17T05:03:19.495255+0000","last_became_active":"2024-09-17T05:03:18.513909+0000","last_became_peered":"2024-09-17T05:03:18.513909+0000","last_unstale":"2024-09-17T05:03:19.495255+0000","last_undegraded":"2024-09-17T05:03:19.495255+0000","last_fullsized":"2024-09-17T05:03:19.495255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485558+0000","last_change":"2024-09-17T05:03:17.489922+0000","last_active":"2024-09-17T05:03:19.485558+0000","last_peered":"2024-09-17T05:03:19.485558+0000","last_clean":"2024-09-17T05:03:19.485558+0000","last_became_active":"2024-09-17T05:03:17.489611+0000","last_became_peered":"2024-09-17T05:03:17.489611+0000","last_unstale":"2024-09-17T05:03:19.485558+0000","last_undegraded":"2024-09-17T05:03:19.485558+0000","last_fullsized":"2024-09-17T05:03:19.485558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:35.801215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796855+0000","last_change":"2024-09-17T05:03:18.485055+0000","last_active":"2024-09-17T05:03:21.796855+0000","last_peered":"2024-09-17T05:03:21.796855+0000","last_clean":"2024-09-17T05:03:21.796855+0000","last_became_active":"2024-09-17T05:03:18.484744+0000","last_became_peered":"2024-09-17T05:03:18.484744+0000","last_unstale":"2024-09-17T05:03:21.796855+0000","last_undegraded":"2024-09-17T05:03:21.796855+0000","last_fullsized":"2024-09-17T05:03:21.796855+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:14.318804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796609+0000","last_change":"2024-09-17T05:03:17.477682+0000","last_active":"2024-09-17T05:03:21.796609+0000","last_peered":"2024-09-17T05:03:21.796609+0000","last_clean":"2024-09-17T05:03:21.796609+0000","last_became_active":"2024-09-17T05:03:17.477257+0000","last_became_peered":"2024-09-17T05:03:17.477257+0000","last_unstale":"2024-09-17T05:03:21.796609+0000","last_undegraded":"2024-09-17T05:03:21.796609+0000","last_fullsized":"2024-09-17T05:03:21.796609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:29.176860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487353+0000","last_change":"2024-09-17T05:03:18.504857+0000","last_active":"2024-09-17T05:03:19.487353+0000","last_peered":"2024-09-17T05:03:19.487353+0000","last_clean":"2024-09-17T05:03:19.487353+0000","last_became_active":"2024-09-17T05:03:18.504185+0000","last_became_peered":"2024-09-17T05:03:18.504185+0000","last_unstale":"2024-09-17T05:03:19.487353+0000","last_undegraded":"2024-09-17T05:03:19.487353+0000","last_fullsized":"2024-09-17T05:03:19.487353+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:51.971141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796357+0000","last_change":"2024-09-17T05:03:17.488461+0000","last_active":"2024-09-17T05:03:21.796357+0000","last_peered":"2024-09-17T05:03:21.796357+0000","last_clean":"2024-09-17T05:03:21.796357+0000","last_became_active":"2024-09-17T05:03:17.488076+0000","last_became_peered":"2024-09-17T05:03:17.488076+0000","last_unstale":"2024-09-17T05:03:21.796357+0000","last_undegraded":"2024-09-17T05:03:21.796357+0000","last_fullsized":"2024-09-17T05:03:21.796357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:41.181926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485103+0000","last_change":"2024-09-17T05:03:18.494837+0000","last_active":"2024-09-17T05:03:19.485103+0000","last_peered":"2024-09-17T05:03:19.485103+0000","last_clean":"2024-09-17T05:03:19.485103+0000","last_became_active":"2024-09-17T05:03:18.494265+0000","last_became_peered":"2024-09-17T05:03:18.494265+0000","last_unstale":"2024-09-17T05:03:19.485103+0000","last_undegraded":"2024-09-17T05:03:19.485103+0000","last_fullsized":"2024-09-17T05:03:19.485103+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.380215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484969+0000","last_change":"2024-09-17T05:03:17.476199+0000","last_active":"2024-09-17T05:03:19.484969+0000","last_peered":"2024-09-17T05:03:19.484969+0000","last_clean":"2024-09-17T05:03:19.484969+0000","last_became_active":"2024-09-17T05:03:17.475834+0000","last_became_peered":"2024-09-17T05:03:17.475834+0000","last_unstale":"2024-09-17T05:03:19.484969+0000","last_undegraded":"2024-09-17T05:03:19.484969+0000","last_fullsized":"2024-09-17T05:03:19.484969+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:21.694015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494962+0000","last_change":"2024-09-17T05:03:18.495879+0000","last_active":"2024-09-17T05:03:19.494962+0000","last_peered":"2024-09-17T05:03:19.494962+0000","last_clean":"2024-09-17T05:03:19.494962+0000","last_became_active":"2024-09-17T05:03:18.495553+0000","last_became_peered":"2024-09-17T05:03:18.495553+0000","last_unstale":"2024-09-17T05:03:19.494962+0000","last_undegraded":"2024-09-17T05:03:19.494962+0000","last_fullsized":"2024-09-17T05:03:19.494962+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485216+0000","last_change":"2024-09-17T05:03:17.484357+0000","last_active":"2024-09-17T05:03:19.485216+0000","last_peered":"2024-09-17T05:03:19.485216+0000","last_clean":"2024-09-17T05:03:19.485216+0000","last_became_active":"2024-09-17T05:03:17.483648+0000","last_became_peered":"2024-09-17T05:03:17.483648+0000","last_unstale":"2024-09-17T05:03:19.485216+0000","last_undegraded":"2024-09-17T05:03:19.485216+0000","last_fullsized":"2024-09-17T05:03:19.485216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484412+0000","last_change":"2024-09-17T05:03:18.496015+0000","last_active":"2024-09-17T05:03:19.484412+0000","last_peered":"2024-09-17T05:03:19.484412+0000","last_clean":"2024-09-17T05:03:19.484412+0000","last_became_active":"2024-09-17T05:03:18.495714+0000","last_became_peered":"2024-09-17T05:03:18.495714+0000","last_unstale":"2024-09-17T05:03:19.484412+0000","last_undegraded":"2024-09-17T05:03:19.484412+0000","last_fullsized":"2024-09-17T05:03:19.484412+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.412751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485074+0000","last_change":"2024-09-17T05:03:17.483837+0000","last_active":"2024-09-17T05:03:19.485074+0000","last_peered":"2024-09-17T05:03:19.485074+0000","last_clean":"2024-09-17T05:03:19.485074+0000","last_became_active":"2024-09-17T05:03:17.483474+0000","last_became_peered":"2024-09-17T05:03:17.483474+0000","last_unstale":"2024-09-17T05:03:19.485074+0000","last_undegraded":"2024-09-17T05:03:19.485074+0000","last_fullsized":"2024-09-17T05:03:19.485074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.383407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795609+0000","last_change":"2024-09-17T05:03:18.513177+0000","last_active":"2024-09-17T05:03:21.795609+0000","last_peered":"2024-09-17T05:03:21.795609+0000","last_clean":"2024-09-17T05:03:21.795609+0000","last_became_active":"2024-09-17T05:03:18.512727+0000","last_became_peered":"2024-09-17T05:03:18.512727+0000","last_unstale":"2024-09-17T05:03:21.795609+0000","last_undegraded":"2024-09-17T05:03:21.795609+0000","last_fullsized":"2024-09-17T05:03:21.795609+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:55.305773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796884+0000","last_change":"2024-09-17T05:03:17.471492+0000","last_active":"2024-09-17T05:03:21.796884+0000","last_peered":"2024-09-17T05:03:21.796884+0000","last_clean":"2024-09-17T05:03:21.796884+0000","last_became_active":"2024-09-17T05:03:17.471119+0000","last_became_peered":"2024-09-17T05:03:17.471119+0000","last_unstale":"2024-09-17T05:03:21.796884+0000","last_undegraded":"2024-09-17T05:03:21.796884+0000","last_fullsized":"2024-09-17T05:03:21.796884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:12.694951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795808+0000","last_change":"2024-09-17T05:03:18.495599+0000","last_active":"2024-09-17T05:03:21.795808+0000","last_peered":"2024-09-17T05:03:21.795808+0000","last_clean":"2024-09-17T05:03:21.795808+0000","last_became_active":"2024-09-17T05:03:18.495038+0000","last_became_peered":"2024-09-17T05:03:18.495038+0000","last_unstale":"2024-09-17T05:03:21.795808+0000","last_undegraded":"2024-09-17T05:03:21.795808+0000","last_fullsized":"2024-09-17T05:03:21.795808+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:26.057553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487172+0000","last_change":"2024-09-17T05:03:17.480393+0000","last_active":"2024-09-17T05:03:19.487172+0000","last_peered":"2024-09-17T05:03:19.487172+0000","last_clean":"2024-09-17T05:03:19.487172+0000","last_became_active":"2024-09-17T05:03:17.479912+0000","last_became_peered":"2024-09-17T05:03:17.479912+0000","last_unstale":"2024-09-17T05:03:19.487172+0000","last_undegraded":"2024-09-17T05:03:19.487172+0000","last_fullsized":"2024-09-17T05:03:19.487172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:06.645919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796839+0000","last_change":"2024-09-17T05:03:18.504212+0000","last_active":"2024-09-17T05:03:21.796839+0000","last_peered":"2024-09-17T05:03:21.796839+0000","last_clean":"2024-09-17T05:03:21.796839+0000","last_became_active":"2024-09-17T05:03:18.503841+0000","last_became_peered":"2024-09-17T05:03:18.503841+0000","last_unstale":"2024-09-17T05:03:21.796839+0000","last_undegraded":"2024-09-17T05:03:21.796839+0000","last_fullsized":"2024-09-17T05:03:21.796839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:51.416667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414644+0000","last_change":"2024-09-17T05:03:17.482800+0000","last_active":"2024-09-17T05:03:20.414644+0000","last_peered":"2024-09-17T05:03:20.414644+0000","last_clean":"2024-09-17T05:03:20.414644+0000","last_became_active":"2024-09-17T05:03:17.482376+0000","last_became_peered":"2024-09-17T05:03:17.482376+0000","last_unstale":"2024-09-17T05:03:20.414644+0000","last_undegraded":"2024-09-17T05:03:20.414644+0000","last_fullsized":"2024-09-17T05:03:20.414644+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:06.028991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795582+0000","last_change":"2024-09-17T05:03:18.506127+0000","last_active":"2024-09-17T05:03:21.795582+0000","last_peered":"2024-09-17T05:03:21.795582+0000","last_clean":"2024-09-17T05:03:21.795582+0000","last_became_active":"2024-09-17T05:03:18.505295+0000","last_became_peered":"2024-09-17T05:03:18.505295+0000","last_unstale":"2024-09-17T05:03:21.795582+0000","last_undegraded":"2024-09-17T05:03:21.795582+0000","last_fullsized":"2024-09-17T05:03:21.795582+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:50.676398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484488+0000","last_change":"2024-09-17T05:03:17.477301+0000","last_active":"2024-09-17T05:03:19.484488+0000","last_peered":"2024-09-17T05:03:19.484488+0000","last_clean":"2024-09-17T05:03:19.484488+0000","last_became_active":"2024-09-17T05:03:17.476888+0000","last_became_peered":"2024-09-17T05:03:17.476888+0000","last_unstale":"2024-09-17T05:03:19.484488+0000","last_undegraded":"2024-09-17T05:03:19.484488+0000","last_fullsized":"2024-09-17T05:03:19.484488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:58.301917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414868+0000","last_change":"2024-09-17T05:03:18.507972+0000","last_active":"2024-09-17T05:03:20.414868+0000","last_peered":"2024-09-17T05:03:20.414868+0000","last_clean":"2024-09-17T05:03:20.414868+0000","last_became_active":"2024-09-17T05:03:18.507423+0000","last_became_peered":"2024-09-17T05:03:18.507423+0000","last_unstale":"2024-09-17T05:03:20.414868+0000","last_undegraded":"2024-09-17T05:03:20.414868+0000","last_fullsized":"2024-09-17T05:03:20.414868+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.380215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795437+0000","last_change":"2024-09-17T05:03:17.481847+0000","last_active":"2024-09-17T05:03:21.795437+0000","last_peered":"2024-09-17T05:03:21.795437+0000","last_clean":"2024-09-17T05:03:21.795437+0000","last_became_active":"2024-09-17T05:03:17.481546+0000","last_became_peered":"2024-09-17T05:03:17.481546+0000","last_unstale":"2024-09-17T05:03:21.795437+0000","last_undegraded":"2024-09-17T05:03:21.795437+0000","last_fullsized":"2024-09-17T05:03:21.795437+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.383407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485912+0000","last_change":"2024-09-17T05:03:18.484589+0000","last_active":"2024-09-17T05:03:19.485912+0000","last_peered":"2024-09-17T05:03:19.485912+0000","last_clean":"2024-09-17T05:03:19.485912+0000","last_became_active":"2024-09-17T05:03:18.483862+0000","last_became_peered":"2024-09-17T05:03:18.483862+0000","last_unstale":"2024-09-17T05:03:19.485912+0000","last_undegraded":"2024-09-17T05:03:19.485912+0000","last_fullsized":"2024-09-17T05:03:19.485912+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.509403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":23,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414651+0000","last_change":"2024-09-17T05:03:17.489475+0000","last_active":"2024-09-17T05:03:20.414651+0000","last_peered":"2024-09-17T05:03:20.414651+0000","last_clean":"2024-09-17T05:03:20.414651+0000","last_became_active":"2024-09-17T05:03:17.489135+0000","last_became_peered":"2024-09-17T05:03:17.489135+0000","last_unstale":"2024-09-17T05:03:20.414651+0000","last_undegraded":"2024-09-17T05:03:20.414651+0000","last_fullsized":"2024-09-17T05:03:20.414651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:35.734369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486537+0000","last_change":"2024-09-17T05:03:18.487882+0000","last_active":"2024-09-17T05:03:19.486537+0000","last_peered":"2024-09-17T05:03:19.486537+0000","last_clean":"2024-09-17T05:03:19.486537+0000","last_became_active":"2024-09-17T05:03:18.487523+0000","last_became_peered":"2024-09-17T05:03:18.487523+0000","last_unstale":"2024-09-17T05:03:19.486537+0000","last_undegraded":"2024-09-17T05:03:19.486537+0000","last_fullsized":"2024-09-17T05:03:19.486537+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:44.737783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414979+0000","last_change":"2024-09-17T05:03:17.489385+0000","last_active":"2024-09-17T05:03:20.414979+0000","last_peered":"2024-09-17T05:03:20.414979+0000","last_clean":"2024-09-17T05:03:20.414979+0000","last_became_active":"2024-09-17T05:03:17.489069+0000","last_became_peered":"2024-09-17T05:03:17.489069+0000","last_unstale":"2024-09-17T05:03:20.414979+0000","last_undegraded":"2024-09-17T05:03:20.414979+0000","last_fullsized":"2024-09-17T05:03:20.414979+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:33.379573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795898+0000","last_change":"2024-09-17T05:03:18.501550+0000","last_active":"2024-09-17T05:03:21.795898+0000","last_peered":"2024-09-17T05:03:21.795898+0000","last_clean":"2024-09-17T05:03:21.795898+0000","last_became_active":"2024-09-17T05:03:18.501150+0000","last_became_peered":"2024-09-17T05:03:18.501150+0000","last_unstale":"2024-09-17T05:03:21.795898+0000","last_undegraded":"2024-09-17T05:03:21.795898+0000","last_fullsized":"2024-09-17T05:03:21.795898+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:30.983417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414870+0000","last_change":"2024-09-17T05:03:17.483869+0000","last_active":"2024-09-17T05:03:20.414870+0000","last_peered":"2024-09-17T05:03:20.414870+0000","last_clean":"2024-09-17T05:03:20.414870+0000","last_became_active":"2024-09-17T05:03:17.483565+0000","last_became_peered":"2024-09-17T05:03:17.483565+0000","last_unstale":"2024-09-17T05:03:20.414870+0000","last_undegraded":"2024-09-17T05:03:20.414870+0000","last_fullsized":"2024-09-17T05:03:20.414870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:48.075582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795516+0000","last_change":"2024-09-17T05:03:18.489876+0000","last_active":"2024-09-17T05:03:21.795516+0000","last_peered":"2024-09-17T05:03:21.795516+0000","last_clean":"2024-09-17T05:03:21.795516+0000","last_became_active":"2024-09-17T05:03:18.488017+0000","last_became_peered":"2024-09-17T05:03:18.488017+0000","last_unstale":"2024-09-17T05:03:21.795516+0000","last_undegraded":"2024-09-17T05:03:21.795516+0000","last_fullsized":"2024-09-17T05:03:21.795516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:13.753545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484788+0000","last_change":"2024-09-17T05:03:17.484821+0000","last_active":"2024-09-17T05:03:19.484788+0000","last_peered":"2024-09-17T05:03:19.484788+0000","last_clean":"2024-09-17T05:03:19.484788+0000","last_became_active":"2024-09-17T05:03:17.484445+0000","last_became_peered":"2024-09-17T05:03:17.484445+0000","last_unstale":"2024-09-17T05:03:19.484788+0000","last_undegraded":"2024-09-17T05:03:19.484788+0000","last_fullsized":"2024-09-17T05:03:19.484788+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:44.206475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414381+0000","last_change":"2024-09-17T05:03:18.508867+0000","last_active":"2024-09-17T05:03:20.414381+0000","last_peered":"2024-09-17T05:03:20.414381+0000","last_clean":"2024-09-17T05:03:20.414381+0000","last_became_active":"2024-09-17T05:03:18.508319+0000","last_became_peered":"2024-09-17T05:03:18.508319+0000","last_unstale":"2024-09-17T05:03:20.414381+0000","last_undegraded":"2024-09-17T05:03:20.414381+0000","last_fullsized":"2024-09-17T05:03:20.414381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:01.699937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796400+0000","last_change":"2024-09-17T05:03:17.469441+0000","last_active":"2024-09-17T05:03:21.796400+0000","last_peered":"2024-09-17T05:03:21.796400+0000","last_clean":"2024-09-17T05:03:21.796400+0000","last_became_active":"2024-09-17T05:03:17.469019+0000","last_became_peered":"2024-09-17T05:03:17.469019+0000","last_unstale":"2024-09-17T05:03:21.796400+0000","last_undegraded":"2024-09-17T05:03:21.796400+0000","last_fullsized":"2024-09-17T05:03:21.796400+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:02.998746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796262+0000","last_change":"2024-09-17T05:03:18.489610+0000","last_active":"2024-09-17T05:03:21.796262+0000","last_peered":"2024-09-17T05:03:21.796262+0000","last_clean":"2024-09-17T05:03:21.796262+0000","last_became_active":"2024-09-17T05:03:18.489271+0000","last_became_peered":"2024-09-17T05:03:18.489271+0000","last_unstale":"2024-09-17T05:03:21.796262+0000","last_undegraded":"2024-09-17T05:03:21.796262+0000","last_fullsized":"2024-09-17T05:03:21.796262+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:50.676398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795797+0000","last_change":"2024-09-17T05:03:17.488710+0000","last_active":"2024-09-17T05:03:21.795797+0000","last_peered":"2024-09-17T05:03:21.795797+0000","last_clean":"2024-09-17T05:03:21.795797+0000","last_became_active":"2024-09-17T05:03:17.488178+0000","last_became_peered":"2024-09-17T05:03:17.488178+0000","last_unstale":"2024-09-17T05:03:21.795797+0000","last_undegraded":"2024-09-17T05:03:21.795797+0000","last_fullsized":"2024-09-17T05:03:21.795797+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:44.044047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484785+0000","last_change":"2024-09-17T05:03:18.484052+0000","last_active":"2024-09-17T05:03:19.484785+0000","last_peered":"2024-09-17T05:03:19.484785+0000","last_clean":"2024-09-17T05:03:19.484785+0000","last_became_active":"2024-09-17T05:03:18.483686+0000","last_became_peered":"2024-09-17T05:03:18.483686+0000","last_unstale":"2024-09-17T05:03:19.484785+0000","last_undegraded":"2024-09-17T05:03:19.484785+0000","last_fullsized":"2024-09-17T05:03:19.484785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485517+0000","last_change":"2024-09-17T05:03:17.487381+0000","last_active":"2024-09-17T05:03:19.485517+0000","last_peered":"2024-09-17T05:03:19.485517+0000","last_clean":"2024-09-17T05:03:19.485517+0000","last_became_active":"2024-09-17T05:03:17.487025+0000","last_became_peered":"2024-09-17T05:03:17.487025+0000","last_unstale":"2024-09-17T05:03:19.485517+0000","last_undegraded":"2024-09-17T05:03:19.485517+0000","last_fullsized":"2024-09-17T05:03:19.485517+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:38.335990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795664+0000","last_change":"2024-09-17T05:03:18.513093+0000","last_active":"2024-09-17T05:03:21.795664+0000","last_peered":"2024-09-17T05:03:21.795664+0000","last_clean":"2024-09-17T05:03:21.795664+0000","last_became_active":"2024-09-17T05:03:18.512779+0000","last_became_peered":"2024-09-17T05:03:18.512779+0000","last_unstale":"2024-09-17T05:03:21.795664+0000","last_undegraded":"2024-09-17T05:03:21.795664+0000","last_fullsized":"2024-09-17T05:03:21.795664+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:07.251450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486706+0000","last_change":"2024-09-17T05:03:17.488730+0000","last_active":"2024-09-17T05:03:19.486706+0000","last_peered":"2024-09-17T05:03:19.486706+0000","last_clean":"2024-09-17T05:03:19.486706+0000","last_became_active":"2024-09-17T05:03:17.487850+0000","last_became_peered":"2024-09-17T05:03:17.487850+0000","last_unstale":"2024-09-17T05:03:19.486706+0000","last_undegraded":"2024-09-17T05:03:19.486706+0000","last_fullsized":"2024-09-17T05:03:19.486706+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:20.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414954+0000","last_change":"2024-09-17T05:03:18.508348+0000","last_active":"2024-09-17T05:03:20.414954+0000","last_peered":"2024-09-17T05:03:20.414954+0000","last_clean":"2024-09-17T05:03:20.414954+0000","last_became_active":"2024-09-17T05:03:18.507958+0000","last_became_peered":"2024-09-17T05:03:18.507958+0000","last_unstale":"2024-09-17T05:03:20.414954+0000","last_undegraded":"2024-09-17T05:03:20.414954+0000","last_fullsized":"2024-09-17T05:03:20.414954+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:41.130167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486411+0000","last_change":"2024-09-17T05:03:17.477112+0000","last_active":"2024-09-17T05:03:19.486411+0000","last_peered":"2024-09-17T05:03:19.486411+0000","last_clean":"2024-09-17T05:03:19.486411+0000","last_became_active":"2024-09-17T05:03:17.476642+0000","last_became_peered":"2024-09-17T05:03:17.476642+0000","last_unstale":"2024-09-17T05:03:19.486411+0000","last_undegraded":"2024-09-17T05:03:19.486411+0000","last_fullsized":"2024-09-17T05:03:19.486411+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:50.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484227+0000","last_change":"2024-09-17T05:03:18.493987+0000","last_active":"2024-09-17T05:03:19.484227+0000","last_peered":"2024-09-17T05:03:19.484227+0000","last_clean":"2024-09-17T05:03:19.484227+0000","last_became_active":"2024-09-17T05:03:18.492993+0000","last_became_peered":"2024-09-17T05:03:18.492993+0000","last_unstale":"2024-09-17T05:03:19.484227+0000","last_undegraded":"2024-09-17T05:03:19.484227+0000","last_fullsized":"2024-09-17T05:03:19.484227+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:01.699937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796817+0000","last_change":"2024-09-17T05:03:17.488680+0000","last_active":"2024-09-17T05:03:21.796817+0000","last_peered":"2024-09-17T05:03:21.796817+0000","last_clean":"2024-09-17T05:03:21.796817+0000","last_became_active":"2024-09-17T05:03:17.488050+0000","last_became_peered":"2024-09-17T05:03:17.488050+0000","last_unstale":"2024-09-17T05:03:21.796817+0000","last_undegraded":"2024-09-17T05:03:21.796817+0000","last_fullsized":"2024-09-17T05:03:21.796817+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:01:44.257805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487087+0000","last_change":"2024-09-17T05:03:18.495602+0000","last_active":"2024-09-17T05:03:19.487087+0000","last_peered":"2024-09-17T05:03:19.487087+0000","last_clean":"2024-09-17T05:03:19.487087+0000","last_became_active":"2024-09-17T05:03:18.495190+0000","last_became_peered":"2024-09-17T05:03:18.495190+0000","last_unstale":"2024-09-17T05:03:19.487087+0000","last_undegraded":"2024-09-17T05:03:19.487087+0000","last_fullsized":"2024-09-17T05:03:19.487087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:49.184314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.797002+0000","last_change":"2024-09-17T05:03:17.471433+0000","last_active":"2024-09-17T05:03:21.797002+0000","last_peered":"2024-09-17T05:03:21.797002+0000","last_clean":"2024-09-17T05:03:21.797002+0000","last_became_active":"2024-09-17T05:03:17.470676+0000","last_became_peered":"2024-09-17T05:03:17.470676+0000","last_unstale":"2024-09-17T05:03:21.797002+0000","last_undegraded":"2024-09-17T05:03:21.797002+0000","last_fullsized":"2024-09-17T05:03:21.797002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:35.470632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485340+0000","last_change":"2024-09-17T05:03:18.499092+0000","last_active":"2024-09-17T05:03:19.485340+0000","last_peered":"2024-09-17T05:03:19.485340+0000","last_clean":"2024-09-17T05:03:19.485340+0000","last_became_active":"2024-09-17T05:03:18.498784+0000","last_became_peered":"2024-09-17T05:03:18.498784+0000","last_unstale":"2024-09-17T05:03:19.485340+0000","last_undegraded":"2024-09-17T05:03:19.485340+0000","last_fullsized":"2024-09-17T05:03:19.485340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:35.871301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485538+0000","last_change":"2024-09-17T05:03:17.481418+0000","last_active":"2024-09-17T05:03:19.485538+0000","last_peered":"2024-09-17T05:03:19.485538+0000","last_clean":"2024-09-17T05:03:19.485538+0000","last_became_active":"2024-09-17T05:03:17.481090+0000","last_became_peered":"2024-09-17T05:03:17.481090+0000","last_unstale":"2024-09-17T05:03:19.485538+0000","last_undegraded":"2024-09-17T05:03:19.485538+0000","last_fullsized":"2024-09-17T05:03:19.485538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.000517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487212+0000","last_change":"2024-09-17T05:03:18.496866+0000","last_active":"2024-09-17T05:03:19.487212+0000","last_peered":"2024-09-17T05:03:19.487212+0000","last_clean":"2024-09-17T05:03:19.487212+0000","last_became_active":"2024-09-17T05:03:18.496533+0000","last_became_peered":"2024-09-17T05:03:18.496533+0000","last_unstale":"2024-09-17T05:03:19.487212+0000","last_undegraded":"2024-09-17T05:03:19.487212+0000","last_fullsized":"2024-09-17T05:03:19.487212+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:06.312704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487198+0000","last_change":"2024-09-17T05:03:17.470269+0000","last_active":"2024-09-17T05:03:19.487198+0000","last_peered":"2024-09-17T05:03:19.487198+0000","last_clean":"2024-09-17T05:03:19.487198+0000","last_became_active":"2024-09-17T05:03:17.469912+0000","last_became_peered":"2024-09-17T05:03:17.469912+0000","last_unstale":"2024-09-17T05:03:19.487198+0000","last_undegraded":"2024-09-17T05:03:19.487198+0000","last_fullsized":"2024-09-17T05:03:19.487198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:33.570989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795830+0000","last_change":"2024-09-17T05:03:18.494624+0000","last_active":"2024-09-17T05:03:21.795830+0000","last_peered":"2024-09-17T05:03:21.795830+0000","last_clean":"2024-09-17T05:03:21.795830+0000","last_became_active":"2024-09-17T05:03:18.494312+0000","last_became_peered":"2024-09-17T05:03:18.494312+0000","last_unstale":"2024-09-17T05:03:21.795830+0000","last_undegraded":"2024-09-17T05:03:21.795830+0000","last_fullsized":"2024-09-17T05:03:21.795830+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:54.573981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485616+0000","last_change":"2024-09-17T05:03:17.486561+0000","last_active":"2024-09-17T05:03:19.485616+0000","last_peered":"2024-09-17T05:03:19.485616+0000","last_clean":"2024-09-17T05:03:19.485616+0000","last_became_active":"2024-09-17T05:03:17.486224+0000","last_became_peered":"2024-09-17T05:03:17.486224+0000","last_unstale":"2024-09-17T05:03:19.485616+0000","last_undegraded":"2024-09-17T05:03:19.485616+0000","last_fullsized":"2024-09-17T05:03:19.485616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:31.418463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487417+0000","last_change":"2024-09-17T05:03:18.504505+0000","last_active":"2024-09-17T05:03:19.487417+0000","last_peered":"2024-09-17T05:03:19.487417+0000","last_clean":"2024-09-17T05:03:19.487417+0000","last_became_active":"2024-09-17T05:03:18.503742+0000","last_became_peered":"2024-09-17T05:03:18.503742+0000","last_unstale":"2024-09-17T05:03:19.487417+0000","last_undegraded":"2024-09-17T05:03:19.487417+0000","last_fullsized":"2024-09-17T05:03:19.487417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795520+0000","last_change":"2024-09-17T05:03:17.479945+0000","last_active":"2024-09-17T05:03:21.795520+0000","last_peered":"2024-09-17T05:03:21.795520+0000","last_clean":"2024-09-17T05:03:21.795520+0000","last_became_active":"2024-09-17T05:03:17.479596+0000","last_became_peered":"2024-09-17T05:03:17.479596+0000","last_unstale":"2024-09-17T05:03:21.795520+0000","last_undegraded":"2024-09-17T05:03:21.795520+0000","last_fullsized":"2024-09-17T05:03:21.795520+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414745+0000","last_change":"2024-09-17T05:03:18.503490+0000","last_active":"2024-09-17T05:03:20.414745+0000","last_peered":"2024-09-17T05:03:20.414745+0000","last_clean":"2024-09-17T05:03:20.414745+0000","last_became_active":"2024-09-17T05:03:18.503217+0000","last_became_peered":"2024-09-17T05:03:18.503217+0000","last_unstale":"2024-09-17T05:03:20.414745+0000","last_undegraded":"2024-09-17T05:03:20.414745+0000","last_fullsized":"2024-09-17T05:03:20.414745+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484236+0000","last_change":"2024-09-17T05:03:17.476961+0000","last_active":"2024-09-17T05:03:19.484236+0000","last_peered":"2024-09-17T05:03:19.484236+0000","last_clean":"2024-09-17T05:03:19.484236+0000","last_became_active":"2024-09-17T05:03:17.476427+0000","last_became_peered":"2024-09-17T05:03:17.476427+0000","last_unstale":"2024-09-17T05:03:19.484236+0000","last_undegraded":"2024-09-17T05:03:19.484236+0000","last_fullsized":"2024-09-17T05:03:19.484236+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:28.438189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494906+0000","last_change":"2024-09-17T05:03:18.509986+0000","last_active":"2024-09-17T05:03:19.494906+0000","last_peered":"2024-09-17T05:03:19.494906+0000","last_clean":"2024-09-17T05:03:19.494906+0000","last_became_active":"2024-09-17T05:03:18.509706+0000","last_became_peered":"2024-09-17T05:03:18.509706+0000","last_unstale":"2024-09-17T05:03:19.494906+0000","last_undegraded":"2024-09-17T05:03:19.494906+0000","last_fullsized":"2024-09-17T05:03:19.494906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494256+0000","last_change":"2024-09-17T05:03:17.483817+0000","last_active":"2024-09-17T05:03:19.494256+0000","last_peered":"2024-09-17T05:03:19.494256+0000","last_clean":"2024-09-17T05:03:19.494256+0000","last_became_active":"2024-09-17T05:03:17.483478+0000","last_became_peered":"2024-09-17T05:03:17.483478+0000","last_unstale":"2024-09-17T05:03:19.494256+0000","last_undegraded":"2024-09-17T05:03:19.494256+0000","last_fullsized":"2024-09-17T05:03:19.494256+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:27.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484317+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:19.484317+0000","last_peered":"2024-09-17T05:03:19.484317+0000","last_clean":"2024-09-17T05:03:19.484317+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:19.484317+0000","last_undegraded":"2024-09-17T05:03:19.484317+0000","last_fullsized":"2024-09-17T05:03:19.484317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":643072,"data_stored":277529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":275915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":275859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":866263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1232896,"data_stored":867895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:23.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:03:23.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 -- 172.21.15.16:0/2645116939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15d630 msgr2=0x7f5a6c0d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 --2- 172.21.15.16:0/2645116939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15d630 0x7f5a6c0d06d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009bc0 tx=0x7f5a6401cac0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 -- 172.21.15.16:0/2645116939 shutdown_connections 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 --2- 172.21.15.16:0/2645116939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c15d630 0x7f5a6c0d06d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 --2- 172.21.15.16:0/2645116939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c160bb0 0x7f5a6c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 -- 172.21.15.16:0/2645116939 >> 172.21.15.16:0/2645116939 conn(0x7f5a6c0cf070 msgr2=0x7f5a6c0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 -- 172.21.15.16:0/2645116939 shutdown_connections 2024-09-17T05:03:23.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 -- 172.21.15.16:0/2645116939 wait complete. 2024-09-17T05:03:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 Processor -- start 2024-09-17T05:03:23.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.757+0000 7f5a72e00640 1 -- start start 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 0x7f5a6c1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c15d630 0x7f5a6c165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c160bb0 0x7f5a6c1fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1699c0 con 0x7f5a6c0cca50 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c169840 con 0x7f5a6c160bb0 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c169b40 con 0x7f5a6c15d630 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 0x7f5a6c1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 0x7f5a6c1656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45778/0 (socket says 172.21.15.16:45778) 2024-09-17T05:03:23.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 -- 172.21.15.16:0/865442611 learned_addr learned my addr 172.21.15.16:0/865442611 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a70e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c160bb0 0x7f5a6c1fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a63400640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c15d630 0x7f5a6c165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 -- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c15d630 msgr2=0x7f5a6c165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c15d630 0x7f5a6c165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 -- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c160bb0 msgr2=0x7f5a6c1fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c160bb0 0x7f5a6c1fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 -- 172.21.15.16:0/865442611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c06e470 con 0x7f5a6c0cca50 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a70e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c160bb0 0x7f5a6c1fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a63400640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c15d630 0x7f5a6c165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a6be00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 0x7f5a6c1656e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00d700 tx=0x7f5a5c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c017070 con 0x7f5a6c0cca50 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a5c009b10 con 0x7f5a6c0cca50 2024-09-17T05:03:23.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c010460 con 0x7f5a6c0cca50 2024-09-17T05:03:23.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c0b7240 con 0x7f5a6c0cca50 2024-09-17T05:03:23.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c0b7520 con 0x7f5a6c0cca50 2024-09-17T05:03:23.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.761+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c0d06d0 con 0x7f5a6c0cca50 2024-09-17T05:03:23.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.765+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a5c01a070 con 0x7f5a6c0cca50 2024-09-17T05:03:23.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.765+0000 7f5a68c00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a48073200 0x7f5a48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:23.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.765+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5a5c094560 con 0x7f5a6c0cca50 2024-09-17T05:03:23.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.765+0000 7f5a63400640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a48073200 0x7f5a48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:23.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.765+0000 7f5a63400640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a48073200 0x7f5a48075710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0c5840 tx=0x7f5a54009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:23.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:23.769+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a5c05f470 con 0x7f5a6c0cca50 2024-09-17T05:03:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.005+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5a6c1546b0 con 0x7f5a6c0cca50 2024-09-17T05:03:24.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.005+0000 7f5a68c00640 1 -- 172.21.15.16:0/865442611 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1217 (secure 0 0 0) 0x7f5a5c05ed70 con 0x7f5a6c0cca50 2024-09-17T05:03:24.147 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:24.147 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":161,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":8,"num_up_osds":8,"osd_up_since":1726549257,"num_in_osds":8,"osd_in_since":1726549246,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":248946688,"bytes_avail":772845166592,"bytes_total":773094113280,"write_bytes_sec":2373,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5460}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T05:03:01.491151+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:03:24.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a48073200 msgr2=0x7f5a48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a48073200 0x7f5a48075710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0c5840 tx=0x7f5a54009040 comp rx=0 tx=0).stop 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 msgr2=0x7f5a6c1656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 0x7f5a6c1656e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00d700 tx=0x7f5a5c00dbc0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 shutdown_connections 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a48073200 0x7f5a48075710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a6c160bb0 0x7f5a6c1fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a6c15d630 0x7f5a6c165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 --2- 172.21.15.16:0/865442611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a6c0cca50 0x7f5a6c1656e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 >> 172.21.15.16:0/865442611 conn(0x7f5a6c0cf070 msgr2=0x7f5a6c0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 shutdown_connections 2024-09-17T05:03:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.009+0000 7f5a72e00640 1 -- 172.21.15.16:0/865442611 wait complete. 2024-09-17T05:03:24.150 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:03:24.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:24.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13ee400640 1 -- 172.21.15.16:0/277651158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc004d50 msgr2=0x7f13cc0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13ee400640 1 --2- 172.21.15.16:0/277651158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13cc004d50 0x7f13cc0051a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009e00 tx=0x7f13e001d170 comp rx=0 tx=0).stop 2024-09-17T05:03:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13e7e00640 1 -- 172.21.15.16:0/277651158 reap_dead start 2024-09-17T05:03:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13ee400640 1 -- 172.21.15.16:0/277651158 shutdown_connections 2024-09-17T05:03:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13ee400640 1 -- 172.21.15.16:0/277651158 >> 172.21.15.16:0/277651158 conn(0x7f13e806f900 msgr2=0x7f13e80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13ee400640 1 -- 172.21.15.16:0/277651158 shutdown_connections 2024-09-17T05:03:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.305+0000 7f13ee400640 1 -- 172.21.15.16:0/277651158 wait complete. 2024-09-17T05:03:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 Processor -- start 2024-09-17T05:03:24.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 -- start start 2024-09-17T05:03:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13cc004d50 0x7f13e8154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 0x7f13e80caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13e8153600 0x7f13e8144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13e8073b90 con 0x7f13e81f95f0 2024-09-17T05:03:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13e815bdf0 con 0x7f13e8153600 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13ee400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f13e8073d10 con 0x7f13cc004d50 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 0x7f13e80caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 0x7f13e80caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45802/0 (socket says 172.21.15.16:45802) 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13dfe00640 1 -- 172.21.15.16:0/3026184184 learned_addr learned my addr 172.21.15.16:0/3026184184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13e7400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13cc004d50 0x7f13e8154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.309+0000 7f13e7e00640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13e8153600 0x7f13e8144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13dfe00640 1 -- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13cc004d50 msgr2=0x7f13e8154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13dfe00640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13cc004d50 0x7f13e8154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13dfe00640 1 -- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13e8153600 msgr2=0x7f13e8144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13dfe00640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13e8153600 0x7f13e8144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13dfe00640 1 -- 172.21.15.16:0/3026184184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e80c9370 con 0x7f13e81f95f0 2024-09-17T05:03:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13dfe00640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 0x7f13e80caeb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13d0001c10 tx=0x7f13d000e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0012850 con 0x7f13e81f95f0 2024-09-17T05:03:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d00129f0 con 0x7f13e81f95f0 2024-09-17T05:03:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0012dc0 con 0x7f13e81f95f0 2024-09-17T05:03:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e80c9650 con 0x7f13e81f95f0 2024-09-17T05:03:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e80cb3f0 con 0x7f13e81f95f0 2024-09-17T05:03:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e80b35e0 con 0x7f13e81f95f0 2024-09-17T05:03:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.313+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13d000d030 con 0x7f13e81f95f0 2024-09-17T05:03:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.317+0000 7f13df400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f13c4073200 0x7f13c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.317+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f13d00974f0 con 0x7f13e81f95f0 2024-09-17T05:03:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.317+0000 7f13e7400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f13c4073200 0x7f13c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:24.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.317+0000 7f13e7400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f13c4073200 0x7f13c4075710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005fd0 tx=0x7f13d800b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:24.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.321+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13d0016030 con 0x7f13e81f95f0 2024-09-17T05:03:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.493+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13e81f9b90 con 0x7f13c4073200 2024-09-17T05:03:24.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.497+0000 7f13df400640 1 -- 172.21.15.16:0/3026184184 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f13e81f9b90 con 0x7f13c4073200 2024-09-17T05:03:24.503 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:24.504 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:24.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f13c4073200 msgr2=0x7f13c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f13c4073200 0x7f13c4075710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005fd0 tx=0x7f13d800b260 comp rx=0 tx=0).stop 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 msgr2=0x7f13e80caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 0x7f13e80caeb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13d0001c10 tx=0x7f13d000e770 comp rx=0 tx=0).stop 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 shutdown_connections 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f13c4073200 0x7f13c4075710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13e8153600 0x7f13e8144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13e81f95f0 0x7f13e80caeb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 --2- 172.21.15.16:0/3026184184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13cc004d50 0x7f13e8154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 >> 172.21.15.16:0/3026184184 conn(0x7f13e806f900 msgr2=0x7f13e81559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 shutdown_connections 2024-09-17T05:03:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:24.501+0000 7f13ee400640 1 -- 172.21.15.16:0/3026184184 wait complete. 2024-09-17T05:03:24.525 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-17T05:03:23.496192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243112,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731608,"statfs":{"total":773094113280,"available":772845166592,"internally_reserved":0,"allocated":6234112,"data_stored":3392156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.041179"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795632+0000","last_change":"2024-09-17T05:03:18.501500+0000","last_active":"2024-09-17T05:03:21.795632+0000","last_peered":"2024-09-17T05:03:21.795632+0000","last_clean":"2024-09-17T05:03:21.795632+0000","last_became_active":"2024-09-17T05:03:18.501008+0000","last_became_peered":"2024-09-17T05:03:18.501008+0000","last_unstale":"2024-09-17T05:03:21.795632+0000","last_undegraded":"2024-09-17T05:03:21.795632+0000","last_fullsized":"2024-09-17T05:03:21.795632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:37.679190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796248+0000","last_change":"2024-09-17T05:03:18.490954+0000","last_active":"2024-09-17T05:03:21.796248+0000","last_peered":"2024-09-17T05:03:21.796248+0000","last_clean":"2024-09-17T05:03:21.796248+0000","last_became_active":"2024-09-17T05:03:18.490645+0000","last_became_peered":"2024-09-17T05:03:18.490645+0000","last_unstale":"2024-09-17T05:03:21.796248+0000","last_undegraded":"2024-09-17T05:03:21.796248+0000","last_fullsized":"2024-09-17T05:03:21.796248+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:07.251450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485314+0000","last_change":"2024-09-17T05:03:18.505087+0000","last_active":"2024-09-17T05:03:19.485314+0000","last_peered":"2024-09-17T05:03:19.485314+0000","last_clean":"2024-09-17T05:03:19.485314+0000","last_became_active":"2024-09-17T05:03:18.504758+0000","last_became_peered":"2024-09-17T05:03:18.504758+0000","last_unstale":"2024-09-17T05:03:19.485314+0000","last_undegraded":"2024-09-17T05:03:19.485314+0000","last_fullsized":"2024-09-17T05:03:19.485314+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:15.676197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484737+0000","last_change":"2024-09-17T05:03:18.498547+0000","last_active":"2024-09-17T05:03:19.484737+0000","last_peered":"2024-09-17T05:03:19.484737+0000","last_clean":"2024-09-17T05:03:19.484737+0000","last_became_active":"2024-09-17T05:03:18.498140+0000","last_became_peered":"2024-09-17T05:03:18.498140+0000","last_unstale":"2024-09-17T05:03:19.484737+0000","last_undegraded":"2024-09-17T05:03:19.484737+0000","last_fullsized":"2024-09-17T05:03:19.484737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795682+0000","last_change":"2024-09-17T05:03:18.493490+0000","last_active":"2024-09-17T05:03:21.795682+0000","last_peered":"2024-09-17T05:03:21.795682+0000","last_clean":"2024-09-17T05:03:21.795682+0000","last_became_active":"2024-09-17T05:03:18.493086+0000","last_became_peered":"2024-09-17T05:03:18.493086+0000","last_unstale":"2024-09-17T05:03:21.795682+0000","last_undegraded":"2024-09-17T05:03:21.795682+0000","last_fullsized":"2024-09-17T05:03:21.795682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:47.377246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414436+0000","last_change":"2024-09-17T05:03:18.503188+0000","last_active":"2024-09-17T05:03:20.414436+0000","last_peered":"2024-09-17T05:03:20.414436+0000","last_clean":"2024-09-17T05:03:20.414436+0000","last_became_active":"2024-09-17T05:03:18.502531+0000","last_became_peered":"2024-09-17T05:03:18.502531+0000","last_unstale":"2024-09-17T05:03:20.414436+0000","last_undegraded":"2024-09-17T05:03:20.414436+0000","last_fullsized":"2024-09-17T05:03:20.414436+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486528+0000","last_change":"2024-09-17T05:03:18.485411+0000","last_active":"2024-09-17T05:03:19.486528+0000","last_peered":"2024-09-17T05:03:19.486528+0000","last_clean":"2024-09-17T05:03:19.486528+0000","last_became_active":"2024-09-17T05:03:18.485074+0000","last_became_peered":"2024-09-17T05:03:18.485074+0000","last_unstale":"2024-09-17T05:03:19.486528+0000","last_undegraded":"2024-09-17T05:03:19.486528+0000","last_fullsized":"2024-09-17T05:03:19.486528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:07.445182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414394+0000","last_change":"2024-09-17T05:03:18.499467+0000","last_active":"2024-09-17T05:03:20.414394+0000","last_peered":"2024-09-17T05:03:20.414394+0000","last_clean":"2024-09-17T05:03:20.414394+0000","last_became_active":"2024-09-17T05:03:18.499137+0000","last_became_peered":"2024-09-17T05:03:18.499137+0000","last_unstale":"2024-09-17T05:03:20.414394+0000","last_undegraded":"2024-09-17T05:03:20.414394+0000","last_fullsized":"2024-09-17T05:03:20.414394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:18.636910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485862+0000","last_change":"2024-09-17T05:03:18.496016+0000","last_active":"2024-09-17T05:03:19.485862+0000","last_peered":"2024-09-17T05:03:19.485862+0000","last_clean":"2024-09-17T05:03:19.485862+0000","last_became_active":"2024-09-17T05:03:18.495706+0000","last_became_peered":"2024-09-17T05:03:18.495706+0000","last_unstale":"2024-09-17T05:03:19.485862+0000","last_undegraded":"2024-09-17T05:03:19.485862+0000","last_fullsized":"2024-09-17T05:03:19.485862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:03.396864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487208+0000","last_change":"2024-09-17T05:03:18.490713+0000","last_active":"2024-09-17T05:03:19.487208+0000","last_peered":"2024-09-17T05:03:19.487208+0000","last_clean":"2024-09-17T05:03:19.487208+0000","last_became_active":"2024-09-17T05:03:18.490369+0000","last_became_peered":"2024-09-17T05:03:18.490369+0000","last_unstale":"2024-09-17T05:03:19.487208+0000","last_undegraded":"2024-09-17T05:03:19.487208+0000","last_fullsized":"2024-09-17T05:03:19.487208+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.356295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795500+0000","last_change":"2024-09-17T05:03:18.502622+0000","last_active":"2024-09-17T05:03:21.795500+0000","last_peered":"2024-09-17T05:03:21.795500+0000","last_clean":"2024-09-17T05:03:21.795500+0000","last_became_active":"2024-09-17T05:03:18.502143+0000","last_became_peered":"2024-09-17T05:03:18.502143+0000","last_unstale":"2024-09-17T05:03:21.795500+0000","last_undegraded":"2024-09-17T05:03:21.795500+0000","last_fullsized":"2024-09-17T05:03:21.795500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:17.126958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414390+0000","last_change":"2024-09-17T05:03:18.499587+0000","last_active":"2024-09-17T05:03:20.414390+0000","last_peered":"2024-09-17T05:03:20.414390+0000","last_clean":"2024-09-17T05:03:20.414390+0000","last_became_active":"2024-09-17T05:03:18.499252+0000","last_became_peered":"2024-09-17T05:03:18.499252+0000","last_unstale":"2024-09-17T05:03:20.414390+0000","last_undegraded":"2024-09-17T05:03:20.414390+0000","last_fullsized":"2024-09-17T05:03:20.414390+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414386+0000","last_change":"2024-09-17T05:03:18.504641+0000","last_active":"2024-09-17T05:03:20.414386+0000","last_peered":"2024-09-17T05:03:20.414386+0000","last_clean":"2024-09-17T05:03:20.414386+0000","last_became_active":"2024-09-17T05:03:18.504311+0000","last_became_peered":"2024-09-17T05:03:18.504311+0000","last_unstale":"2024-09-17T05:03:20.414386+0000","last_undegraded":"2024-09-17T05:03:20.414386+0000","last_fullsized":"2024-09-17T05:03:20.414386+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:52.037853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795414+0000","last_change":"2024-09-17T05:03:18.499614+0000","last_active":"2024-09-17T05:03:21.795414+0000","last_peered":"2024-09-17T05:03:21.795414+0000","last_clean":"2024-09-17T05:03:21.795414+0000","last_became_active":"2024-09-17T05:03:18.498970+0000","last_became_peered":"2024-09-17T05:03:18.498970+0000","last_unstale":"2024-09-17T05:03:21.795414+0000","last_undegraded":"2024-09-17T05:03:21.795414+0000","last_fullsized":"2024-09-17T05:03:21.795414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:01.114758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484404+0000","last_change":"2024-09-17T05:03:18.486986+0000","last_active":"2024-09-17T05:03:19.484404+0000","last_peered":"2024-09-17T05:03:19.484404+0000","last_clean":"2024-09-17T05:03:19.484404+0000","last_became_active":"2024-09-17T05:03:18.486638+0000","last_became_peered":"2024-09-17T05:03:18.486638+0000","last_unstale":"2024-09-17T05:03:19.484404+0000","last_undegraded":"2024-09-17T05:03:19.484404+0000","last_fullsized":"2024-09-17T05:03:19.484404+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:04.033462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414318+0000","last_change":"2024-09-17T05:03:18.508226+0000","last_active":"2024-09-17T05:03:20.414318+0000","last_peered":"2024-09-17T05:03:20.414318+0000","last_clean":"2024-09-17T05:03:20.414318+0000","last_became_active":"2024-09-17T05:03:18.506993+0000","last_became_peered":"2024-09-17T05:03:18.506993+0000","last_unstale":"2024-09-17T05:03:20.414318+0000","last_undegraded":"2024-09-17T05:03:20.414318+0000","last_fullsized":"2024-09-17T05:03:20.414318+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:01.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796065+0000","last_change":"2024-09-17T05:03:18.501235+0000","last_active":"2024-09-17T05:03:21.796065+0000","last_peered":"2024-09-17T05:03:21.796065+0000","last_clean":"2024-09-17T05:03:21.796065+0000","last_became_active":"2024-09-17T05:03:18.500966+0000","last_became_peered":"2024-09-17T05:03:18.500966+0000","last_unstale":"2024-09-17T05:03:21.796065+0000","last_undegraded":"2024-09-17T05:03:21.796065+0000","last_fullsized":"2024-09-17T05:03:21.796065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:40.933886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485015+0000","last_change":"2024-09-17T05:03:18.499110+0000","last_active":"2024-09-17T05:03:19.485015+0000","last_peered":"2024-09-17T05:03:19.485015+0000","last_clean":"2024-09-17T05:03:19.485015+0000","last_became_active":"2024-09-17T05:03:18.498458+0000","last_became_peered":"2024-09-17T05:03:18.498458+0000","last_unstale":"2024-09-17T05:03:19.485015+0000","last_undegraded":"2024-09-17T05:03:19.485015+0000","last_fullsized":"2024-09-17T05:03:19.485015+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:19.040116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485907+0000","last_change":"2024-09-17T05:03:18.498320+0000","last_active":"2024-09-17T05:03:19.485907+0000","last_peered":"2024-09-17T05:03:19.485907+0000","last_clean":"2024-09-17T05:03:19.485907+0000","last_became_active":"2024-09-17T05:03:18.498041+0000","last_became_peered":"2024-09-17T05:03:18.498041+0000","last_unstale":"2024-09-17T05:03:19.485907+0000","last_undegraded":"2024-09-17T05:03:19.485907+0000","last_fullsized":"2024-09-17T05:03:19.485907+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:49.128082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796031+0000","last_change":"2024-09-17T05:03:18.509422+0000","last_active":"2024-09-17T05:03:21.796031+0000","last_peered":"2024-09-17T05:03:21.796031+0000","last_clean":"2024-09-17T05:03:21.796031+0000","last_became_active":"2024-09-17T05:03:18.506222+0000","last_became_peered":"2024-09-17T05:03:18.506222+0000","last_unstale":"2024-09-17T05:03:21.796031+0000","last_undegraded":"2024-09-17T05:03:21.796031+0000","last_fullsized":"2024-09-17T05:03:21.796031+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:01.114758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484565+0000","last_change":"2024-09-17T05:03:18.488562+0000","last_active":"2024-09-17T05:03:19.484565+0000","last_peered":"2024-09-17T05:03:19.484565+0000","last_clean":"2024-09-17T05:03:19.484565+0000","last_became_active":"2024-09-17T05:03:18.488267+0000","last_became_peered":"2024-09-17T05:03:18.488267+0000","last_unstale":"2024-09-17T05:03:19.484565+0000","last_undegraded":"2024-09-17T05:03:19.484565+0000","last_fullsized":"2024-09-17T05:03:19.484565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485314+0000","last_change":"2024-09-17T05:03:18.491046+0000","last_active":"2024-09-17T05:03:19.485314+0000","last_peered":"2024-09-17T05:03:19.485314+0000","last_clean":"2024-09-17T05:03:19.485314+0000","last_became_active":"2024-09-17T05:03:18.490734+0000","last_became_peered":"2024-09-17T05:03:18.490734+0000","last_unstale":"2024-09-17T05:03:19.485314+0000","last_undegraded":"2024-09-17T05:03:19.485314+0000","last_fullsized":"2024-09-17T05:03:19.485314+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:49.725834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485069+0000","last_change":"2024-09-17T05:03:18.495503+0000","last_active":"2024-09-17T05:03:19.485069+0000","last_peered":"2024-09-17T05:03:19.485069+0000","last_clean":"2024-09-17T05:03:19.485069+0000","last_became_active":"2024-09-17T05:03:18.494911+0000","last_became_peered":"2024-09-17T05:03:18.494911+0000","last_unstale":"2024-09-17T05:03:19.485069+0000","last_undegraded":"2024-09-17T05:03:19.485069+0000","last_fullsized":"2024-09-17T05:03:19.485069+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:02.757954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485067+0000","last_change":"2024-09-17T05:03:18.492326+0000","last_active":"2024-09-17T05:03:19.485067+0000","last_peered":"2024-09-17T05:03:19.485067+0000","last_clean":"2024-09-17T05:03:19.485067+0000","last_became_active":"2024-09-17T05:03:18.492003+0000","last_became_peered":"2024-09-17T05:03:18.492003+0000","last_unstale":"2024-09-17T05:03:19.485067+0000","last_undegraded":"2024-09-17T05:03:19.485067+0000","last_fullsized":"2024-09-17T05:03:19.485067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:47.441012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795905+0000","last_change":"2024-09-17T05:03:18.493346+0000","last_active":"2024-09-17T05:03:21.795905+0000","last_peered":"2024-09-17T05:03:21.795905+0000","last_clean":"2024-09-17T05:03:21.795905+0000","last_became_active":"2024-09-17T05:03:18.490847+0000","last_became_peered":"2024-09-17T05:03:18.490847+0000","last_unstale":"2024-09-17T05:03:21.795905+0000","last_undegraded":"2024-09-17T05:03:21.795905+0000","last_fullsized":"2024-09-17T05:03:21.795905+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:23.594082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485645+0000","last_change":"2024-09-17T05:03:18.492283+0000","last_active":"2024-09-17T05:03:19.485645+0000","last_peered":"2024-09-17T05:03:19.485645+0000","last_clean":"2024-09-17T05:03:19.485645+0000","last_became_active":"2024-09-17T05:03:18.491925+0000","last_became_peered":"2024-09-17T05:03:18.491925+0000","last_unstale":"2024-09-17T05:03:19.485645+0000","last_undegraded":"2024-09-17T05:03:19.485645+0000","last_fullsized":"2024-09-17T05:03:19.485645+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:36.834342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414831+0000","last_change":"2024-09-17T05:03:18.509121+0000","last_active":"2024-09-17T05:03:20.414831+0000","last_peered":"2024-09-17T05:03:20.414831+0000","last_clean":"2024-09-17T05:03:20.414831+0000","last_became_active":"2024-09-17T05:03:18.508808+0000","last_became_peered":"2024-09-17T05:03:18.508808+0000","last_unstale":"2024-09-17T05:03:20.414831+0000","last_undegraded":"2024-09-17T05:03:20.414831+0000","last_fullsized":"2024-09-17T05:03:20.414831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:19.040116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796106+0000","last_change":"2024-09-17T05:03:18.497050+0000","last_active":"2024-09-17T05:03:21.796106+0000","last_peered":"2024-09-17T05:03:21.796106+0000","last_clean":"2024-09-17T05:03:21.796106+0000","last_became_active":"2024-09-17T05:03:18.496820+0000","last_became_peered":"2024-09-17T05:03:18.496820+0000","last_unstale":"2024-09-17T05:03:21.796106+0000","last_undegraded":"2024-09-17T05:03:21.796106+0000","last_fullsized":"2024-09-17T05:03:21.796106+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:17.126958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795843+0000","last_change":"2024-09-17T05:03:18.500297+0000","last_active":"2024-09-17T05:03:21.795843+0000","last_peered":"2024-09-17T05:03:21.795843+0000","last_clean":"2024-09-17T05:03:21.795843+0000","last_became_active":"2024-09-17T05:03:18.499968+0000","last_became_peered":"2024-09-17T05:03:18.499968+0000","last_unstale":"2024-09-17T05:03:21.795843+0000","last_undegraded":"2024-09-17T05:03:21.795843+0000","last_fullsized":"2024-09-17T05:03:21.795843+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:36.618926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796625+0000","last_change":"2024-09-17T05:03:18.484183+0000","last_active":"2024-09-17T05:03:21.796625+0000","last_peered":"2024-09-17T05:03:21.796625+0000","last_clean":"2024-09-17T05:03:21.796625+0000","last_became_active":"2024-09-17T05:03:18.483876+0000","last_became_peered":"2024-09-17T05:03:18.483876+0000","last_unstale":"2024-09-17T05:03:21.796625+0000","last_undegraded":"2024-09-17T05:03:21.796625+0000","last_fullsized":"2024-09-17T05:03:21.796625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:49.128082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485437+0000","last_change":"2024-09-17T05:03:18.495213+0000","last_active":"2024-09-17T05:03:19.485437+0000","last_peered":"2024-09-17T05:03:19.485437+0000","last_clean":"2024-09-17T05:03:19.485437+0000","last_became_active":"2024-09-17T05:03:18.494507+0000","last_became_peered":"2024-09-17T05:03:18.494507+0000","last_unstale":"2024-09-17T05:03:19.485437+0000","last_undegraded":"2024-09-17T05:03:19.485437+0000","last_fullsized":"2024-09-17T05:03:19.485437+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:27.674507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795374+0000","last_change":"2024-09-17T05:03:18.491751+0000","last_active":"2024-09-17T05:03:21.795374+0000","last_peered":"2024-09-17T05:03:21.795374+0000","last_clean":"2024-09-17T05:03:21.795374+0000","last_became_active":"2024-09-17T05:03:18.491440+0000","last_became_peered":"2024-09-17T05:03:18.491440+0000","last_unstale":"2024-09-17T05:03:21.795374+0000","last_undegraded":"2024-09-17T05:03:21.795374+0000","last_fullsized":"2024-09-17T05:03:21.795374+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:02.045773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796105+0000","last_change":"2024-09-17T05:03:18.500083+0000","last_active":"2024-09-17T05:03:21.796105+0000","last_peered":"2024-09-17T05:03:21.796105+0000","last_clean":"2024-09-17T05:03:21.796105+0000","last_became_active":"2024-09-17T05:03:18.499799+0000","last_became_peered":"2024-09-17T05:03:18.499799+0000","last_unstale":"2024-09-17T05:03:21.796105+0000","last_undegraded":"2024-09-17T05:03:21.796105+0000","last_fullsized":"2024-09-17T05:03:21.796105+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:36.834342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.495433+0000","last_change":"2024-09-17T05:03:17.487458+0000","last_active":"2024-09-17T05:03:19.495433+0000","last_peered":"2024-09-17T05:03:19.495433+0000","last_clean":"2024-09-17T05:03:19.495433+0000","last_became_active":"2024-09-17T05:03:17.487174+0000","last_became_peered":"2024-09-17T05:03:17.487174+0000","last_unstale":"2024-09-17T05:03:19.495433+0000","last_undegraded":"2024-09-17T05:03:19.495433+0000","last_fullsized":"2024-09-17T05:03:19.495433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.060756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796394+0000","last_change":"2024-09-17T05:03:18.497880+0000","last_active":"2024-09-17T05:03:21.796394+0000","last_peered":"2024-09-17T05:03:21.796394+0000","last_clean":"2024-09-17T05:03:21.796394+0000","last_became_active":"2024-09-17T05:03:18.497468+0000","last_became_peered":"2024-09-17T05:03:18.497468+0000","last_unstale":"2024-09-17T05:03:21.796394+0000","last_undegraded":"2024-09-17T05:03:21.796394+0000","last_fullsized":"2024-09-17T05:03:21.796394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:03.135305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414697+0000","last_change":"2024-09-17T05:03:17.489555+0000","last_active":"2024-09-17T05:03:20.414697+0000","last_peered":"2024-09-17T05:03:20.414697+0000","last_clean":"2024-09-17T05:03:20.414697+0000","last_became_active":"2024-09-17T05:03:17.489182+0000","last_became_peered":"2024-09-17T05:03:17.489182+0000","last_unstale":"2024-09-17T05:03:20.414697+0000","last_undegraded":"2024-09-17T05:03:20.414697+0000","last_fullsized":"2024-09-17T05:03:20.414697+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796408+0000","last_change":"2024-09-17T05:03:18.500202+0000","last_active":"2024-09-17T05:03:21.796408+0000","last_peered":"2024-09-17T05:03:21.796408+0000","last_clean":"2024-09-17T05:03:21.796408+0000","last_became_active":"2024-09-17T05:03:18.499893+0000","last_became_peered":"2024-09-17T05:03:18.499893+0000","last_unstale":"2024-09-17T05:03:21.796408+0000","last_undegraded":"2024-09-17T05:03:21.796408+0000","last_fullsized":"2024-09-17T05:03:21.796408+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:03.396864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487289+0000","last_change":"2024-09-17T05:03:17.473668+0000","last_active":"2024-09-17T05:03:19.487289+0000","last_peered":"2024-09-17T05:03:19.487289+0000","last_clean":"2024-09-17T05:03:19.487289+0000","last_became_active":"2024-09-17T05:03:17.473283+0000","last_became_peered":"2024-09-17T05:03:17.473283+0000","last_unstale":"2024-09-17T05:03:19.487289+0000","last_undegraded":"2024-09-17T05:03:19.487289+0000","last_fullsized":"2024-09-17T05:03:19.487289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:34.241801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487050+0000","last_change":"2024-09-17T05:03:18.495083+0000","last_active":"2024-09-17T05:03:19.487050+0000","last_peered":"2024-09-17T05:03:19.487050+0000","last_clean":"2024-09-17T05:03:19.487050+0000","last_became_active":"2024-09-17T05:03:18.494736+0000","last_became_peered":"2024-09-17T05:03:18.494736+0000","last_unstale":"2024-09-17T05:03:19.487050+0000","last_undegraded":"2024-09-17T05:03:19.487050+0000","last_fullsized":"2024-09-17T05:03:19.487050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:24.301106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796281+0000","last_change":"2024-09-17T05:03:17.484272+0000","last_active":"2024-09-17T05:03:21.796281+0000","last_peered":"2024-09-17T05:03:21.796281+0000","last_clean":"2024-09-17T05:03:21.796281+0000","last_became_active":"2024-09-17T05:03:17.483583+0000","last_became_peered":"2024-09-17T05:03:17.483583+0000","last_unstale":"2024-09-17T05:03:21.796281+0000","last_undegraded":"2024-09-17T05:03:21.796281+0000","last_fullsized":"2024-09-17T05:03:21.796281+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:53.765242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.495255+0000","last_change":"2024-09-17T05:03:18.514276+0000","last_active":"2024-09-17T05:03:19.495255+0000","last_peered":"2024-09-17T05:03:19.495255+0000","last_clean":"2024-09-17T05:03:19.495255+0000","last_became_active":"2024-09-17T05:03:18.513909+0000","last_became_peered":"2024-09-17T05:03:18.513909+0000","last_unstale":"2024-09-17T05:03:19.495255+0000","last_undegraded":"2024-09-17T05:03:19.495255+0000","last_fullsized":"2024-09-17T05:03:19.495255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485558+0000","last_change":"2024-09-17T05:03:17.489922+0000","last_active":"2024-09-17T05:03:19.485558+0000","last_peered":"2024-09-17T05:03:19.485558+0000","last_clean":"2024-09-17T05:03:19.485558+0000","last_became_active":"2024-09-17T05:03:17.489611+0000","last_became_peered":"2024-09-17T05:03:17.489611+0000","last_unstale":"2024-09-17T05:03:19.485558+0000","last_undegraded":"2024-09-17T05:03:19.485558+0000","last_fullsized":"2024-09-17T05:03:19.485558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:35.801215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796855+0000","last_change":"2024-09-17T05:03:18.485055+0000","last_active":"2024-09-17T05:03:21.796855+0000","last_peered":"2024-09-17T05:03:21.796855+0000","last_clean":"2024-09-17T05:03:21.796855+0000","last_became_active":"2024-09-17T05:03:18.484744+0000","last_became_peered":"2024-09-17T05:03:18.484744+0000","last_unstale":"2024-09-17T05:03:21.796855+0000","last_undegraded":"2024-09-17T05:03:21.796855+0000","last_fullsized":"2024-09-17T05:03:21.796855+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:14.318804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796609+0000","last_change":"2024-09-17T05:03:17.477682+0000","last_active":"2024-09-17T05:03:21.796609+0000","last_peered":"2024-09-17T05:03:21.796609+0000","last_clean":"2024-09-17T05:03:21.796609+0000","last_became_active":"2024-09-17T05:03:17.477257+0000","last_became_peered":"2024-09-17T05:03:17.477257+0000","last_unstale":"2024-09-17T05:03:21.796609+0000","last_undegraded":"2024-09-17T05:03:21.796609+0000","last_fullsized":"2024-09-17T05:03:21.796609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:29.176860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487353+0000","last_change":"2024-09-17T05:03:18.504857+0000","last_active":"2024-09-17T05:03:19.487353+0000","last_peered":"2024-09-17T05:03:19.487353+0000","last_clean":"2024-09-17T05:03:19.487353+0000","last_became_active":"2024-09-17T05:03:18.504185+0000","last_became_peered":"2024-09-17T05:03:18.504185+0000","last_unstale":"2024-09-17T05:03:19.487353+0000","last_undegraded":"2024-09-17T05:03:19.487353+0000","last_fullsized":"2024-09-17T05:03:19.487353+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:51.971141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796357+0000","last_change":"2024-09-17T05:03:17.488461+0000","last_active":"2024-09-17T05:03:21.796357+0000","last_peered":"2024-09-17T05:03:21.796357+0000","last_clean":"2024-09-17T05:03:21.796357+0000","last_became_active":"2024-09-17T05:03:17.488076+0000","last_became_peered":"2024-09-17T05:03:17.488076+0000","last_unstale":"2024-09-17T05:03:21.796357+0000","last_undegraded":"2024-09-17T05:03:21.796357+0000","last_fullsized":"2024-09-17T05:03:21.796357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:41.181926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485103+0000","last_change":"2024-09-17T05:03:18.494837+0000","last_active":"2024-09-17T05:03:19.485103+0000","last_peered":"2024-09-17T05:03:19.485103+0000","last_clean":"2024-09-17T05:03:19.485103+0000","last_became_active":"2024-09-17T05:03:18.494265+0000","last_became_peered":"2024-09-17T05:03:18.494265+0000","last_unstale":"2024-09-17T05:03:19.485103+0000","last_undegraded":"2024-09-17T05:03:19.485103+0000","last_fullsized":"2024-09-17T05:03:19.485103+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.380215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484969+0000","last_change":"2024-09-17T05:03:17.476199+0000","last_active":"2024-09-17T05:03:19.484969+0000","last_peered":"2024-09-17T05:03:19.484969+0000","last_clean":"2024-09-17T05:03:19.484969+0000","last_became_active":"2024-09-17T05:03:17.475834+0000","last_became_peered":"2024-09-17T05:03:17.475834+0000","last_unstale":"2024-09-17T05:03:19.484969+0000","last_undegraded":"2024-09-17T05:03:19.484969+0000","last_fullsized":"2024-09-17T05:03:19.484969+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:21.694015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494962+0000","last_change":"2024-09-17T05:03:18.495879+0000","last_active":"2024-09-17T05:03:19.494962+0000","last_peered":"2024-09-17T05:03:19.494962+0000","last_clean":"2024-09-17T05:03:19.494962+0000","last_became_active":"2024-09-17T05:03:18.495553+0000","last_became_peered":"2024-09-17T05:03:18.495553+0000","last_unstale":"2024-09-17T05:03:19.494962+0000","last_undegraded":"2024-09-17T05:03:19.494962+0000","last_fullsized":"2024-09-17T05:03:19.494962+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485216+0000","last_change":"2024-09-17T05:03:17.484357+0000","last_active":"2024-09-17T05:03:19.485216+0000","last_peered":"2024-09-17T05:03:19.485216+0000","last_clean":"2024-09-17T05:03:19.485216+0000","last_became_active":"2024-09-17T05:03:17.483648+0000","last_became_peered":"2024-09-17T05:03:17.483648+0000","last_unstale":"2024-09-17T05:03:19.485216+0000","last_undegraded":"2024-09-17T05:03:19.485216+0000","last_fullsized":"2024-09-17T05:03:19.485216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484412+0000","last_change":"2024-09-17T05:03:18.496015+0000","last_active":"2024-09-17T05:03:19.484412+0000","last_peered":"2024-09-17T05:03:19.484412+0000","last_clean":"2024-09-17T05:03:19.484412+0000","last_became_active":"2024-09-17T05:03:18.495714+0000","last_became_peered":"2024-09-17T05:03:18.495714+0000","last_unstale":"2024-09-17T05:03:19.484412+0000","last_undegraded":"2024-09-17T05:03:19.484412+0000","last_fullsized":"2024-09-17T05:03:19.484412+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.412751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485074+0000","last_change":"2024-09-17T05:03:17.483837+0000","last_active":"2024-09-17T05:03:19.485074+0000","last_peered":"2024-09-17T05:03:19.485074+0000","last_clean":"2024-09-17T05:03:19.485074+0000","last_became_active":"2024-09-17T05:03:17.483474+0000","last_became_peered":"2024-09-17T05:03:17.483474+0000","last_unstale":"2024-09-17T05:03:19.485074+0000","last_undegraded":"2024-09-17T05:03:19.485074+0000","last_fullsized":"2024-09-17T05:03:19.485074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.383407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795609+0000","last_change":"2024-09-17T05:03:18.513177+0000","last_active":"2024-09-17T05:03:21.795609+0000","last_peered":"2024-09-17T05:03:21.795609+0000","last_clean":"2024-09-17T05:03:21.795609+0000","last_became_active":"2024-09-17T05:03:18.512727+0000","last_became_peered":"2024-09-17T05:03:18.512727+0000","last_unstale":"2024-09-17T05:03:21.795609+0000","last_undegraded":"2024-09-17T05:03:21.795609+0000","last_fullsized":"2024-09-17T05:03:21.795609+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:55.305773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796884+0000","last_change":"2024-09-17T05:03:17.471492+0000","last_active":"2024-09-17T05:03:21.796884+0000","last_peered":"2024-09-17T05:03:21.796884+0000","last_clean":"2024-09-17T05:03:21.796884+0000","last_became_active":"2024-09-17T05:03:17.471119+0000","last_became_peered":"2024-09-17T05:03:17.471119+0000","last_unstale":"2024-09-17T05:03:21.796884+0000","last_undegraded":"2024-09-17T05:03:21.796884+0000","last_fullsized":"2024-09-17T05:03:21.796884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:12.694951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795808+0000","last_change":"2024-09-17T05:03:18.495599+0000","last_active":"2024-09-17T05:03:21.795808+0000","last_peered":"2024-09-17T05:03:21.795808+0000","last_clean":"2024-09-17T05:03:21.795808+0000","last_became_active":"2024-09-17T05:03:18.495038+0000","last_became_peered":"2024-09-17T05:03:18.495038+0000","last_unstale":"2024-09-17T05:03:21.795808+0000","last_undegraded":"2024-09-17T05:03:21.795808+0000","last_fullsized":"2024-09-17T05:03:21.795808+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:26.057553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487172+0000","last_change":"2024-09-17T05:03:17.480393+0000","last_active":"2024-09-17T05:03:19.487172+0000","last_peered":"2024-09-17T05:03:19.487172+0000","last_clean":"2024-09-17T05:03:19.487172+0000","last_became_active":"2024-09-17T05:03:17.479912+0000","last_became_peered":"2024-09-17T05:03:17.479912+0000","last_unstale":"2024-09-17T05:03:19.487172+0000","last_undegraded":"2024-09-17T05:03:19.487172+0000","last_fullsized":"2024-09-17T05:03:19.487172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:06.645919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796839+0000","last_change":"2024-09-17T05:03:18.504212+0000","last_active":"2024-09-17T05:03:21.796839+0000","last_peered":"2024-09-17T05:03:21.796839+0000","last_clean":"2024-09-17T05:03:21.796839+0000","last_became_active":"2024-09-17T05:03:18.503841+0000","last_became_peered":"2024-09-17T05:03:18.503841+0000","last_unstale":"2024-09-17T05:03:21.796839+0000","last_undegraded":"2024-09-17T05:03:21.796839+0000","last_fullsized":"2024-09-17T05:03:21.796839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:51.416667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414644+0000","last_change":"2024-09-17T05:03:17.482800+0000","last_active":"2024-09-17T05:03:20.414644+0000","last_peered":"2024-09-17T05:03:20.414644+0000","last_clean":"2024-09-17T05:03:20.414644+0000","last_became_active":"2024-09-17T05:03:17.482376+0000","last_became_peered":"2024-09-17T05:03:17.482376+0000","last_unstale":"2024-09-17T05:03:20.414644+0000","last_undegraded":"2024-09-17T05:03:20.414644+0000","last_fullsized":"2024-09-17T05:03:20.414644+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:06.028991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795582+0000","last_change":"2024-09-17T05:03:18.506127+0000","last_active":"2024-09-17T05:03:21.795582+0000","last_peered":"2024-09-17T05:03:21.795582+0000","last_clean":"2024-09-17T05:03:21.795582+0000","last_became_active":"2024-09-17T05:03:18.505295+0000","last_became_peered":"2024-09-17T05:03:18.505295+0000","last_unstale":"2024-09-17T05:03:21.795582+0000","last_undegraded":"2024-09-17T05:03:21.795582+0000","last_fullsized":"2024-09-17T05:03:21.795582+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:50.676398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484488+0000","last_change":"2024-09-17T05:03:17.477301+0000","last_active":"2024-09-17T05:03:19.484488+0000","last_peered":"2024-09-17T05:03:19.484488+0000","last_clean":"2024-09-17T05:03:19.484488+0000","last_became_active":"2024-09-17T05:03:17.476888+0000","last_became_peered":"2024-09-17T05:03:17.476888+0000","last_unstale":"2024-09-17T05:03:19.484488+0000","last_undegraded":"2024-09-17T05:03:19.484488+0000","last_fullsized":"2024-09-17T05:03:19.484488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:58.301917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414868+0000","last_change":"2024-09-17T05:03:18.507972+0000","last_active":"2024-09-17T05:03:20.414868+0000","last_peered":"2024-09-17T05:03:20.414868+0000","last_clean":"2024-09-17T05:03:20.414868+0000","last_became_active":"2024-09-17T05:03:18.507423+0000","last_became_peered":"2024-09-17T05:03:18.507423+0000","last_unstale":"2024-09-17T05:03:20.414868+0000","last_undegraded":"2024-09-17T05:03:20.414868+0000","last_fullsized":"2024-09-17T05:03:20.414868+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.380215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795437+0000","last_change":"2024-09-17T05:03:17.481847+0000","last_active":"2024-09-17T05:03:21.795437+0000","last_peered":"2024-09-17T05:03:21.795437+0000","last_clean":"2024-09-17T05:03:21.795437+0000","last_became_active":"2024-09-17T05:03:17.481546+0000","last_became_peered":"2024-09-17T05:03:17.481546+0000","last_unstale":"2024-09-17T05:03:21.795437+0000","last_undegraded":"2024-09-17T05:03:21.795437+0000","last_fullsized":"2024-09-17T05:03:21.795437+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.383407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485912+0000","last_change":"2024-09-17T05:03:18.484589+0000","last_active":"2024-09-17T05:03:19.485912+0000","last_peered":"2024-09-17T05:03:19.485912+0000","last_clean":"2024-09-17T05:03:19.485912+0000","last_became_active":"2024-09-17T05:03:18.483862+0000","last_became_peered":"2024-09-17T05:03:18.483862+0000","last_unstale":"2024-09-17T05:03:19.485912+0000","last_undegraded":"2024-09-17T05:03:19.485912+0000","last_fullsized":"2024-09-17T05:03:19.485912+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.509403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":23,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414651+0000","last_change":"2024-09-17T05:03:17.489475+0000","last_active":"2024-09-17T05:03:20.414651+0000","last_peered":"2024-09-17T05:03:20.414651+0000","last_clean":"2024-09-17T05:03:20.414651+0000","last_became_active":"2024-09-17T05:03:17.489135+0000","last_became_peered":"2024-09-17T05:03:17.489135+0000","last_unstale":"2024-09-17T05:03:20.414651+0000","last_undegraded":"2024-09-17T05:03:20.414651+0000","last_fullsized":"2024-09-17T05:03:20.414651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:35.734369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486537+0000","last_change":"2024-09-17T05:03:18.487882+0000","last_active":"2024-09-17T05:03:19.486537+0000","last_peered":"2024-09-17T05:03:19.486537+0000","last_clean":"2024-09-17T05:03:19.486537+0000","last_became_active":"2024-09-17T05:03:18.487523+0000","last_became_peered":"2024-09-17T05:03:18.487523+0000","last_unstale":"2024-09-17T05:03:19.486537+0000","last_undegraded":"2024-09-17T05:03:19.486537+0000","last_fullsized":"2024-09-17T05:03:19.486537+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:44.737783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414979+0000","last_change":"2024-09-17T05:03:17.489385+0000","last_active":"2024-09-17T05:03:20.414979+0000","last_peered":"2024-09-17T05:03:20.414979+0000","last_clean":"2024-09-17T05:03:20.414979+0000","last_became_active":"2024-09-17T05:03:17.489069+0000","last_became_peered":"2024-09-17T05:03:17.489069+0000","last_unstale":"2024-09-17T05:03:20.414979+0000","last_undegraded":"2024-09-17T05:03:20.414979+0000","last_fullsized":"2024-09-17T05:03:20.414979+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:33.379573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795898+0000","last_change":"2024-09-17T05:03:18.501550+0000","last_active":"2024-09-17T05:03:21.795898+0000","last_peered":"2024-09-17T05:03:21.795898+0000","last_clean":"2024-09-17T05:03:21.795898+0000","last_became_active":"2024-09-17T05:03:18.501150+0000","last_became_peered":"2024-09-17T05:03:18.501150+0000","last_unstale":"2024-09-17T05:03:21.795898+0000","last_undegraded":"2024-09-17T05:03:21.795898+0000","last_fullsized":"2024-09-17T05:03:21.795898+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:30.983417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414870+0000","last_change":"2024-09-17T05:03:17.483869+0000","last_active":"2024-09-17T05:03:20.414870+0000","last_peered":"2024-09-17T05:03:20.414870+0000","last_clean":"2024-09-17T05:03:20.414870+0000","last_became_active":"2024-09-17T05:03:17.483565+0000","last_became_peered":"2024-09-17T05:03:17.483565+0000","last_unstale":"2024-09-17T05:03:20.414870+0000","last_undegraded":"2024-09-17T05:03:20.414870+0000","last_fullsized":"2024-09-17T05:03:20.414870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:48.075582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795516+0000","last_change":"2024-09-17T05:03:18.489876+0000","last_active":"2024-09-17T05:03:21.795516+0000","last_peered":"2024-09-17T05:03:21.795516+0000","last_clean":"2024-09-17T05:03:21.795516+0000","last_became_active":"2024-09-17T05:03:18.488017+0000","last_became_peered":"2024-09-17T05:03:18.488017+0000","last_unstale":"2024-09-17T05:03:21.795516+0000","last_undegraded":"2024-09-17T05:03:21.795516+0000","last_fullsized":"2024-09-17T05:03:21.795516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:13.753545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484788+0000","last_change":"2024-09-17T05:03:17.484821+0000","last_active":"2024-09-17T05:03:19.484788+0000","last_peered":"2024-09-17T05:03:19.484788+0000","last_clean":"2024-09-17T05:03:19.484788+0000","last_became_active":"2024-09-17T05:03:17.484445+0000","last_became_peered":"2024-09-17T05:03:17.484445+0000","last_unstale":"2024-09-17T05:03:19.484788+0000","last_undegraded":"2024-09-17T05:03:19.484788+0000","last_fullsized":"2024-09-17T05:03:19.484788+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:44.206475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414381+0000","last_change":"2024-09-17T05:03:18.508867+0000","last_active":"2024-09-17T05:03:20.414381+0000","last_peered":"2024-09-17T05:03:20.414381+0000","last_clean":"2024-09-17T05:03:20.414381+0000","last_became_active":"2024-09-17T05:03:18.508319+0000","last_became_peered":"2024-09-17T05:03:18.508319+0000","last_unstale":"2024-09-17T05:03:20.414381+0000","last_undegraded":"2024-09-17T05:03:20.414381+0000","last_fullsized":"2024-09-17T05:03:20.414381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:01.699937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796400+0000","last_change":"2024-09-17T05:03:17.469441+0000","last_active":"2024-09-17T05:03:21.796400+0000","last_peered":"2024-09-17T05:03:21.796400+0000","last_clean":"2024-09-17T05:03:21.796400+0000","last_became_active":"2024-09-17T05:03:17.469019+0000","last_became_peered":"2024-09-17T05:03:17.469019+0000","last_unstale":"2024-09-17T05:03:21.796400+0000","last_undegraded":"2024-09-17T05:03:21.796400+0000","last_fullsized":"2024-09-17T05:03:21.796400+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:02.998746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796262+0000","last_change":"2024-09-17T05:03:18.489610+0000","last_active":"2024-09-17T05:03:21.796262+0000","last_peered":"2024-09-17T05:03:21.796262+0000","last_clean":"2024-09-17T05:03:21.796262+0000","last_became_active":"2024-09-17T05:03:18.489271+0000","last_became_peered":"2024-09-17T05:03:18.489271+0000","last_unstale":"2024-09-17T05:03:21.796262+0000","last_undegraded":"2024-09-17T05:03:21.796262+0000","last_fullsized":"2024-09-17T05:03:21.796262+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:50.676398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795797+0000","last_change":"2024-09-17T05:03:17.488710+0000","last_active":"2024-09-17T05:03:21.795797+0000","last_peered":"2024-09-17T05:03:21.795797+0000","last_clean":"2024-09-17T05:03:21.795797+0000","last_became_active":"2024-09-17T05:03:17.488178+0000","last_became_peered":"2024-09-17T05:03:17.488178+0000","last_unstale":"2024-09-17T05:03:21.795797+0000","last_undegraded":"2024-09-17T05:03:21.795797+0000","last_fullsized":"2024-09-17T05:03:21.795797+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:44.044047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484785+0000","last_change":"2024-09-17T05:03:18.484052+0000","last_active":"2024-09-17T05:03:19.484785+0000","last_peered":"2024-09-17T05:03:19.484785+0000","last_clean":"2024-09-17T05:03:19.484785+0000","last_became_active":"2024-09-17T05:03:18.483686+0000","last_became_peered":"2024-09-17T05:03:18.483686+0000","last_unstale":"2024-09-17T05:03:19.484785+0000","last_undegraded":"2024-09-17T05:03:19.484785+0000","last_fullsized":"2024-09-17T05:03:19.484785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485517+0000","last_change":"2024-09-17T05:03:17.487381+0000","last_active":"2024-09-17T05:03:19.485517+0000","last_peered":"2024-09-17T05:03:19.485517+0000","last_clean":"2024-09-17T05:03:19.485517+0000","last_became_active":"2024-09-17T05:03:17.487025+0000","last_became_peered":"2024-09-17T05:03:17.487025+0000","last_unstale":"2024-09-17T05:03:19.485517+0000","last_undegraded":"2024-09-17T05:03:19.485517+0000","last_fullsized":"2024-09-17T05:03:19.485517+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:38.335990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795664+0000","last_change":"2024-09-17T05:03:18.513093+0000","last_active":"2024-09-17T05:03:21.795664+0000","last_peered":"2024-09-17T05:03:21.795664+0000","last_clean":"2024-09-17T05:03:21.795664+0000","last_became_active":"2024-09-17T05:03:18.512779+0000","last_became_peered":"2024-09-17T05:03:18.512779+0000","last_unstale":"2024-09-17T05:03:21.795664+0000","last_undegraded":"2024-09-17T05:03:21.795664+0000","last_fullsized":"2024-09-17T05:03:21.795664+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:07.251450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486706+0000","last_change":"2024-09-17T05:03:17.488730+0000","last_active":"2024-09-17T05:03:19.486706+0000","last_peered":"2024-09-17T05:03:19.486706+0000","last_clean":"2024-09-17T05:03:19.486706+0000","last_became_active":"2024-09-17T05:03:17.487850+0000","last_became_peered":"2024-09-17T05:03:17.487850+0000","last_unstale":"2024-09-17T05:03:19.486706+0000","last_undegraded":"2024-09-17T05:03:19.486706+0000","last_fullsized":"2024-09-17T05:03:19.486706+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:20.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414954+0000","last_change":"2024-09-17T05:03:18.508348+0000","last_active":"2024-09-17T05:03:20.414954+0000","last_peered":"2024-09-17T05:03:20.414954+0000","last_clean":"2024-09-17T05:03:20.414954+0000","last_became_active":"2024-09-17T05:03:18.507958+0000","last_became_peered":"2024-09-17T05:03:18.507958+0000","last_unstale":"2024-09-17T05:03:20.414954+0000","last_undegraded":"2024-09-17T05:03:20.414954+0000","last_fullsized":"2024-09-17T05:03:20.414954+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:41.130167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486411+0000","last_change":"2024-09-17T05:03:17.477112+0000","last_active":"2024-09-17T05:03:19.486411+0000","last_peered":"2024-09-17T05:03:19.486411+0000","last_clean":"2024-09-17T05:03:19.486411+0000","last_became_active":"2024-09-17T05:03:17.476642+0000","last_became_peered":"2024-09-17T05:03:17.476642+0000","last_unstale":"2024-09-17T05:03:19.486411+0000","last_undegraded":"2024-09-17T05:03:19.486411+0000","last_fullsized":"2024-09-17T05:03:19.486411+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:50.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484227+0000","last_change":"2024-09-17T05:03:18.493987+0000","last_active":"2024-09-17T05:03:19.484227+0000","last_peered":"2024-09-17T05:03:19.484227+0000","last_clean":"2024-09-17T05:03:19.484227+0000","last_became_active":"2024-09-17T05:03:18.492993+0000","last_became_peered":"2024-09-17T05:03:18.492993+0000","last_unstale":"2024-09-17T05:03:19.484227+0000","last_undegraded":"2024-09-17T05:03:19.484227+0000","last_fullsized":"2024-09-17T05:03:19.484227+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:01.699937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796817+0000","last_change":"2024-09-17T05:03:17.488680+0000","last_active":"2024-09-17T05:03:21.796817+0000","last_peered":"2024-09-17T05:03:21.796817+0000","last_clean":"2024-09-17T05:03:21.796817+0000","last_became_active":"2024-09-17T05:03:17.488050+0000","last_became_peered":"2024-09-17T05:03:17.488050+0000","last_unstale":"2024-09-17T05:03:21.796817+0000","last_undegraded":"2024-09-17T05:03:21.796817+0000","last_fullsized":"2024-09-17T05:03:21.796817+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:01:44.257805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487087+0000","last_change":"2024-09-17T05:03:18.495602+0000","last_active":"2024-09-17T05:03:19.487087+0000","last_peered":"2024-09-17T05:03:19.487087+0000","last_clean":"2024-09-17T05:03:19.487087+0000","last_became_active":"2024-09-17T05:03:18.495190+0000","last_became_peered":"2024-09-17T05:03:18.495190+0000","last_unstale":"2024-09-17T05:03:19.487087+0000","last_undegraded":"2024-09-17T05:03:19.487087+0000","last_fullsized":"2024-09-17T05:03:19.487087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:49.184314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.797002+0000","last_change":"2024-09-17T05:03:17.471433+0000","last_active":"2024-09-17T05:03:21.797002+0000","last_peered":"2024-09-17T05:03:21.797002+0000","last_clean":"2024-09-17T05:03:21.797002+0000","last_became_active":"2024-09-17T05:03:17.470676+0000","last_became_peered":"2024-09-17T05:03:17.470676+0000","last_unstale":"2024-09-17T05:03:21.797002+0000","last_undegraded":"2024-09-17T05:03:21.797002+0000","last_fullsized":"2024-09-17T05:03:21.797002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:35.470632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485340+0000","last_change":"2024-09-17T05:03:18.499092+0000","last_active":"2024-09-17T05:03:19.485340+0000","last_peered":"2024-09-17T05:03:19.485340+0000","last_clean":"2024-09-17T05:03:19.485340+0000","last_became_active":"2024-09-17T05:03:18.498784+0000","last_became_peered":"2024-09-17T05:03:18.498784+0000","last_unstale":"2024-09-17T05:03:19.485340+0000","last_undegraded":"2024-09-17T05:03:19.485340+0000","last_fullsized":"2024-09-17T05:03:19.485340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:35.871301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485538+0000","last_change":"2024-09-17T05:03:17.481418+0000","last_active":"2024-09-17T05:03:19.485538+0000","last_peered":"2024-09-17T05:03:19.485538+0000","last_clean":"2024-09-17T05:03:19.485538+0000","last_became_active":"2024-09-17T05:03:17.481090+0000","last_became_peered":"2024-09-17T05:03:17.481090+0000","last_unstale":"2024-09-17T05:03:19.485538+0000","last_undegraded":"2024-09-17T05:03:19.485538+0000","last_fullsized":"2024-09-17T05:03:19.485538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.000517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487212+0000","last_change":"2024-09-17T05:03:18.496866+0000","last_active":"2024-09-17T05:03:19.487212+0000","last_peered":"2024-09-17T05:03:19.487212+0000","last_clean":"2024-09-17T05:03:19.487212+0000","last_became_active":"2024-09-17T05:03:18.496533+0000","last_became_peered":"2024-09-17T05:03:18.496533+0000","last_unstale":"2024-09-17T05:03:19.487212+0000","last_undegraded":"2024-09-17T05:03:19.487212+0000","last_fullsized":"2024-09-17T05:03:19.487212+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:06.312704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487198+0000","last_change":"2024-09-17T05:03:17.470269+0000","last_active":"2024-09-17T05:03:19.487198+0000","last_peered":"2024-09-17T05:03:19.487198+0000","last_clean":"2024-09-17T05:03:19.487198+0000","last_became_active":"2024-09-17T05:03:17.469912+0000","last_became_peered":"2024-09-17T05:03:17.469912+0000","last_unstale":"2024-09-17T05:03:19.487198+0000","last_undegraded":"2024-09-17T05:03:19.487198+0000","last_fullsized":"2024-09-17T05:03:19.487198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:33.570989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795830+0000","last_change":"2024-09-17T05:03:18.494624+0000","last_active":"2024-09-17T05:03:21.795830+0000","last_peered":"2024-09-17T05:03:21.795830+0000","last_clean":"2024-09-17T05:03:21.795830+0000","last_became_active":"2024-09-17T05:03:18.494312+0000","last_became_peered":"2024-09-17T05:03:18.494312+0000","last_unstale":"2024-09-17T05:03:21.795830+0000","last_undegraded":"2024-09-17T05:03:21.795830+0000","last_fullsized":"2024-09-17T05:03:21.795830+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:54.573981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485616+0000","last_change":"2024-09-17T05:03:17.486561+0000","last_active":"2024-09-17T05:03:19.485616+0000","last_peered":"2024-09-17T05:03:19.485616+0000","last_clean":"2024-09-17T05:03:19.485616+0000","last_became_active":"2024-09-17T05:03:17.486224+0000","last_became_peered":"2024-09-17T05:03:17.486224+0000","last_unstale":"2024-09-17T05:03:19.485616+0000","last_undegraded":"2024-09-17T05:03:19.485616+0000","last_fullsized":"2024-09-17T05:03:19.485616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:31.418463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487417+0000","last_change":"2024-09-17T05:03:18.504505+0000","last_active":"2024-09-17T05:03:19.487417+0000","last_peered":"2024-09-17T05:03:19.487417+0000","last_clean":"2024-09-17T05:03:19.487417+0000","last_became_active":"2024-09-17T05:03:18.503742+0000","last_became_peered":"2024-09-17T05:03:18.503742+0000","last_unstale":"2024-09-17T05:03:19.487417+0000","last_undegraded":"2024-09-17T05:03:19.487417+0000","last_fullsized":"2024-09-17T05:03:19.487417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795520+0000","last_change":"2024-09-17T05:03:17.479945+0000","last_active":"2024-09-17T05:03:21.795520+0000","last_peered":"2024-09-17T05:03:21.795520+0000","last_clean":"2024-09-17T05:03:21.795520+0000","last_became_active":"2024-09-17T05:03:17.479596+0000","last_became_peered":"2024-09-17T05:03:17.479596+0000","last_unstale":"2024-09-17T05:03:21.795520+0000","last_undegraded":"2024-09-17T05:03:21.795520+0000","last_fullsized":"2024-09-17T05:03:21.795520+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414745+0000","last_change":"2024-09-17T05:03:18.503490+0000","last_active":"2024-09-17T05:03:20.414745+0000","last_peered":"2024-09-17T05:03:20.414745+0000","last_clean":"2024-09-17T05:03:20.414745+0000","last_became_active":"2024-09-17T05:03:18.503217+0000","last_became_peered":"2024-09-17T05:03:18.503217+0000","last_unstale":"2024-09-17T05:03:20.414745+0000","last_undegraded":"2024-09-17T05:03:20.414745+0000","last_fullsized":"2024-09-17T05:03:20.414745+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484236+0000","last_change":"2024-09-17T05:03:17.476961+0000","last_active":"2024-09-17T05:03:19.484236+0000","last_peered":"2024-09-17T05:03:19.484236+0000","last_clean":"2024-09-17T05:03:19.484236+0000","last_became_active":"2024-09-17T05:03:17.476427+0000","last_became_peered":"2024-09-17T05:03:17.476427+0000","last_unstale":"2024-09-17T05:03:19.484236+0000","last_undegraded":"2024-09-17T05:03:19.484236+0000","last_fullsized":"2024-09-17T05:03:19.484236+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:28.438189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494906+0000","last_change":"2024-09-17T05:03:18.509986+0000","last_active":"2024-09-17T05:03:19.494906+0000","last_peered":"2024-09-17T05:03:19.494906+0000","last_clean":"2024-09-17T05:03:19.494906+0000","last_became_active":"2024-09-17T05:03:18.509706+0000","last_became_peered":"2024-09-17T05:03:18.509706+0000","last_unstale":"2024-09-17T05:03:19.494906+0000","last_undegraded":"2024-09-17T05:03:19.494906+0000","last_fullsized":"2024-09-17T05:03:19.494906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494256+0000","last_change":"2024-09-17T05:03:17.483817+0000","last_active":"2024-09-17T05:03:19.494256+0000","last_peered":"2024-09-17T05:03:19.494256+0000","last_clean":"2024-09-17T05:03:19.494256+0000","last_became_active":"2024-09-17T05:03:17.483478+0000","last_became_peered":"2024-09-17T05:03:17.483478+0000","last_unstale":"2024-09-17T05:03:19.494256+0000","last_undegraded":"2024-09-17T05:03:19.494256+0000","last_fullsized":"2024-09-17T05:03:19.494256+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:27.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484317+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:19.484317+0000","last_peered":"2024-09-17T05:03:19.484317+0000","last_clean":"2024-09-17T05:03:19.484317+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:19.484317+0000","last_undegraded":"2024-09-17T05:03:19.484317+0000","last_fullsized":"2024-09-17T05:03:19.484317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":643072,"data_stored":277529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":275915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":275859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":866263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1232896,"data_stored":867895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:27.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 -- 172.21.15.16:0/3795250918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa00015f150 msgr2=0x7fa00015f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 --2- 172.21.15.16:0/3795250918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa00015f150 0x7fa00015f560 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa0000ccfb0 tx=0x7f9ff801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 -- 172.21.15.16:0/3795250918 shutdown_connections 2024-09-17T05:03:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 --2- 172.21.15.16:0/3795250918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa00015f150 0x7fa00015f560 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 --2- 172.21.15.16:0/3795250918 >> v2:172.21.15.104:3300/0 conn(0x7fa00015c0a0 0x7fa00015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 --2- 172.21.15.16:0/3795250918 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0000b2f50 0x7fa00015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 -- 172.21.15.16:0/3795250918 >> 172.21.15.16:0/3795250918 conn(0x7fa00006f900 msgr2=0x7fa0000b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 -- 172.21.15.16:0/3795250918 shutdown_connections 2024-09-17T05:03:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 -- 172.21.15.16:0/3795250918 wait complete. 2024-09-17T05:03:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 Processor -- start 2024-09-17T05:03:27.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.705+0000 7fa006a00640 1 -- start start 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 0x7fa000157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa00015c0a0 0x7fa0001414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa00015f150 0x7fa000159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa000073f50 con 0x7fa0000b2f50 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa000073dd0 con 0x7fa00015c0a0 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa0000740d0 con 0x7fa00015f150 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 0x7fa000157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 0x7fa000157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58308/0 (socket says 172.21.15.16:58308) 2024-09-17T05:03:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 -- 172.21.15.16:0/2995153624 learned_addr learned my addr 172.21.15.16:0/2995153624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 -- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa00015f150 msgr2=0x7fa000159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa00015f150 0x7fa000159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 -- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa00015c0a0 msgr2=0x7fa0001414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa00015c0a0 0x7fa0001414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 -- 172.21.15.16:0/2995153624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0001f6860 con 0x7fa0000b2f50 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9fffe00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 0x7fa000157080 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d690 tx=0x7f9ff000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0017070 con 0x7fa0000b2f50 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ff0005310 con 0x7fa0000b2f50 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0009820 con 0x7fa0000b2f50 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa000157a90 con 0x7fa0000b2f50 2024-09-17T05:03:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa000158050 con 0x7fa0000b2f50 2024-09-17T05:03:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.709+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa00015c4b0 con 0x7fa0000b2f50 2024-09-17T05:03:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.713+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ff00099c0 con 0x7fa0000b2f50 2024-09-17T05:03:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.713+0000 7f9ffcc00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9fdc073200 0x7f9fdc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.713+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9ff0094530 con 0x7fa0000b2f50 2024-09-17T05:03:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.713+0000 7f9ff7e00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9fdc073200 0x7f9fdc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:27.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.713+0000 7f9ff7e00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9fdc073200 0x7f9fdc075710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa0000cc840 tx=0x7f9fe8005cf0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.717+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ff005f320 con 0x7fa0000b2f50 2024-09-17T05:03:27.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.889+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa000152c80 con 0x7f9fdc073200 2024-09-17T05:03:27.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.893+0000 7f9ffcc00640 1 -- 172.21.15.16:0/2995153624 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7fa000152c80 con 0x7f9fdc073200 2024-09-17T05:03:27.900 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9fdc073200 msgr2=0x7f9fdc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9fdc073200 0x7f9fdc075710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa0000cc840 tx=0x7f9fe8005cf0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 msgr2=0x7fa000157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 0x7fa000157080 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000d690 tx=0x7f9ff000db50 comp rx=0 tx=0).stop 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 shutdown_connections 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9fdc073200 0x7f9fdc075710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa00015f150 0x7fa000159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa00015c0a0 0x7fa0001414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 --2- 172.21.15.16:0/2995153624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0000b2f50 0x7fa000157080 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 >> 172.21.15.16:0/2995153624 conn(0x7fa00006f900 msgr2=0x7fa000165a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 shutdown_connections 2024-09-17T05:03:27.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:27.897+0000 7fa006a00640 1 -- 172.21.15.16:0/2995153624 wait complete. 2024-09-17T05:03:27.920 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-17T05:03:27.497204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243112,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731608,"statfs":{"total":773094113280,"available":772845166592,"internally_reserved":0,"allocated":6234112,"data_stored":3392156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.026993"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795632+0000","last_change":"2024-09-17T05:03:18.501500+0000","last_active":"2024-09-17T05:03:21.795632+0000","last_peered":"2024-09-17T05:03:21.795632+0000","last_clean":"2024-09-17T05:03:21.795632+0000","last_became_active":"2024-09-17T05:03:18.501008+0000","last_became_peered":"2024-09-17T05:03:18.501008+0000","last_unstale":"2024-09-17T05:03:21.795632+0000","last_undegraded":"2024-09-17T05:03:21.795632+0000","last_fullsized":"2024-09-17T05:03:21.795632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:37.679190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796248+0000","last_change":"2024-09-17T05:03:18.490954+0000","last_active":"2024-09-17T05:03:21.796248+0000","last_peered":"2024-09-17T05:03:21.796248+0000","last_clean":"2024-09-17T05:03:21.796248+0000","last_became_active":"2024-09-17T05:03:18.490645+0000","last_became_peered":"2024-09-17T05:03:18.490645+0000","last_unstale":"2024-09-17T05:03:21.796248+0000","last_undegraded":"2024-09-17T05:03:21.796248+0000","last_fullsized":"2024-09-17T05:03:21.796248+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:07.251450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485314+0000","last_change":"2024-09-17T05:03:18.505087+0000","last_active":"2024-09-17T05:03:19.485314+0000","last_peered":"2024-09-17T05:03:19.485314+0000","last_clean":"2024-09-17T05:03:19.485314+0000","last_became_active":"2024-09-17T05:03:18.504758+0000","last_became_peered":"2024-09-17T05:03:18.504758+0000","last_unstale":"2024-09-17T05:03:19.485314+0000","last_undegraded":"2024-09-17T05:03:19.485314+0000","last_fullsized":"2024-09-17T05:03:19.485314+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:15.676197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484737+0000","last_change":"2024-09-17T05:03:18.498547+0000","last_active":"2024-09-17T05:03:19.484737+0000","last_peered":"2024-09-17T05:03:19.484737+0000","last_clean":"2024-09-17T05:03:19.484737+0000","last_became_active":"2024-09-17T05:03:18.498140+0000","last_became_peered":"2024-09-17T05:03:18.498140+0000","last_unstale":"2024-09-17T05:03:19.484737+0000","last_undegraded":"2024-09-17T05:03:19.484737+0000","last_fullsized":"2024-09-17T05:03:19.484737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795682+0000","last_change":"2024-09-17T05:03:18.493490+0000","last_active":"2024-09-17T05:03:21.795682+0000","last_peered":"2024-09-17T05:03:21.795682+0000","last_clean":"2024-09-17T05:03:21.795682+0000","last_became_active":"2024-09-17T05:03:18.493086+0000","last_became_peered":"2024-09-17T05:03:18.493086+0000","last_unstale":"2024-09-17T05:03:21.795682+0000","last_undegraded":"2024-09-17T05:03:21.795682+0000","last_fullsized":"2024-09-17T05:03:21.795682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:47.377246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414436+0000","last_change":"2024-09-17T05:03:18.503188+0000","last_active":"2024-09-17T05:03:20.414436+0000","last_peered":"2024-09-17T05:03:20.414436+0000","last_clean":"2024-09-17T05:03:20.414436+0000","last_became_active":"2024-09-17T05:03:18.502531+0000","last_became_peered":"2024-09-17T05:03:18.502531+0000","last_unstale":"2024-09-17T05:03:20.414436+0000","last_undegraded":"2024-09-17T05:03:20.414436+0000","last_fullsized":"2024-09-17T05:03:20.414436+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486528+0000","last_change":"2024-09-17T05:03:18.485411+0000","last_active":"2024-09-17T05:03:19.486528+0000","last_peered":"2024-09-17T05:03:19.486528+0000","last_clean":"2024-09-17T05:03:19.486528+0000","last_became_active":"2024-09-17T05:03:18.485074+0000","last_became_peered":"2024-09-17T05:03:18.485074+0000","last_unstale":"2024-09-17T05:03:19.486528+0000","last_undegraded":"2024-09-17T05:03:19.486528+0000","last_fullsized":"2024-09-17T05:03:19.486528+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:07.445182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414394+0000","last_change":"2024-09-17T05:03:18.499467+0000","last_active":"2024-09-17T05:03:20.414394+0000","last_peered":"2024-09-17T05:03:20.414394+0000","last_clean":"2024-09-17T05:03:20.414394+0000","last_became_active":"2024-09-17T05:03:18.499137+0000","last_became_peered":"2024-09-17T05:03:18.499137+0000","last_unstale":"2024-09-17T05:03:20.414394+0000","last_undegraded":"2024-09-17T05:03:20.414394+0000","last_fullsized":"2024-09-17T05:03:20.414394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:18.636910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485862+0000","last_change":"2024-09-17T05:03:18.496016+0000","last_active":"2024-09-17T05:03:19.485862+0000","last_peered":"2024-09-17T05:03:19.485862+0000","last_clean":"2024-09-17T05:03:19.485862+0000","last_became_active":"2024-09-17T05:03:18.495706+0000","last_became_peered":"2024-09-17T05:03:18.495706+0000","last_unstale":"2024-09-17T05:03:19.485862+0000","last_undegraded":"2024-09-17T05:03:19.485862+0000","last_fullsized":"2024-09-17T05:03:19.485862+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:03.396864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487208+0000","last_change":"2024-09-17T05:03:18.490713+0000","last_active":"2024-09-17T05:03:19.487208+0000","last_peered":"2024-09-17T05:03:19.487208+0000","last_clean":"2024-09-17T05:03:19.487208+0000","last_became_active":"2024-09-17T05:03:18.490369+0000","last_became_peered":"2024-09-17T05:03:18.490369+0000","last_unstale":"2024-09-17T05:03:19.487208+0000","last_undegraded":"2024-09-17T05:03:19.487208+0000","last_fullsized":"2024-09-17T05:03:19.487208+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.356295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795500+0000","last_change":"2024-09-17T05:03:18.502622+0000","last_active":"2024-09-17T05:03:21.795500+0000","last_peered":"2024-09-17T05:03:21.795500+0000","last_clean":"2024-09-17T05:03:21.795500+0000","last_became_active":"2024-09-17T05:03:18.502143+0000","last_became_peered":"2024-09-17T05:03:18.502143+0000","last_unstale":"2024-09-17T05:03:21.795500+0000","last_undegraded":"2024-09-17T05:03:21.795500+0000","last_fullsized":"2024-09-17T05:03:21.795500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:17.126958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414390+0000","last_change":"2024-09-17T05:03:18.499587+0000","last_active":"2024-09-17T05:03:20.414390+0000","last_peered":"2024-09-17T05:03:20.414390+0000","last_clean":"2024-09-17T05:03:20.414390+0000","last_became_active":"2024-09-17T05:03:18.499252+0000","last_became_peered":"2024-09-17T05:03:18.499252+0000","last_unstale":"2024-09-17T05:03:20.414390+0000","last_undegraded":"2024-09-17T05:03:20.414390+0000","last_fullsized":"2024-09-17T05:03:20.414390+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414386+0000","last_change":"2024-09-17T05:03:18.504641+0000","last_active":"2024-09-17T05:03:20.414386+0000","last_peered":"2024-09-17T05:03:20.414386+0000","last_clean":"2024-09-17T05:03:20.414386+0000","last_became_active":"2024-09-17T05:03:18.504311+0000","last_became_peered":"2024-09-17T05:03:18.504311+0000","last_unstale":"2024-09-17T05:03:20.414386+0000","last_undegraded":"2024-09-17T05:03:20.414386+0000","last_fullsized":"2024-09-17T05:03:20.414386+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:52.037853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795414+0000","last_change":"2024-09-17T05:03:18.499614+0000","last_active":"2024-09-17T05:03:21.795414+0000","last_peered":"2024-09-17T05:03:21.795414+0000","last_clean":"2024-09-17T05:03:21.795414+0000","last_became_active":"2024-09-17T05:03:18.498970+0000","last_became_peered":"2024-09-17T05:03:18.498970+0000","last_unstale":"2024-09-17T05:03:21.795414+0000","last_undegraded":"2024-09-17T05:03:21.795414+0000","last_fullsized":"2024-09-17T05:03:21.795414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:01.114758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484404+0000","last_change":"2024-09-17T05:03:18.486986+0000","last_active":"2024-09-17T05:03:19.484404+0000","last_peered":"2024-09-17T05:03:19.484404+0000","last_clean":"2024-09-17T05:03:19.484404+0000","last_became_active":"2024-09-17T05:03:18.486638+0000","last_became_peered":"2024-09-17T05:03:18.486638+0000","last_unstale":"2024-09-17T05:03:19.484404+0000","last_undegraded":"2024-09-17T05:03:19.484404+0000","last_fullsized":"2024-09-17T05:03:19.484404+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:22:04.033462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414318+0000","last_change":"2024-09-17T05:03:18.508226+0000","last_active":"2024-09-17T05:03:20.414318+0000","last_peered":"2024-09-17T05:03:20.414318+0000","last_clean":"2024-09-17T05:03:20.414318+0000","last_became_active":"2024-09-17T05:03:18.506993+0000","last_became_peered":"2024-09-17T05:03:18.506993+0000","last_unstale":"2024-09-17T05:03:20.414318+0000","last_undegraded":"2024-09-17T05:03:20.414318+0000","last_fullsized":"2024-09-17T05:03:20.414318+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:01.068407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796065+0000","last_change":"2024-09-17T05:03:18.501235+0000","last_active":"2024-09-17T05:03:21.796065+0000","last_peered":"2024-09-17T05:03:21.796065+0000","last_clean":"2024-09-17T05:03:21.796065+0000","last_became_active":"2024-09-17T05:03:18.500966+0000","last_became_peered":"2024-09-17T05:03:18.500966+0000","last_unstale":"2024-09-17T05:03:21.796065+0000","last_undegraded":"2024-09-17T05:03:21.796065+0000","last_fullsized":"2024-09-17T05:03:21.796065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:40.933886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485015+0000","last_change":"2024-09-17T05:03:18.499110+0000","last_active":"2024-09-17T05:03:19.485015+0000","last_peered":"2024-09-17T05:03:19.485015+0000","last_clean":"2024-09-17T05:03:19.485015+0000","last_became_active":"2024-09-17T05:03:18.498458+0000","last_became_peered":"2024-09-17T05:03:18.498458+0000","last_unstale":"2024-09-17T05:03:19.485015+0000","last_undegraded":"2024-09-17T05:03:19.485015+0000","last_fullsized":"2024-09-17T05:03:19.485015+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:19.040116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485907+0000","last_change":"2024-09-17T05:03:18.498320+0000","last_active":"2024-09-17T05:03:19.485907+0000","last_peered":"2024-09-17T05:03:19.485907+0000","last_clean":"2024-09-17T05:03:19.485907+0000","last_became_active":"2024-09-17T05:03:18.498041+0000","last_became_peered":"2024-09-17T05:03:18.498041+0000","last_unstale":"2024-09-17T05:03:19.485907+0000","last_undegraded":"2024-09-17T05:03:19.485907+0000","last_fullsized":"2024-09-17T05:03:19.485907+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:49.128082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796031+0000","last_change":"2024-09-17T05:03:18.509422+0000","last_active":"2024-09-17T05:03:21.796031+0000","last_peered":"2024-09-17T05:03:21.796031+0000","last_clean":"2024-09-17T05:03:21.796031+0000","last_became_active":"2024-09-17T05:03:18.506222+0000","last_became_peered":"2024-09-17T05:03:18.506222+0000","last_unstale":"2024-09-17T05:03:21.796031+0000","last_undegraded":"2024-09-17T05:03:21.796031+0000","last_fullsized":"2024-09-17T05:03:21.796031+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:01.114758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484565+0000","last_change":"2024-09-17T05:03:18.488562+0000","last_active":"2024-09-17T05:03:19.484565+0000","last_peered":"2024-09-17T05:03:19.484565+0000","last_clean":"2024-09-17T05:03:19.484565+0000","last_became_active":"2024-09-17T05:03:18.488267+0000","last_became_peered":"2024-09-17T05:03:18.488267+0000","last_unstale":"2024-09-17T05:03:19.484565+0000","last_undegraded":"2024-09-17T05:03:19.484565+0000","last_fullsized":"2024-09-17T05:03:19.484565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485314+0000","last_change":"2024-09-17T05:03:18.491046+0000","last_active":"2024-09-17T05:03:19.485314+0000","last_peered":"2024-09-17T05:03:19.485314+0000","last_clean":"2024-09-17T05:03:19.485314+0000","last_became_active":"2024-09-17T05:03:18.490734+0000","last_became_peered":"2024-09-17T05:03:18.490734+0000","last_unstale":"2024-09-17T05:03:19.485314+0000","last_undegraded":"2024-09-17T05:03:19.485314+0000","last_fullsized":"2024-09-17T05:03:19.485314+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:49.725834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485069+0000","last_change":"2024-09-17T05:03:18.495503+0000","last_active":"2024-09-17T05:03:19.485069+0000","last_peered":"2024-09-17T05:03:19.485069+0000","last_clean":"2024-09-17T05:03:19.485069+0000","last_became_active":"2024-09-17T05:03:18.494911+0000","last_became_peered":"2024-09-17T05:03:18.494911+0000","last_unstale":"2024-09-17T05:03:19.485069+0000","last_undegraded":"2024-09-17T05:03:19.485069+0000","last_fullsized":"2024-09-17T05:03:19.485069+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:02.757954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485067+0000","last_change":"2024-09-17T05:03:18.492326+0000","last_active":"2024-09-17T05:03:19.485067+0000","last_peered":"2024-09-17T05:03:19.485067+0000","last_clean":"2024-09-17T05:03:19.485067+0000","last_became_active":"2024-09-17T05:03:18.492003+0000","last_became_peered":"2024-09-17T05:03:18.492003+0000","last_unstale":"2024-09-17T05:03:19.485067+0000","last_undegraded":"2024-09-17T05:03:19.485067+0000","last_fullsized":"2024-09-17T05:03:19.485067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:47.441012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795905+0000","last_change":"2024-09-17T05:03:18.493346+0000","last_active":"2024-09-17T05:03:21.795905+0000","last_peered":"2024-09-17T05:03:21.795905+0000","last_clean":"2024-09-17T05:03:21.795905+0000","last_became_active":"2024-09-17T05:03:18.490847+0000","last_became_peered":"2024-09-17T05:03:18.490847+0000","last_unstale":"2024-09-17T05:03:21.795905+0000","last_undegraded":"2024-09-17T05:03:21.795905+0000","last_fullsized":"2024-09-17T05:03:21.795905+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:23.594082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485645+0000","last_change":"2024-09-17T05:03:18.492283+0000","last_active":"2024-09-17T05:03:19.485645+0000","last_peered":"2024-09-17T05:03:19.485645+0000","last_clean":"2024-09-17T05:03:19.485645+0000","last_became_active":"2024-09-17T05:03:18.491925+0000","last_became_peered":"2024-09-17T05:03:18.491925+0000","last_unstale":"2024-09-17T05:03:19.485645+0000","last_undegraded":"2024-09-17T05:03:19.485645+0000","last_fullsized":"2024-09-17T05:03:19.485645+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:36.834342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414831+0000","last_change":"2024-09-17T05:03:18.509121+0000","last_active":"2024-09-17T05:03:20.414831+0000","last_peered":"2024-09-17T05:03:20.414831+0000","last_clean":"2024-09-17T05:03:20.414831+0000","last_became_active":"2024-09-17T05:03:18.508808+0000","last_became_peered":"2024-09-17T05:03:18.508808+0000","last_unstale":"2024-09-17T05:03:20.414831+0000","last_undegraded":"2024-09-17T05:03:20.414831+0000","last_fullsized":"2024-09-17T05:03:20.414831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:19.040116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796106+0000","last_change":"2024-09-17T05:03:18.497050+0000","last_active":"2024-09-17T05:03:21.796106+0000","last_peered":"2024-09-17T05:03:21.796106+0000","last_clean":"2024-09-17T05:03:21.796106+0000","last_became_active":"2024-09-17T05:03:18.496820+0000","last_became_peered":"2024-09-17T05:03:18.496820+0000","last_unstale":"2024-09-17T05:03:21.796106+0000","last_undegraded":"2024-09-17T05:03:21.796106+0000","last_fullsized":"2024-09-17T05:03:21.796106+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:17.126958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795843+0000","last_change":"2024-09-17T05:03:18.500297+0000","last_active":"2024-09-17T05:03:21.795843+0000","last_peered":"2024-09-17T05:03:21.795843+0000","last_clean":"2024-09-17T05:03:21.795843+0000","last_became_active":"2024-09-17T05:03:18.499968+0000","last_became_peered":"2024-09-17T05:03:18.499968+0000","last_unstale":"2024-09-17T05:03:21.795843+0000","last_undegraded":"2024-09-17T05:03:21.795843+0000","last_fullsized":"2024-09-17T05:03:21.795843+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:36.618926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796625+0000","last_change":"2024-09-17T05:03:18.484183+0000","last_active":"2024-09-17T05:03:21.796625+0000","last_peered":"2024-09-17T05:03:21.796625+0000","last_clean":"2024-09-17T05:03:21.796625+0000","last_became_active":"2024-09-17T05:03:18.483876+0000","last_became_peered":"2024-09-17T05:03:18.483876+0000","last_unstale":"2024-09-17T05:03:21.796625+0000","last_undegraded":"2024-09-17T05:03:21.796625+0000","last_fullsized":"2024-09-17T05:03:21.796625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:49.128082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485437+0000","last_change":"2024-09-17T05:03:18.495213+0000","last_active":"2024-09-17T05:03:19.485437+0000","last_peered":"2024-09-17T05:03:19.485437+0000","last_clean":"2024-09-17T05:03:19.485437+0000","last_became_active":"2024-09-17T05:03:18.494507+0000","last_became_peered":"2024-09-17T05:03:18.494507+0000","last_unstale":"2024-09-17T05:03:19.485437+0000","last_undegraded":"2024-09-17T05:03:19.485437+0000","last_fullsized":"2024-09-17T05:03:19.485437+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:27.674507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795374+0000","last_change":"2024-09-17T05:03:18.491751+0000","last_active":"2024-09-17T05:03:21.795374+0000","last_peered":"2024-09-17T05:03:21.795374+0000","last_clean":"2024-09-17T05:03:21.795374+0000","last_became_active":"2024-09-17T05:03:18.491440+0000","last_became_peered":"2024-09-17T05:03:18.491440+0000","last_unstale":"2024-09-17T05:03:21.795374+0000","last_undegraded":"2024-09-17T05:03:21.795374+0000","last_fullsized":"2024-09-17T05:03:21.795374+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:51:02.045773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796105+0000","last_change":"2024-09-17T05:03:18.500083+0000","last_active":"2024-09-17T05:03:21.796105+0000","last_peered":"2024-09-17T05:03:21.796105+0000","last_clean":"2024-09-17T05:03:21.796105+0000","last_became_active":"2024-09-17T05:03:18.499799+0000","last_became_peered":"2024-09-17T05:03:18.499799+0000","last_unstale":"2024-09-17T05:03:21.796105+0000","last_undegraded":"2024-09-17T05:03:21.796105+0000","last_fullsized":"2024-09-17T05:03:21.796105+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:36.834342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.495433+0000","last_change":"2024-09-17T05:03:17.487458+0000","last_active":"2024-09-17T05:03:19.495433+0000","last_peered":"2024-09-17T05:03:19.495433+0000","last_clean":"2024-09-17T05:03:19.495433+0000","last_became_active":"2024-09-17T05:03:17.487174+0000","last_became_peered":"2024-09-17T05:03:17.487174+0000","last_unstale":"2024-09-17T05:03:19.495433+0000","last_undegraded":"2024-09-17T05:03:19.495433+0000","last_fullsized":"2024-09-17T05:03:19.495433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:00.060756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796394+0000","last_change":"2024-09-17T05:03:18.497880+0000","last_active":"2024-09-17T05:03:21.796394+0000","last_peered":"2024-09-17T05:03:21.796394+0000","last_clean":"2024-09-17T05:03:21.796394+0000","last_became_active":"2024-09-17T05:03:18.497468+0000","last_became_peered":"2024-09-17T05:03:18.497468+0000","last_unstale":"2024-09-17T05:03:21.796394+0000","last_undegraded":"2024-09-17T05:03:21.796394+0000","last_fullsized":"2024-09-17T05:03:21.796394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:03.135305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414697+0000","last_change":"2024-09-17T05:03:17.489555+0000","last_active":"2024-09-17T05:03:20.414697+0000","last_peered":"2024-09-17T05:03:20.414697+0000","last_clean":"2024-09-17T05:03:20.414697+0000","last_became_active":"2024-09-17T05:03:17.489182+0000","last_became_peered":"2024-09-17T05:03:17.489182+0000","last_unstale":"2024-09-17T05:03:20.414697+0000","last_undegraded":"2024-09-17T05:03:20.414697+0000","last_fullsized":"2024-09-17T05:03:20.414697+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.412025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796408+0000","last_change":"2024-09-17T05:03:18.500202+0000","last_active":"2024-09-17T05:03:21.796408+0000","last_peered":"2024-09-17T05:03:21.796408+0000","last_clean":"2024-09-17T05:03:21.796408+0000","last_became_active":"2024-09-17T05:03:18.499893+0000","last_became_peered":"2024-09-17T05:03:18.499893+0000","last_unstale":"2024-09-17T05:03:21.796408+0000","last_undegraded":"2024-09-17T05:03:21.796408+0000","last_fullsized":"2024-09-17T05:03:21.796408+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:03.396864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487289+0000","last_change":"2024-09-17T05:03:17.473668+0000","last_active":"2024-09-17T05:03:19.487289+0000","last_peered":"2024-09-17T05:03:19.487289+0000","last_clean":"2024-09-17T05:03:19.487289+0000","last_became_active":"2024-09-17T05:03:17.473283+0000","last_became_peered":"2024-09-17T05:03:17.473283+0000","last_unstale":"2024-09-17T05:03:19.487289+0000","last_undegraded":"2024-09-17T05:03:19.487289+0000","last_fullsized":"2024-09-17T05:03:19.487289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:34.241801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487050+0000","last_change":"2024-09-17T05:03:18.495083+0000","last_active":"2024-09-17T05:03:19.487050+0000","last_peered":"2024-09-17T05:03:19.487050+0000","last_clean":"2024-09-17T05:03:19.487050+0000","last_became_active":"2024-09-17T05:03:18.494736+0000","last_became_peered":"2024-09-17T05:03:18.494736+0000","last_unstale":"2024-09-17T05:03:19.487050+0000","last_undegraded":"2024-09-17T05:03:19.487050+0000","last_fullsized":"2024-09-17T05:03:19.487050+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:58:24.301106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796281+0000","last_change":"2024-09-17T05:03:17.484272+0000","last_active":"2024-09-17T05:03:21.796281+0000","last_peered":"2024-09-17T05:03:21.796281+0000","last_clean":"2024-09-17T05:03:21.796281+0000","last_became_active":"2024-09-17T05:03:17.483583+0000","last_became_peered":"2024-09-17T05:03:17.483583+0000","last_unstale":"2024-09-17T05:03:21.796281+0000","last_undegraded":"2024-09-17T05:03:21.796281+0000","last_fullsized":"2024-09-17T05:03:21.796281+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:53.765242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.495255+0000","last_change":"2024-09-17T05:03:18.514276+0000","last_active":"2024-09-17T05:03:19.495255+0000","last_peered":"2024-09-17T05:03:19.495255+0000","last_clean":"2024-09-17T05:03:19.495255+0000","last_became_active":"2024-09-17T05:03:18.513909+0000","last_became_peered":"2024-09-17T05:03:18.513909+0000","last_unstale":"2024-09-17T05:03:19.495255+0000","last_undegraded":"2024-09-17T05:03:19.495255+0000","last_fullsized":"2024-09-17T05:03:19.495255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485558+0000","last_change":"2024-09-17T05:03:17.489922+0000","last_active":"2024-09-17T05:03:19.485558+0000","last_peered":"2024-09-17T05:03:19.485558+0000","last_clean":"2024-09-17T05:03:19.485558+0000","last_became_active":"2024-09-17T05:03:17.489611+0000","last_became_peered":"2024-09-17T05:03:17.489611+0000","last_unstale":"2024-09-17T05:03:19.485558+0000","last_undegraded":"2024-09-17T05:03:19.485558+0000","last_fullsized":"2024-09-17T05:03:19.485558+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:35.801215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796855+0000","last_change":"2024-09-17T05:03:18.485055+0000","last_active":"2024-09-17T05:03:21.796855+0000","last_peered":"2024-09-17T05:03:21.796855+0000","last_clean":"2024-09-17T05:03:21.796855+0000","last_became_active":"2024-09-17T05:03:18.484744+0000","last_became_peered":"2024-09-17T05:03:18.484744+0000","last_unstale":"2024-09-17T05:03:21.796855+0000","last_undegraded":"2024-09-17T05:03:21.796855+0000","last_fullsized":"2024-09-17T05:03:21.796855+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:14.318804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796609+0000","last_change":"2024-09-17T05:03:17.477682+0000","last_active":"2024-09-17T05:03:21.796609+0000","last_peered":"2024-09-17T05:03:21.796609+0000","last_clean":"2024-09-17T05:03:21.796609+0000","last_became_active":"2024-09-17T05:03:17.477257+0000","last_became_peered":"2024-09-17T05:03:17.477257+0000","last_unstale":"2024-09-17T05:03:21.796609+0000","last_undegraded":"2024-09-17T05:03:21.796609+0000","last_fullsized":"2024-09-17T05:03:21.796609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:29.176860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487353+0000","last_change":"2024-09-17T05:03:18.504857+0000","last_active":"2024-09-17T05:03:19.487353+0000","last_peered":"2024-09-17T05:03:19.487353+0000","last_clean":"2024-09-17T05:03:19.487353+0000","last_became_active":"2024-09-17T05:03:18.504185+0000","last_became_peered":"2024-09-17T05:03:18.504185+0000","last_unstale":"2024-09-17T05:03:19.487353+0000","last_undegraded":"2024-09-17T05:03:19.487353+0000","last_fullsized":"2024-09-17T05:03:19.487353+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:51.971141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796357+0000","last_change":"2024-09-17T05:03:17.488461+0000","last_active":"2024-09-17T05:03:21.796357+0000","last_peered":"2024-09-17T05:03:21.796357+0000","last_clean":"2024-09-17T05:03:21.796357+0000","last_became_active":"2024-09-17T05:03:17.488076+0000","last_became_peered":"2024-09-17T05:03:17.488076+0000","last_unstale":"2024-09-17T05:03:21.796357+0000","last_undegraded":"2024-09-17T05:03:21.796357+0000","last_fullsized":"2024-09-17T05:03:21.796357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:41.181926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485103+0000","last_change":"2024-09-17T05:03:18.494837+0000","last_active":"2024-09-17T05:03:19.485103+0000","last_peered":"2024-09-17T05:03:19.485103+0000","last_clean":"2024-09-17T05:03:19.485103+0000","last_became_active":"2024-09-17T05:03:18.494265+0000","last_became_peered":"2024-09-17T05:03:18.494265+0000","last_unstale":"2024-09-17T05:03:19.485103+0000","last_undegraded":"2024-09-17T05:03:19.485103+0000","last_fullsized":"2024-09-17T05:03:19.485103+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.380215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484969+0000","last_change":"2024-09-17T05:03:17.476199+0000","last_active":"2024-09-17T05:03:19.484969+0000","last_peered":"2024-09-17T05:03:19.484969+0000","last_clean":"2024-09-17T05:03:19.484969+0000","last_became_active":"2024-09-17T05:03:17.475834+0000","last_became_peered":"2024-09-17T05:03:17.475834+0000","last_unstale":"2024-09-17T05:03:19.484969+0000","last_undegraded":"2024-09-17T05:03:19.484969+0000","last_fullsized":"2024-09-17T05:03:19.484969+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:21.694015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494962+0000","last_change":"2024-09-17T05:03:18.495879+0000","last_active":"2024-09-17T05:03:19.494962+0000","last_peered":"2024-09-17T05:03:19.494962+0000","last_clean":"2024-09-17T05:03:19.494962+0000","last_became_active":"2024-09-17T05:03:18.495553+0000","last_became_peered":"2024-09-17T05:03:18.495553+0000","last_unstale":"2024-09-17T05:03:19.494962+0000","last_undegraded":"2024-09-17T05:03:19.494962+0000","last_fullsized":"2024-09-17T05:03:19.494962+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:29.809649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485216+0000","last_change":"2024-09-17T05:03:17.484357+0000","last_active":"2024-09-17T05:03:19.485216+0000","last_peered":"2024-09-17T05:03:19.485216+0000","last_clean":"2024-09-17T05:03:19.485216+0000","last_became_active":"2024-09-17T05:03:17.483648+0000","last_became_peered":"2024-09-17T05:03:17.483648+0000","last_unstale":"2024-09-17T05:03:19.485216+0000","last_undegraded":"2024-09-17T05:03:19.485216+0000","last_fullsized":"2024-09-17T05:03:19.485216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484412+0000","last_change":"2024-09-17T05:03:18.496015+0000","last_active":"2024-09-17T05:03:19.484412+0000","last_peered":"2024-09-17T05:03:19.484412+0000","last_clean":"2024-09-17T05:03:19.484412+0000","last_became_active":"2024-09-17T05:03:18.495714+0000","last_became_peered":"2024-09-17T05:03:18.495714+0000","last_unstale":"2024-09-17T05:03:19.484412+0000","last_undegraded":"2024-09-17T05:03:19.484412+0000","last_fullsized":"2024-09-17T05:03:19.484412+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.412751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485074+0000","last_change":"2024-09-17T05:03:17.483837+0000","last_active":"2024-09-17T05:03:19.485074+0000","last_peered":"2024-09-17T05:03:19.485074+0000","last_clean":"2024-09-17T05:03:19.485074+0000","last_became_active":"2024-09-17T05:03:17.483474+0000","last_became_peered":"2024-09-17T05:03:17.483474+0000","last_unstale":"2024-09-17T05:03:19.485074+0000","last_undegraded":"2024-09-17T05:03:19.485074+0000","last_fullsized":"2024-09-17T05:03:19.485074+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.383407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795609+0000","last_change":"2024-09-17T05:03:18.513177+0000","last_active":"2024-09-17T05:03:21.795609+0000","last_peered":"2024-09-17T05:03:21.795609+0000","last_clean":"2024-09-17T05:03:21.795609+0000","last_became_active":"2024-09-17T05:03:18.512727+0000","last_became_peered":"2024-09-17T05:03:18.512727+0000","last_unstale":"2024-09-17T05:03:21.795609+0000","last_undegraded":"2024-09-17T05:03:21.795609+0000","last_fullsized":"2024-09-17T05:03:21.795609+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:55.305773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796884+0000","last_change":"2024-09-17T05:03:17.471492+0000","last_active":"2024-09-17T05:03:21.796884+0000","last_peered":"2024-09-17T05:03:21.796884+0000","last_clean":"2024-09-17T05:03:21.796884+0000","last_became_active":"2024-09-17T05:03:17.471119+0000","last_became_peered":"2024-09-17T05:03:17.471119+0000","last_unstale":"2024-09-17T05:03:21.796884+0000","last_undegraded":"2024-09-17T05:03:21.796884+0000","last_fullsized":"2024-09-17T05:03:21.796884+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:12.694951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795808+0000","last_change":"2024-09-17T05:03:18.495599+0000","last_active":"2024-09-17T05:03:21.795808+0000","last_peered":"2024-09-17T05:03:21.795808+0000","last_clean":"2024-09-17T05:03:21.795808+0000","last_became_active":"2024-09-17T05:03:18.495038+0000","last_became_peered":"2024-09-17T05:03:18.495038+0000","last_unstale":"2024-09-17T05:03:21.795808+0000","last_undegraded":"2024-09-17T05:03:21.795808+0000","last_fullsized":"2024-09-17T05:03:21.795808+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:26.057553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":21,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487172+0000","last_change":"2024-09-17T05:03:17.480393+0000","last_active":"2024-09-17T05:03:19.487172+0000","last_peered":"2024-09-17T05:03:19.487172+0000","last_clean":"2024-09-17T05:03:19.487172+0000","last_became_active":"2024-09-17T05:03:17.479912+0000","last_became_peered":"2024-09-17T05:03:17.479912+0000","last_unstale":"2024-09-17T05:03:19.487172+0000","last_undegraded":"2024-09-17T05:03:19.487172+0000","last_fullsized":"2024-09-17T05:03:19.487172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:06.645919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796839+0000","last_change":"2024-09-17T05:03:18.504212+0000","last_active":"2024-09-17T05:03:21.796839+0000","last_peered":"2024-09-17T05:03:21.796839+0000","last_clean":"2024-09-17T05:03:21.796839+0000","last_became_active":"2024-09-17T05:03:18.503841+0000","last_became_peered":"2024-09-17T05:03:18.503841+0000","last_unstale":"2024-09-17T05:03:21.796839+0000","last_undegraded":"2024-09-17T05:03:21.796839+0000","last_fullsized":"2024-09-17T05:03:21.796839+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:51.416667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414644+0000","last_change":"2024-09-17T05:03:17.482800+0000","last_active":"2024-09-17T05:03:20.414644+0000","last_peered":"2024-09-17T05:03:20.414644+0000","last_clean":"2024-09-17T05:03:20.414644+0000","last_became_active":"2024-09-17T05:03:17.482376+0000","last_became_peered":"2024-09-17T05:03:17.482376+0000","last_unstale":"2024-09-17T05:03:20.414644+0000","last_undegraded":"2024-09-17T05:03:20.414644+0000","last_fullsized":"2024-09-17T05:03:20.414644+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:06.028991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795582+0000","last_change":"2024-09-17T05:03:18.506127+0000","last_active":"2024-09-17T05:03:21.795582+0000","last_peered":"2024-09-17T05:03:21.795582+0000","last_clean":"2024-09-17T05:03:21.795582+0000","last_became_active":"2024-09-17T05:03:18.505295+0000","last_became_peered":"2024-09-17T05:03:18.505295+0000","last_unstale":"2024-09-17T05:03:21.795582+0000","last_undegraded":"2024-09-17T05:03:21.795582+0000","last_fullsized":"2024-09-17T05:03:21.795582+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:50.676398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484488+0000","last_change":"2024-09-17T05:03:17.477301+0000","last_active":"2024-09-17T05:03:19.484488+0000","last_peered":"2024-09-17T05:03:19.484488+0000","last_clean":"2024-09-17T05:03:19.484488+0000","last_became_active":"2024-09-17T05:03:17.476888+0000","last_became_peered":"2024-09-17T05:03:17.476888+0000","last_unstale":"2024-09-17T05:03:19.484488+0000","last_undegraded":"2024-09-17T05:03:19.484488+0000","last_fullsized":"2024-09-17T05:03:19.484488+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:58.301917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414868+0000","last_change":"2024-09-17T05:03:18.507972+0000","last_active":"2024-09-17T05:03:20.414868+0000","last_peered":"2024-09-17T05:03:20.414868+0000","last_clean":"2024-09-17T05:03:20.414868+0000","last_became_active":"2024-09-17T05:03:18.507423+0000","last_became_peered":"2024-09-17T05:03:18.507423+0000","last_unstale":"2024-09-17T05:03:20.414868+0000","last_undegraded":"2024-09-17T05:03:20.414868+0000","last_fullsized":"2024-09-17T05:03:20.414868+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:03.380215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795437+0000","last_change":"2024-09-17T05:03:17.481847+0000","last_active":"2024-09-17T05:03:21.795437+0000","last_peered":"2024-09-17T05:03:21.795437+0000","last_clean":"2024-09-17T05:03:21.795437+0000","last_became_active":"2024-09-17T05:03:17.481546+0000","last_became_peered":"2024-09-17T05:03:17.481546+0000","last_unstale":"2024-09-17T05:03:21.795437+0000","last_undegraded":"2024-09-17T05:03:21.795437+0000","last_fullsized":"2024-09-17T05:03:21.795437+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.383407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485912+0000","last_change":"2024-09-17T05:03:18.484589+0000","last_active":"2024-09-17T05:03:19.485912+0000","last_peered":"2024-09-17T05:03:19.485912+0000","last_clean":"2024-09-17T05:03:19.485912+0000","last_became_active":"2024-09-17T05:03:18.483862+0000","last_became_peered":"2024-09-17T05:03:18.483862+0000","last_unstale":"2024-09-17T05:03:19.485912+0000","last_undegraded":"2024-09-17T05:03:19.485912+0000","last_fullsized":"2024-09-17T05:03:19.485912+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.509403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":23,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414651+0000","last_change":"2024-09-17T05:03:17.489475+0000","last_active":"2024-09-17T05:03:20.414651+0000","last_peered":"2024-09-17T05:03:20.414651+0000","last_clean":"2024-09-17T05:03:20.414651+0000","last_became_active":"2024-09-17T05:03:17.489135+0000","last_became_peered":"2024-09-17T05:03:17.489135+0000","last_unstale":"2024-09-17T05:03:20.414651+0000","last_undegraded":"2024-09-17T05:03:20.414651+0000","last_fullsized":"2024-09-17T05:03:20.414651+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:41:35.734369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486537+0000","last_change":"2024-09-17T05:03:18.487882+0000","last_active":"2024-09-17T05:03:19.486537+0000","last_peered":"2024-09-17T05:03:19.486537+0000","last_clean":"2024-09-17T05:03:19.486537+0000","last_became_active":"2024-09-17T05:03:18.487523+0000","last_became_peered":"2024-09-17T05:03:18.487523+0000","last_unstale":"2024-09-17T05:03:19.486537+0000","last_undegraded":"2024-09-17T05:03:19.486537+0000","last_fullsized":"2024-09-17T05:03:19.486537+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:44.737783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414979+0000","last_change":"2024-09-17T05:03:17.489385+0000","last_active":"2024-09-17T05:03:20.414979+0000","last_peered":"2024-09-17T05:03:20.414979+0000","last_clean":"2024-09-17T05:03:20.414979+0000","last_became_active":"2024-09-17T05:03:17.489069+0000","last_became_peered":"2024-09-17T05:03:17.489069+0000","last_unstale":"2024-09-17T05:03:20.414979+0000","last_undegraded":"2024-09-17T05:03:20.414979+0000","last_fullsized":"2024-09-17T05:03:20.414979+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:52:33.379573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795898+0000","last_change":"2024-09-17T05:03:18.501550+0000","last_active":"2024-09-17T05:03:21.795898+0000","last_peered":"2024-09-17T05:03:21.795898+0000","last_clean":"2024-09-17T05:03:21.795898+0000","last_became_active":"2024-09-17T05:03:18.501150+0000","last_became_peered":"2024-09-17T05:03:18.501150+0000","last_unstale":"2024-09-17T05:03:21.795898+0000","last_undegraded":"2024-09-17T05:03:21.795898+0000","last_fullsized":"2024-09-17T05:03:21.795898+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:30.983417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414870+0000","last_change":"2024-09-17T05:03:17.483869+0000","last_active":"2024-09-17T05:03:20.414870+0000","last_peered":"2024-09-17T05:03:20.414870+0000","last_clean":"2024-09-17T05:03:20.414870+0000","last_became_active":"2024-09-17T05:03:17.483565+0000","last_became_peered":"2024-09-17T05:03:17.483565+0000","last_unstale":"2024-09-17T05:03:20.414870+0000","last_undegraded":"2024-09-17T05:03:20.414870+0000","last_fullsized":"2024-09-17T05:03:20.414870+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:48.075582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795516+0000","last_change":"2024-09-17T05:03:18.489876+0000","last_active":"2024-09-17T05:03:21.795516+0000","last_peered":"2024-09-17T05:03:21.795516+0000","last_clean":"2024-09-17T05:03:21.795516+0000","last_became_active":"2024-09-17T05:03:18.488017+0000","last_became_peered":"2024-09-17T05:03:18.488017+0000","last_unstale":"2024-09-17T05:03:21.795516+0000","last_undegraded":"2024-09-17T05:03:21.795516+0000","last_fullsized":"2024-09-17T05:03:21.795516+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:13.753545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484788+0000","last_change":"2024-09-17T05:03:17.484821+0000","last_active":"2024-09-17T05:03:19.484788+0000","last_peered":"2024-09-17T05:03:19.484788+0000","last_clean":"2024-09-17T05:03:19.484788+0000","last_became_active":"2024-09-17T05:03:17.484445+0000","last_became_peered":"2024-09-17T05:03:17.484445+0000","last_unstale":"2024-09-17T05:03:19.484788+0000","last_undegraded":"2024-09-17T05:03:19.484788+0000","last_fullsized":"2024-09-17T05:03:19.484788+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:44.206475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414381+0000","last_change":"2024-09-17T05:03:18.508867+0000","last_active":"2024-09-17T05:03:20.414381+0000","last_peered":"2024-09-17T05:03:20.414381+0000","last_clean":"2024-09-17T05:03:20.414381+0000","last_became_active":"2024-09-17T05:03:18.508319+0000","last_became_peered":"2024-09-17T05:03:18.508319+0000","last_unstale":"2024-09-17T05:03:20.414381+0000","last_undegraded":"2024-09-17T05:03:20.414381+0000","last_fullsized":"2024-09-17T05:03:20.414381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:01.699937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796400+0000","last_change":"2024-09-17T05:03:17.469441+0000","last_active":"2024-09-17T05:03:21.796400+0000","last_peered":"2024-09-17T05:03:21.796400+0000","last_clean":"2024-09-17T05:03:21.796400+0000","last_became_active":"2024-09-17T05:03:17.469019+0000","last_became_peered":"2024-09-17T05:03:17.469019+0000","last_unstale":"2024-09-17T05:03:21.796400+0000","last_undegraded":"2024-09-17T05:03:21.796400+0000","last_fullsized":"2024-09-17T05:03:21.796400+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:02.998746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796262+0000","last_change":"2024-09-17T05:03:18.489610+0000","last_active":"2024-09-17T05:03:21.796262+0000","last_peered":"2024-09-17T05:03:21.796262+0000","last_clean":"2024-09-17T05:03:21.796262+0000","last_became_active":"2024-09-17T05:03:18.489271+0000","last_became_peered":"2024-09-17T05:03:18.489271+0000","last_unstale":"2024-09-17T05:03:21.796262+0000","last_undegraded":"2024-09-17T05:03:21.796262+0000","last_fullsized":"2024-09-17T05:03:21.796262+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:50.676398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795797+0000","last_change":"2024-09-17T05:03:17.488710+0000","last_active":"2024-09-17T05:03:21.795797+0000","last_peered":"2024-09-17T05:03:21.795797+0000","last_clean":"2024-09-17T05:03:21.795797+0000","last_became_active":"2024-09-17T05:03:17.488178+0000","last_became_peered":"2024-09-17T05:03:17.488178+0000","last_unstale":"2024-09-17T05:03:21.795797+0000","last_undegraded":"2024-09-17T05:03:21.795797+0000","last_fullsized":"2024-09-17T05:03:21.795797+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:14:44.044047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484785+0000","last_change":"2024-09-17T05:03:18.484052+0000","last_active":"2024-09-17T05:03:19.484785+0000","last_peered":"2024-09-17T05:03:19.484785+0000","last_clean":"2024-09-17T05:03:19.484785+0000","last_became_active":"2024-09-17T05:03:18.483686+0000","last_became_peered":"2024-09-17T05:03:18.483686+0000","last_unstale":"2024-09-17T05:03:19.484785+0000","last_undegraded":"2024-09-17T05:03:19.484785+0000","last_fullsized":"2024-09-17T05:03:19.484785+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485517+0000","last_change":"2024-09-17T05:03:17.487381+0000","last_active":"2024-09-17T05:03:19.485517+0000","last_peered":"2024-09-17T05:03:19.485517+0000","last_clean":"2024-09-17T05:03:19.485517+0000","last_became_active":"2024-09-17T05:03:17.487025+0000","last_became_peered":"2024-09-17T05:03:17.487025+0000","last_unstale":"2024-09-17T05:03:19.485517+0000","last_undegraded":"2024-09-17T05:03:19.485517+0000","last_fullsized":"2024-09-17T05:03:19.485517+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:38.335990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795664+0000","last_change":"2024-09-17T05:03:18.513093+0000","last_active":"2024-09-17T05:03:21.795664+0000","last_peered":"2024-09-17T05:03:21.795664+0000","last_clean":"2024-09-17T05:03:21.795664+0000","last_became_active":"2024-09-17T05:03:18.512779+0000","last_became_peered":"2024-09-17T05:03:18.512779+0000","last_unstale":"2024-09-17T05:03:21.795664+0000","last_undegraded":"2024-09-17T05:03:21.795664+0000","last_fullsized":"2024-09-17T05:03:21.795664+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:07.251450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486706+0000","last_change":"2024-09-17T05:03:17.488730+0000","last_active":"2024-09-17T05:03:19.486706+0000","last_peered":"2024-09-17T05:03:19.486706+0000","last_clean":"2024-09-17T05:03:19.486706+0000","last_became_active":"2024-09-17T05:03:17.487850+0000","last_became_peered":"2024-09-17T05:03:17.487850+0000","last_unstale":"2024-09-17T05:03:19.486706+0000","last_undegraded":"2024-09-17T05:03:19.486706+0000","last_fullsized":"2024-09-17T05:03:19.486706+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:20.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414954+0000","last_change":"2024-09-17T05:03:18.508348+0000","last_active":"2024-09-17T05:03:20.414954+0000","last_peered":"2024-09-17T05:03:20.414954+0000","last_clean":"2024-09-17T05:03:20.414954+0000","last_became_active":"2024-09-17T05:03:18.507958+0000","last_became_peered":"2024-09-17T05:03:18.507958+0000","last_unstale":"2024-09-17T05:03:20.414954+0000","last_undegraded":"2024-09-17T05:03:20.414954+0000","last_fullsized":"2024-09-17T05:03:20.414954+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:41.130167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.486411+0000","last_change":"2024-09-17T05:03:17.477112+0000","last_active":"2024-09-17T05:03:19.486411+0000","last_peered":"2024-09-17T05:03:19.486411+0000","last_clean":"2024-09-17T05:03:19.486411+0000","last_became_active":"2024-09-17T05:03:17.476642+0000","last_became_peered":"2024-09-17T05:03:17.476642+0000","last_unstale":"2024-09-17T05:03:19.486411+0000","last_undegraded":"2024-09-17T05:03:19.486411+0000","last_fullsized":"2024-09-17T05:03:19.486411+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:50.045021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484227+0000","last_change":"2024-09-17T05:03:18.493987+0000","last_active":"2024-09-17T05:03:19.484227+0000","last_peered":"2024-09-17T05:03:19.484227+0000","last_clean":"2024-09-17T05:03:19.484227+0000","last_became_active":"2024-09-17T05:03:18.492993+0000","last_became_peered":"2024-09-17T05:03:18.492993+0000","last_unstale":"2024-09-17T05:03:19.484227+0000","last_undegraded":"2024-09-17T05:03:19.484227+0000","last_fullsized":"2024-09-17T05:03:19.484227+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:31:01.699937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.796817+0000","last_change":"2024-09-17T05:03:17.488680+0000","last_active":"2024-09-17T05:03:21.796817+0000","last_peered":"2024-09-17T05:03:21.796817+0000","last_clean":"2024-09-17T05:03:21.796817+0000","last_became_active":"2024-09-17T05:03:17.488050+0000","last_became_peered":"2024-09-17T05:03:17.488050+0000","last_unstale":"2024-09-17T05:03:21.796817+0000","last_undegraded":"2024-09-17T05:03:21.796817+0000","last_fullsized":"2024-09-17T05:03:21.796817+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:01:44.257805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487087+0000","last_change":"2024-09-17T05:03:18.495602+0000","last_active":"2024-09-17T05:03:19.487087+0000","last_peered":"2024-09-17T05:03:19.487087+0000","last_clean":"2024-09-17T05:03:19.487087+0000","last_became_active":"2024-09-17T05:03:18.495190+0000","last_became_peered":"2024-09-17T05:03:18.495190+0000","last_unstale":"2024-09-17T05:03:19.487087+0000","last_undegraded":"2024-09-17T05:03:19.487087+0000","last_fullsized":"2024-09-17T05:03:19.487087+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:49.184314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.797002+0000","last_change":"2024-09-17T05:03:17.471433+0000","last_active":"2024-09-17T05:03:21.797002+0000","last_peered":"2024-09-17T05:03:21.797002+0000","last_clean":"2024-09-17T05:03:21.797002+0000","last_became_active":"2024-09-17T05:03:17.470676+0000","last_became_peered":"2024-09-17T05:03:17.470676+0000","last_unstale":"2024-09-17T05:03:21.797002+0000","last_undegraded":"2024-09-17T05:03:21.797002+0000","last_fullsized":"2024-09-17T05:03:21.797002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:35.470632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485340+0000","last_change":"2024-09-17T05:03:18.499092+0000","last_active":"2024-09-17T05:03:19.485340+0000","last_peered":"2024-09-17T05:03:19.485340+0000","last_clean":"2024-09-17T05:03:19.485340+0000","last_became_active":"2024-09-17T05:03:18.498784+0000","last_became_peered":"2024-09-17T05:03:18.498784+0000","last_unstale":"2024-09-17T05:03:19.485340+0000","last_undegraded":"2024-09-17T05:03:19.485340+0000","last_fullsized":"2024-09-17T05:03:19.485340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:35.871301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485538+0000","last_change":"2024-09-17T05:03:17.481418+0000","last_active":"2024-09-17T05:03:19.485538+0000","last_peered":"2024-09-17T05:03:19.485538+0000","last_clean":"2024-09-17T05:03:19.485538+0000","last_became_active":"2024-09-17T05:03:17.481090+0000","last_became_peered":"2024-09-17T05:03:17.481090+0000","last_unstale":"2024-09-17T05:03:19.485538+0000","last_undegraded":"2024-09-17T05:03:19.485538+0000","last_fullsized":"2024-09-17T05:03:19.485538+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.000517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487212+0000","last_change":"2024-09-17T05:03:18.496866+0000","last_active":"2024-09-17T05:03:19.487212+0000","last_peered":"2024-09-17T05:03:19.487212+0000","last_clean":"2024-09-17T05:03:19.487212+0000","last_became_active":"2024-09-17T05:03:18.496533+0000","last_became_peered":"2024-09-17T05:03:18.496533+0000","last_unstale":"2024-09-17T05:03:19.487212+0000","last_undegraded":"2024-09-17T05:03:19.487212+0000","last_fullsized":"2024-09-17T05:03:19.487212+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:06.312704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487198+0000","last_change":"2024-09-17T05:03:17.470269+0000","last_active":"2024-09-17T05:03:19.487198+0000","last_peered":"2024-09-17T05:03:19.487198+0000","last_clean":"2024-09-17T05:03:19.487198+0000","last_became_active":"2024-09-17T05:03:17.469912+0000","last_became_peered":"2024-09-17T05:03:17.469912+0000","last_unstale":"2024-09-17T05:03:19.487198+0000","last_undegraded":"2024-09-17T05:03:19.487198+0000","last_fullsized":"2024-09-17T05:03:19.487198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:33.570989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795830+0000","last_change":"2024-09-17T05:03:18.494624+0000","last_active":"2024-09-17T05:03:21.795830+0000","last_peered":"2024-09-17T05:03:21.795830+0000","last_clean":"2024-09-17T05:03:21.795830+0000","last_became_active":"2024-09-17T05:03:18.494312+0000","last_became_peered":"2024-09-17T05:03:18.494312+0000","last_unstale":"2024-09-17T05:03:21.795830+0000","last_undegraded":"2024-09-17T05:03:21.795830+0000","last_fullsized":"2024-09-17T05:03:21.795830+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:54.573981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.485616+0000","last_change":"2024-09-17T05:03:17.486561+0000","last_active":"2024-09-17T05:03:19.485616+0000","last_peered":"2024-09-17T05:03:19.485616+0000","last_clean":"2024-09-17T05:03:19.485616+0000","last_became_active":"2024-09-17T05:03:17.486224+0000","last_became_peered":"2024-09-17T05:03:17.486224+0000","last_unstale":"2024-09-17T05:03:19.485616+0000","last_undegraded":"2024-09-17T05:03:19.485616+0000","last_fullsized":"2024-09-17T05:03:19.485616+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:53:31.418463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.487417+0000","last_change":"2024-09-17T05:03:18.504505+0000","last_active":"2024-09-17T05:03:19.487417+0000","last_peered":"2024-09-17T05:03:19.487417+0000","last_clean":"2024-09-17T05:03:19.487417+0000","last_became_active":"2024-09-17T05:03:18.503742+0000","last_became_peered":"2024-09-17T05:03:18.503742+0000","last_unstale":"2024-09-17T05:03:19.487417+0000","last_undegraded":"2024-09-17T05:03:19.487417+0000","last_fullsized":"2024-09-17T05:03:19.487417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:07.839715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:21.795520+0000","last_change":"2024-09-17T05:03:17.479945+0000","last_active":"2024-09-17T05:03:21.795520+0000","last_peered":"2024-09-17T05:03:21.795520+0000","last_clean":"2024-09-17T05:03:21.795520+0000","last_became_active":"2024-09-17T05:03:17.479596+0000","last_became_peered":"2024-09-17T05:03:17.479596+0000","last_unstale":"2024-09-17T05:03:21.795520+0000","last_undegraded":"2024-09-17T05:03:21.795520+0000","last_fullsized":"2024-09-17T05:03:21.795520+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:48.105450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:20.414745+0000","last_change":"2024-09-17T05:03:18.503490+0000","last_active":"2024-09-17T05:03:20.414745+0000","last_peered":"2024-09-17T05:03:20.414745+0000","last_clean":"2024-09-17T05:03:20.414745+0000","last_became_active":"2024-09-17T05:03:18.503217+0000","last_became_peered":"2024-09-17T05:03:18.503217+0000","last_unstale":"2024-09-17T05:03:20.414745+0000","last_undegraded":"2024-09-17T05:03:20.414745+0000","last_fullsized":"2024-09-17T05:03:20.414745+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:21:51.694352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484236+0000","last_change":"2024-09-17T05:03:17.476961+0000","last_active":"2024-09-17T05:03:19.484236+0000","last_peered":"2024-09-17T05:03:19.484236+0000","last_clean":"2024-09-17T05:03:19.484236+0000","last_became_active":"2024-09-17T05:03:17.476427+0000","last_became_peered":"2024-09-17T05:03:17.476427+0000","last_unstale":"2024-09-17T05:03:19.484236+0000","last_undegraded":"2024-09-17T05:03:19.484236+0000","last_fullsized":"2024-09-17T05:03:19.484236+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:58:28.438189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494906+0000","last_change":"2024-09-17T05:03:18.509986+0000","last_active":"2024-09-17T05:03:19.494906+0000","last_peered":"2024-09-17T05:03:19.494906+0000","last_clean":"2024-09-17T05:03:19.494906+0000","last_became_active":"2024-09-17T05:03:18.509706+0000","last_became_peered":"2024-09-17T05:03:18.509706+0000","last_unstale":"2024-09-17T05:03:19.494906+0000","last_undegraded":"2024-09-17T05:03:19.494906+0000","last_fullsized":"2024-09-17T05:03:19.494906+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:17.450231+0000","last_clean_scrub_stamp":"2024-09-17T05:03:17.450231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:00.496228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.494256+0000","last_change":"2024-09-17T05:03:17.483817+0000","last_active":"2024-09-17T05:03:19.494256+0000","last_peered":"2024-09-17T05:03:19.494256+0000","last_clean":"2024-09-17T05:03:19.494256+0000","last_became_active":"2024-09-17T05:03:17.483478+0000","last_became_peered":"2024-09-17T05:03:17.483478+0000","last_unstale":"2024-09-17T05:03:19.494256+0000","last_undegraded":"2024-09-17T05:03:19.494256+0000","last_fullsized":"2024-09-17T05:03:19.494256+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:16.442580+0000","last_clean_scrub_stamp":"2024-09-17T05:03:16.442580+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:27.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":172,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T05:03:19.484317+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:19.484317+0000","last_peered":"2024-09-17T05:03:19.484317+0000","last_clean":"2024-09-17T05:03:19.484317+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:19.484317+0000","last_undegraded":"2024-09-17T05:03:19.484317+0000","last_fullsized":"2024-09-17T05:03:19.484317+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":275881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":643072,"data_stored":277529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":275915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":275859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":866263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509473,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":626688,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1232896,"data_stored":867895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:27.924 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:03:27.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:03:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 -- 172.21.15.16:0/3739246415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df80b2f50 msgr2=0x7f4df815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 --2- 172.21.15.16:0/3739246415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df80b2f50 0x7f4df815bb60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009bc0 tx=0x7f4de801cb10 comp rx=0 tx=0).stop 2024-09-17T05:03:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 -- 172.21.15.16:0/3739246415 shutdown_connections 2024-09-17T05:03:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 --2- 172.21.15.16:0/3739246415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df80b2f50 0x7f4df815bb60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 -- 172.21.15.16:0/3739246415 >> 172.21.15.16:0/3739246415 conn(0x7f4df806f900 msgr2=0x7f4df80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 -- 172.21.15.16:0/3739246415 shutdown_connections 2024-09-17T05:03:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 -- 172.21.15.16:0/3739246415 wait complete. 2024-09-17T05:03:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 Processor -- start 2024-09-17T05:03:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.129+0000 7f4dfda00640 1 -- start start 2024-09-17T05:03:28.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 0x7f4df81556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4df80b2f50 0x7f4df8166470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df815c0a0 0x7f4df81669b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8169c90 con 0x7f4df8073240 2024-09-17T05:03:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8169b10 con 0x7f4df815c0a0 2024-09-17T05:03:28.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4df8169e10 con 0x7f4df80b2f50 2024-09-17T05:03:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 0x7f4df81556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 0x7f4df81556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58322/0 (socket says 172.21.15.16:58322) 2024-09-17T05:03:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 -- 172.21.15.16:0/1372383782 learned_addr learned my addr 172.21.15.16:0/1372383782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df7400640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df815c0a0 0x7f4df81669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dee000640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4df80b2f50 0x7f4df8166470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 -- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4df80b2f50 msgr2=0x7f4df8166470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4df80b2f50 0x7f4df8166470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 -- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df815c0a0 msgr2=0x7f4df81669b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df815c0a0 0x7f4df81669b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 -- 172.21.15.16:0/1372383782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df81410f0 con 0x7f4df8073240 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df7400640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df815c0a0 0x7f4df81669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dee000640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4df80b2f50 0x7f4df8166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:28.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4df6a00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 0x7f4df81556a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009b10 tx=0x7f4de8005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80b2040 con 0x7f4df8073240 2024-09-17T05:03:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4de8004d40 con 0x7f4df8073240 2024-09-17T05:03:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004240 con 0x7f4df8073240 2024-09-17T05:03:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df81413d0 con 0x7f4df8073240 2024-09-17T05:03:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df8141990 con 0x7f4df8073240 2024-09-17T05:03:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.133+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df806e280 con 0x7f4df8073240 2024-09-17T05:03:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.137+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4de80bd060 con 0x7f4df8073240 2024-09-17T05:03:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.137+0000 7f4def400640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4dd4073260 0x7f4dd4075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.137+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f4de812df70 con 0x7f4df8073240 2024-09-17T05:03:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.137+0000 7f4dee000640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4dd4073260 0x7f4dd4075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.137+0000 7f4dee000640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4dd4073260 0x7f4dd4075770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4df8152240 tx=0x7f4de0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.141+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4de8016790 con 0x7f4df8073240 2024-09-17T05:03:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.377+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4df806dfa0 con 0x7f4df8073240 2024-09-17T05:03:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.381+0000 7f4def400640 1 -- 172.21.15.16:0/1372383782 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7f4de80f8e90 con 0x7f4df8073240 2024-09-17T05:03:28.384 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:03:28.384 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:03:28.385 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:03:28.385 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:03:28.385 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:03:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:03:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4dd4073260 msgr2=0x7f4dd4075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4dd4073260 0x7f4dd4075770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4df8152240 tx=0x7f4de0009040 comp rx=0 tx=0).stop 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 msgr2=0x7f4df81556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 0x7f4df81556a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009b10 tx=0x7f4de8005ae0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 shutdown_connections 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4dd4073260 0x7f4dd4075770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4df815c0a0 0x7f4df81669b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4df80b2f50 0x7f4df8166470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 --2- 172.21.15.16:0/1372383782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df8073240 0x7f4df81556a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 >> 172.21.15.16:0/1372383782 conn(0x7f4df806f900 msgr2=0x7f4df80d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 shutdown_connections 2024-09-17T05:03:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.385+0000 7f4dfda00640 1 -- 172.21.15.16:0/1372383782 wait complete. 2024-09-17T05:03:28.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 -- 172.21.15.16:0/1546849661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7240b2f50 msgr2=0x7fe72415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 --2- 172.21.15.16:0/1546849661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7240b2f50 0x7fe72415bb60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe714009900 tx=0x7fe71401c850 comp rx=0 tx=0).stop 2024-09-17T05:03:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 -- 172.21.15.16:0/1546849661 shutdown_connections 2024-09-17T05:03:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 --2- 172.21.15.16:0/1546849661 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe72415c0a0 0x7fe72415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 --2- 172.21.15.16:0/1546849661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7240b2f50 0x7fe72415bb60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 -- 172.21.15.16:0/1546849661 >> 172.21.15.16:0/1546849661 conn(0x7fe72406f900 msgr2=0x7fe7240b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 -- 172.21.15.16:0/1546849661 shutdown_connections 2024-09-17T05:03:28.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.565+0000 7fe728c00640 1 -- 172.21.15.16:0/1546849661 wait complete. 2024-09-17T05:03:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 Processor -- start 2024-09-17T05:03:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 -- start start 2024-09-17T05:03:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 0x7fe724164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe72415c0a0 0x7fe724150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe72415f150 0x7fe7241583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe724169150 con 0x7fe7240b2f50 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe724168fd0 con 0x7fe72415c0a0 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe728c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe7241692d0 con 0x7fe72415f150 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 0x7fe724164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 0x7fe724164690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58334/0 (socket says 172.21.15.16:58334) 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 -- 172.21.15.16:0/4243368171 learned_addr learned my addr 172.21.15.16:0/4243368171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe719400640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe72415c0a0 0x7fe724150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe722800640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe72415f150 0x7fe7241583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 -- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe72415f150 msgr2=0x7fe7241583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe72415f150 0x7fe7241583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 -- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe72415c0a0 msgr2=0x7fe724150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe72415c0a0 0x7fe724150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.569+0000 7fe721e00640 1 -- 172.21.15.16:0/4243368171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe724164ee0 con 0x7fe7240b2f50 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe721e00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 0x7fe724164690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe714009850 tx=0x7fe714004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140b2040 con 0x7fe7240b2f50 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe714004430 con 0x7fe7240b2f50 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7240cdf70 con 0x7fe7240b2f50 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714005280 con 0x7fe7240b2f50 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe724150640 con 0x7fe7240b2f50 2024-09-17T05:03:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72415bb60 con 0x7fe7240b2f50 2024-09-17T05:03:28.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe714005420 con 0x7fe7240b2f50 2024-09-17T05:03:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.573+0000 7fe71aa00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe700073200 0x7fe700075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.577+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fe71412dd70 con 0x7fe7240b2f50 2024-09-17T05:03:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.577+0000 7fe719400640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe700073200 0x7fe700075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.577+0000 7fe719400640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe700073200 0x7fe700075710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe72415b670 tx=0x7fe70c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:28.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.581+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7140164d0 con 0x7fe7240b2f50 2024-09-17T05:03:28.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.785+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe724159fe0 con 0x7fe7240b2f50 2024-09-17T05:03:28.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.785+0000 7fe71aa00640 1 -- 172.21.15.16:0/4243368171 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7fe7140f8630 con 0x7fe7240b2f50 2024-09-17T05:03:28.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:28.790 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:17.889311+0000","modified":"2024-09-17T05:03:20.917921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5460":{"gid":5460,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3845424541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3845424541},{"type":"v1","addr":"172.21.15.16:6834","nonce":3845424541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5460,"qdb_cluster":[5460]},"id":3}]} 2024-09-17T05:03:28.790 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-17T05:03:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe700073200 msgr2=0x7fe700075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe700073200 0x7fe700075710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe72415b670 tx=0x7fe70c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 msgr2=0x7fe724164690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 0x7fe724164690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe714009850 tx=0x7fe714004040 comp rx=0 tx=0).stop 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 shutdown_connections 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe700073200 0x7fe700075710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe72415f150 0x7fe7241583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe72415c0a0 0x7fe724150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 --2- 172.21.15.16:0/4243368171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7240b2f50 0x7fe724164690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 >> 172.21.15.16:0/4243368171 conn(0x7fe72406f900 msgr2=0x7fe724155870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 shutdown_connections 2024-09-17T05:03:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:28.789+0000 7fe728c00640 1 -- 172.21.15.16:0/4243368171 wait complete. 2024-09-17T05:03:29.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:03:17.889311+0000', 'modified': '2024-09-17T05:03:20.917921+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5460': {'gid': 5460, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6834/3845424541', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 3845424541}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 3845424541}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5460, 'qdb_cluster': [5460]} 2024-09-17T05:03:29.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:03:29.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:03:29.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd15400640 1 -- 172.21.15.16:0/1678593228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccf4004da0 msgr2=0x7fccf4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd15400640 1 --2- 172.21.15.16:0/1678593228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccf4004da0 0x7fccf4005230 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009eb0 tx=0x7fcd0801d5d0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd0f200640 1 -- 172.21.15.16:0/1678593228 reap_dead start 2024-09-17T05:03:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd15400640 1 -- 172.21.15.16:0/1678593228 shutdown_connections 2024-09-17T05:03:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd15400640 1 -- 172.21.15.16:0/1678593228 >> 172.21.15.16:0/1678593228 conn(0x7fcd100b2250 msgr2=0x7fcd100d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd15400640 1 -- 172.21.15.16:0/1678593228 shutdown_connections 2024-09-17T05:03:29.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.373+0000 7fcd15400640 1 -- 172.21.15.16:0/1678593228 wait complete. 2024-09-17T05:03:29.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 Processor -- start 2024-09-17T05:03:29.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 -- start start 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccf4004da0 0x7fcd10154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd100c7110 0x7fcd10156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 0x7fcd100c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd100cf070 con 0x7fcd100c5d60 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd100ceef0 con 0x7fccf4004da0 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcd100cf1f0 con 0x7fcd100c7110 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 0x7fcd100c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 0x7fcd100c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58354/0 (socket says 172.21.15.16:58354) 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 -- 172.21.15.16:0/3410426550 learned_addr learned my addr 172.21.15.16:0/3410426550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0e800640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccf4004da0 0x7fcd10154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd05e00640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd100c7110 0x7fcd10156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 -- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd100c7110 msgr2=0x7fcd10156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd100c7110 0x7fcd10156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 -- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccf4004da0 msgr2=0x7fcd10154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccf4004da0 0x7fcd10154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 -- 172.21.15.16:0/3410426550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd100c66b0 con 0x7fcd100c5d60 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd0f200640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 0x7fcd100c6170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd080ad040 tx=0x7fcd080abad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd07400640 1 -- 172.21.15.16:0/3410426550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08009ab0 con 0x7fcd100c5d60 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd07400640 1 -- 172.21.15.16:0/3410426550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd080ba070 con 0x7fcd100c5d60 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd100c7880 con 0x7fcd100c5d60 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd07400640 1 -- 172.21.15.16:0/3410426550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08009200 con 0x7fcd100c5d60 2024-09-17T05:03:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.377+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd100c7e40 con 0x7fcd100c5d60 2024-09-17T05:03:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.381+0000 7fcd07400640 1 -- 172.21.15.16:0/3410426550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd080a9050 con 0x7fcd100c5d60 2024-09-17T05:03:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.381+0000 7fcd07400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fccec073200 0x7fccec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.381+0000 7fcd07400640 1 -- 172.21.15.16:0/3410426550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fcd0812f7d0 con 0x7fcd100c5d60 2024-09-17T05:03:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.381+0000 7fcd0e800640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fccec073200 0x7fccec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.381+0000 7fcd1c9d2000 1 Processor -- start 2024-09-17T05:03:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.381+0000 7fcd1c9d2000 1 -- start start 2024-09-17T05:03:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd0e800640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fccec073200 0x7fccec075710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006fd0 tx=0x7fcd00006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f445a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f445a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f445a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f4447a410 con 0x555f44454950 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555f443b7b40 con 0x555f4443dca0 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x555f443a6f70 con 0x555f44064460 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f445a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f445a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58362/0 (socket says 172.21.15.16:58362) 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 -- 172.21.15.16:0/2681617185 learned_addr learned my addr 172.21.15.16:0/2681617185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd0f200640 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f445a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f445a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 -- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 msgr2=0x555f445a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f445a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 -- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 msgr2=0x555f445a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f445a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 -- 172.21.15.16:0/2681617185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4436e5b0 con 0x555f44454950 2024-09-17T05:03:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f445a5be0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fccf8013800 tx=0x7fccf8013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d08b9085b353564 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fccff000640 1 -- 172.21.15.16:0/2681617185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80132f0 con 0x555f44454950 2024-09-17T05:03:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fccff000640 1 -- 172.21.15.16:0/2681617185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccf8013e50 con 0x555f44454950 2024-09-17T05:03:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fccff000640 1 -- 172.21.15.16:0/2681617185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8011ca0 con 0x555f44454950 2024-09-17T05:03:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2681617185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 msgr2=0x555f445a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f445a5be0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fccf8013800 tx=0x7fccf8013c00 comp rx=0 tx=0).stop 2024-09-17T05:03:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2681617185 shutdown_connections 2024-09-17T05:03:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f445a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f445a5be0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2681617185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f445a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2681617185 >> 172.21.15.16:0/2681617185 conn(0x555f4452ad90 msgr2=0x555f4455a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2681617185 shutdown_connections 2024-09-17T05:03:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.385+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2681617185 wait complete. 2024-09-17T05:03:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 -- start start 2024-09-17T05:03:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 Processor -- start 2024-09-17T05:03:29.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f445b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f4443dca0 0x555f445b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f44454950 0x555f445b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f443415f0 con 0x555f44064460 2024-09-17T05:03:29.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555f443aad80 con 0x555f44454950 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x555f4434d520 con 0x555f4443dca0 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 10 client.0 _ll_register_callbacks cb 0x555f44351830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f445b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f445b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58368/0 (socket says 172.21.15.16:58368) 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 -- 172.21.15.16:0/2213670058 learned_addr learned my addr 172.21.15.16:0/2213670058 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f4443dca0 0x555f445b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0f200640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f44454950 0x555f445b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 -- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f4443dca0 msgr2=0x555f445b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f4443dca0 0x555f445b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 -- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f44454950 msgr2=0x555f445b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f44454950 0x555f445b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 -- 172.21.15.16:0/2213670058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f44345190 con 0x555f44064460 2024-09-17T05:03:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f4443dca0 0x555f445b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f445b0d00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcd00000c10 tx=0x7fcd00001370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909afddc1cb75577 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fccf3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fccf3400640 1 -- 172.21.15.16:0/2213670058 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00012dd0 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fccf3400640 1 -- 172.21.15.16:0/2213670058 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fcd00025a20 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f4433f110 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fccf3400640 1 -- 172.21.15.16:0/2213670058 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd00030050 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 10 client.5592 fetch_fsmap learned FSMap version 26 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.389+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x555f4442bf10 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fccf3400640 1 -- 172.21.15.16:0/2213670058 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555f4433f110 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fccf3400640 1 -- 172.21.15.16:0/2213670058 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fcd00025c20 con 0x555f44064460 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 10 client.5592 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T05:03:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 10 client.5592 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:03:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 20 client.5592 populate_metadata read hostname 'smithi016' 2024-09-17T05:03:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f440497d0 0x555f445bb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 4 client.5592 mds_command: new command op to 5460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:03:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 --> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f443b9a90 con 0x555f440497d0 2024-09-17T05:03:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f440497d0 0x555f445bb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f440497d0 0x555f445bb0e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74161911dd393486 server_cookie=cfae8b08d99d311f in_seq=0 out_seq=0 2024-09-17T05:03:29.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fccf3400640 10 client.5592 ms_handle_connect on v2:172.21.15.16:6832/3845424541 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fccf3400640 1 -- 172.21.15.16:0/2213670058 <== mds.0 v2:172.21.15.16:6832/3845424541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555f443b9a90 con 0x555f440497d0 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fccf3400640 10 client.5592 handle_command_reply: tid=0 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 client.5592 shutdown 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 msgr2=0x555f445b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f445b0d00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcd00000c10 tx=0x7fcd00001370 comp rx=0 tx=0).stop 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 shutdown_connections 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f44454950 0x555f445b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f4443dca0 0x555f445b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f440497d0 0x555f445bb0e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2213670058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f445b0d00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fccf3400640 0 client.5592 ms_handle_reset on v2:172.21.15.16:6832/3845424541 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 >> 172.21.15.16:0/2213670058 conn(0x555f4452ad90 msgr2=0x555f445a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 shutdown_connections 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2213670058 wait complete. 2024-09-17T05:03:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.393+0000 7fcd1c9d2000 20 client.5592 trim_cache size 0 max 16384 2024-09-17T05:03:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 Processor -- start 2024-09-17T05:03:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 -- start start 2024-09-17T05:03:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f4461bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f4461c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44454950 0x555f4461c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101feb00 con 0x555f44064460 2024-09-17T05:03:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd100b4e80 con 0x555f4443dca0 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcd100b41e0 con 0x555f44454950 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f4461bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f4461bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58372/0 (socket says 172.21.15.16:58372) 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 -- 172.21.15.16:0/2403787196 learned_addr learned my addr 172.21.15.16:0/2403787196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd05e00640 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f4461c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0f200640 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44454950 0x555f4461c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 -- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44454950 msgr2=0x555f4461c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44454950 0x555f4461c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 -- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 msgr2=0x555f4461c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f4461c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.413+0000 7fcd0e800640 1 -- 172.21.15.16:0/2403787196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f44402db0 con 0x555f44064460 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd0e800640 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f4461bc10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006110 tx=0x7fcd00001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f954f2648a9b5070 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fccfe600640 1 -- 172.21.15.16:0/2403787196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00012bd0 con 0x555f44064460 2024-09-17T05:03:29.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fccfe600640 1 -- 172.21.15.16:0/2403787196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd00038070 con 0x555f44064460 2024-09-17T05:03:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fccfe600640 1 -- 172.21.15.16:0/2403787196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000257e0 con 0x555f44064460 2024-09-17T05:03:29.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2403787196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 msgr2=0x555f4461bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f4461bc10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006110 tx=0x7fcd00001300 comp rx=0 tx=0).stop 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2403787196 shutdown_connections 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44454950 0x555f4461c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f4461c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/2403787196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44064460 0x555f4461bc10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2403787196 >> 172.21.15.16:0/2403787196 conn(0x555f4452ad90 msgr2=0x555f4455d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2403787196 shutdown_connections 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/2403787196 wait complete. 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- start start 2024-09-17T05:03:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 Processor -- start 2024-09-17T05:03:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f4461fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f446200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f44620610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f443415f0 con 0x555f44454950 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd100b4830 con 0x555f4443dca0 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x555f44466750 con 0x555f44064460 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.417+0000 7fcd1c9d2000 10 client.0 _ll_register_callbacks cb 0x7fcd00006060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f44620610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f44620610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58386/0 (socket says 172.21.15.16:58386) 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 -- 172.21.15.16:0/356100614 learned_addr learned my addr 172.21.15.16:0/356100614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0e800640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f4461fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd05e00640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f446200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 -- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 msgr2=0x555f4461fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f4461fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 -- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 msgr2=0x555f446200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f446200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 -- 172.21.15.16:0/356100614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4436dab0 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0f200640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f44620610 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcd080f9d00 tx=0x7fcd080f9950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d8d8303d510f9f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 1 -- 172.21.15.16:0/356100614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080c4040 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 1 -- 172.21.15.16:0/356100614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fcd080ffbc0 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd00038070 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 1 -- 172.21.15.16:0/356100614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd080fa000 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 10 client.5598 fetch_fsmap learned FSMap version 26 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x555f4434c070 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 1 -- 172.21.15.16:0/356100614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd00038070 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 1 -- 172.21.15.16:0/356100614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fcd080c7060 con 0x555f44454950 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 10 client.5598 fetch_fsmap finished waiting for FSMap version 26 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 10 client.5598 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:03:29.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 20 client.5598 populate_metadata read hostname 'smithi016' 2024-09-17T05:03:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f44037830 0x555f445db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 4 client.5598 mds_command: new command op to 5460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:03:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 --> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] -- command(tid 0: {"prefix": "status"}) -- 0x555f445c1400 con 0x555f44037830 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0e800640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f44037830 0x555f445db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 10 client.5598 ms_handle_connect on v2:172.21.15.16:6832/3845424541 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd0e800640 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f44037830 0x555f445db9f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddf95f60c61f0d5 server_cookie=14503326695bd31e in_seq=0 out_seq=0 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 1 -- 172.21.15.16:0/356100614 <== mds.0 v2:172.21.15.16:6832/3845424541 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x555f445c1400 con 0x555f44037830 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 10 client.5598 handle_command_reply: tid=0 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 client.5598 shutdown 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 msgr2=0x555f44620610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f44620610 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcd080f9d00 tx=0x7fcd080f9950 comp rx=0 tx=0).stop 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 shutdown_connections 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:6832/3845424541,v1:172.21.15.16:6834/3845424541] conn(0x555f44037830 0x555f445db9f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f44454950 0x555f44620610 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f4443dca0 0x555f446200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 --2- 172.21.15.16:0/356100614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x555f44064460 0x555f4461fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fccf3400640 0 client.5598 ms_handle_reset on v2:172.21.15.16:6832/3845424541 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 >> 172.21.15.16:0/356100614 conn(0x555f4452ad90 msgr2=0x555f4455d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 shutdown_connections 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 1 -- 172.21.15.16:0/356100614 wait complete. 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd1c9d2000 20 client.5598 trim_cache size 0 max 16384 2024-09-17T05:03:29.428 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5460, 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.943163054999999, 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 26, 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 53, 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 53, 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.328607867000001 2024-09-17T05:03:29.429 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:03:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fccec073200 msgr2=0x7fccec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd15400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fccec073200 0x7fccec075710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006fd0 tx=0x7fcd00006b20 comp rx=0 tx=0).stop 2024-09-17T05:03:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 msgr2=0x7fcd100c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.421+0000 7fcd15400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 0x7fcd100c6170 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcd080ad040 tx=0x7fcd080abad0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 shutdown_connections 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fccec073200 0x7fccec075710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd100c5d60 0x7fcd100c6170 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd100c7110 0x7fcd10156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 --2- 172.21.15.16:0/3410426550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccf4004da0 0x7fcd10154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 >> 172.21.15.16:0/3410426550 conn(0x7fcd100b2250 msgr2=0x7fcd1006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 shutdown_connections 2024-09-17T05:03:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.425+0000 7fcd15400640 1 -- 172.21.15.16:0/3410426550 wait complete. 2024-09-17T05:03:29.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:03:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 -- 172.21.15.16:0/1148139427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057c0039f0 msgr2=0x7f057c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 --2- 172.21.15.16:0/1148139427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f057c0039f0 0x7f057c003e20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f058800c390 tx=0x7f058801ce90 comp rx=0 tx=0).stop 2024-09-17T05:03:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f0596800640 1 -- 172.21.15.16:0/1148139427 reap_dead start 2024-09-17T05:03:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 -- 172.21.15.16:0/1148139427 shutdown_connections 2024-09-17T05:03:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 -- 172.21.15.16:0/1148139427 >> 172.21.15.16:0/1148139427 conn(0x7f05980cf070 msgr2=0x7f05980cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 -- 172.21.15.16:0/1148139427 shutdown_connections 2024-09-17T05:03:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 -- 172.21.15.16:0/1148139427 wait complete. 2024-09-17T05:03:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 Processor -- start 2024-09-17T05:03:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.605+0000 7f059ca00640 1 -- start start 2024-09-17T05:03:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f057c0039f0 0x7f05980c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05981494f0 0x7f05980c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 0x7f05980c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0598168ed0 con 0x7f0598149900 2024-09-17T05:03:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0598168d50 con 0x7f05981494f0 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0598169050 con 0x7f057c0039f0 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 0x7f05980c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 0x7f05980c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58398/0 (socket says 172.21.15.16:58398) 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 -- 172.21.15.16:0/230001105 learned_addr learned my addr 172.21.15.16:0/230001105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0595e00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f057c0039f0 0x7f05980c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f058d400640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05981494f0 0x7f05980c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 -- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f057c0039f0 msgr2=0x7f05980c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f057c0039f0 0x7f05980c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 -- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05981494f0 msgr2=0x7f05980c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05981494f0 0x7f05980c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 -- 172.21.15.16:0/230001105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05980cb510 con 0x7f0598149900 2024-09-17T05:03:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f0596800640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 0x7f05980c6420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f059815e4a0 tx=0x7f0590001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590011870 con 0x7f0598149900 2024-09-17T05:03:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0590017070 con 0x7f0598149900 2024-09-17T05:03:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590011d80 con 0x7f0598149900 2024-09-17T05:03:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05980cb7f0 con 0x7f0598149900 2024-09-17T05:03:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05980cbdb0 con 0x7f0598149900 2024-09-17T05:03:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.609+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05980d06d0 con 0x7f0598149900 2024-09-17T05:03:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.613+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f059001c460 con 0x7f0598149900 2024-09-17T05:03:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.613+0000 7f058ea00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0574073200 0x7f0574075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:29.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.613+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f0590094b50 con 0x7f0598149900 2024-09-17T05:03:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.613+0000 7f0595e00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0574073200 0x7f0574075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:29.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.613+0000 7f0595e00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0574073200 0x7f0574075710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05880ab000 tx=0x7f0588019cd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:29.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.617+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f059005f940 con 0x7f0598149900 2024-09-17T05:03:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.829+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f05980cc190 con 0x7f0598149900 2024-09-17T05:03:29.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.829+0000 7f058ea00640 1 -- 172.21.15.16:0/230001105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f059005f240 con 0x7f0598149900 2024-09-17T05:03:29.832 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:03:29.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0574073200 msgr2=0x7f0574075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0574073200 0x7f0574075710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f05880ab000 tx=0x7f0588019cd0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 msgr2=0x7f05980c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:29.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 0x7f05980c6420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f059815e4a0 tx=0x7f0590001ac0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 shutdown_connections 2024-09-17T05:03:29.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0574073200 0x7f0574075710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0598149900 0x7f05980c6420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05981494f0 0x7f05980c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 --2- 172.21.15.16:0/230001105 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f057c0039f0 0x7f05980c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 >> 172.21.15.16:0/230001105 conn(0x7f05980cf070 msgr2=0x7f059815b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 shutdown_connections 2024-09-17T05:03:29.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:29.833+0000 7f059ca00640 1 -- 172.21.15.16:0/230001105 wait complete. 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:03:29.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:03:29.856 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:03:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:29.856 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:03:29.865 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:03:29.866 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42283sec preferred_lft 42283sec 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:03:29.867 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.868 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:03:29.869 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:29.869 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:03:29.869 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.869 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:03:29.869 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:29.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:29.870 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:03:29.932 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:03:29.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:29.934 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:03:29.987 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:03:29.988 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:03:29.988 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:03:29.988 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:03:29.988 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:03:29.988 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:03:29.988 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:03:29.988 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:03:29.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:03:29.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:03:29.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:03:29.989 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:03:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:03:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:30.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:30.126 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:03:30.134 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:03:30.135 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:03:30.135 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:03:30.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:30.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:03:30.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:03:30.202 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:30.253 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:03:30.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:03:30.254 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:30.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:03:30.306 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:03:30.309 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:03:30.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:30.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:30.454 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:03:30.455 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:03:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:03:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:03:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:03:30.473 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:03:30.483 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:03:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:03:30.487 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:03:30.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:30.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:30.508 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:03:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:03:30.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:30.612 INFO:teuthology.orchestra.run.smithi016.stdout:client5470 2024-09-17T05:03:30.613 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:30.613 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:30.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:03:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 -- 172.21.15.16:0/195598225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22300b2f50 msgr2=0x7f223015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 --2- 172.21.15.16:0/195598225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22300b2f50 0x7f223015bb60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f223015cc70 tx=0x7f222001c830 comp rx=0 tx=0).stop 2024-09-17T05:03:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 -- 172.21.15.16:0/195598225 shutdown_connections 2024-09-17T05:03:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 --2- 172.21.15.16:0/195598225 >> v2:172.21.15.104:3300/0 conn(0x7f223015f150 0x7f223015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:30.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 --2- 172.21.15.16:0/195598225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f223015c0a0 0x7f223015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 --2- 172.21.15.16:0/195598225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22300b2f50 0x7f223015bb60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 -- 172.21.15.16:0/195598225 >> 172.21.15.16:0/195598225 conn(0x7f223006f900 msgr2=0x7f22300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 -- 172.21.15.16:0/195598225 shutdown_connections 2024-09-17T05:03:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.825+0000 7f2234c00640 1 -- 172.21.15.16:0/195598225 wait complete. 2024-09-17T05:03:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 Processor -- start 2024-09-17T05:03:30.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 -- start start 2024-09-17T05:03:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f22300b2f50 0x7f223015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 0x7f223014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f223015f150 0x7f22300cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f223006e430 con 0x7f223015c0a0 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f223006e2b0 con 0x7f223015f150 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f223006e5b0 con 0x7f22300b2f50 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 0x7f223014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 0x7f223014d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58420/0 (socket says 172.21.15.16:58420) 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 -- 172.21.15.16:0/2193555139 learned_addr learned my addr 172.21.15.16:0/2193555139 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f222e800640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f223015f150 0x7f22300cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f222de00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f22300b2f50 0x7f223015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 -- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f22300b2f50 msgr2=0x7f223015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:30.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f22300b2f50 0x7f223015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 -- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f223015f150 msgr2=0x7f22300cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f223015f150 0x7f22300cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 -- 172.21.15.16:0/2193555139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223015ea40 con 0x7f223015c0a0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2225400640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 0x7f223014d0d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f221800d610 tx=0x7f221800dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2218017070 con 0x7f223015c0a0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2218005310 con 0x7f223015c0a0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22180047b0 con 0x7f223015c0a0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.829+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223015cef0 con 0x7f223015c0a0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.833+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223015d4b0 con 0x7f223015c0a0 2024-09-17T05:03:30.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.833+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f223015c4b0 con 0x7f223015c0a0 2024-09-17T05:03:30.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.833+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2218004950 con 0x7f223015c0a0 2024-09-17T05:03:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.833+0000 7f2226a00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f220c073200 0x7f220c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.833+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2218094330 con 0x7f223015c0a0 2024-09-17T05:03:30.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.833+0000 7f222de00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f220c073200 0x7f220c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:30.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.837+0000 7f222de00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f220c073200 0x7f220c075710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22200af000 tx=0x7f22200a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:30.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:30.837+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f221805eb80 con 0x7f223015c0a0 2024-09-17T05:03:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.049+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f22300ce2c0 con 0x7f223015c0a0 2024-09-17T05:03:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.049+0000 7f2226a00640 1 -- 172.21.15.16:0/2193555139 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f22180635b0 con 0x7f223015c0a0 2024-09-17T05:03:31.052 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:03:31.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f220c073200 msgr2=0x7f220c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f220c073200 0x7f220c075710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f22200af000 tx=0x7f22200a7040 comp rx=0 tx=0).stop 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 msgr2=0x7f223014d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 0x7f223014d0d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f221800d610 tx=0x7f221800dad0 comp rx=0 tx=0).stop 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 shutdown_connections 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f220c073200 0x7f220c075710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f223015f150 0x7f22300cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f223015c0a0 0x7f223014d0d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 --2- 172.21.15.16:0/2193555139 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f22300b2f50 0x7f223015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 >> 172.21.15.16:0/2193555139 conn(0x7f223006f900 msgr2=0x7f22300ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 shutdown_connections 2024-09-17T05:03:31.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:31.053+0000 7f2234c00640 1 -- 172.21.15.16:0/2193555139 wait complete. 2024-09-17T05:03:31.074 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:03:31.075 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:03:31.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.075 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.085 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:03:31.086 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.087 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:03:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:31.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.089 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:03:31.152 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:03:31.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.154 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:03:31.212 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:03:31.212 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:03:31.212 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:03:31.212 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:03:31.212 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:03:31.212 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:03:31.213 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:03:31.213 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:03:31.213 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:03:31.213 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:03:31.213 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:03:31.213 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:03:31.213 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:03:31.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:03:31.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:03:31.213 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:03:31.214 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:03:31.307 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:03:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:03:31.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:31.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.356 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:03:31.364 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:03:31.364 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:03:31.366 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:03:31.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:03:31.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:03:31.428 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:03:31.480 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:03:31.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:03:31.480 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:03:31.527 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:03:31.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.527 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:03:31.628 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:03:31.629 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:03:31.629 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:03:31.646 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:03:31.646 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:03:31.646 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:03:31.647 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:03:31.647 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:03:31.657 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:03:31.658 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:03:31.661 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:03:31.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:31.661 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:03:31.680 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:03:31.770 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:03:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:03:31.774 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:31.780 INFO:teuthology.orchestra.run.smithi104.stdout:client5613 2024-09-17T05:03:31.781 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:31.781 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:31.833 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:03:31.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:31.912 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:31.995 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:03:31.995 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:31.995 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:31.995 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:32.005 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:32.005 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:32.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:03:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 -- 172.21.15.16:0/1017201052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73815ba50 msgr2=0x7fa73815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 --2- 172.21.15.16:0/1017201052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73815ba50 0x7fa73815be60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa738071010 tx=0x7fa73001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 -- 172.21.15.16:0/1017201052 shutdown_connections 2024-09-17T05:03:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 --2- 172.21.15.16:0/1017201052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa73815ba50 0x7fa73815be60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 --2- 172.21.15.16:0/1017201052 >> v2:172.21.15.104:3300/0 conn(0x7fa738074e30 0x7fa73815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 --2- 172.21.15.16:0/1017201052 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa738074a20 0x7fa73815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 -- 172.21.15.16:0/1017201052 >> 172.21.15.16:0/1017201052 conn(0x7fa7380b2250 msgr2=0x7fa7380d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 -- 172.21.15.16:0/1017201052 shutdown_connections 2024-09-17T05:03:32.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.269+0000 7fa73ee00640 1 -- 172.21.15.16:0/1017201052 wait complete. 2024-09-17T05:03:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 Processor -- start 2024-09-17T05:03:32.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 -- start start 2024-09-17T05:03:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 0x7fa7381505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa738074e30 0x7fa7380b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa73815ba50 0x7fa73815a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380cf3c0 con 0x7fa738074a20 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380cf240 con 0x7fa738074e30 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa7381681d0 con 0x7fa73815ba50 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 0x7fa7381505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 0x7fa7381505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58434/0 (socket says 172.21.15.16:58434) 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 -- 172.21.15.16:0/262800687 learned_addr learned my addr 172.21.15.16:0/262800687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:32.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ce00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa73815ba50 0x7fa73815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa72f400640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa738074e30 0x7fa7380b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 -- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa73815ba50 msgr2=0x7fa73815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa73815ba50 0x7fa73815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 -- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa738074e30 msgr2=0x7fa7380b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa738074e30 0x7fa7380b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 -- 172.21.15.16:0/262800687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738160050 con 0x7fa738074a20 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa737e00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 0x7fa7381505f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa72800eab0 tx=0x7fa72800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72800cea0 con 0x7fa738074a20 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa728004c20 con 0x7fa738074a20 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728005020 con 0x7fa738074a20 2024-09-17T05:03:32.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa738150b30 con 0x7fa738074a20 2024-09-17T05:03:32.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.273+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7380c7820 con 0x7fa738074a20 2024-09-17T05:03:32.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.277+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73815be60 con 0x7fa738074a20 2024-09-17T05:03:32.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.277+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7280051c0 con 0x7fa738074a20 2024-09-17T05:03:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.277+0000 7fa734c00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa714073230 0x7fa714075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.277+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fa7280951d0 con 0x7fa738074a20 2024-09-17T05:03:32.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.277+0000 7fa72f400640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa714073230 0x7fa714075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.281+0000 7fa72f400640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa714073230 0x7fa714075740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ca5d0 tx=0x7fa7200091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:32.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.285+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa72805f970 con 0x7fa738074a20 2024-09-17T05:03:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.469+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa7380c7ae0 con 0x7fa738074a20 2024-09-17T05:03:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.469+0000 7fa734c00640 1 -- 172.21.15.16:0/262800687 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7fa7280643a0 con 0x7fa738074a20 2024-09-17T05:03:32.472 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:03:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa714073230 msgr2=0x7fa714075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa714073230 0x7fa714075740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7380ca5d0 tx=0x7fa7200091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 msgr2=0x7fa7381505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 0x7fa7381505f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa72800eab0 tx=0x7fa72800ef70 comp rx=0 tx=0).stop 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 shutdown_connections 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa714073230 0x7fa714075740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa73815ba50 0x7fa73815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa738074e30 0x7fa7380b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 --2- 172.21.15.16:0/262800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa738074a20 0x7fa7381505f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 >> 172.21.15.16:0/262800687 conn(0x7fa7380b2250 msgr2=0x7fa738151550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 shutdown_connections 2024-09-17T05:03:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.473+0000 7fa73ee00640 1 -- 172.21.15.16:0/262800687 wait complete. 2024-09-17T05:03:32.494 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:03:32.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:32.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:32.521 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:03:32.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:32.521 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:03:32.574 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:32.575 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:32.575 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:32.586 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:32.586 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:32.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 -- 172.21.15.16:0/2964337397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 msgr2=0x7f3cb415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/2964337397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb415f310 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009c80 tx=0x7f3cac01cda0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 -- 172.21.15.16:0/2964337397 shutdown_connections 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/2964337397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb415f310 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/2964337397 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3cb4155c10 0x7f3cb40af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 -- 172.21.15.16:0/2964337397 >> 172.21.15.16:0/2964337397 conn(0x7f3cb40cd980 msgr2=0x7f3cb40cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 -- 172.21.15.16:0/2964337397 shutdown_connections 2024-09-17T05:03:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.797+0000 7f3cb9c00640 1 -- 172.21.15.16:0/2964337397 wait complete. 2024-09-17T05:03:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 Processor -- start 2024-09-17T05:03:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- start start 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb40b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3cb40ce4d0 0x7f3cb41fc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb4155c10 0x7f3cb41fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4071b00 con 0x7f3cb40afa00 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4071980 con 0x7f3cb4155c10 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb4071c80 con 0x7f3cb40ce4d0 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb40b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb40b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58452/0 (socket says 172.21.15.16:58452) 2024-09-17T05:03:32.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 -- 172.21.15.16:0/808520328 learned_addr learned my addr 172.21.15.16:0/808520328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3caa000640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3cb40ce4d0 0x7f3cb41fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb3400640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb4155c10 0x7f3cb41fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 -- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3cb40ce4d0 msgr2=0x7f3cb41fc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3cb40ce4d0 0x7f3cb41fc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 -- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb4155c10 msgr2=0x7f3cb41fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb4155c10 0x7f3cb41fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 -- 172.21.15.16:0/808520328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb40b5ea0 con 0x7f3cb40afa00 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb2a00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb40b53e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400c3c0 tx=0x7f3ca400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4017070 con 0x7f3cb40afa00 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ca40053c0 con 0x7f3cb40afa00 2024-09-17T05:03:32.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4003860 con 0x7f3cb40afa00 2024-09-17T05:03:32.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb40a9460 con 0x7f3cb40afa00 2024-09-17T05:03:32.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb40b6420 con 0x7f3cb40afa00 2024-09-17T05:03:32.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.801+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb415f310 con 0x7f3cb40afa00 2024-09-17T05:03:32.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.805+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ca401a070 con 0x7f3cb40afa00 2024-09-17T05:03:32.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.805+0000 7f3cab400640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3c90073200 0x7f3c90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:32.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.805+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f3ca40937d0 con 0x7f3cb40afa00 2024-09-17T05:03:32.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.805+0000 7f3caa000640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3c90073200 0x7f3c90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:32.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.805+0000 7f3caa000640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3c90073200 0x7f3c90075710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40b5130 tx=0x7f3c9c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:32.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.809+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3cb40ce8e0 con 0x7f3cb40afa00 2024-09-17T05:03:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.993+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3cb40b65f0 con 0x7f3cb40afa00 2024-09-17T05:03:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.993+0000 7f3cab400640 1 -- 172.21.15.16:0/808520328 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) ==== 80+0+0 (secure 0 0 0) 0x7f3ca405e650 con 0x7f3cb40afa00 2024-09-17T05:03:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3c90073200 msgr2=0x7f3c90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3c90073200 0x7f3c90075710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40b5130 tx=0x7f3c9c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 msgr2=0x7f3cb40b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb40b53e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400c3c0 tx=0x7f3ca400c880 comp rx=0 tx=0).stop 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 shutdown_connections 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3c90073200 0x7f3c90075710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cb4155c10 0x7f3cb41fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:32.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3cb40ce4d0 0x7f3cb41fc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 --2- 172.21.15.16:0/808520328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cb40afa00 0x7f3cb40b53e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 >> 172.21.15.16:0/808520328 conn(0x7f3cb40cd980 msgr2=0x7f3cb40cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 shutdown_connections 2024-09-17T05:03:33.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:32.997+0000 7f3cb9c00640 1 -- 172.21.15.16:0/808520328 wait complete. 2024-09-17T05:03:33.019 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:03:33.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:33.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:03:33.045 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:03:33.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:33.045 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:03:33.097 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:33.097 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:33.097 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:33.106 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:03:33.107 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:03:33.107 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:33.107 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:33.148 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:03:33.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 -- 172.21.15.16:0/4211197171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc15c270 msgr2=0x7faabc15c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 --2- 172.21.15.16:0/4211197171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc15c270 0x7faabc15c680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009840 tx=0x7faaa801c790 comp rx=0 tx=0).stop 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 -- 172.21.15.16:0/4211197171 shutdown_connections 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 --2- 172.21.15.16:0/4211197171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc15c270 0x7faabc15c680 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 -- 172.21.15.16:0/4211197171 >> 172.21.15.16:0/4211197171 conn(0x7faabc06f900 msgr2=0x7faabc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 -- 172.21.15.16:0/4211197171 shutdown_connections 2024-09-17T05:03:33.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 -- 172.21.15.16:0/4211197171 wait complete. 2024-09-17T05:03:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 Processor -- start 2024-09-17T05:03:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.305+0000 7faac4600640 1 -- start start 2024-09-17T05:03:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc073240 0x7faabc14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faabc0b2f50 0x7faabc160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 0x7faabc15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faabc169cc0 con 0x7faabc15c270 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faabc169b40 con 0x7faabc073240 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7faabc169e40 con 0x7faabc0b2f50 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 0x7faabc15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 0x7faabc15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58456/0 (socket says 172.21.15.16:58456) 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 -- 172.21.15.16:0/424871380 learned_addr learned my addr 172.21.15.16:0/424871380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac1c00640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc073240 0x7faabc14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac1200640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faabc0b2f50 0x7faabc160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 -- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faabc0b2f50 msgr2=0x7faabc160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faabc0b2f50 0x7faabc160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 -- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc073240 msgr2=0x7faabc14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc073240 0x7faabc14f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 -- 172.21.15.16:0/424871380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc06e350 con 0x7faabc15c270 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac1c00640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc073240 0x7faabc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac1200640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faabc0b2f50 0x7faabc160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac2600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 0x7faabc15a630 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faab400c6e0 tx=0x7faab400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4003b80 con 0x7faabc15c270 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc06d2b0 con 0x7faabc15c270 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faab4003d20 con 0x7faabc15c270 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc06d870 con 0x7faabc15c270 2024-09-17T05:03:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40052d0 con 0x7faabc15c270 2024-09-17T05:03:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.309+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc1fcc40 con 0x7faabc15c270 2024-09-17T05:03:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.313+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faab4005470 con 0x7faabc15c270 2024-09-17T05:03:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.313+0000 7faab9600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faa9c073230 0x7faa9c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.313+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7faab4094190 con 0x7faabc15c270 2024-09-17T05:03:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.313+0000 7faac1c00640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faa9c073230 0x7faa9c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.313+0000 7faac1c00640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faa9c073230 0x7faa9c075740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faab00046c0 tx=0x7faab0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.317+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faab405efa0 con 0x7faabc15c270 2024-09-17T05:03:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.529+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faabc15c680 con 0x7faabc15c270 2024-09-17T05:03:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.529+0000 7faab9600640 1 -- 172.21.15.16:0/424871380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7faab405e8a0 con 0x7faabc15c270 2024-09-17T05:03:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:33.533 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:17.889311+0000","modified":"2024-09-17T05:03:20.917921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5460":{"gid":5460,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3845424541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3845424541},{"type":"v1","addr":"172.21.15.16:6834","nonce":3845424541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5460,"qdb_cluster":[5460]},"id":3}]} 2024-09-17T05:03:33.534 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-17T05:03:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faa9c073230 msgr2=0x7faa9c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faa9c073230 0x7faa9c075740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faab00046c0 tx=0x7faab0009040 comp rx=0 tx=0).stop 2024-09-17T05:03:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 msgr2=0x7faabc15a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 0x7faabc15a630 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faab400c6e0 tx=0x7faab400cba0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 shutdown_connections 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faa9c073230 0x7faa9c075740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faabc15c270 0x7faabc15a630 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faabc0b2f50 0x7faabc160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 --2- 172.21.15.16:0/424871380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faabc073240 0x7faabc14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 >> 172.21.15.16:0/424871380 conn(0x7faabc06f900 msgr2=0x7faabc15cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 shutdown_connections 2024-09-17T05:03:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.533+0000 7faac4600640 1 -- 172.21.15.16:0/424871380 wait complete. 2024-09-17T05:03:33.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847f200640 1 -- 172.21.15.16:0/267849182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 msgr2=0x7f845c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847f200640 1 --2- 172.21.15.16:0/267849182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f845c003ea0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f846800c290 tx=0x7f846801cfa0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847d200640 1 -- 172.21.15.16:0/267849182 reap_dead start 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847f200640 1 -- 172.21.15.16:0/267849182 shutdown_connections 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847f200640 1 -- 172.21.15.16:0/267849182 >> 172.21.15.16:0/267849182 conn(0x7f84780d10c0 msgr2=0x7f84780d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847f200640 1 -- 172.21.15.16:0/267849182 shutdown_connections 2024-09-17T05:03:33.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.713+0000 7f847f200640 1 -- 172.21.15.16:0/267849182 wait complete. 2024-09-17T05:03:33.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 Processor -- start 2024-09-17T05:03:33.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 -- start start 2024-09-17T05:03:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f8478155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478165610 0x7f84781f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8478154050 0x7f8478152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8478166970 con 0x7f845c003a10 2024-09-17T05:03:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f84781667f0 con 0x7f8478165610 2024-09-17T05:03:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8478166af0 con 0x7f8478154050 2024-09-17T05:03:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f8478155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f8478155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58472/0 (socket says 172.21.15.16:58472) 2024-09-17T05:03:33.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 -- 172.21.15.16:0/3625567326 learned_addr learned my addr 172.21.15.16:0/3625567326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847d200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8478154050 0x7f8478152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f846fe00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478165610 0x7f84781f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 -- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8478154050 msgr2=0x7f8478152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8478154050 0x7f8478152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 -- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478165610 msgr2=0x7f84781f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478165610 0x7f84781f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 -- 172.21.15.16:0/3625567326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781548f0 con 0x7f845c003a10 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f846fe00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478165610 0x7f84781f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8477e00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f8478155990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8468098920 tx=0x7f8468006cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680a5c00 con 0x7f845c003a10 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84680014c0 con 0x7f845c003a10 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8478154bd0 con 0x7f845c003a10 2024-09-17T05:03:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468006330 con 0x7f845c003a10 2024-09-17T05:03:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.717+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8478155190 con 0x7f845c003a10 2024-09-17T05:03:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.721+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84780b4610 con 0x7f845c003a10 2024-09-17T05:03:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.721+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8468003600 con 0x7f845c003a10 2024-09-17T05:03:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.721+0000 7f8474c00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8454073200 0x7f8454075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.721+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f846812dd10 con 0x7f845c003a10 2024-09-17T05:03:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.721+0000 7f846fe00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8454073200 0x7f8454075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:33.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.721+0000 7f846fe00640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8454073200 0x7f8454075710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84780b57d0 tx=0x7f84600091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:33.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.725+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8468133020 con 0x7f845c003a10 2024-09-17T05:03:33.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.933+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84780b5800 con 0x7f845c003a10 2024-09-17T05:03:33.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f8474c00640 1 -- 172.21.15.16:0/3625567326 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f8468016d50 con 0x7f845c003a10 2024-09-17T05:03:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:33.939 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:17.889311+0000","modified":"2024-09-17T05:03:20.917921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5460":{"gid":5460,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3845424541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3845424541},{"type":"v1","addr":"172.21.15.16:6834","nonce":3845424541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5460,"qdb_cluster":[5460]},"id":3}]} 2024-09-17T05:03:33.939 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8454073200 msgr2=0x7f8454075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8454073200 0x7f8454075710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84780b57d0 tx=0x7f84600091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 msgr2=0x7f8478155990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f8478155990 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8468098920 tx=0x7f8468006cf0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 shutdown_connections 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8454073200 0x7f8454075710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8478154050 0x7f8478152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8478165610 0x7f84781f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 --2- 172.21.15.16:0/3625567326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f845c003a10 0x7f8478155990 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 >> 172.21.15.16:0/3625567326 conn(0x7f84780d10c0 msgr2=0x7f8478151b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 shutdown_connections 2024-09-17T05:03:33.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:33.937+0000 7f847f200640 1 -- 172.21.15.16:0/3625567326 wait complete. 2024-09-17T05:03:33.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:34.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 -- 172.21.15.16:0/1295377909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54071920 msgr2=0x7feb540b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 --2- 172.21.15.16:0/1295377909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54071920 0x7feb540b61e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feb5415caf0 tx=0x7feb3c01c890 comp rx=0 tx=0).stop 2024-09-17T05:03:34.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 -- 172.21.15.16:0/1295377909 shutdown_connections 2024-09-17T05:03:34.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 --2- 172.21.15.16:0/1295377909 >> v2:172.21.15.104:3300/0 conn(0x7feb5406e030 0x7feb5406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 --2- 172.21.15.16:0/1295377909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54071920 0x7feb540b61e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 -- 172.21.15.16:0/1295377909 >> 172.21.15.16:0/1295377909 conn(0x7feb540d10f0 msgr2=0x7feb540d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:34.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 -- 172.21.15.16:0/1295377909 shutdown_connections 2024-09-17T05:03:34.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.117+0000 7feb5a000640 1 -- 172.21.15.16:0/1295377909 wait complete. 2024-09-17T05:03:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 Processor -- start 2024-09-17T05:03:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 -- start start 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb5406e030 0x7feb5414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 0x7feb54156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb54159f30 0x7feb5406d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb54168ef0 con 0x7feb54152740 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb54168d70 con 0x7feb54159f30 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7feb54169070 con 0x7feb5406e030 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 0x7feb54156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 0x7feb54156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58490/0 (socket says 172.21.15.16:58490) 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 -- 172.21.15.16:0/3780948830 learned_addr learned my addr 172.21.15.16:0/3780948830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb58c00640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb5406e030 0x7feb5414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb59600640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb54159f30 0x7feb5406d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 -- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb5406e030 msgr2=0x7feb5414b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb5406e030 0x7feb5414b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 -- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb54159f30 msgr2=0x7feb5406d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb54159f30 0x7feb5406d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 -- 172.21.15.16:0/3780948830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5414bd80 con 0x7feb54152740 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb4be00640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 0x7feb54156060 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0b0040 tx=0x7feb3c0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0b2040 con 0x7feb54152740 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb3c0044b0 con 0x7feb54152740 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5414c200 con 0x7feb54152740 2024-09-17T05:03:34.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c003860 con 0x7feb54152740 2024-09-17T05:03:34.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.121+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb54159450 con 0x7feb54152740 2024-09-17T05:03:34.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.125+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5414bf50 con 0x7feb54152740 2024-09-17T05:03:34.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.125+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb3c003a00 con 0x7feb54152740 2024-09-17T05:03:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.125+0000 7feb51600640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb30073200 0x7feb30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.125+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7feb3c12e730 con 0x7feb54152740 2024-09-17T05:03:34.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.125+0000 7feb58c00640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb30073200 0x7feb30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.125+0000 7feb58c00640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb30073200 0x7feb30075710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb44000f70 tx=0x7feb44009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:34.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.129+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb3c0164b0 con 0x7feb54152740 2024-09-17T05:03:34.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.305+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb5406e440 con 0x7feb54152740 2024-09-17T05:03:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.305+0000 7feb51600640 1 -- 172.21.15.16:0/3780948830 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16930 (secure 0 0 0) 0x7feb3c0f9650 con 0x7feb54152740 2024-09-17T05:03:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:34.309 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:19.480354+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:15.637532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:16.849662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb30073200 msgr2=0x7feb30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb30073200 0x7feb30075710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb44000f70 tx=0x7feb44009040 comp rx=0 tx=0).stop 2024-09-17T05:03:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 msgr2=0x7feb54156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 0x7feb54156060 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0b0040 tx=0x7feb3c0a57d0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 shutdown_connections 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb30073200 0x7feb30075710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb54159f30 0x7feb5406d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54152740 0x7feb54156060 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 --2- 172.21.15.16:0/3780948830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb5406e030 0x7feb5414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 >> 172.21.15.16:0/3780948830 conn(0x7feb540d10f0 msgr2=0x7feb5414c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 shutdown_connections 2024-09-17T05:03:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.309+0000 7feb5a000640 1 -- 172.21.15.16:0/3780948830 wait complete. 2024-09-17T05:03:34.332 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:03:34.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 -- 172.21.15.16:0/2996082153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de006dfa0 msgr2=0x7f0de006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 --2- 172.21.15.16:0/2996082153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de006dfa0 0x7f0de006e3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009840 tx=0x7f0dcc01c620 comp rx=0 tx=0).stop 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 -- 172.21.15.16:0/2996082153 shutdown_connections 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 --2- 172.21.15.16:0/2996082153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de006dfa0 0x7f0de006e3b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 -- 172.21.15.16:0/2996082153 >> 172.21.15.16:0/2996082153 conn(0x7f0de00cdb80 msgr2=0x7f0de00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 -- 172.21.15.16:0/2996082153 shutdown_connections 2024-09-17T05:03:34.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.489+0000 7f0de8400640 1 -- 172.21.15.16:0/2996082153 wait complete. 2024-09-17T05:03:34.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 Processor -- start 2024-09-17T05:03:34.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 -- start start 2024-09-17T05:03:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0de006dfa0 0x7f0de00c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 0x7f0de00b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de00cc6f0 0x7f0de00b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0169f50 con 0x7f0de00749d0 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0169dd0 con 0x7f0de00cc6f0 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0de016a0d0 con 0x7f0de006dfa0 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 0x7f0de00b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0de006dfa0 0x7f0de00c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 0x7f0de00b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58502/0 (socket says 172.21.15.16:58502) 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 -- 172.21.15.16:0/587418118 learned_addr learned my addr 172.21.15.16:0/587418118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:34.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de6400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de00cc6f0 0x7f0de00b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 -- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0de006dfa0 msgr2=0x7f0de00c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0de006dfa0 0x7f0de00c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 -- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de00cc6f0 msgr2=0x7f0de00b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de00cc6f0 0x7f0de00b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 -- 172.21.15.16:0/587418118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de00c9b80 con 0x7f0de00749d0 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de5000640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 0x7f0de00b65e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc01acd0 tx=0x7f0dcc005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0ad070 con 0x7f0de00749d0 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dcc0aab20 con 0x7f0de00749d0 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0046d0 con 0x7f0de00749d0 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de00c9e60 con 0x7f0de00749d0 2024-09-17T05:03:34.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.493+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de00ca420 con 0x7f0de00749d0 2024-09-17T05:03:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.497+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de00c98f0 con 0x7f0de00749d0 2024-09-17T05:03:34.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.497+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0dcc0b0070 con 0x7f0de00749d0 2024-09-17T05:03:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.497+0000 7f0ddd600640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0dc0073200 0x7f0dc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.497+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f0dcc12e7c0 con 0x7f0de00749d0 2024-09-17T05:03:34.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.497+0000 7f0de5a00640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0dc0073200 0x7f0dc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.501+0000 7f0de5a00640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0dc0073200 0x7f0dc0075710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40010f0 tx=0x7f0dd4001930 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:34.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.501+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0dcc0b8070 con 0x7f0de00749d0 2024-09-17T05:03:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.713+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0de006e3b0 con 0x7f0de00749d0 2024-09-17T05:03:34.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0ddd600640 1 -- 172.21.15.16:0/587418118 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f0dcc016330 con 0x7f0de00749d0 2024-09-17T05:03:34.719 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:34.719 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:17.889311+0000","modified":"2024-09-17T05:03:20.917921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5460":{"gid":5460,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3845424541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3845424541},{"type":"v1","addr":"172.21.15.16:6834","nonce":3845424541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5460,"qdb_cluster":[5460]},"id":3}]} 2024-09-17T05:03:34.719 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-17T05:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0dc0073200 msgr2=0x7f0dc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0dc0073200 0x7f0dc0075710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40010f0 tx=0x7f0dd4001930 comp rx=0 tx=0).stop 2024-09-17T05:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 msgr2=0x7f0de00b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 0x7f0de00b65e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc01acd0 tx=0x7f0dcc005750 comp rx=0 tx=0).stop 2024-09-17T05:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 shutdown_connections 2024-09-17T05:03:34.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0dc0073200 0x7f0dc0075710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0de00cc6f0 0x7f0de00b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de00749d0 0x7f0de00b65e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 --2- 172.21.15.16:0/587418118 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0de006dfa0 0x7f0de00c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 >> 172.21.15.16:0/587418118 conn(0x7f0de00cdb80 msgr2=0x7f0de01515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 shutdown_connections 2024-09-17T05:03:34.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.717+0000 7f0de8400640 1 -- 172.21.15.16:0/587418118 wait complete. 2024-09-17T05:03:34.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 -- 172.21.15.16:0/1116150893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc406ebb0 msgr2=0x7f3dc40b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 --2- 172.21.15.16:0/1116150893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc406ebb0 0x7f3dc40b4610 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3dc415c740 tx=0x7f3dbc01cd60 comp rx=0 tx=0).stop 2024-09-17T05:03:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 -- 172.21.15.16:0/1116150893 shutdown_connections 2024-09-17T05:03:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 --2- 172.21.15.16:0/1116150893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc406ebb0 0x7f3dc40b4610 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 --2- 172.21.15.16:0/1116150893 >> v2:172.21.15.104:3300/0 conn(0x7f3dc406e7a0 0x7f3dc40b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 --2- 172.21.15.16:0/1116150893 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc4152540 0x7f3dc415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 -- 172.21.15.16:0/1116150893 >> 172.21.15.16:0/1116150893 conn(0x7f3dc40d10c0 msgr2=0x7f3dc40d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 -- 172.21.15.16:0/1116150893 shutdown_connections 2024-09-17T05:03:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.897+0000 7f3dc9800640 1 -- 172.21.15.16:0/1116150893 wait complete. 2024-09-17T05:03:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 Processor -- start 2024-09-17T05:03:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 -- start start 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dc406e7a0 0x7f3dc406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc406ebb0 0x7f3dc413f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 0x7f3dc4152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc415f8f0 con 0x7f3dc4152540 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc415f770 con 0x7f3dc406e7a0 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc415fa70 con 0x7f3dc406ebb0 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 0x7f3dc4152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 0x7f3dc4152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58512/0 (socket says 172.21.15.16:58512) 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 -- 172.21.15.16:0/693832978 learned_addr learned my addr 172.21.15.16:0/693832978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc2a00640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dc406e7a0 0x7f3dc406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dbbe00640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc406ebb0 0x7f3dc413f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 -- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc406ebb0 msgr2=0x7f3dc413f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc406ebb0 0x7f3dc413f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 -- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dc406e7a0 msgr2=0x7f3dc406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dc406e7a0 0x7f3dc406ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 -- 172.21.15.16:0/693832978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc41f94f0 con 0x7f3dc4152540 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dbbe00640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc406ebb0 0x7f3dc413f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc3400640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 0x7f3dc4152a70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc01b410 tx=0x7f3dbc005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0bc070 con 0x7f3dc4152540 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dbc004330 con 0x7f3dc4152540 2024-09-17T05:03:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc4141d90 con 0x7f3dc4152540 2024-09-17T05:03:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0b2040 con 0x7f3dc4152540 2024-09-17T05:03:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.901+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc4142350 con 0x7f3dc4152540 2024-09-17T05:03:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.905+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc4154a50 con 0x7f3dc4152540 2024-09-17T05:03:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.905+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3dbc0bf070 con 0x7f3dc4152540 2024-09-17T05:03:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.905+0000 7f3dbaa00640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3da0073230 0x7f3da0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.905+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f3dbc12df50 con 0x7f3dc4152540 2024-09-17T05:03:34.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.905+0000 7f3dc2a00640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3da0073230 0x7f3da0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:34.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.905+0000 7f3dc2a00640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3da0073230 0x7f3da0075740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3db4004630 tx=0x7f3db400b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:34.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:34.909+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3dbc016950 con 0x7f3dc4152540 2024-09-17T05:03:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc406d6c0 con 0x7f3dc4152540 2024-09-17T05:03:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dbaa00640 1 -- 172.21.15.16:0/693832978 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f3dbc0f8790 con 0x7f3dc4152540 2024-09-17T05:03:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:35.123 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-17T05:03:21:499645+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:17.889311+0000","modified":"2024-09-17T05:03:20.917921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5460":{"gid":5460,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/3845424541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3845424541},{"type":"v1","addr":"172.21.15.16:6834","nonce":3845424541}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5460,"qdb_cluster":[5460]},"id":3}]} 2024-09-17T05:03:35.123 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3da0073230 msgr2=0x7f3da0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3da0073230 0x7f3da0075740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3db4004630 tx=0x7f3db400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 msgr2=0x7f3dc4152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 0x7f3dc4152a70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc01b410 tx=0x7f3dbc005b10 comp rx=0 tx=0).stop 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 shutdown_connections 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3da0073230 0x7f3da0075740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dc4152540 0x7f3dc4152a70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3dc406ebb0 0x7f3dc413f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 --2- 172.21.15.16:0/693832978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dc406e7a0 0x7f3dc406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 >> 172.21.15.16:0/693832978 conn(0x7f3dc40d10c0 msgr2=0x7f3dc4151ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 shutdown_connections 2024-09-17T05:03:35.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.121+0000 7f3dc9800640 1 -- 172.21.15.16:0/693832978 wait complete. 2024-09-17T05:03:35.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 -- 172.21.15.16:0/2149895281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 msgr2=0x7f135c0cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 --2- 172.21.15.16:0/2149895281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 0x7f135c0cc8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f134c009840 tx=0x7f134c01c820 comp rx=0 tx=0).stop 2024-09-17T05:03:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 -- 172.21.15.16:0/2149895281 shutdown_connections 2024-09-17T05:03:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 --2- 172.21.15.16:0/2149895281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 0x7f135c0cc8f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 -- 172.21.15.16:0/2149895281 >> 172.21.15.16:0/2149895281 conn(0x7f135c06f900 msgr2=0x7f135c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 -- 172.21.15.16:0/2149895281 shutdown_connections 2024-09-17T05:03:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.301+0000 7f1362a00640 1 -- 172.21.15.16:0/2149895281 wait complete. 2024-09-17T05:03:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 Processor -- start 2024-09-17T05:03:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 -- start start 2024-09-17T05:03:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f135c06d530 0x7f135c13b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 0x7f135c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 0x7f135c0a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f135c16b3f0 con 0x7f135c0cedc0 2024-09-17T05:03:35.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f135c16b270 con 0x7f135c15d680 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1362a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f135c16b570 con 0x7f135c06d530 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 0x7f135c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 0x7f135c0a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58528/0 (socket says 172.21.15.16:58528) 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 -- 172.21.15.16:0/550092735 learned_addr learned my addr 172.21.15.16:0/550092735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f135be00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f135c06d530 0x7f135c13b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1360a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 0x7f135c0a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 -- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f135c06d530 msgr2=0x7f135c13b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f135c06d530 0x7f135c13b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 -- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 msgr2=0x7f135c0a9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 0x7f135c0a9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.305+0000 7f1353e00640 1 -- 172.21.15.16:0/550092735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c13b9d0 con 0x7f135c0cedc0 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1353e00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 0x7f135c0a9450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f134400c3c0 tx=0x7f134400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344017070 con 0x7f135c0cedc0 2024-09-17T05:03:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13440053c0 con 0x7f135c0cedc0 2024-09-17T05:03:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c156220 con 0x7f135c0cedc0 2024-09-17T05:03:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344003860 con 0x7f135c0cedc0 2024-09-17T05:03:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c1567e0 con 0x7f135c0cedc0 2024-09-17T05:03:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135c155f90 con 0x7f135c0cedc0 2024-09-17T05:03:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f134401a070 con 0x7f135c0cedc0 2024-09-17T05:03:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1358c00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1338073230 0x7f1338075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.309+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1344093550 con 0x7f135c0cedc0 2024-09-17T05:03:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.313+0000 7f135be00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1338073230 0x7f1338075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.313+0000 7f135be00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1338073230 0x7f1338075740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f134c0af000 tx=0x7f134c01b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:35.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.317+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f134405de20 con 0x7f135c0cedc0 2024-09-17T05:03:35.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.493+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f135c06d940 con 0x7f135c0cedc0 2024-09-17T05:03:35.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.493+0000 7f1358c00640 1 -- 172.21.15.16:0/550092735 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16930 (secure 0 0 0) 0x7f1344062850 con 0x7f135c0cedc0 2024-09-17T05:03:35.496 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:35.496 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":53,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:19.480354+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:15.637532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:16.849662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.493+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1338073230 msgr2=0x7f1338075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.493+0000 7f1362a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1338073230 0x7f1338075740 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f134c0af000 tx=0x7f134c01b060 comp rx=0 tx=0).stop 2024-09-17T05:03:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.493+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 msgr2=0x7f135c0a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.493+0000 7f1362a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 0x7f135c0a9450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f134400c3c0 tx=0x7f134400c880 comp rx=0 tx=0).stop 2024-09-17T05:03:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 shutdown_connections 2024-09-17T05:03:35.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1338073230 0x7f1338075740 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f135c15d680 0x7f135c0a9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f135c0cedc0 0x7f135c0a9450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 --2- 172.21.15.16:0/550092735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f135c06d530 0x7f135c13b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 >> 172.21.15.16:0/550092735 conn(0x7f135c06f900 msgr2=0x7f135c0b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 shutdown_connections 2024-09-17T05:03:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.497+0000 7f1362a00640 1 -- 172.21.15.16:0/550092735 wait complete. 2024-09-17T05:03:35.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 -- 172.21.15.16:0/3437155582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978070630 msgr2=0x7fc9780cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 --2- 172.21.15.16:0/3437155582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978070630 0x7fc9780cb760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc960009840 tx=0x7fc96001c7f0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 -- 172.21.15.16:0/3437155582 shutdown_connections 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 --2- 172.21.15.16:0/3437155582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978070630 0x7fc9780cb760 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 -- 172.21.15.16:0/3437155582 >> 172.21.15.16:0/3437155582 conn(0x7fc9780cd980 msgr2=0x7fc9780cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 -- 172.21.15.16:0/3437155582 shutdown_connections 2024-09-17T05:03:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 -- 172.21.15.16:0/3437155582 wait complete. 2024-09-17T05:03:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.677+0000 7fc97f200640 1 Processor -- start 2024-09-17T05:03:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- start start 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc97806ce90 0x7fc9781526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9780cb350 0x7fc9781558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 0x7fc9780aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc97816ae90 con 0x7fc978152d40 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc97816ad10 con 0x7fc9780cb350 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc97816b010 con 0x7fc97806ce90 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 0x7fc9780aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 0x7fc9780aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58552/0 (socket says 172.21.15.16:58552) 2024-09-17T05:03:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 -- 172.21.15.16:0/1368993164 learned_addr learned my addr 172.21.15.16:0/1368993164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc977e00640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc97806ce90 0x7fc9781526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc96fe00640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9780cb350 0x7fc9781558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 -- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc97806ce90 msgr2=0x7fc9781526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc97806ce90 0x7fc9781526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 -- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9780cb350 msgr2=0x7fc9781558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9780cb350 0x7fc9781558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 -- 172.21.15.16:0/1368993164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc978156050 con 0x7fc978152d40 2024-09-17T05:03:35.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97d200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 0x7fc9780aeb10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc97000cb10 tx=0x7fc97000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:35.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970016020 con 0x7fc978152d40 2024-09-17T05:03:35.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc97000ce50 con 0x7fc978152d40 2024-09-17T05:03:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97000ade0 con 0x7fc978152d40 2024-09-17T05:03:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9781fbfc0 con 0x7fc978152d40 2024-09-17T05:03:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9781fc580 con 0x7fc978152d40 2024-09-17T05:03:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.681+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9780cb760 con 0x7fc978152d40 2024-09-17T05:03:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.685+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc970019020 con 0x7fc978152d40 2024-09-17T05:03:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.685+0000 7fc974c00640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc954073200 0x7fc954075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.685+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc970094220 con 0x7fc978152d40 2024-09-17T05:03:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.685+0000 7fc977e00640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc954073200 0x7fc954075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.685+0000 7fc977e00640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc954073200 0x7fc954075710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc968007fd0 tx=0x7fc968009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:35.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.689+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc970020030 con 0x7fc978152d40 2024-09-17T05:03:35.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.897+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc97806a180 con 0x7fc978152d40 2024-09-17T05:03:35.901 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:03:35.897+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:03:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.901+0000 7fc974c00640 1 -- 172.21.15.16:0/1368993164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fc97005e9c0 con 0x7fc978152d40 2024-09-17T05:03:35.903 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:03:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc954073200 msgr2=0x7fc954075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc954073200 0x7fc954075710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc968007fd0 tx=0x7fc968009040 comp rx=0 tx=0).stop 2024-09-17T05:03:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 msgr2=0x7fc9780aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:35.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 0x7fc9780aeb10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc97000cb10 tx=0x7fc97000cc00 comp rx=0 tx=0).stop 2024-09-17T05:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 shutdown_connections 2024-09-17T05:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc954073200 0x7fc954075710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978152d40 0x7fc9780aeb10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9780cb350 0x7fc9781558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 --2- 172.21.15.16:0/1368993164 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc97806ce90 0x7fc9781526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:35.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 >> 172.21.15.16:0/1368993164 conn(0x7fc9780cd980 msgr2=0x7fc978070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 shutdown_connections 2024-09-17T05:03:35.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:35.905+0000 7fc97f200640 1 -- 172.21.15.16:0/1368993164 wait complete. 2024-09-17T05:03:35.913 INFO:tasks.ceph.mds.c.smithi016.stderr: -2317> 2024-09-17T05:03:20.441+0000 7f60cda00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:03:35.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:03:36.033 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 -- 172.21.15.16:0/1483902670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 msgr2=0x7fbad40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 --2- 172.21.15.16:0/1483902670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad40ce5d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbad40ce2c0 tx=0x7fbabc01c820 comp rx=0 tx=0).stop 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 -- 172.21.15.16:0/1483902670 shutdown_connections 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 --2- 172.21.15.16:0/1483902670 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad415f6f0 0x7fbad40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 --2- 172.21.15.16:0/1483902670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad40ce5d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 -- 172.21.15.16:0/1483902670 >> 172.21.15.16:0/1483902670 conn(0x7fbad40b2250 msgr2=0x7fbad40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 -- 172.21.15.16:0/1483902670 shutdown_connections 2024-09-17T05:03:36.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.093+0000 7fbadaa00640 1 -- 172.21.15.16:0/1483902670 wait complete. 2024-09-17T05:03:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 Processor -- start 2024-09-17T05:03:36.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 -- start start 2024-09-17T05:03:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 0x7fbad41599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad41542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad415f6f0 0x7fbad4154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4168ed0 con 0x7fbad4074a20 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbad4168d50 con 0x7fbad4074e30 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbad4169050 con 0x7fbad415f6f0 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 0x7fbad41599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 0x7fbad41599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58572/0 (socket says 172.21.15.16:58572) 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 -- 172.21.15.16:0/2188086827 learned_addr learned my addr 172.21.15.16:0/2188086827 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbacb400640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad41542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:36.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad8a00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad415f6f0 0x7fbad4154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 -- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad415f6f0 msgr2=0x7fbad4154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad415f6f0 0x7fbad4154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 -- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 msgr2=0x7fbad41542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad41542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 -- 172.21.15.16:0/2188086827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad40af360 con 0x7fbad4074a20 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbacb400640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad41542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad3e00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 0x7fbad41599e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbac400c340 tx=0x7fbac400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4017070 con 0x7fbad4074a20 2024-09-17T05:03:36.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbac4003600 con 0x7fbad4074a20 2024-09-17T05:03:36.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad40b3230 con 0x7fbad4074a20 2024-09-17T05:03:36.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.097+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4003a50 con 0x7fbad4074a20 2024-09-17T05:03:36.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.101+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad40b37f0 con 0x7fbad4074a20 2024-09-17T05:03:36.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.101+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad415efe0 con 0x7fbad4074a20 2024-09-17T05:03:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.101+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbac401a070 con 0x7fbad4074a20 2024-09-17T05:03:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.101+0000 7fbad0c00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.101+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6187+0+0 (secure 0 0 0) 0x7fbac4093780 con 0x7fbad4074a20 2024-09-17T05:03:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.101+0000 7fbacb400640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.105+0000 7fbacb400640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0af000 tx=0x7fbabc01b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:36.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.105+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbac405ded0 con 0x7fbad4074a20 2024-09-17T05:03:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.325+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbad40ccb10 con 0x7fbad4074a20 2024-09-17T05:03:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.901+0000 7fbad0c00640 1 -- 172.21.15.16:0/2188086827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fbac4062900 con 0x7fbad4074a20 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 msgr2=0x7fbab0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0af000 tx=0x7fbabc01b060 comp rx=0 tx=0).stop 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 msgr2=0x7fbad41599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 0x7fbad41599e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbac400c340 tx=0x7fbac400c800 comp rx=0 tx=0).stop 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 shutdown_connections 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad415f6f0 0x7fbad4154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad4074e30 0x7fbad41542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 --2- 172.21.15.16:0/2188086827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad4074a20 0x7fbad41599e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 >> 172.21.15.16:0/2188086827 conn(0x7fbad40b2250 msgr2=0x7fbad415de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 shutdown_connections 2024-09-17T05:03:36.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:36.905+0000 7fbadaa00640 1 -- 172.21.15.16:0/2188086827 wait complete. 2024-09-17T05:03:36.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:03:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 -- 172.21.15.16:0/4223599762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0cc0f0 msgr2=0x7fe41c0adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 --2- 172.21.15.16:0/4223599762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0cc0f0 0x7fe41c0adef0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009840 tx=0x7fe40c01c790 comp rx=0 tx=0).stop 2024-09-17T05:03:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 -- 172.21.15.16:0/4223599762 shutdown_connections 2024-09-17T05:03:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 --2- 172.21.15.16:0/4223599762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0cc0f0 0x7fe41c0adef0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 -- 172.21.15.16:0/4223599762 >> 172.21.15.16:0/4223599762 conn(0x7fe41c0d0050 msgr2=0x7fe41c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 -- 172.21.15.16:0/4223599762 shutdown_connections 2024-09-17T05:03:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.093+0000 7fe421c00640 1 -- 172.21.15.16:0/4223599762 wait complete. 2024-09-17T05:03:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 Processor -- start 2024-09-17T05:03:37.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 -- start start 2024-09-17T05:03:37.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 0x7fe41c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:37.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe41c0709f0 0x7fe41c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:37.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c0cc0f0 0x7fe41c151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:37.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c169d60 con 0x7fe41c0705e0 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe41c169be0 con 0x7fe41c0cc0f0 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe421c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe41c169ee0 con 0x7fe41c0709f0 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 0x7fe41c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 0x7fe41c0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41130/0 (socket says 172.21.15.16:41130) 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 -- 172.21.15.16:0/202230867 learned_addr learned my addr 172.21.15.16:0/202230867 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe412000640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe41c0709f0 0x7fe41c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41b400640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c0cc0f0 0x7fe41c151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 -- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe41c0709f0 msgr2=0x7fe41c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe41c0709f0 0x7fe41c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 -- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c0cc0f0 msgr2=0x7fe41c151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c0cc0f0 0x7fe41c151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe41aa00640 1 -- 172.21.15.16:0/202230867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41c0cda10 con 0x7fe41c0705e0 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.097+0000 7fe412000640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe41c0709f0 0x7fe41c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe41aa00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 0x7fe41c0cde10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009790 tx=0x7fe40c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c0be070 con 0x7fe41c0705e0 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe40c0a6560 con 0x7fe41c0705e0 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c0a6af0 con 0x7fe41c0705e0 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41c15b0e0 con 0x7fe41c0705e0 2024-09-17T05:03:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41c14c160 con 0x7fe41c0705e0 2024-09-17T05:03:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe41c0adef0 con 0x7fe41c0705e0 2024-09-17T05:03:37.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.101+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe40c0a6c90 con 0x7fe41c0705e0 2024-09-17T05:03:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.105+0000 7fe413400640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3f8073200 0x7fe3f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.105+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6187+0+0 (secure 0 0 0) 0x7fe40c12e130 con 0x7fe41c0705e0 2024-09-17T05:03:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.105+0000 7fe412000640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3f8073200 0x7fe3f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.105+0000 7fe412000640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3f8073200 0x7fe3f8075710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe41c14e9c0 tx=0x7fe404009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.109+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe40c0ac030 con 0x7fe41c0705e0 2024-09-17T05:03:37.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.285+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe41c1fc0c0 con 0x7fe41c0705e0 2024-09-17T05:03:37.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.909+0000 7fe413400640 1 -- 172.21.15.16:0/202230867 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) ==== 181+0+0 (secure 0 0 0) 0x7fe40c016410 con 0x7fe41c0705e0 2024-09-17T05:03:37.912 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:03:37.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3f8073200 msgr2=0x7fe3f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3f8073200 0x7fe3f8075710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe41c14e9c0 tx=0x7fe404009040 comp rx=0 tx=0).stop 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 msgr2=0x7fe41c0cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 0x7fe41c0cde10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009790 tx=0x7fe40c005db0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 shutdown_connections 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3f8073200 0x7fe3f8075710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe41c0cc0f0 0x7fe41c151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe41c0709f0 0x7fe41c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 --2- 172.21.15.16:0/202230867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe41c0705e0 0x7fe41c0cde10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 >> 172.21.15.16:0/202230867 conn(0x7fe41c0d0050 msgr2=0x7fe41c0cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:37.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 shutdown_connections 2024-09-17T05:03:37.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:37.913+0000 7fe421c00640 1 -- 172.21.15.16:0/202230867 wait complete. 2024-09-17T05:03:37.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 -- 172.21.15.16:0/963317908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 msgr2=0x7f83a015f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 --2- 172.21.15.16:0/963317908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a015f6b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f83a00b2c20 tx=0x7f839801cd50 comp rx=0 tx=0).stop 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 -- 172.21.15.16:0/963317908 shutdown_connections 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 --2- 172.21.15.16:0/963317908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a015f6b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 --2- 172.21.15.16:0/963317908 >> v2:172.21.15.104:3300/0 conn(0x7f83a00b4620 0x7f83a006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 -- 172.21.15.16:0/963317908 >> 172.21.15.16:0/963317908 conn(0x7f83a00d0050 msgr2=0x7f83a00d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:38.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 -- 172.21.15.16:0/963317908 shutdown_connections 2024-09-17T05:03:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 -- 172.21.15.16:0/963317908 wait complete. 2024-09-17T05:03:38.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 Processor -- start 2024-09-17T05:03:38.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.109+0000 7f83a4c00640 1 -- start start 2024-09-17T05:03:38.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83a006e4e0 0x7f83a0157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a009d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83a00b4620 0x7f83a009d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83a015fd70 con 0x7f83a0074bf0 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f83a015fbf0 con 0x7f83a00b4620 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f83a0168920 con 0x7f83a006e4e0 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a009d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a009d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41150/0 (socket says 172.21.15.16:41150) 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 -- 172.21.15.16:0/2064479383 learned_addr learned my addr 172.21.15.16:0/2064479383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839de00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83a006e4e0 0x7f83a0157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839e800640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83a00b4620 0x7f83a009d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 -- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83a006e4e0 msgr2=0x7f83a0157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:38.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83a006e4e0 0x7f83a0157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 -- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83a00b4620 msgr2=0x7f83a009d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83a00b4620 0x7f83a009d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 -- 172.21.15.16:0/2064479383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a01f9d10 con 0x7f83a0074bf0 2024-09-17T05:03:38.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f839d400640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a009d1a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c3f0 tx=0x7f838c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c0052d0 con 0x7f83a0074bf0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f838c007020 con 0x7f83a0074bf0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a01f9fc0 con 0x7f83a0074bf0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c007470 con 0x7f83a0074bf0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a01fa580 con 0x7f83a0074bf0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.113+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a006dfa0 con 0x7f83a0074bf0 2024-09-17T05:03:38.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.117+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f838c0067e0 con 0x7f83a0074bf0 2024-09-17T05:03:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.117+0000 7f8396000640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8380073230 0x7f8380075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.117+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5749+0+0 (secure 0 0 0) 0x7f838c093f90 con 0x7f83a0074bf0 2024-09-17T05:03:38.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.117+0000 7f839de00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8380073230 0x7f8380075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:38.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.117+0000 7f839de00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8380073230 0x7f8380075740 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8390009b40 tx=0x7f83900091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:38.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.121+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f83a006e8f0 con 0x7f83a0074bf0 2024-09-17T05:03:38.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.297+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f83a015bb00 con 0x7f83a0074bf0 2024-09-17T05:03:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.909+0000 7f8396000640 1 -- 172.21.15.16:0/2064479383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) ==== 169+0+0 (secure 0 0 0) 0x7f838c001090 con 0x7f83a0074bf0 2024-09-17T05:03:38.913 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:03:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8380073230 msgr2=0x7f8380075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8380073230 0x7f8380075740 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8390009b40 tx=0x7f83900091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 msgr2=0x7f83a009d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a009d1a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c3f0 tx=0x7f838c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 shutdown_connections 2024-09-17T05:03:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8380073230 0x7f8380075740 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f83a00b4620 0x7f83a009d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a0074bf0 0x7f83a009d1a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 --2- 172.21.15.16:0/2064479383 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f83a006e4e0 0x7f83a0157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:38.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 >> 172.21.15.16:0/2064479383 conn(0x7f83a00d0050 msgr2=0x7f83a006d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 shutdown_connections 2024-09-17T05:03:38.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:38.913+0000 7f83a4c00640 1 -- 172.21.15.16:0/2064479383 wait complete. 2024-09-17T05:03:38.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:03:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.097+0000 7fbfb4200640 1 -- 172.21.15.16:0/946762090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac0cacb0 msgr2=0x7fbfac0cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.097+0000 7fbfb4200640 1 --2- 172.21.15.16:0/946762090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac0cacb0 0x7fbfac0cb0c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009bc0 tx=0x7fbfa401cba0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 -- 172.21.15.16:0/946762090 shutdown_connections 2024-09-17T05:03:39.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 --2- 172.21.15.16:0/946762090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac0cacb0 0x7fbfac0cb0c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 --2- 172.21.15.16:0/946762090 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbfac0b5a00 0x7fbfac15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 -- 172.21.15.16:0/946762090 >> 172.21.15.16:0/946762090 conn(0x7fbfac0b2250 msgr2=0x7fbfac0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 -- 172.21.15.16:0/946762090 shutdown_connections 2024-09-17T05:03:39.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 -- 172.21.15.16:0/946762090 wait complete. 2024-09-17T05:03:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 Processor -- start 2024-09-17T05:03:39.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 -- start start 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac073170 0x7fbfac15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.101+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbfac0b5a00 0x7fbfac15b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 0x7fbfac14f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac169d50 con 0x7fbfac0cacb0 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac169bd0 con 0x7fbfac073170 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbfac169ed0 con 0x7fbfac0b5a00 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 0x7fbfac14f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 0x7fbfac14f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41154/0 (socket says 172.21.15.16:41154) 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 -- 172.21.15.16:0/1821493815 learned_addr learned my addr 172.21.15.16:0/1821493815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb1800640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac073170 0x7fbfac15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb0e00640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbfac0b5a00 0x7fbfac15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 -- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbfac0b5a00 msgr2=0x7fbfac15b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbfac0b5a00 0x7fbfac15b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 -- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac073170 msgr2=0x7fbfac15e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac073170 0x7fbfac15e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 -- 172.21.15.16:0/1821493815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac1f9e60 con 0x7fbfac0cacb0 2024-09-17T05:03:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb1800640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac073170 0x7fbfac15e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:03:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb2200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 0x7fbfac14f550 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40b0040 tx=0x7fbfa4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40b2040 con 0x7fbfac0cacb0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfa4004040 con 0x7fbfac0cacb0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4004430 con 0x7fbfac0cacb0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac15eef0 con 0x7fbfac0cacb0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac15f1d0 con 0x7fbfac0cacb0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.105+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac15ec60 con 0x7fbfac0cacb0 2024-09-17T05:03:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.109+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbfa40bd030 con 0x7fbfac0cacb0 2024-09-17T05:03:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.109+0000 7fbfa9600640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf8c073230 0x7fbf8c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.109+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fbfa412dd10 con 0x7fbfac0cacb0 2024-09-17T05:03:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.109+0000 7fbfb1800640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf8c073230 0x7fbf8c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.109+0000 7fbfb1800640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf8c073230 0x7fbf8c075740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009600 tx=0x7fbfa00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.113+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbfa40c4060 con 0x7fbfac0cacb0 2024-09-17T05:03:39.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.289+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fbfac0cd830 con 0x7fbfac0cacb0 2024-09-17T05:03:39.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfa9600640 1 -- 172.21.15.16:0/1821493815 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7fbfa40a5e20 con 0x7fbfac0cacb0 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf8c073230 msgr2=0x7fbf8c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf8c073230 0x7fbf8c075740 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009600 tx=0x7fbfa00091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 msgr2=0x7fbfac14f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 0x7fbfac14f550 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40b0040 tx=0x7fbfa4003f40 comp rx=0 tx=0).stop 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 shutdown_connections 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf8c073230 0x7fbf8c075740 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfac0cacb0 0x7fbfac14f550 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbfac0b5a00 0x7fbfac15b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 --2- 172.21.15.16:0/1821493815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfac073170 0x7fbfac15e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 >> 172.21.15.16:0/1821493815 conn(0x7fbfac0b2250 msgr2=0x7fbfac0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:39.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 shutdown_connections 2024-09-17T05:03:39.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.293+0000 7fbfb4200640 1 -- 172.21.15.16:0/1821493815 wait complete. 2024-09-17T05:03:39.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-09-17T05:03:39.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 -- 172.21.15.16:0/1688724193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a406ce90 msgr2=0x7f16a40cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 --2- 172.21.15.16:0/1688724193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a406ce90 0x7f16a40cf320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1698009900 tx=0x7f169801c700 comp rx=0 tx=0).stop 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 -- 172.21.15.16:0/1688724193 shutdown_connections 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 --2- 172.21.15.16:0/1688724193 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a40cf860 0x7f16a415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 --2- 172.21.15.16:0/1688724193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a406ce90 0x7f16a40cf320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 -- 172.21.15.16:0/1688724193 >> 172.21.15.16:0/1688724193 conn(0x7f16a40d0860 msgr2=0x7f16a40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 -- 172.21.15.16:0/1688724193 shutdown_connections 2024-09-17T05:03:39.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 -- 172.21.15.16:0/1688724193 wait complete. 2024-09-17T05:03:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.473+0000 7f16ac600640 1 Processor -- start 2024-09-17T05:03:39.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- start start 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a406ce90 0x7f16a414f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 0x7f16a41fc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a415f580 0x7f16a4159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41689e0 con 0x7f16a40cf860 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40b5a80 con 0x7f16a415f580 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f16a4168b60 con 0x7f16a406ce90 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 0x7f16a41fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 0x7f16a41fc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41164/0 (socket says 172.21.15.16:41164) 2024-09-17T05:03:39.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 -- 172.21.15.16:0/798515957 learned_addr learned my addr 172.21.15.16:0/798515957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9c00640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a406ce90 0x7f16a414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16aa600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a415f580 0x7f16a4159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 -- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a406ce90 msgr2=0x7f16a414f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a406ce90 0x7f16a414f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 -- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a415f580 msgr2=0x7f16a4159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a415f580 0x7f16a4159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 -- 172.21.15.16:0/798515957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a4151c00 con 0x7f16a40cf860 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16aa600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a415f580 0x7f16a4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9c00640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a406ce90 0x7f16a414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:39.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a9200640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 0x7f16a41fc710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f169000c9a0 tx=0x7f169000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690007bb0 con 0x7f16a40cf860 2024-09-17T05:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1690007e40 con 0x7f16a40cf860 2024-09-17T05:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000f6b0 con 0x7f16a40cf860 2024-09-17T05:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a41fcc50 con 0x7f16a40cf860 2024-09-17T05:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a4152150 con 0x7f16a40cf860 2024-09-17T05:03:39.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.477+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a40cf320 con 0x7f16a40cf860 2024-09-17T05:03:39.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.481+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f169000f850 con 0x7f16a40cf860 2024-09-17T05:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.481+0000 7f16a1600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1684073200 0x7f1684075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.481+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f1690094870 con 0x7f16a40cf860 2024-09-17T05:03:39.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.481+0000 7f16a9c00640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1684073200 0x7f1684075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:39.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.481+0000 7f16a9c00640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1684073200 0x7f1684075710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f16980af000 tx=0x7f169801af40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:39.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.485+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f16a41fcc50 con 0x7f16a40cf860 2024-09-17T05:03:39.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.733+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f16a4151210 con 0x7f16a40cf860 2024-09-17T05:03:39.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16a1600640 1 -- 172.21.15.16:0/798515957 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v146) ==== 162+0+0 (secure 0 0 0) 0x7f1690014030 con 0x7f16a40cf860 2024-09-17T05:03:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1684073200 msgr2=0x7f1684075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1684073200 0x7f1684075710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f16980af000 tx=0x7f169801af40 comp rx=0 tx=0).stop 2024-09-17T05:03:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 msgr2=0x7f16a41fc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 0x7f16a41fc710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f169000c9a0 tx=0x7f169000ce60 comp rx=0 tx=0).stop 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 shutdown_connections 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1684073200 0x7f1684075710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a415f580 0x7f16a4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a40cf860 0x7f16a41fc710 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 --2- 172.21.15.16:0/798515957 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16a406ce90 0x7f16a414f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 >> 172.21.15.16:0/798515957 conn(0x7f16a40d0860 msgr2=0x7f16a4153a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 shutdown_connections 2024-09-17T05:03:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:39.925+0000 7f16ac600640 1 -- 172.21.15.16:0/798515957 wait complete. 2024-09-17T05:03:39.949 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... skipped 'Require FUSE client to inject client release failure' 2024-09-17T05:03:39.949 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:03:39.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 -- 172.21.15.16:0/1960502140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c003a40 msgr2=0x7f588c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 --2- 172.21.15.16:0/1960502140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c003a40 0x7f588c003eb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f589800c310 tx=0x7f589801cf50 comp rx=0 tx=0).stop 2024-09-17T05:03:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58a7e00640 1 -- 172.21.15.16:0/1960502140 reap_dead start 2024-09-17T05:03:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 -- 172.21.15.16:0/1960502140 shutdown_connections 2024-09-17T05:03:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 -- 172.21.15.16:0/1960502140 >> 172.21.15.16:0/1960502140 conn(0x7f58a806f900 msgr2=0x7f58a80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 -- 172.21.15.16:0/1960502140 shutdown_connections 2024-09-17T05:03:40.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 -- 172.21.15.16:0/1960502140 wait complete. 2024-09-17T05:03:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 Processor -- start 2024-09-17T05:03:40.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.109+0000 7f58ae600640 1 -- start start 2024-09-17T05:03:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c003a40 0x7f58a813ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 0x7f58a815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a815aee0 0x7f58a8164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8167670 con 0x7f58a813f6f0 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81674f0 con 0x7f58a815aee0 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f58a81677f0 con 0x7f588c003a40 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 0x7f58a815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 0x7f58a815a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41190/0 (socket says 172.21.15.16:41190) 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 -- 172.21.15.16:0/3441769728 learned_addr learned my addr 172.21.15.16:0/3441769728 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58a7400640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c003a40 0x7f58a813ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58a7e00640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a815aee0 0x7f58a8164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 -- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c003a40 msgr2=0x7f58a813ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c003a40 0x7f58a813ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 -- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a815aee0 msgr2=0x7f58a8164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a815aee0 0x7f58a8164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 -- 172.21.15.16:0/3441769728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a81f8880 con 0x7f58a813f6f0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58a7400640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c003a40 0x7f58a813ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589fe00640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 0x7f58a815a9a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5890003ef0 tx=0x7f5890001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890011640 con 0x7f58a813f6f0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5890016020 con 0x7f58a813f6f0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a81f8590 con 0x7f58a813f6f0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890011a80 con 0x7f58a813f6f0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81f8e40 con 0x7f58a813f6f0 2024-09-17T05:03:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.113+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a80cf0b0 con 0x7f58a813f6f0 2024-09-17T05:03:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.117+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5890011c20 con 0x7f58a813f6f0 2024-09-17T05:03:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.117+0000 7f589f400640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073200 0x7f5884075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.117+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f5890094dd0 con 0x7f58a813f6f0 2024-09-17T05:03:40.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.117+0000 7f58a7400640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073200 0x7f5884075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.117+0000 7f58a7400640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073200 0x7f5884075710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f58980ac000 tx=0x7f5898019da0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.121+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5890020020 con 0x7f58a813f6f0 2024-09-17T05:03:40.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.297+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58a815dc00 con 0x7f58a813f6f0 2024-09-17T05:03:40.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f589f400640 1 -- 172.21.15.16:0/3441769728 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+13293 (secure 0 0 0) 0x7f589005ffe0 con 0x7f58a813f6f0 2024-09-17T05:03:40.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:40.303 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:38.912899+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/3845424541":"2024-09-18T05:03:35.902877+0000","172.21.15.16:6832/3845424541":"2024-09-18T05:03:35.902877+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073200 msgr2=0x7f5884075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073200 0x7f5884075710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f58980ac000 tx=0x7f5898019da0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 msgr2=0x7f58a815a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 0x7f58a815a9a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5890003ef0 tx=0x7f5890001640 comp rx=0 tx=0).stop 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 shutdown_connections 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5884073200 0x7f5884075710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58a815aee0 0x7f58a8164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a813f6f0 0x7f58a815a9a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 --2- 172.21.15.16:0/3441769728 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f588c003a40 0x7f58a813ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 >> 172.21.15.16:0/3441769728 conn(0x7f58a806f900 msgr2=0x7f58a80b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 shutdown_connections 2024-09-17T05:03:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.301+0000 7f58ae600640 1 -- 172.21.15.16:0/3441769728 wait complete. 2024-09-17T05:03:40.325 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:03:40.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 -- 172.21.15.16:0/4074664332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d015f520 msgr2=0x7fd3d015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 --2- 172.21.15.16:0/4074664332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d015f520 0x7fd3d015c160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009bc0 tx=0x7fd3c801cba0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 -- 172.21.15.16:0/4074664332 shutdown_connections 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 --2- 172.21.15.16:0/4074664332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d015f520 0x7fd3d015c160 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 --2- 172.21.15.16:0/4074664332 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3d0074a20 0x7fd3d015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 -- 172.21.15.16:0/4074664332 >> 172.21.15.16:0/4074664332 conn(0x7fd3d00b2250 msgr2=0x7fd3d00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:40.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 -- 172.21.15.16:0/4074664332 shutdown_connections 2024-09-17T05:03:40.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 -- 172.21.15.16:0/4074664332 wait complete. 2024-09-17T05:03:40.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 Processor -- start 2024-09-17T05:03:40.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.489+0000 7fd3d6400640 1 -- start start 2024-09-17T05:03:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3d0074a20 0x7fd3d0153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d0074e30 0x7fd3d01511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 0x7fd3d0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0168f30 con 0x7fd3d015f520 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0168db0 con 0x7fd3d0074e30 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd3d01690b0 con 0x7fd3d0074a20 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 0x7fd3d0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 0x7fd3d0151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41200/0 (socket says 172.21.15.16:41200) 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 -- 172.21.15.16:0/2899545260 learned_addr learned my addr 172.21.15.16:0/2899545260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cf400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3d0074a20 0x7fd3d0153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3c7e00640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d0074e30 0x7fd3d01511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 -- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3d0074a20 msgr2=0x7fd3d0153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3d0074a20 0x7fd3d0153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 -- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d0074e30 msgr2=0x7fd3d01511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d0074e30 0x7fd3d01511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 -- 172.21.15.16:0/2899545260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d01fc700 con 0x7fd3d015f520 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3cfe00640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 0x7fd3d0151730 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd3c801a7d0 tx=0x7fd3c8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80bb070 con 0x7fd3d015f520 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3c8004d40 con 0x7fd3d015f520 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d01fc9e0 con 0x7fd3d015f520 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80b2040 con 0x7fd3d015f520 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d0150cb0 con 0x7fd3d015f520 2024-09-17T05:03:40.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.493+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d015b390 con 0x7fd3d015f520 2024-09-17T05:03:40.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.497+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd3c80be070 con 0x7fd3d015f520 2024-09-17T05:03:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.497+0000 7fd3c7400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd3ac073230 0x7fd3ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.497+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fd3c812dbd0 con 0x7fd3d015f520 2024-09-17T05:03:40.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.497+0000 7fd3cf400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd3ac073230 0x7fd3ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.497+0000 7fd3cf400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd3ac073230 0x7fd3ac075740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009b40 tx=0x7fd3c00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:40.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.501+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd3c8016790 con 0x7fd3d015f520 2024-09-17T05:03:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.677+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd3d0151c90 con 0x7fd3d015f520 2024-09-17T05:03:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.677+0000 7fd3c7400640 1 -- 172.21.15.16:0/2899545260 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7fd3c80f8700 con 0x7fd3d015f520 2024-09-17T05:03:40.680 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:03:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd3ac073230 msgr2=0x7fd3ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd3ac073230 0x7fd3ac075740 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009b40 tx=0x7fd3c00091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 msgr2=0x7fd3d0151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 0x7fd3d0151730 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd3c801a7d0 tx=0x7fd3c8003f60 comp rx=0 tx=0).stop 2024-09-17T05:03:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 shutdown_connections 2024-09-17T05:03:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd3ac073230 0x7fd3ac075740 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3d015f520 0x7fd3d0151730 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d0074e30 0x7fd3d01511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 --2- 172.21.15.16:0/2899545260 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd3d0074a20 0x7fd3d0153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 >> 172.21.15.16:0/2899545260 conn(0x7fd3d00b2250 msgr2=0x7fd3d00ca900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 shutdown_connections 2024-09-17T05:03:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.681+0000 7fd3d6400640 1 -- 172.21.15.16:0/2899545260 wait complete. 2024-09-17T05:03:40.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-09-17T05:03:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 -- 172.21.15.16:0/724429017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00cc8c0 msgr2=0x7f8de015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 --2- 172.21.15.16:0/724429017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00cc8c0 0x7f8de015cab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8de00ce800 tx=0x7f8dd401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 -- 172.21.15.16:0/724429017 shutdown_connections 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 --2- 172.21.15.16:0/724429017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de00d0010 0x7f8de00d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 --2- 172.21.15.16:0/724429017 >> v2:172.21.15.104:3300/0 conn(0x7f8de00d1580 0x7f8de00d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 --2- 172.21.15.16:0/724429017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00cc8c0 0x7f8de015cab0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 -- 172.21.15.16:0/724429017 >> 172.21.15.16:0/724429017 conn(0x7f8de00d2270 msgr2=0x7f8de00d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 -- 172.21.15.16:0/724429017 shutdown_connections 2024-09-17T05:03:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 -- 172.21.15.16:0/724429017 wait complete. 2024-09-17T05:03:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 Processor -- start 2024-09-17T05:03:40.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 -- start start 2024-09-17T05:03:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de00cc8c0 0x7f8de015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.861+0000 7f8de7800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de00d0010 0x7f8de0152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 0x7f8de01fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8de00ca120 con 0x7f8de00d1580 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8de00c9fa0 con 0x7f8de00cc8c0 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8de00ca2a0 con 0x7f8de00d0010 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 0x7f8de01fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 0x7f8de01fa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41224/0 (socket says 172.21.15.16:41224) 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 -- 172.21.15.16:0/3051097134 learned_addr learned my addr 172.21.15.16:0/3051097134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de4e00640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de00cc8c0 0x7f8de015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8ddfe00640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de00d0010 0x7f8de0152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 -- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de00d0010 msgr2=0x7f8de0152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de00d0010 0x7f8de0152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 -- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de00cc8c0 msgr2=0x7f8de015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de00cc8c0 0x7f8de015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 -- 172.21.15.16:0/3051097134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de015d130 con 0x7f8de00d1580 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de5800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 0x7f8de01fa2c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e970 tx=0x7f8dd800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8004d30 con 0x7f8de00d1580 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dd800cb20 con 0x7f8de00d1580 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de015d410 con 0x7f8de00d1580 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8005020 con 0x7f8de00d1580 2024-09-17T05:03:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de015d9d0 con 0x7f8de00d1580 2024-09-17T05:03:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de00d0420 con 0x7f8de00d1580 2024-09-17T05:03:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.865+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8dd80051c0 con 0x7f8de00d1580 2024-09-17T05:03:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.869+0000 7f8ddcc00640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dc0073200 0x7f8dc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.869+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f8dd8094970 con 0x7f8de00d1580 2024-09-17T05:03:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.869+0000 7f8de4e00640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dc0073200 0x7f8dc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:40.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.869+0000 7f8de4e00640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dc0073200 0x7f8dc0075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40af000 tx=0x7f8dd40a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:40.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:40.873+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8dd8016030 con 0x7f8de00d1580 2024-09-17T05:03:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.109+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f8de0070ec0 con 0x7f8de00d1580 2024-09-17T05:03:41.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.933+0000 7f8ddcc00640 1 -- 172.21.15.16:0/3051097134 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v148) ==== 153+0+0 (secure 0 0 0) 0x7f8dd805f4a0 con 0x7f8de00d1580 2024-09-17T05:03:41.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.933+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dc0073200 msgr2=0x7f8dc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:41.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.933+0000 7f8de7800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dc0073200 0x7f8dc0075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40af000 tx=0x7f8dd40a7040 comp rx=0 tx=0).stop 2024-09-17T05:03:41.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.933+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 msgr2=0x7f8de01fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:41.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.933+0000 7f8de7800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 0x7f8de01fa2c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800e970 tx=0x7f8dd800ee30 comp rx=0 tx=0).stop 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 shutdown_connections 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dc0073200 0x7f8dc0075710 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00d1580 0x7f8de01fa2c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de00d0010 0x7f8de0152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 --2- 172.21.15.16:0/3051097134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de00cc8c0 0x7f8de015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 >> 172.21.15.16:0/3051097134 conn(0x7f8de00d2270 msgr2=0x7f8de015bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 shutdown_connections 2024-09-17T05:03:41.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:41.937+0000 7f8de7800640 1 -- 172.21.15.16:0/3051097134 wait complete. 2024-09-17T05:03:41.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 -- 172.21.15.16:0/1244878444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3015c0a0 msgr2=0x7f6e3015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 --2- 172.21.15.16:0/1244878444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3015c0a0 0x7f6e3015c4b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009900 tx=0x7f6e1801c850 comp rx=0 tx=0).stop 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 -- 172.21.15.16:0/1244878444 shutdown_connections 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 --2- 172.21.15.16:0/1244878444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e3015c0a0 0x7f6e3015c4b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 --2- 172.21.15.16:0/1244878444 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e300b2f50 0x7f6e3015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 -- 172.21.15.16:0/1244878444 >> 172.21.15.16:0/1244878444 conn(0x7f6e3006f900 msgr2=0x7f6e300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:42.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 -- 172.21.15.16:0/1244878444 shutdown_connections 2024-09-17T05:03:42.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 -- 172.21.15.16:0/1244878444 wait complete. 2024-09-17T05:03:42.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 Processor -- start 2024-09-17T05:03:42.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.125+0000 7f6e34e00640 1 -- start start 2024-09-17T05:03:42.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 0x7f6e3014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3015c0a0 0x7f6e30150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e3015f150 0x7f6e30147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30169150 con 0x7f6e300b2f50 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30168fd0 con 0x7f6e3015c0a0 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6e301692d0 con 0x7f6e3015f150 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 0x7f6e3014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 0x7f6e3014c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41250/0 (socket says 172.21.15.16:41250) 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 -- 172.21.15.16:0/4282608579 learned_addr learned my addr 172.21.15.16:0/4282608579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e25400640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3015c0a0 0x7f6e30150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2e800640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e3015f150 0x7f6e30147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 -- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e3015f150 msgr2=0x7f6e30147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e3015f150 0x7f6e30147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 -- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3015c0a0 msgr2=0x7f6e30150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3015c0a0 0x7f6e30150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 -- 172.21.15.16:0/4282608579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3014de90 con 0x7f6e300b2f50 2024-09-17T05:03:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e25400640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3015c0a0 0x7f6e30150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2e800640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e3015f150 0x7f6e30147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:42.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e2de00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 0x7f6e3014c640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000daf0 tx=0x7f6e2000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20017070 con 0x7f6e300b2f50 2024-09-17T05:03:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e20004b30 con 0x7f6e300b2f50 2024-09-17T05:03:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e301fa4e0 con 0x7f6e300b2f50 2024-09-17T05:03:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20005020 con 0x7f6e300b2f50 2024-09-17T05:03:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e301fa780 con 0x7f6e300b2f50 2024-09-17T05:03:42.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.129+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3015bb60 con 0x7f6e300b2f50 2024-09-17T05:03:42.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.133+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6e200051c0 con 0x7f6e300b2f50 2024-09-17T05:03:42.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.133+0000 7f6e26a00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e0c073200 0x7f6e0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.133+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f6e200945f0 con 0x7f6e300b2f50 2024-09-17T05:03:42.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.133+0000 7f6e25400640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e0c073200 0x7f6e0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.133+0000 7f6e25400640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e0c073200 0x7f6e0c075710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e180af000 tx=0x7f6e1801b090 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:42.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.137+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6e20020070 con 0x7f6e300b2f50 2024-09-17T05:03:42.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.313+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f6e3015d280 con 0x7f6e300b2f50 2024-09-17T05:03:42.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.313+0000 7f6e26a00640 1 -- 172.21.15.16:0/4282608579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7f6e2005f770 con 0x7f6e300b2f50 2024-09-17T05:03:42.316 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:03:42.316 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:03:42.316 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:03:42.316 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:03:42.316 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:03:42.317 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:03:42.318 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:03:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:03:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:03:42.319 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e0c073200 msgr2=0x7f6e0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e0c073200 0x7f6e0c075710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6e180af000 tx=0x7f6e1801b090 comp rx=0 tx=0).stop 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 msgr2=0x7f6e3014c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 0x7f6e3014c640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000daf0 tx=0x7f6e2000b260 comp rx=0 tx=0).stop 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 shutdown_connections 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e0c073200 0x7f6e0c075710 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e3015f150 0x7f6e30147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3015c0a0 0x7f6e30150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 --2- 172.21.15.16:0/4282608579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e300b2f50 0x7f6e3014c640 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 >> 172.21.15.16:0/4282608579 conn(0x7f6e3006f900 msgr2=0x7f6e3015e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 shutdown_connections 2024-09-17T05:03:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.317+0000 7f6e34e00640 1 -- 172.21.15.16:0/4282608579 wait complete. 2024-09-17T05:03:42.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:03:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 -- 172.21.15.16:0/1564557223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df015c490 msgr2=0x7f6df0073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 --2- 172.21.15.16:0/1564557223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df015c490 0x7f6df0073dc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009840 tx=0x7f6dd801c820 comp rx=0 tx=0).stop 2024-09-17T05:03:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 -- 172.21.15.16:0/1564557223 shutdown_connections 2024-09-17T05:03:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 --2- 172.21.15.16:0/1564557223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df015c490 0x7f6df0073dc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 -- 172.21.15.16:0/1564557223 >> 172.21.15.16:0/1564557223 conn(0x7f6df00b2dd0 msgr2=0x7f6df00b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 -- 172.21.15.16:0/1564557223 shutdown_connections 2024-09-17T05:03:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.497+0000 7f6df6e00640 1 -- 172.21.15.16:0/1564557223 wait complete. 2024-09-17T05:03:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 Processor -- start 2024-09-17T05:03:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 -- start start 2024-09-17T05:03:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6df006f5f0 0x7f6df0148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 0x7f6df01f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6df015c490 0x7f6df014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6df016b220 con 0x7f6df0074300 2024-09-17T05:03:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6df016b0a0 con 0x7f6df015c490 2024-09-17T05:03:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df6e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6df016b3a0 con 0x7f6df006f5f0 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6de7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 0x7f6df01f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6de7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 0x7f6df01f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41266/0 (socket says 172.21.15.16:41266) 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6de7400640 1 -- 172.21.15.16:0/46016779 learned_addr learned my addr 172.21.15.16:0/46016779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6df4e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6df015c490 0x7f6df014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.501+0000 7f6defe00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6df006f5f0 0x7f6df0148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6de7400640 1 -- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6df006f5f0 msgr2=0x7f6df0148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6de7400640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6df006f5f0 0x7f6df0148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6de7400640 1 -- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6df015c490 msgr2=0x7f6df014da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6de7400640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6df015c490 0x7f6df014da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6de7400640 1 -- 172.21.15.16:0/46016779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df015bda0 con 0x7f6df0074300 2024-09-17T05:03:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6de7400640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 0x7f6df01f9730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80b0040 tx=0x7f6dd80a6f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004240 con 0x7f6df0074300 2024-09-17T05:03:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dd80bf070 con 0x7f6df0074300 2024-09-17T05:03:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df0153b90 con 0x7f6df0074300 2024-09-17T05:03:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80044f0 con 0x7f6df0074300 2024-09-17T05:03:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df01f8f30 con 0x7f6df0074300 2024-09-17T05:03:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df01f8d20 con 0x7f6df0074300 2024-09-17T05:03:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.505+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6dd8003860 con 0x7f6df0074300 2024-09-17T05:03:42.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.509+0000 7f6decc00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dcc073200 0x7f6dcc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:42.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.509+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f6dd812ec70 con 0x7f6df0074300 2024-09-17T05:03:42.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.509+0000 7f6defe00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dcc073200 0x7f6dcc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.509+0000 7f6defe00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dcc073200 0x7f6dcc075710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6de0001e40 tx=0x7f6de0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.513+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6dd80c8020 con 0x7f6df0074300 2024-09-17T05:03:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.689+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6df0073dc0 con 0x7f6df0074300 2024-09-17T05:03:42.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.693+0000 7f6decc00640 1 -- 172.21.15.16:0/46016779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f6dd8016410 con 0x7f6df0074300 2024-09-17T05:03:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dcc073200 msgr2=0x7f6dcc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dcc073200 0x7f6dcc075710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6de0001e40 tx=0x7f6de0009040 comp rx=0 tx=0).stop 2024-09-17T05:03:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 msgr2=0x7f6df01f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 0x7f6df01f9730 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6dd80b0040 tx=0x7f6dd80a6f90 comp rx=0 tx=0).stop 2024-09-17T05:03:42.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 shutdown_connections 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dcc073200 0x7f6dcc075710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6df015c490 0x7f6df014da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0074300 0x7f6df01f9730 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 --2- 172.21.15.16:0/46016779 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6df006f5f0 0x7f6df0148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 >> 172.21.15.16:0/46016779 conn(0x7f6df00b2dd0 msgr2=0x7f6df015a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 shutdown_connections 2024-09-17T05:03:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:42.697+0000 7f6df6e00640 1 -- 172.21.15.16:0/46016779 wait complete. 2024-09-17T05:03:42.719 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:03:42.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:42.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:03:42.923 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:03:42.923 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:42.924 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:42.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:03:42.933 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:42.933 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:42.943 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:03:42.943 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:42.944 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:42.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:03:42.982 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:03:42.982 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:42.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:03:42.997 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:03:42.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 -- 172.21.15.16:0/1358968439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce0073240 msgr2=0x7fdce0073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 --2- 172.21.15.16:0/1358968439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce0073240 0x7fdce0073650 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdce0072860 tx=0x7fdcd801ca60 comp rx=0 tx=0).stop 2024-09-17T05:03:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 -- 172.21.15.16:0/1358968439 shutdown_connections 2024-09-17T05:03:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 --2- 172.21.15.16:0/1358968439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce0073240 0x7fdce0073650 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 --2- 172.21.15.16:0/1358968439 >> v2:172.21.15.104:3300/0 conn(0x7fdce0074b00 0x7fdce00b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 -- 172.21.15.16:0/1358968439 >> 172.21.15.16:0/1358968439 conn(0x7fdce006f900 msgr2=0x7fdce00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 -- 172.21.15.16:0/1358968439 shutdown_connections 2024-09-17T05:03:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.181+0000 7fdce5400640 1 -- 172.21.15.16:0/1358968439 wait complete. 2024-09-17T05:03:43.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 Processor -- start 2024-09-17T05:03:43.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 -- start start 2024-09-17T05:03:43.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce0073240 0x7fdce0148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce0074b00 0x7fdce01645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 0x7fdce0164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0168bb0 con 0x7fdce015f080 2024-09-17T05:03:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0168a30 con 0x7fdce0073240 2024-09-17T05:03:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdce0168d30 con 0x7fdce0074b00 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 0x7fdce0164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 0x7fdce0164b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41274/0 (socket says 172.21.15.16:41274) 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 -- 172.21.15.16:0/1438479963 learned_addr learned my addr 172.21.15.16:0/1438479963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcde800640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce0073240 0x7fdce0148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcd7e00640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce0074b00 0x7fdce01645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 -- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce0074b00 msgr2=0x7fdce01645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce0074b00 0x7fdce01645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 -- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce0073240 msgr2=0x7fdce0148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce0073240 0x7fdce0148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 -- 172.21.15.16:0/1438479963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce00cfae0 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcde800640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce0073240 0x7fdce0148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcdf200640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 0x7fdce0164b00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009b10 tx=0x7fdcd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b2040 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd8004200 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8004630 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce00b0d30 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.185+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce00b1810 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.189+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce0073650 con 0x7fdce015f080 2024-09-17T05:03:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.189+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdcd80ad440 con 0x7fdce015f080 2024-09-17T05:03:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.189+0000 7fdcd6a00640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073200 0x7fdcbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.189+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7fdcd812e790 con 0x7fdce015f080 2024-09-17T05:03:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.189+0000 7fdcde800640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073200 0x7fdcbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.189+0000 7fdcde800640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073200 0x7fdcbc075710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009b90 tx=0x7fdcd00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:43.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.193+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdcd8016770 con 0x7fdce015f080 2024-09-17T05:03:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.401+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdce006d720 con 0x7fdce015f080 2024-09-17T05:03:43.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.401+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1438479963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3911 (secure 0 0 0) 0x7fdcd812e090 con 0x7fdce015f080 2024-09-17T05:03:43.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:43.406 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"btime":"2024-09-17T05:03:36:905276+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4997,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/462457506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":462457506},{"type":"v1","addr":"172.21.15.104:6835","nonce":462457506}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5317,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/3766633919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":3766633919},{"type":"v1","addr":"172.21.15.104:6837","nonce":3766633919}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5323,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4162445006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4162445006},{"type":"v1","addr":"172.21.15.16:6835","nonce":4162445006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/692427110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":692427110},{"type":"v1","addr":"172.21.15.16:6834","nonce":692427110}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-09-17T05:03:43.406 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 28 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073200 msgr2=0x7fdcbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073200 0x7fdcbc075710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009b90 tx=0x7fdcd00091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 msgr2=0x7fdce0164b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 0x7fdce0164b00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009b10 tx=0x7fdcd8009260 comp rx=0 tx=0).stop 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 shutdown_connections 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073200 0x7fdcbc075710 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015f080 0x7fdce0164b00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce0074b00 0x7fdce01645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 --2- 172.21.15.16:0/1438479963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce0073240 0x7fdce0148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 >> 172.21.15.16:0/1438479963 conn(0x7fdce006f900 msgr2=0x7fdce014b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 shutdown_connections 2024-09-17T05:03:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.405+0000 7fdce5400640 1 -- 172.21.15.16:0/1438479963 wait complete. 2024-09-17T05:03:43.428 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:03:43.428 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:03:43.428 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:03:43.428 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:43.429 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:03:43.429 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:03:43.429 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:03:43.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:43.429 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:03:43.429 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:03:43.429 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:03:43.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:43.430 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:03:43.430 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:03:43.430 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:03:43.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:03:43.441 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:03:43.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:03:43.446 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:03:43.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:03:43.449 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:03:43.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:03:43.450 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:03:43.450 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:03:43.491 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:03:43.492 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:43.494 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:03:43.494 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:43.495 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:03:43.495 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:43.496 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:03:43.497 DEBUG:teuthology.parallel:result is None 2024-09-17T05:03:43.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:03:43.593 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:03:43.596 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:03:43.601 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:03:43.602 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:03:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 -- 172.21.15.16:0/3168160094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 msgr2=0x7f4a700cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 --2- 172.21.15.16:0/3168160094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700cf870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009bc0 tx=0x7f4a6801cba0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 -- 172.21.15.16:0/3168160094 shutdown_connections 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 --2- 172.21.15.16:0/3168160094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700cf870 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 --2- 172.21.15.16:0/3168160094 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a700caf50 0x7f4a700cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 -- 172.21.15.16:0/3168160094 >> 172.21.15.16:0/3168160094 conn(0x7f4a700cd980 msgr2=0x7f4a700cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 -- 172.21.15.16:0/3168160094 shutdown_connections 2024-09-17T05:03:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 -- 172.21.15.16:0/3168160094 wait complete. 2024-09-17T05:03:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 Processor -- start 2024-09-17T05:03:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.661+0000 7f4a74a00640 1 -- start start 2024-09-17T05:03:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a700caf50 0x7f4a70150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a7015e990 0x7f4a70070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7016c5d0 con 0x7f4a700cf460 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7016c450 con 0x7f4a700caf50 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7016c750 con 0x7f4a7015e990 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41306/0 (socket says 172.21.15.16:41306) 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 -- 172.21.15.16:0/2798341924 learned_addr learned my addr 172.21.15.16:0/2798341924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a6de00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a700caf50 0x7f4a70150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a6e800640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a7015e990 0x7f4a70070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 -- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a7015e990 msgr2=0x7f4a70070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a7015e990 0x7f4a70070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 -- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a700caf50 msgr2=0x7f4a70150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a700caf50 0x7f4a70150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 -- 172.21.15.16:0/2798341924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a70151ed0 con 0x7f4a700cf460 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a67e00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700705f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800e710 tx=0x7f4a5800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800cab0 con 0x7f4a700cf460 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a5800cc50 con 0x7f4a700cf460 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a700b41e0 con 0x7f4a700cf460 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58005280 con 0x7f4a700cf460 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a700b4480 con 0x7f4a700cf460 2024-09-17T05:03:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.665+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a700b65c0 con 0x7f4a700cf460 2024-09-17T05:03:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.669+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a580047b0 con 0x7f4a700cf460 2024-09-17T05:03:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.669+0000 7f4a66000640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a4c073200 0x7f4a4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.669+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5335+0+0 (secure 0 0 0) 0x7f4a58094b40 con 0x7f4a700cf460 2024-09-17T05:03:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.669+0000 7f4a6de00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a4c073200 0x7f4a4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.669+0000 7f4a6de00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a4c073200 0x7f4a4c075710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007d70 tx=0x7f4a60007d00 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:43.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.673+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4a70151ed0 con 0x7f4a700cf460 2024-09-17T05:03:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.861+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4a700cb360 con 0x7f4a700cf460 2024-09-17T05:03:43.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.945+0000 7f4a66000640 1 -- 172.21.15.16:0/2798341924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) ==== 97+0+0 (secure 0 0 0) 0x7f4a5805f5e0 con 0x7f4a700cf460 2024-09-17T05:03:43.949 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:03:43.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a4c073200 msgr2=0x7f4a4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a4c073200 0x7f4a4c075710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a60007d70 tx=0x7f4a60007d00 comp rx=0 tx=0).stop 2024-09-17T05:03:43.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 msgr2=0x7f4a700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:43.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700705f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800e710 tx=0x7f4a5800ebd0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 shutdown_connections 2024-09-17T05:03:43.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a4c073200 0x7f4a4c075710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a7015e990 0x7f4a70070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a700cf460 0x7f4a700705f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 --2- 172.21.15.16:0/2798341924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a700caf50 0x7f4a70150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:43.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 >> 172.21.15.16:0/2798341924 conn(0x7f4a700cd980 msgr2=0x7f4a7015df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:43.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 shutdown_connections 2024-09-17T05:03:43.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:43.949+0000 7f4a74a00640 1 -- 172.21.15.16:0/2798341924 wait complete. 2024-09-17T05:03:43.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:03:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 -- 172.21.15.16:0/661128426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c15d410 msgr2=0x7f642c15ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- 172.21.15.16:0/661128426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c15d410 0x7f642c15ff40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f642c0cf400 tx=0x7f642001c740 comp rx=0 tx=0).stop 2024-09-17T05:03:44.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 -- 172.21.15.16:0/661128426 shutdown_connections 2024-09-17T05:03:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- 172.21.15.16:0/661128426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f642c074e40 0x7f642c15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- 172.21.15.16:0/661128426 >> v2:172.21.15.104:3300/0 conn(0x7f642c160480 0x7f642c074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- 172.21.15.16:0/661128426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c15d410 0x7f642c15ff40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 -- 172.21.15.16:0/661128426 >> 172.21.15.16:0/661128426 conn(0x7f642c073de0 msgr2=0x7f642c0741f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 -- 172.21.15.16:0/661128426 shutdown_connections 2024-09-17T05:03:44.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 -- 172.21.15.16:0/661128426 wait complete. 2024-09-17T05:03:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 Processor -- start 2024-09-17T05:03:44.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 -- start start 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f642c074e40 0x7f642c1fc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f642c15d410 0x7f642c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.133+0000 7f6434400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 0x7f642c1468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6434400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f642c0d0960 con 0x7f642c160480 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6434400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f642c0d07e0 con 0x7f642c074e40 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6434400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f642c0d0ae0 con 0x7f642c15d410 2024-09-17T05:03:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 0x7f642c1468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 0x7f642c1468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41330/0 (socket says 172.21.15.16:41330) 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 -- 172.21.15.16:0/1206689356 learned_addr learned my addr 172.21.15.16:0/1206689356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6431a00640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f642c074e40 0x7f642c1fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6431000640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f642c15d410 0x7f642c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 -- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f642c15d410 msgr2=0x7f642c1565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f642c15d410 0x7f642c1565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 -- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f642c074e40 msgr2=0x7f642c1fc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f642c074e40 0x7f642c1fc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 -- 172.21.15.16:0/1206689356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c0b5cc0 con 0x7f642c160480 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6432400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 0x7f642c1468a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f642400e990 tx=0x7f642400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424004d30 con 0x7f642c160480 2024-09-17T05:03:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f642400caa0 con 0x7f642c160480 2024-09-17T05:03:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c0b5fa0 con 0x7f642c160480 2024-09-17T05:03:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424005020 con 0x7f642c160480 2024-09-17T05:03:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c06dfe0 con 0x7f642c160480 2024-09-17T05:03:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.137+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c15ff40 con 0x7f642c160480 2024-09-17T05:03:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.141+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f64240051c0 con 0x7f642c160480 2024-09-17T05:03:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.141+0000 7f6429600640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f640c073230 0x7f640c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.141+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7f6424094c90 con 0x7f642c160480 2024-09-17T05:03:44.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.141+0000 7f6431a00640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f640c073230 0x7f640c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.141+0000 7f6431a00640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f640c073230 0x7f640c075740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64200af000 tx=0x7f64200a8040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:44.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.145+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6424016030 con 0x7f642c160480 2024-09-17T05:03:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f642c072770 con 0x7f642c160480 2024-09-17T05:03:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6429600640 1 -- 172.21.15.16:0/1206689356 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) ==== 80+0+5304 (secure 0 0 0) 0x7f642405f800 con 0x7f642c160480 2024-09-17T05:03:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:44.387 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:03:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:03:44.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:03:44.388 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.389 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.390 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:03:44.391 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:03:44.393 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.394 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.395 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:03:44.396 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:03:44.397 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:03:44.398 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.399 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.400 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.402 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:03:44.402 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:03:44.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f640c073230 msgr2=0x7f640c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f640c073230 0x7f640c075740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f64200af000 tx=0x7f64200a8040 comp rx=0 tx=0).stop 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 msgr2=0x7f642c1468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 0x7f642c1468a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f642400e990 tx=0x7f642400ee50 comp rx=0 tx=0).stop 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 shutdown_connections 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f640c073230 0x7f640c075740 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f642c160480 0x7f642c1468a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f642c15d410 0x7f642c1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 --2- 172.21.15.16:0/1206689356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f642c074e40 0x7f642c1fc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 >> 172.21.15.16:0/1206689356 conn(0x7f642c073de0 msgr2=0x7f642c1525d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 shutdown_connections 2024-09-17T05:03:44.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.385+0000 7f6434400640 1 -- 172.21.15.16:0/1206689356 wait complete. 2024-09-17T05:03:44.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:03:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 -- 172.21.15.16:0/1304486311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c0b2f50 msgr2=0x7fbc0c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 --2- 172.21.15.16:0/1304486311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c0b2f50 0x7fbc0c15bb60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009840 tx=0x7fbbfc01c790 comp rx=0 tx=0).stop 2024-09-17T05:03:44.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 -- 172.21.15.16:0/1304486311 shutdown_connections 2024-09-17T05:03:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 --2- 172.21.15.16:0/1304486311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc0c073240 0x7fbc0c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 --2- 172.21.15.16:0/1304486311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c0b2f50 0x7fbc0c15bb60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 -- 172.21.15.16:0/1304486311 >> 172.21.15.16:0/1304486311 conn(0x7fbc0c06f900 msgr2=0x7fbc0c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 -- 172.21.15.16:0/1304486311 shutdown_connections 2024-09-17T05:03:44.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.565+0000 7fbc10c00640 1 -- 172.21.15.16:0/1304486311 wait complete. 2024-09-17T05:03:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 Processor -- start 2024-09-17T05:03:44.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 -- start start 2024-09-17T05:03:44.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc0c073240 0x7fbc0c0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc0c0b2f50 0x7fbc0c164710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 0x7fbc0c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c168cc0 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c168b40 con 0x7fbc0c073240 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c168e40 con 0x7fbc0c0b2f50 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 0x7fbc0c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 0x7fbc0c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41358/0 (socket says 172.21.15.16:41358) 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 -- 172.21.15.16:0/118861835 learned_addr learned my addr 172.21.15.16:0/118861835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc09e00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc0c073240 0x7fbc0c0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc01400640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc0c0b2f50 0x7fbc0c164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 -- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc0c0b2f50 msgr2=0x7fbc0c164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc0c0b2f50 0x7fbc0c164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 -- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc0c073240 msgr2=0x7fbc0c0cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc0c073240 0x7fbc0c0cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 -- 172.21.15.16:0/118861835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c164c50 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc09e00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc0c073240 0x7fbc0c0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc0a800640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 0x7fbc0c1625d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400e710 tx=0x7fbc0400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0400ca50 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc0400cbf0 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04005020 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c165160 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.569+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c165720 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.573+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c073650 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.573+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc040051c0 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.573+0000 7fbc02a00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbbe8073200 0x7fbbe8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.573+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7fbc04094ab0 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.573+0000 7fbc09e00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbbe8073200 0x7fbbe8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.577+0000 7fbc09e00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbbe8073200 0x7fbbe8075710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0af000 tx=0x7fbbfc01afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:44.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.577+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc0405fc80 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbc0c0ccd60 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc02a00640 1 -- 172.21.15.16:0/118861835 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) ==== 84+0+65 (secure 0 0 0) 0x7fbc0405f580 con 0x7fbc0c15c0a0 2024-09-17T05:03:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:44.759 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:03:44.759 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:03:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:03:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:03:44.760 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:03:44.760 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:03:44.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbbe8073200 msgr2=0x7fbbe8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbbe8073200 0x7fbbe8075710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0af000 tx=0x7fbbfc01afd0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 msgr2=0x7fbc0c1625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 0x7fbc0c1625d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400e710 tx=0x7fbc0400ebd0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 shutdown_connections 2024-09-17T05:03:44.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbbe8073200 0x7fbbe8075710 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0c15c0a0 0x7fbc0c1625d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc0c0b2f50 0x7fbc0c164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 --2- 172.21.15.16:0/118861835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc0c073240 0x7fbc0c0cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 >> 172.21.15.16:0/118861835 conn(0x7fbc0c06f900 msgr2=0x7fbc0c151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:44.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 shutdown_connections 2024-09-17T05:03:44.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.757+0000 7fbc10c00640 1 -- 172.21.15.16:0/118861835 wait complete. 2024-09-17T05:03:44.781 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:03:44.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:03:44.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 -- 172.21.15.16:0/4233985166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 msgr2=0x7fcf640d1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 --2- 172.21.15.16:0/4233985166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf640d1880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcf54009840 tx=0x7fcf5401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 -- 172.21.15.16:0/4233985166 shutdown_connections 2024-09-17T05:03:44.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 --2- 172.21.15.16:0/4233985166 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf640d1dc0 0x7fcf640cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 --2- 172.21.15.16:0/4233985166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf640d1880 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 -- 172.21.15.16:0/4233985166 >> 172.21.15.16:0/4233985166 conn(0x7fcf640715e0 msgr2=0x7fcf640719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:44.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 -- 172.21.15.16:0/4233985166 shutdown_connections 2024-09-17T05:03:44.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.941+0000 7fcf6a600640 1 -- 172.21.15.16:0/4233985166 wait complete. 2024-09-17T05:03:44.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 Processor -- start 2024-09-17T05:03:44.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 -- start start 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf640cf170 0x7fcf641551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf6406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf640d1dc0 0x7fcf6406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64169dc0 con 0x7fcf640cfdf0 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64169c40 con 0x7fcf640cf170 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf6a600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcf64169f40 con 0x7fcf640d1dc0 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf6406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf6406dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41378/0 (socket says 172.21.15.16:41378) 2024-09-17T05:03:44.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 -- 172.21.15.16:0/2079554504 learned_addr learned my addr 172.21.15.16:0/2079554504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf63400640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf640cf170 0x7fcf641551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf63e00640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf640d1dc0 0x7fcf6406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 -- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf640d1dc0 msgr2=0x7fcf6406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf640d1dc0 0x7fcf6406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 -- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf640cf170 msgr2=0x7fcf641551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf640cf170 0x7fcf641551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 -- 172.21.15.16:0/2079554504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf641fc0c0 con 0x7fcf640cfdf0 2024-09-17T05:03:44.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5aa00640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf6406dfa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00e7a0 tx=0x7fcf4c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00ccc0 con 0x7fcf640cfdf0 2024-09-17T05:03:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.945+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf4c010430 con 0x7fcf640cfdf0 2024-09-17T05:03:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf641fc3a0 con 0x7fcf640cfdf0 2024-09-17T05:03:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c010860 con 0x7fcf640cfdf0 2024-09-17T05:03:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf641fc960 con 0x7fcf640cfdf0 2024-09-17T05:03:44.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf640d1880 con 0x7fcf640cfdf0 2024-09-17T05:03:44.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcf4c010a00 con 0x7fcf640cfdf0 2024-09-17T05:03:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf5be00640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf40073200 0x7fcf40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5249+0+0 (secure 0 0 0) 0x7fcf4c094b00 con 0x7fcf640cfdf0 2024-09-17T05:03:44.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.949+0000 7fcf63400640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf40073200 0x7fcf40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:44.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.953+0000 7fcf63400640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf40073200 0x7fcf40075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf540af000 tx=0x7fcf5401b030 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:44.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:44.957+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcf4c05fcd0 con 0x7fcf640cfdf0 2024-09-17T05:03:45.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.133+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcf64155720 con 0x7fcf640cfdf0 2024-09-17T05:03:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.957+0000 7fcf5be00640 1 -- 172.21.15.16:0/2079554504 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) ==== 138+0+0 (secure 0 0 0) 0x7fcf4c094e20 con 0x7fcf640cfdf0 2024-09-17T05:03:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:03:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf40073200 msgr2=0x7fcf40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf40073200 0x7fcf40075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf540af000 tx=0x7fcf5401b030 comp rx=0 tx=0).stop 2024-09-17T05:03:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 msgr2=0x7fcf6406dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf6406dfa0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00e7a0 tx=0x7fcf4c00ec60 comp rx=0 tx=0).stop 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 shutdown_connections 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcf40073200 0x7fcf40075710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcf640d1dc0 0x7fcf6406e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf640cfdf0 0x7fcf6406dfa0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 --2- 172.21.15.16:0/2079554504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf640cf170 0x7fcf641551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 >> 172.21.15.16:0/2079554504 conn(0x7fcf640715e0 msgr2=0x7fcf641660b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 shutdown_connections 2024-09-17T05:03:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:45.961+0000 7fcf6a600640 1 -- 172.21.15.16:0/2079554504 wait complete. 2024-09-17T05:03:45.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 -- 172.21.15.16:0/456832072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 msgr2=0x7fc8c416f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 --2- 172.21.15.16:0/456832072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c416f7d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40d10a0 tx=0x7fc8bc01ccc0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 -- 172.21.15.16:0/456832072 shutdown_connections 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 --2- 172.21.15.16:0/456832072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c416f7d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 --2- 172.21.15.16:0/456832072 >> v2:172.21.15.104:3300/0 conn(0x7fc8c4074160 0x7fc8c416ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 --2- 172.21.15.16:0/456832072 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8c4073250 0x7fc8c4073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 -- 172.21.15.16:0/456832072 >> 172.21.15.16:0/456832072 conn(0x7fc8c40cffa0 msgr2=0x7fc8c40d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 -- 172.21.15.16:0/456832072 shutdown_connections 2024-09-17T05:03:46.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 -- 172.21.15.16:0/456832072 wait complete. 2024-09-17T05:03:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 Processor -- start 2024-09-17T05:03:46.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.165+0000 7fc8cba00640 1 -- start start 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8c4073250 0x7fc8c420a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c4074160 0x7fc8c4169260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4176ce0 con 0x7fc8c416f3c0 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4176b60 con 0x7fc8c4074160 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c4176e60 con 0x7fc8c4073250 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c40ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41392/0 (socket says 172.21.15.16:41392) 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 -- 172.21.15.16:0/476525822 learned_addr learned my addr 172.21.15.16:0/476525822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9000640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8c4073250 0x7fc8c420a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c3e00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c4074160 0x7fc8c4169260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 -- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8c4073250 msgr2=0x7fc8c420a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8c4073250 0x7fc8c420a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 -- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c4074160 msgr2=0x7fc8c4169260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c4074160 0x7fc8c4169260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 -- 172.21.15.16:0/476525822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c40cce00 con 0x7fc8c416f3c0 2024-09-17T05:03:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c9a00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c40ae840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009bd0 tx=0x7fc8bc0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc0bb070 con 0x7fc8c416f3c0 2024-09-17T05:03:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8bc004d40 con 0x7fc8c416f3c0 2024-09-17T05:03:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc0b2040 con 0x7fc8c416f3c0 2024-09-17T05:03:46.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c40cd0a0 con 0x7fc8c416f3c0 2024-09-17T05:03:46.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c40cd380 con 0x7fc8c416f3c0 2024-09-17T05:03:46.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.169+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c42096b0 con 0x7fc8c416f3c0 2024-09-17T05:03:46.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.173+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc8bc004200 con 0x7fc8c416f3c0 2024-09-17T05:03:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.173+0000 7fc8c0c00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8a4073200 0x7fc8a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.173+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5619+0+0 (secure 0 0 0) 0x7fc8bc12e360 con 0x7fc8c416f3c0 2024-09-17T05:03:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.173+0000 7fc8c9000640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8a4073200 0x7fc8a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:46.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.173+0000 7fc8c9000640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8a4073200 0x7fc8a4075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bb50 tx=0x7fc8b000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:46.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.177+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc8bc0c4020 con 0x7fc8c416f3c0 2024-09-17T05:03:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.353+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc8c416ee80 con 0x7fc8c416f3c0 2024-09-17T05:03:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.961+0000 7fc8c0c00640 1 -- 172.21.15.16:0/476525822 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) ==== 170+0+0 (secure 0 0 0) 0x7fc8bc016970 con 0x7fc8c416f3c0 2024-09-17T05:03:46.964 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:03:46.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8a4073200 msgr2=0x7fc8a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8a4073200 0x7fc8a4075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bb50 tx=0x7fc8b000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 msgr2=0x7fc8c40ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c40ae840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009bd0 tx=0x7fc8bc0092c0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 shutdown_connections 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8a4073200 0x7fc8a4075710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8c416f3c0 0x7fc8c40ae840 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c4074160 0x7fc8c4169260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 --2- 172.21.15.16:0/476525822 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc8c4073250 0x7fc8c420a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 >> 172.21.15.16:0/476525822 conn(0x7fc8c40cffa0 msgr2=0x7fc8c40b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 shutdown_connections 2024-09-17T05:03:46.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:46.965+0000 7fc8cba00640 1 -- 172.21.15.16:0/476525822 wait complete. 2024-09-17T05:03:47.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:03:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 -- 172.21.15.16:0/1369921524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c15a820 msgr2=0x7f4f8c0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 --2- 172.21.15.16:0/1369921524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c15a820 0x7f4f8c0cac90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4f740092b0 tx=0x7f4f7401c820 comp rx=0 tx=0).stop 2024-09-17T05:03:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 -- 172.21.15.16:0/1369921524 shutdown_connections 2024-09-17T05:03:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 --2- 172.21.15.16:0/1369921524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c15a820 0x7f4f8c0cac90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 --2- 172.21.15.16:0/1369921524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f8c0ca6b0 0x7f4f8c074be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 -- 172.21.15.16:0/1369921524 >> 172.21.15.16:0/1369921524 conn(0x7f4f8c0b2250 msgr2=0x7f4f8c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 -- 172.21.15.16:0/1369921524 shutdown_connections 2024-09-17T05:03:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 -- 172.21.15.16:0/1369921524 wait complete. 2024-09-17T05:03:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 Processor -- start 2024-09-17T05:03:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.173+0000 7f4f90c00640 1 -- start start 2024-09-17T05:03:47.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 0x7f4f8c1f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c0cb1d0 0x7f4f8c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f8c15a820 0x7f4f8c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c167de0 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c167c60 con 0x7f4f8c0cb1d0 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4f8c167f60 con 0x7f4f8c15a820 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 0x7f4f8c1f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 0x7f4f8c1f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33830/0 (socket says 172.21.15.16:33830) 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 -- 172.21.15.16:0/2555185818 learned_addr learned my addr 172.21.15.16:0/2555185818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f83e00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c0cb1d0 0x7f4f8c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f8a800640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f8c15a820 0x7f4f8c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 -- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f8c15a820 msgr2=0x7f4f8c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:47.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f8c15a820 0x7f4f8c0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 -- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c0cb1d0 msgr2=0x7f4f8c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c0cb1d0 0x7f4f8c0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 -- 172.21.15.16:0/2555185818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c1f8d50 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f89e00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 0x7f4f8c1f8810 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0058d0 tx=0x7f4f7c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c01b070 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f7c00ae50 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c00e9c0 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c148860 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c148e20 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.177+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c074be0 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.181+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4f7c01e070 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.181+0000 7f4f82000640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f68073230 0x7f4f68075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.181+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6001+0+0 (secure 0 0 0) 0x7f4f7c094650 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.181+0000 7f4f83e00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f68073230 0x7f4f68075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.181+0000 7f4f83e00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f68073230 0x7f4f68075740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f740af000 tx=0x7f4f740a8040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:47.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.185+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4f7c00de60 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.397+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4f8c068db0 con 0x7f4f8c0ca6b0 2024-09-17T05:03:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:47.993+0000 7f4f82000640 1 -- 172.21.15.16:0/2555185818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f4f7c05eee0 con 0x7f4f8c0ca6b0 2024-09-17T05:03:48.003 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 9 and data pool 10 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f68073230 msgr2=0x7f4f68075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f68073230 0x7f4f68075740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f740af000 tx=0x7f4f740a8040 comp rx=0 tx=0).stop 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 msgr2=0x7f4f8c1f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 0x7f4f8c1f8810 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0058d0 tx=0x7f4f7c00a620 comp rx=0 tx=0).stop 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 shutdown_connections 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4f68073230 0x7f4f68075740 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4f740af000 tx=0x7f4f740a8040 comp rx=0 tx=0).stop 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4f8c15a820 0x7f4f8c0b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f8c0cb1d0 0x7f4f8c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 --2- 172.21.15.16:0/2555185818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8c0ca6b0 0x7f4f8c1f8810 secure :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0058d0 tx=0x7f4f7c00a620 comp rx=0 tx=0).stop 2024-09-17T05:03:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.005+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 >> 172.21.15.16:0/2555185818 conn(0x7f4f8c0b2250 msgr2=0x7f4f8c1565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:48.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.009+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 shutdown_connections 2024-09-17T05:03:48.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.009+0000 7f4f90c00640 1 -- 172.21.15.16:0/2555185818 wait complete. 2024-09-17T05:03:48.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:48.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 -- 172.21.15.16:0/186870969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415c0a0 msgr2=0x7f529415c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 --2- 172.21.15.16:0/186870969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415c0a0 0x7f529415c4b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f529415cc70 tx=0x7f527c01c830 comp rx=0 tx=0).stop 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 -- 172.21.15.16:0/186870969 shutdown_connections 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 --2- 172.21.15.16:0/186870969 >> v2:172.21.15.104:3300/0 conn(0x7f529415f150 0x7f529415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 --2- 172.21.15.16:0/186870969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415c0a0 0x7f529415c4b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 --2- 172.21.15.16:0/186870969 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f52940b2f50 0x7f529415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 -- 172.21.15.16:0/186870969 >> 172.21.15.16:0/186870969 conn(0x7f529406f900 msgr2=0x7f52940b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 -- 172.21.15.16:0/186870969 shutdown_connections 2024-09-17T05:03:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.209+0000 7f5299e00640 1 -- 172.21.15.16:0/186870969 wait complete. 2024-09-17T05:03:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 Processor -- start 2024-09-17T05:03:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 -- start start 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f52940b2f50 0x7f529413f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f529415c0a0 0x7f52941f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 0x7f529415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5294073ef0 con 0x7f529415f150 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5294073d70 con 0x7f529415c0a0 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5299e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5294074070 con 0x7f52940b2f50 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5293400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 0x7f529415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5293400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 0x7f529415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33844/0 (socket says 172.21.15.16:33844) 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5293400640 1 -- 172.21.15.16:0/2066129368 learned_addr learned my addr 172.21.15.16:0/2066129368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:48.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f5292a00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f52940b2f50 0x7f529413f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.213+0000 7f528a000640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f529415c0a0 0x7f52941f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5293400640 1 -- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f52940b2f50 msgr2=0x7f529413f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5293400640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f52940b2f50 0x7f529413f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5293400640 1 -- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f529415c0a0 msgr2=0x7f52941f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5293400640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f529415c0a0 0x7f52941f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5293400640 1 -- 172.21.15.16:0/2066129368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52941f9700 con 0x7f529415f150 2024-09-17T05:03:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5293400640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 0x7f529415a630 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f528c00e970 tx=0x7f528c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c004d30 con 0x7f529415f150 2024-09-17T05:03:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f528c00cb20 con 0x7f529415f150 2024-09-17T05:03:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c005020 con 0x7f529415f150 2024-09-17T05:03:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52941f99e0 con 0x7f529415f150 2024-09-17T05:03:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f529413fc30 con 0x7f529415f150 2024-09-17T05:03:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529413ffc0 con 0x7f529415f150 2024-09-17T05:03:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.217+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f528c0051c0 con 0x7f529415f150 2024-09-17T05:03:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.221+0000 7f528be00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5270073200 0x7f5270075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.221+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7f528c094d00 con 0x7f529415f150 2024-09-17T05:03:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.221+0000 7f5292a00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5270073200 0x7f5270075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.221+0000 7f5292a00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5270073200 0x7f5270075710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5284009ac0 tx=0x7f52840091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:48.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.225+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f528c016030 con 0x7f529415f150 2024-09-17T05:03:48.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.401+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52941f9490 con 0x7f529415f150 2024-09-17T05:03:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.401+0000 7f528be00640 1 -- 172.21.15.16:0/2066129368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16993 (secure 0 0 0) 0x7f528c05f4a0 con 0x7f529415f150 2024-09-17T05:03:48.404 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:48.405 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:47.404238+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:45.146869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:46.365938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5270073200 msgr2=0x7f5270075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5270073200 0x7f5270075710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5284009ac0 tx=0x7f52840091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 msgr2=0x7f529415a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 0x7f529415a630 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f528c00e970 tx=0x7f528c00ee30 comp rx=0 tx=0).stop 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 shutdown_connections 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5270073200 0x7f5270075710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f529415f150 0x7f529415a630 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f529415c0a0 0x7f52941f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 --2- 172.21.15.16:0/2066129368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f52940b2f50 0x7f529413f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 >> 172.21.15.16:0/2066129368 conn(0x7f529406f900 msgr2=0x7f529415dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 shutdown_connections 2024-09-17T05:03:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.405+0000 7f5299e00640 1 -- 172.21.15.16:0/2066129368 wait complete. 2024-09-17T05:03:48.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:48.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 -- 172.21.15.16:0/1777549354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3300b2f50 msgr2=0x7fc33015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 --2- 172.21.15.16:0/1777549354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3300b2f50 0x7fc33015bb60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc324009870 tx=0x7fc32401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 -- 172.21.15.16:0/1777549354 shutdown_connections 2024-09-17T05:03:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 --2- 172.21.15.16:0/1777549354 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3300b2f50 0x7fc33015bb60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 -- 172.21.15.16:0/1777549354 >> 172.21.15.16:0/1777549354 conn(0x7fc33006f900 msgr2=0x7fc3300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 -- 172.21.15.16:0/1777549354 shutdown_connections 2024-09-17T05:03:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 -- 172.21.15.16:0/1777549354 wait complete. 2024-09-17T05:03:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 Processor -- start 2024-09-17T05:03:48.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.585+0000 7fc337800640 1 -- start start 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc330073240 0x7fc330144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3300b2f50 0x7fc3301f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 0x7fc330144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc330169e50 con 0x7fc33015c0a0 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc330169cd0 con 0x7fc3300b2f50 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc330169fd0 con 0x7fc330073240 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 0x7fc330144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 0x7fc330144220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33846/0 (socket says 172.21.15.16:33846) 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 -- 172.21.15.16:0/1697993758 learned_addr learned my addr 172.21.15.16:0/1697993758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc334e00640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc330073240 0x7fc330144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc32fe00640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3300b2f50 0x7fc3301f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 -- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc330073240 msgr2=0x7fc330144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc330073240 0x7fc330144db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 -- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3300b2f50 msgr2=0x7fc3301f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3300b2f50 0x7fc3301f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 -- 172.21.15.16:0/1697993758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc330164e40 con 0x7fc33015c0a0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc335800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 0x7fc330144220 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc32800ca70 tx=0x7fc32800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328016020 con 0x7fc33015c0a0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc32800a8d0 con 0x7fc33015c0a0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc330165120 con 0x7fc33015c0a0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32800ada0 con 0x7fc33015c0a0 2024-09-17T05:03:48.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3301656e0 con 0x7fc33015c0a0 2024-09-17T05:03:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.589+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3300b0370 con 0x7fc33015c0a0 2024-09-17T05:03:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.593+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc328019020 con 0x7fc33015c0a0 2024-09-17T05:03:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.593+0000 7fc32cc00640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc310073200 0x7fc310075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.593+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fc328094560 con 0x7fc33015c0a0 2024-09-17T05:03:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.593+0000 7fc334e00640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc310073200 0x7fc310075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.593+0000 7fc334e00640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc310073200 0x7fc310075710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3240af000 tx=0x7fc32401b010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:48.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.597+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc328020020 con 0x7fc33015c0a0 2024-09-17T05:03:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.769+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3300b0090 con 0x7fc33015c0a0 2024-09-17T05:03:48.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc32cc00640 1 -- 172.21.15.16:0/1697993758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16993 (secure 0 0 0) 0x7fc32805f3e0 con 0x7fc33015c0a0 2024-09-17T05:03:48.774 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:48.774 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":63,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:47.404238+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:45.146869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:46.365938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc310073200 msgr2=0x7fc310075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc310073200 0x7fc310075710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3240af000 tx=0x7fc32401b010 comp rx=0 tx=0).stop 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 msgr2=0x7fc330144220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 0x7fc330144220 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc32800ca70 tx=0x7fc32800cb60 comp rx=0 tx=0).stop 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 shutdown_connections 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc310073200 0x7fc310075710 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33015c0a0 0x7fc330144220 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3300b2f50 0x7fc3301f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 --2- 172.21.15.16:0/1697993758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc330073240 0x7fc330144db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 >> 172.21.15.16:0/1697993758 conn(0x7fc33006f900 msgr2=0x7fc3300cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 shutdown_connections 2024-09-17T05:03:48.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.773+0000 7fc337800640 1 -- 172.21.15.16:0/1697993758 wait complete. 2024-09-17T05:03:48.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 -- 172.21.15.16:0/2942274450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0744d0 msgr2=0x7fc2ac0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 --2- 172.21.15.16:0/2942274450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0744d0 0x7fc2ac0cd210 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc298009900 tx=0x7fc29801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 -- 172.21.15.16:0/2942274450 shutdown_connections 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 --2- 172.21.15.16:0/2942274450 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc2ac06dfa0 0x7fc2ac06e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 --2- 172.21.15.16:0/2942274450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0744d0 0x7fc2ac0cd210 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 -- 172.21.15.16:0/2942274450 >> 172.21.15.16:0/2942274450 conn(0x7fc2ac0b2dd0 msgr2=0x7fc2ac0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 -- 172.21.15.16:0/2942274450 shutdown_connections 2024-09-17T05:03:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 -- 172.21.15.16:0/2942274450 wait complete. 2024-09-17T05:03:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 Processor -- start 2024-09-17T05:03:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.957+0000 7fc2b4000640 1 -- start start 2024-09-17T05:03:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2ac06dfa0 0x7fc2ac0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 0x7fc2ac0b6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc2ac14ccc0 0x7fc2ac150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac168a60 con 0x7fc2ac0cce00 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1688e0 con 0x7fc2ac06dfa0 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac168be0 con 0x7fc2ac14ccc0 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 0x7fc2ac0b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 0x7fc2ac0b6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33874/0 (socket says 172.21.15.16:33874) 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 -- 172.21.15.16:0/3771073402 learned_addr learned my addr 172.21.15.16:0/3771073402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b2000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc2ac14ccc0 0x7fc2ac150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b1600640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2ac06dfa0 0x7fc2ac0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 -- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc2ac14ccc0 msgr2=0x7fc2ac150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc2ac14ccc0 0x7fc2ac150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 -- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2ac06dfa0 msgr2=0x7fc2ac0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2ac06dfa0 0x7fc2ac0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 -- 172.21.15.16:0/3771073402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac0b70b0 con 0x7fc2ac0cce00 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b1600640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2ac06dfa0 0x7fc2ac0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b0c00640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 0x7fc2ac0b6910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc298009850 tx=0x7fc298005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2980b2040 con 0x7fc2ac0cce00 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc298004330 con 0x7fc2ac0cce00 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc298005290 con 0x7fc2ac0cce00 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac1f87c0 con 0x7fc2ac0cce00 2024-09-17T05:03:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1f8d80 con 0x7fc2ac0cce00 2024-09-17T05:03:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.961+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac06e3b0 con 0x7fc2ac0cce00 2024-09-17T05:03:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.965+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc298005430 con 0x7fc2ac0cce00 2024-09-17T05:03:48.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.965+0000 7fc2a9600640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc28c073230 0x7fc28c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.965+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6101+0+0 (secure 0 0 0) 0x7fc29812db10 con 0x7fc2ac0cce00 2024-09-17T05:03:48.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.965+0000 7fc2b1600640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc28c073230 0x7fc28c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.965+0000 7fc2b1600640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc28c073230 0x7fc28c075740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00016b0 tx=0x7fc2a0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:48.969+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc2980bc060 con 0x7fc2ac0cce00 2024-09-17T05:03:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:49.185+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc2ac0cc5b0 con 0x7fc2ac0cce00 2024-09-17T05:03:49.961 INFO:tasks.ceph.mds.d.smithi104.stderr:2024-09-17T05:03:49.958+0000 7f9a06e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:03:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.009+0000 7fc2a9600640 1 -- 172.21.15.16:0/3771073402 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7fc2980164d0 con 0x7fc2ac0cce00 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc28c073230 msgr2=0x7fc28c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc28c073230 0x7fc28c075740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00016b0 tx=0x7fc2a0009040 comp rx=0 tx=0).stop 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 msgr2=0x7fc2ac0b6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 0x7fc2ac0b6910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc298009850 tx=0x7fc298005760 comp rx=0 tx=0).stop 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 shutdown_connections 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc28c073230 0x7fc28c075740 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc2ac14ccc0 0x7fc2ac150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac0cce00 0x7fc2ac0b6910 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 --2- 172.21.15.16:0/3771073402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2ac06dfa0 0x7fc2ac0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 >> 172.21.15.16:0/3771073402 conn(0x7fc2ac0b2dd0 msgr2=0x7fc2ac0c6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 shutdown_connections 2024-09-17T05:03:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.013+0000 7fc2b4000640 1 -- 172.21.15.16:0/3771073402 wait complete. 2024-09-17T05:03:50.041 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:03:50.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:03:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 -- 172.21.15.16:0/1463329282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d0004d50 msgr2=0x7f30d0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 --2- 172.21.15.16:0/1463329282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30d0004d50 0x7f30d0005160 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0afa90 tx=0x7f30e401d2c0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30ea800640 1 -- 172.21.15.16:0/1463329282 reap_dead start 2024-09-17T05:03:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 -- 172.21.15.16:0/1463329282 shutdown_connections 2024-09-17T05:03:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 -- 172.21.15.16:0/1463329282 >> 172.21.15.16:0/1463329282 conn(0x7f30ec0b2250 msgr2=0x7f30ec0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 -- 172.21.15.16:0/1463329282 shutdown_connections 2024-09-17T05:03:50.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 -- 172.21.15.16:0/1463329282 wait complete. 2024-09-17T05:03:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 Processor -- start 2024-09-17T05:03:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.209+0000 7f30f1000640 1 -- start start 2024-09-17T05:03:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30d0004d50 0x7f30ec1f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30ec14e5d0 0x7f30ec0c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 0x7f30ec0ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0750c0 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec074f40 con 0x7f30d0004d50 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec15fc20 con 0x7f30ec14e5d0 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 0x7f30ec0ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 0x7f30ec0ae9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33880/0 (socket says 172.21.15.16:33880) 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 -- 172.21.15.16:0/1715559618 learned_addr learned my addr 172.21.15.16:0/1715559618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30e3e00640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30ec14e5d0 0x7f30ec0c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30e9e00640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30d0004d50 0x7f30ec1f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 -- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30ec14e5d0 msgr2=0x7f30ec0c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30ec14e5d0 0x7f30ec0c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 -- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30d0004d50 msgr2=0x7f30ec1f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30d0004d50 0x7f30ec1f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 -- 172.21.15.16:0/1715559618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec0c6a00 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30ea800640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 0x7f30ec0ae9b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30e4098e90 tx=0x7f30e40a5f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4004e60 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e4003d90 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec0aeef0 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e40062a0 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1f5ca0 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec15ee70 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.213+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30e4006440 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.217+0000 7f30e2000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30c8073230 0x7f30c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.217+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f30e412eb00 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.217+0000 7f30e9e00640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30c8073230 0x7f30c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.217+0000 7f30e9e00640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30c8073230 0x7f30c8075740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009ce0 tx=0x7f30dc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.221+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30ec0aeef0 con 0x7f30ec0ae5a0 2024-09-17T05:03:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:50.433+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f30ec14e9e0 con 0x7f30ec0ae5a0 2024-09-17T05:03:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.017+0000 7f30e2000640 1 -- 172.21.15.16:0/1715559618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f30e4017070 con 0x7f30ec0ae5a0 2024-09-17T05:03:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30c8073230 msgr2=0x7f30c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30c8073230 0x7f30c8075740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009ce0 tx=0x7f30dc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 msgr2=0x7f30ec0ae9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 0x7f30ec0ae9b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30e4098e90 tx=0x7f30e40a5f70 comp rx=0 tx=0).stop 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 shutdown_connections 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30c8073230 0x7f30c8075740 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0ae5a0 0x7f30ec0ae9b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30ec14e5d0 0x7f30ec0c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 --2- 172.21.15.16:0/1715559618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30d0004d50 0x7f30ec1f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 >> 172.21.15.16:0/1715559618 conn(0x7f30ec0b2250 msgr2=0x7f30ec0ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 shutdown_connections 2024-09-17T05:03:51.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.021+0000 7f30f1000640 1 -- 172.21.15.16:0/1715559618 wait complete. 2024-09-17T05:03:51.050 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:03:51.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 -- 172.21.15.16:0/582740124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c15c270 msgr2=0x7f317c15c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 --2- 172.21.15.16:0/582740124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c15c270 0x7f317c15c680 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3164009840 tx=0x7f316401c790 comp rx=0 tx=0).stop 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 -- 172.21.15.16:0/582740124 shutdown_connections 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 --2- 172.21.15.16:0/582740124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c15c270 0x7f317c15c680 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 -- 172.21.15.16:0/582740124 >> 172.21.15.16:0/582740124 conn(0x7f317c06f900 msgr2=0x7f317c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 -- 172.21.15.16:0/582740124 shutdown_connections 2024-09-17T05:03:51.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.209+0000 7f3181000640 1 -- 172.21.15.16:0/582740124 wait complete. 2024-09-17T05:03:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 Processor -- start 2024-09-17T05:03:51.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 -- start start 2024-09-17T05:03:51.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f317c073240 0x7f317c14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 0x7f317c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f317c15c270 0x7f317c06e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f317c169cf0 con 0x7f317c0b2f50 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f317c169b70 con 0x7f317c15c270 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f317c169e70 con 0x7f317c073240 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 0x7f317c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 0x7f317c06dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33896/0 (socket says 172.21.15.16:33896) 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 -- 172.21.15.16:0/3473257778 learned_addr learned my addr 172.21.15.16:0/3473257778 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3179e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f317c073240 0x7f317c14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f317a800640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f317c15c270 0x7f317c06e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 -- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f317c073240 msgr2=0x7f317c14f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f317c073240 0x7f317c14f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 -- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f317c15c270 msgr2=0x7f317c06e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f317c15c270 0x7f317c06e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 -- 172.21.15.16:0/3473257778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c14f8e0 con 0x7f317c0b2f50 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3179e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f317c073240 0x7f317c14f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f317a800640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f317c15c270 0x7f317c06e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3173e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 0x7f317c06dc00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f316401cc70 tx=0x7f31640a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31640b2040 con 0x7f317c0b2f50 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31640042b0 con 0x7f317c0b2f50 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c0a9be0 con 0x7f317c0b2f50 2024-09-17T05:03:51.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164003600 con 0x7f317c0b2f50 2024-09-17T05:03:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.213+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c0afe10 con 0x7f317c0b2f50 2024-09-17T05:03:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.217+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c15c680 con 0x7f317c0b2f50 2024-09-17T05:03:51.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.217+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31640bd030 con 0x7f317c0b2f50 2024-09-17T05:03:51.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.217+0000 7f3172000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3158073230 0x7f3158075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.217+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f316412dee0 con 0x7f317c0b2f50 2024-09-17T05:03:51.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.217+0000 7f3179e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3158073230 0x7f3158075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.221+0000 7f3179e00640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3158073230 0x7f3158075740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f316c000f70 tx=0x7f316c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.225+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31640c4060 con 0x7f317c0b2f50 2024-09-17T05:03:51.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.429+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f317c0d02b0 con 0x7f317c0b2f50 2024-09-17T05:03:51.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.429+0000 7f3172000640 1 -- 172.21.15.16:0/3473257778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5230 (secure 0 0 0) 0x7f3164016410 con 0x7f317c0b2f50 2024-09-17T05:03:51.433 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:51.433 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:47.404183+0000","modified":"2024-09-17T05:03:50.438145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/3720312961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3720312961},{"type":"v1","addr":"172.21.15.104:6836","nonce":3720312961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5775,"qdb_cluster":[5775]},"id":4}]} 2024-09-17T05:03:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-17T05:03:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3158073230 msgr2=0x7f3158075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3158073230 0x7f3158075740 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f316c000f70 tx=0x7f316c009040 comp rx=0 tx=0).stop 2024-09-17T05:03:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 msgr2=0x7f317c06dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 0x7f317c06dc00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f316401cc70 tx=0x7f31640a57d0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 shutdown_connections 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3158073230 0x7f3158075740 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f317c15c270 0x7f317c06e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f317c0b2f50 0x7f317c06dc00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 --2- 172.21.15.16:0/3473257778 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f317c073240 0x7f317c14f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 >> 172.21.15.16:0/3473257778 conn(0x7f317c06f900 msgr2=0x7f317c15e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 shutdown_connections 2024-09-17T05:03:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.433+0000 7f3181000640 1 -- 172.21.15.16:0/3473257778 wait complete. 2024-09-17T05:03:51.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 -- 172.21.15.16:0/3238249223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c8073240 msgr2=0x7f99c8073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 --2- 172.21.15.16:0/3238249223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c8073240 0x7f99c8073650 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f99c0009bc0 tx=0x7f99c001cb30 comp rx=0 tx=0).stop 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 -- 172.21.15.16:0/3238249223 shutdown_connections 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 --2- 172.21.15.16:0/3238249223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c8073240 0x7f99c8073650 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 --2- 172.21.15.16:0/3238249223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99c80b2f50 0x7f99c815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 -- 172.21.15.16:0/3238249223 >> 172.21.15.16:0/3238249223 conn(0x7f99c806f900 msgr2=0x7f99c80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 -- 172.21.15.16:0/3238249223 shutdown_connections 2024-09-17T05:03:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 -- 172.21.15.16:0/3238249223 wait complete. 2024-09-17T05:03:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 Processor -- start 2024-09-17T05:03:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.613+0000 7f99cf600640 1 -- start start 2024-09-17T05:03:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c8073240 0x7f99c81643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99c80b2f50 0x7f99c8151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 0x7f99c8151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8168d60 con 0x7f99c815c0a0 2024-09-17T05:03:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99c8168be0 con 0x7f99c8073240 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f99c8168ee0 con 0x7f99c80b2f50 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 0x7f99c8151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 0x7f99c8151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33920/0 (socket says 172.21.15.16:33920) 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 -- 172.21.15.16:0/2107798925 learned_addr learned my addr 172.21.15.16:0/2107798925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99bfe00640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99c80b2f50 0x7f99c8151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99ccc00640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c8073240 0x7f99c81643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 -- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99c80b2f50 msgr2=0x7f99c8151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99c80b2f50 0x7f99c8151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 -- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c8073240 msgr2=0x7f99c81643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c8073240 0x7f99c81643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 -- 172.21.15.16:0/2107798925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c81f6860 con 0x7f99c815c0a0 2024-09-17T05:03:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99ccc00640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c8073240 0x7f99c81643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cd600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 0x7f99c8151850 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f99c00b0040 tx=0x7f99c0009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00b2040 con 0x7f99c815c0a0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99c0004130 con 0x7f99c815c0a0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c81f6b40 con 0x7f99c815c0a0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0004560 con 0x7f99c815c0a0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c8164910 con 0x7f99c815c0a0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.617+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c8164b40 con 0x7f99c815c0a0 2024-09-17T05:03:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.621+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f99c00bd030 con 0x7f99c815c0a0 2024-09-17T05:03:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.621+0000 7f99c5600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99a4073230 0x7f99a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.621+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f99c012e170 con 0x7f99c815c0a0 2024-09-17T05:03:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.621+0000 7f99ccc00640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99a4073230 0x7f99a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.621+0000 7f99ccc00640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99a4073230 0x7f99a4075740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005fd0 tx=0x7f99b8005f20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.625+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f99c00c4060 con 0x7f99c815c0a0 2024-09-17T05:03:51.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.801+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99c8073650 con 0x7f99c815c0a0 2024-09-17T05:03:51.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.801+0000 7f99c5600640 1 -- 172.21.15.16:0/2107798925 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16980 (secure 0 0 0) 0x7f99c00167b0 con 0x7f99c815c0a0 2024-09-17T05:03:51.805 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:51.805 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:48.995211+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:45.146869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:46.365938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:03:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99a4073230 msgr2=0x7f99a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99a4073230 0x7f99a4075740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99b8005fd0 tx=0x7f99b8005f20 comp rx=0 tx=0).stop 2024-09-17T05:03:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 msgr2=0x7f99c8151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 0x7f99c8151850 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f99c00b0040 tx=0x7f99c0009200 comp rx=0 tx=0).stop 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 shutdown_connections 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99a4073230 0x7f99a4075740 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99c815c0a0 0x7f99c8151850 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f99c80b2f50 0x7f99c8151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 --2- 172.21.15.16:0/2107798925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c8073240 0x7f99c81643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 >> 172.21.15.16:0/2107798925 conn(0x7f99c806f900 msgr2=0x7f99c815e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 shutdown_connections 2024-09-17T05:03:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.805+0000 7f99cf600640 1 -- 172.21.15.16:0/2107798925 wait complete. 2024-09-17T05:03:51.827 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:03:51.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:51.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 -- 172.21.15.16:0/2211839256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88071e30 msgr2=0x7f3b880cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2211839256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88071e30 0x7f3b880cce00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009900 tx=0x7f3b7801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 -- 172.21.15.16:0/2211839256 shutdown_connections 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2211839256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 0x7f3b880d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2211839256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88071e30 0x7f3b880cce00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 -- 172.21.15.16:0/2211839256 >> 172.21.15.16:0/2211839256 conn(0x7f3b880ceb00 msgr2=0x7f3b880cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 -- 172.21.15.16:0/2211839256 shutdown_connections 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 -- 172.21.15.16:0/2211839256 wait complete. 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 Processor -- start 2024-09-17T05:03:51.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.985+0000 7f3b8e200640 1 -- start start 2024-09-17T05:03:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 0x7f3b8806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b88071e30 0x7f3b881529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 0x7f3b88152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88169d80 con 0x7f3b88070680 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b88169c00 con 0x7f3b88071e30 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3b88169f00 con 0x7f3b880d01c0 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 0x7f3b8806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 0x7f3b8806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33936/0 (socket says 172.21.15.16:33936) 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 -- 172.21.15.16:0/2972687151 learned_addr learned my addr 172.21.15.16:0/2972687151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8d800640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 0x7f3b88152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b7fe00640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b88071e30 0x7f3b881529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 -- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 msgr2=0x7f3b88152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 0x7f3b88152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 -- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b88071e30 msgr2=0x7f3b881529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b88071e30 0x7f3b881529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 -- 172.21.15.16:0/2972687151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b881f9860 con 0x7f3b88070680 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8d800640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 0x7f3b88152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8ce00640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 0x7f3b8806fa90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3b780b0040 tx=0x7f3b780a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b780b2040 con 0x7f3b88070680 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b780043b0 con 0x7f3b88070680 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78005290 con 0x7f3b88070680 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8806ffd0 con 0x7f3b88070680 2024-09-17T05:03:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881f9b00 con 0x7f3b88070680 2024-09-17T05:03:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.989+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b880cce00 con 0x7f3b88070680 2024-09-17T05:03:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.993+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3b780be060 con 0x7f3b88070680 2024-09-17T05:03:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.993+0000 7f3b85600640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b64073200 0x7f3b64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.993+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f3b7812d820 con 0x7f3b88070680 2024-09-17T05:03:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.993+0000 7f3b7fe00640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b64073200 0x7f3b64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:51.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.993+0000 7f3b7fe00640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b64073200 0x7f3b64075710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3b8815bdc0 tx=0x7f3b70009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:51.997+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3b8806ffd0 con 0x7f3b88070680 2024-09-17T05:03:52.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.169+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b88150bd0 con 0x7f3b64073200 2024-09-17T05:03:52.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.177+0000 7f3b85600640 1 -- 172.21.15.16:0/2972687151 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247250 (secure 0 0 0) 0x7f3b88150bd0 con 0x7f3b64073200 2024-09-17T05:03:52.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:52.183 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b64073200 msgr2=0x7f3b64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b64073200 0x7f3b64075710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3b8815bdc0 tx=0x7f3b70009040 comp rx=0 tx=0).stop 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 msgr2=0x7f3b8806fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 0x7f3b8806fa90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3b780b0040 tx=0x7f3b780a5730 comp rx=0 tx=0).stop 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 shutdown_connections 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3b64073200 0x7f3b64075710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3b880d01c0 0x7f3b88152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b88071e30 0x7f3b881529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 --2- 172.21.15.16:0/2972687151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b88070680 0x7f3b8806fa90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 >> 172.21.15.16:0/2972687151 conn(0x7f3b880ceb00 msgr2=0x7f3b8815a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 shutdown_connections 2024-09-17T05:03:52.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.181+0000 7f3b8e200640 1 -- 172.21.15.16:0/2972687151 wait complete. 2024-09-17T05:03:52.203 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-17T05:03:51.503375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":111,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244008,"kb_used_data":6936,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730712,"statfs":{"total":773094113280,"available":772844249088,"internally_reserved":0,"allocated":7102464,"data_stored":3798808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.541178"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978241+0000","last_change":"2024-09-17T05:03:46.978241+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978241+0000","last_undegraded":"2024-09-17T05:03:46.978241+0000","last_fullsized":"2024-09-17T05:03:46.978241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001275+0000","last_change":"2024-09-17T05:03:48.027266+0000","last_active":"2024-09-17T05:03:49.001275+0000","last_peered":"2024-09-17T05:03:49.001275+0000","last_clean":"2024-09-17T05:03:49.001275+0000","last_became_active":"2024-09-17T05:03:48.021805+0000","last_became_peered":"2024-09-17T05:03:48.021805+0000","last_unstale":"2024-09-17T05:03:49.001275+0000","last_undegraded":"2024-09-17T05:03:49.001275+0000","last_fullsized":"2024-09-17T05:03:49.001275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:19.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.984275+0000","last_change":"2024-09-17T05:03:46.984275+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.984275+0000","last_undegraded":"2024-09-17T05:03:46.984275+0000","last_fullsized":"2024-09-17T05:03:46.984275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977027+0000","last_change":"2024-09-17T05:03:46.977027+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977027+0000","last_undegraded":"2024-09-17T05:03:46.977027+0000","last_fullsized":"2024-09-17T05:03:46.977027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979040+0000","last_change":"2024-09-17T05:03:46.979040+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979040+0000","last_undegraded":"2024-09-17T05:03:46.979040+0000","last_fullsized":"2024-09-17T05:03:46.979040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000614+0000","last_change":"2024-09-17T05:03:48.031572+0000","last_active":"2024-09-17T05:03:49.000614+0000","last_peered":"2024-09-17T05:03:49.000614+0000","last_clean":"2024-09-17T05:03:49.000614+0000","last_became_active":"2024-09-17T05:03:48.031253+0000","last_became_peered":"2024-09-17T05:03:48.031253+0000","last_unstale":"2024-09-17T05:03:49.000614+0000","last_undegraded":"2024-09-17T05:03:49.000614+0000","last_fullsized":"2024-09-17T05:03:49.000614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:16.025505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978918+0000","last_change":"2024-09-17T05:03:46.978918+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978918+0000","last_undegraded":"2024-09-17T05:03:46.978918+0000","last_fullsized":"2024-09-17T05:03:46.978918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.983530+0000","last_change":"2024-09-17T05:03:46.983530+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.983530+0000","last_undegraded":"2024-09-17T05:03:46.983530+0000","last_fullsized":"2024-09-17T05:03:46.983530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979383+0000","last_change":"2024-09-17T05:03:46.979383+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979383+0000","last_undegraded":"2024-09-17T05:03:46.979383+0000","last_fullsized":"2024-09-17T05:03:46.979383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974260+0000","last_change":"2024-09-17T05:03:46.974260+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974260+0000","last_undegraded":"2024-09-17T05:03:46.974260+0000","last_fullsized":"2024-09-17T05:03:46.974260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978382+0000","last_change":"2024-09-17T05:03:46.978382+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978382+0000","last_undegraded":"2024-09-17T05:03:46.978382+0000","last_fullsized":"2024-09-17T05:03:46.978382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976518+0000","last_change":"2024-09-17T05:03:46.976518+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976518+0000","last_undegraded":"2024-09-17T05:03:46.976518+0000","last_fullsized":"2024-09-17T05:03:46.976518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986571+0000","last_change":"2024-09-17T05:03:46.986571+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986571+0000","last_undegraded":"2024-09-17T05:03:46.986571+0000","last_fullsized":"2024-09-17T05:03:46.986571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978501+0000","last_change":"2024-09-17T05:03:46.978501+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978501+0000","last_undegraded":"2024-09-17T05:03:46.978501+0000","last_fullsized":"2024-09-17T05:03:46.978501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976005+0000","last_change":"2024-09-17T05:03:46.976005+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976005+0000","last_undegraded":"2024-09-17T05:03:46.976005+0000","last_fullsized":"2024-09-17T05:03:46.976005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978421+0000","last_change":"2024-09-17T05:03:46.978421+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978421+0000","last_undegraded":"2024-09-17T05:03:46.978421+0000","last_fullsized":"2024-09-17T05:03:46.978421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969790+0000","last_change":"2024-09-17T05:03:46.969790+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969790+0000","last_undegraded":"2024-09-17T05:03:46.969790+0000","last_fullsized":"2024-09-17T05:03:46.969790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979878+0000","last_change":"2024-09-17T05:03:46.979878+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979878+0000","last_undegraded":"2024-09-17T05:03:46.979878+0000","last_fullsized":"2024-09-17T05:03:46.979878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977684+0000","last_change":"2024-09-17T05:03:46.977684+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977684+0000","last_undegraded":"2024-09-17T05:03:46.977684+0000","last_fullsized":"2024-09-17T05:03:46.977684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.971210+0000","last_change":"2024-09-17T05:03:46.971210+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.971210+0000","last_undegraded":"2024-09-17T05:03:46.971210+0000","last_fullsized":"2024-09-17T05:03:46.971210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982766+0000","last_change":"2024-09-17T05:03:46.982766+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982766+0000","last_undegraded":"2024-09-17T05:03:46.982766+0000","last_fullsized":"2024-09-17T05:03:46.982766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000912+0000","last_change":"2024-09-17T05:03:48.013364+0000","last_active":"2024-09-17T05:03:49.000912+0000","last_peered":"2024-09-17T05:03:49.000912+0000","last_clean":"2024-09-17T05:03:49.000912+0000","last_became_active":"2024-09-17T05:03:48.012273+0000","last_became_peered":"2024-09-17T05:03:48.012273+0000","last_unstale":"2024-09-17T05:03:49.000912+0000","last_undegraded":"2024-09-17T05:03:49.000912+0000","last_fullsized":"2024-09-17T05:03:49.000912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:46.993665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980950+0000","last_change":"2024-09-17T05:03:46.980950+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980950+0000","last_undegraded":"2024-09-17T05:03:46.980950+0000","last_fullsized":"2024-09-17T05:03:46.980950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982993+0000","last_change":"2024-09-17T05:03:46.982993+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982993+0000","last_undegraded":"2024-09-17T05:03:46.982993+0000","last_fullsized":"2024-09-17T05:03:46.982993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978589+0000","last_change":"2024-09-17T05:03:46.978589+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978589+0000","last_undegraded":"2024-09-17T05:03:46.978589+0000","last_fullsized":"2024-09-17T05:03:46.978589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979580+0000","last_change":"2024-09-17T05:03:46.979580+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979580+0000","last_undegraded":"2024-09-17T05:03:46.979580+0000","last_fullsized":"2024-09-17T05:03:46.979580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991002+0000","last_change":"2024-09-17T05:03:46.991002+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991002+0000","last_undegraded":"2024-09-17T05:03:46.991002+0000","last_fullsized":"2024-09-17T05:03:46.991002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980311+0000","last_change":"2024-09-17T05:03:46.980311+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980311+0000","last_undegraded":"2024-09-17T05:03:46.980311+0000","last_fullsized":"2024-09-17T05:03:46.980311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976442+0000","last_change":"2024-09-17T05:03:46.976442+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976442+0000","last_undegraded":"2024-09-17T05:03:46.976442+0000","last_fullsized":"2024-09-17T05:03:46.976442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975398+0000","last_change":"2024-09-17T05:03:46.975398+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975398+0000","last_undegraded":"2024-09-17T05:03:46.975398+0000","last_fullsized":"2024-09-17T05:03:46.975398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980342+0000","last_change":"2024-09-17T05:03:46.980342+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980342+0000","last_undegraded":"2024-09-17T05:03:46.980342+0000","last_fullsized":"2024-09-17T05:03:46.980342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977212+0000","last_change":"2024-09-17T05:03:46.977212+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977212+0000","last_undegraded":"2024-09-17T05:03:46.977212+0000","last_fullsized":"2024-09-17T05:03:46.977212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976032+0000","last_change":"2024-09-17T05:03:46.976032+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976032+0000","last_undegraded":"2024-09-17T05:03:46.976032+0000","last_fullsized":"2024-09-17T05:03:46.976032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993555+0000","last_change":"2024-09-17T05:03:46.987624+0000","last_active":"2024-09-17T05:03:46.993555+0000","last_peered":"2024-09-17T05:03:46.993555+0000","last_clean":"2024-09-17T05:03:46.993555+0000","last_became_active":"2024-09-17T05:03:46.987266+0000","last_became_peered":"2024-09-17T05:03:46.987266+0000","last_unstale":"2024-09-17T05:03:46.993555+0000","last_undegraded":"2024-09-17T05:03:46.993555+0000","last_fullsized":"2024-09-17T05:03:46.993555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:28.329239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.983834+0000","last_change":"2024-09-17T05:03:46.983834+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.983834+0000","last_undegraded":"2024-09-17T05:03:46.983834+0000","last_fullsized":"2024-09-17T05:03:46.983834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.996739+0000","last_change":"2024-09-17T05:03:46.993282+0000","last_active":"2024-09-17T05:03:46.996739+0000","last_peered":"2024-09-17T05:03:46.996739+0000","last_clean":"2024-09-17T05:03:46.996739+0000","last_became_active":"2024-09-17T05:03:46.992953+0000","last_became_peered":"2024-09-17T05:03:46.992953+0000","last_unstale":"2024-09-17T05:03:46.996739+0000","last_undegraded":"2024-09-17T05:03:46.996739+0000","last_fullsized":"2024-09-17T05:03:46.996739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:51.511991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001473+0000","last_change":"2024-09-17T05:03:48.023120+0000","last_active":"2024-09-17T05:03:49.001473+0000","last_peered":"2024-09-17T05:03:49.001473+0000","last_clean":"2024-09-17T05:03:49.001473+0000","last_became_active":"2024-09-17T05:03:48.022823+0000","last_became_peered":"2024-09-17T05:03:48.022823+0000","last_unstale":"2024-09-17T05:03:49.001473+0000","last_undegraded":"2024-09-17T05:03:49.001473+0000","last_fullsized":"2024-09-17T05:03:49.001473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:09.131399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.996709+0000","last_change":"2024-09-17T05:03:46.993381+0000","last_active":"2024-09-17T05:03:46.996709+0000","last_peered":"2024-09-17T05:03:46.996709+0000","last_clean":"2024-09-17T05:03:46.996709+0000","last_became_active":"2024-09-17T05:03:46.993086+0000","last_became_peered":"2024-09-17T05:03:46.993086+0000","last_unstale":"2024-09-17T05:03:46.996709+0000","last_undegraded":"2024-09-17T05:03:46.996709+0000","last_fullsized":"2024-09-17T05:03:46.996709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:00.606684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978829+0000","last_change":"2024-09-17T05:03:46.978829+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978829+0000","last_undegraded":"2024-09-17T05:03:46.978829+0000","last_fullsized":"2024-09-17T05:03:46.978829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001194+0000","last_change":"2024-09-17T05:03:46.989740+0000","last_active":"2024-09-17T05:03:49.001194+0000","last_peered":"2024-09-17T05:03:49.001194+0000","last_clean":"2024-09-17T05:03:49.001194+0000","last_became_active":"2024-09-17T05:03:46.988950+0000","last_became_peered":"2024-09-17T05:03:46.988950+0000","last_unstale":"2024-09-17T05:03:49.001194+0000","last_undegraded":"2024-09-17T05:03:49.001194+0000","last_fullsized":"2024-09-17T05:03:49.001194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:55.361318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969266+0000","last_change":"2024-09-17T05:03:46.969266+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969266+0000","last_undegraded":"2024-09-17T05:03:46.969266+0000","last_fullsized":"2024-09-17T05:03:46.969266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.989830+0000","last_change":"2024-09-17T05:03:46.984870+0000","last_active":"2024-09-17T05:03:46.989830+0000","last_peered":"2024-09-17T05:03:46.989830+0000","last_clean":"2024-09-17T05:03:46.989830+0000","last_became_active":"2024-09-17T05:03:46.984556+0000","last_became_peered":"2024-09-17T05:03:46.984556+0000","last_unstale":"2024-09-17T05:03:46.989830+0000","last_undegraded":"2024-09-17T05:03:46.989830+0000","last_fullsized":"2024-09-17T05:03:46.989830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:22.081970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978356+0000","last_change":"2024-09-17T05:03:46.978356+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978356+0000","last_undegraded":"2024-09-17T05:03:46.978356+0000","last_fullsized":"2024-09-17T05:03:46.978356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001024+0000","last_change":"2024-09-17T05:03:46.975859+0000","last_active":"2024-09-17T05:03:49.001024+0000","last_peered":"2024-09-17T05:03:49.001024+0000","last_clean":"2024-09-17T05:03:49.001024+0000","last_became_active":"2024-09-17T05:03:46.975513+0000","last_became_peered":"2024-09-17T05:03:46.975513+0000","last_unstale":"2024-09-17T05:03:49.001024+0000","last_undegraded":"2024-09-17T05:03:49.001024+0000","last_fullsized":"2024-09-17T05:03:49.001024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:24.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975768+0000","last_change":"2024-09-17T05:03:46.975768+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975768+0000","last_undegraded":"2024-09-17T05:03:46.975768+0000","last_fullsized":"2024-09-17T05:03:46.975768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001950+0000","last_change":"2024-09-17T05:03:46.993788+0000","last_active":"2024-09-17T05:03:47.001950+0000","last_peered":"2024-09-17T05:03:47.001950+0000","last_clean":"2024-09-17T05:03:47.001950+0000","last_became_active":"2024-09-17T05:03:46.993492+0000","last_became_peered":"2024-09-17T05:03:46.993492+0000","last_unstale":"2024-09-17T05:03:47.001950+0000","last_undegraded":"2024-09-17T05:03:47.001950+0000","last_fullsized":"2024-09-17T05:03:47.001950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:25.235304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.970441+0000","last_change":"2024-09-17T05:03:46.970441+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.970441+0000","last_undegraded":"2024-09-17T05:03:46.970441+0000","last_fullsized":"2024-09-17T05:03:46.970441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997457+0000","last_change":"2024-09-17T05:03:46.990873+0000","last_active":"2024-09-17T05:03:46.997457+0000","last_peered":"2024-09-17T05:03:46.997457+0000","last_clean":"2024-09-17T05:03:46.997457+0000","last_became_active":"2024-09-17T05:03:46.990380+0000","last_became_peered":"2024-09-17T05:03:46.990380+0000","last_unstale":"2024-09-17T05:03:46.997457+0000","last_undegraded":"2024-09-17T05:03:46.997457+0000","last_fullsized":"2024-09-17T05:03:46.997457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:06.392650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979741+0000","last_change":"2024-09-17T05:03:46.979741+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979741+0000","last_undegraded":"2024-09-17T05:03:46.979741+0000","last_fullsized":"2024-09-17T05:03:46.979741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997808+0000","last_change":"2024-09-17T05:03:46.994888+0000","last_active":"2024-09-17T05:03:46.997808+0000","last_peered":"2024-09-17T05:03:46.997808+0000","last_clean":"2024-09-17T05:03:46.997808+0000","last_became_active":"2024-09-17T05:03:46.994186+0000","last_became_peered":"2024-09-17T05:03:46.994186+0000","last_unstale":"2024-09-17T05:03:46.997808+0000","last_undegraded":"2024-09-17T05:03:46.997808+0000","last_fullsized":"2024-09-17T05:03:46.997808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:07.847464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969974+0000","last_change":"2024-09-17T05:03:46.969974+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969974+0000","last_undegraded":"2024-09-17T05:03:46.969974+0000","last_fullsized":"2024-09-17T05:03:46.969974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001298+0000","last_change":"2024-09-17T05:03:46.996818+0000","last_active":"2024-09-17T05:03:47.001298+0000","last_peered":"2024-09-17T05:03:47.001298+0000","last_clean":"2024-09-17T05:03:47.001298+0000","last_became_active":"2024-09-17T05:03:46.996508+0000","last_became_peered":"2024-09-17T05:03:46.996508+0000","last_unstale":"2024-09-17T05:03:47.001298+0000","last_undegraded":"2024-09-17T05:03:47.001298+0000","last_fullsized":"2024-09-17T05:03:47.001298+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:06.183013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991028+0000","last_change":"2024-09-17T05:03:46.991028+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991028+0000","last_undegraded":"2024-09-17T05:03:46.991028+0000","last_fullsized":"2024-09-17T05:03:46.991028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000750+0000","last_change":"2024-09-17T05:03:46.998029+0000","last_active":"2024-09-17T05:03:47.000750+0000","last_peered":"2024-09-17T05:03:47.000750+0000","last_clean":"2024-09-17T05:03:47.000750+0000","last_became_active":"2024-09-17T05:03:46.997688+0000","last_became_peered":"2024-09-17T05:03:46.997688+0000","last_unstale":"2024-09-17T05:03:47.000750+0000","last_undegraded":"2024-09-17T05:03:47.000750+0000","last_fullsized":"2024-09-17T05:03:47.000750+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:58.197722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982691+0000","last_change":"2024-09-17T05:03:46.982691+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982691+0000","last_undegraded":"2024-09-17T05:03:46.982691+0000","last_fullsized":"2024-09-17T05:03:46.982691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001031+0000","last_change":"2024-09-17T05:03:46.990156+0000","last_active":"2024-09-17T05:03:49.001031+0000","last_peered":"2024-09-17T05:03:49.001031+0000","last_clean":"2024-09-17T05:03:49.001031+0000","last_became_active":"2024-09-17T05:03:46.989449+0000","last_became_peered":"2024-09-17T05:03:46.989449+0000","last_unstale":"2024-09-17T05:03:49.001031+0000","last_undegraded":"2024-09-17T05:03:49.001031+0000","last_fullsized":"2024-09-17T05:03:49.001031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:45.812214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.973898+0000","last_change":"2024-09-17T05:03:46.973898+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.973898+0000","last_undegraded":"2024-09-17T05:03:46.973898+0000","last_fullsized":"2024-09-17T05:03:46.973898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.991309+0000","last_change":"2024-09-17T05:03:46.988141+0000","last_active":"2024-09-17T05:03:46.991309+0000","last_peered":"2024-09-17T05:03:46.991309+0000","last_clean":"2024-09-17T05:03:46.991309+0000","last_became_active":"2024-09-17T05:03:46.987762+0000","last_became_peered":"2024-09-17T05:03:46.987762+0000","last_unstale":"2024-09-17T05:03:46.991309+0000","last_undegraded":"2024-09-17T05:03:46.991309+0000","last_fullsized":"2024-09-17T05:03:46.991309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:20.658053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974975+0000","last_change":"2024-09-17T05:03:46.974975+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974975+0000","last_undegraded":"2024-09-17T05:03:46.974975+0000","last_fullsized":"2024-09-17T05:03:46.974975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.988663+0000","last_change":"2024-09-17T05:03:46.985782+0000","last_active":"2024-09-17T05:03:46.988663+0000","last_peered":"2024-09-17T05:03:46.988663+0000","last_clean":"2024-09-17T05:03:46.988663+0000","last_became_active":"2024-09-17T05:03:46.985426+0000","last_became_peered":"2024-09-17T05:03:46.985426+0000","last_unstale":"2024-09-17T05:03:46.988663+0000","last_undegraded":"2024-09-17T05:03:46.988663+0000","last_fullsized":"2024-09-17T05:03:46.988663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:15.869043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974547+0000","last_change":"2024-09-17T05:03:46.974547+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974547+0000","last_undegraded":"2024-09-17T05:03:46.974547+0000","last_fullsized":"2024-09-17T05:03:46.974547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000603+0000","last_change":"2024-09-17T05:03:46.992544+0000","last_active":"2024-09-17T05:03:47.000603+0000","last_peered":"2024-09-17T05:03:47.000603+0000","last_clean":"2024-09-17T05:03:47.000603+0000","last_became_active":"2024-09-17T05:03:46.992203+0000","last_became_peered":"2024-09-17T05:03:46.992203+0000","last_unstale":"2024-09-17T05:03:47.000603+0000","last_undegraded":"2024-09-17T05:03:47.000603+0000","last_fullsized":"2024-09-17T05:03:47.000603+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:07.389673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978996+0000","last_change":"2024-09-17T05:03:46.978996+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978996+0000","last_undegraded":"2024-09-17T05:03:46.978996+0000","last_fullsized":"2024-09-17T05:03:46.978996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998333+0000","last_change":"2024-09-17T05:03:46.989951+0000","last_active":"2024-09-17T05:03:46.998333+0000","last_peered":"2024-09-17T05:03:46.998333+0000","last_clean":"2024-09-17T05:03:46.998333+0000","last_became_active":"2024-09-17T05:03:46.989489+0000","last_became_peered":"2024-09-17T05:03:46.989489+0000","last_unstale":"2024-09-17T05:03:46.998333+0000","last_undegraded":"2024-09-17T05:03:46.998333+0000","last_fullsized":"2024-09-17T05:03:46.998333+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.475208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975516+0000","last_change":"2024-09-17T05:03:46.975516+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975516+0000","last_undegraded":"2024-09-17T05:03:46.975516+0000","last_fullsized":"2024-09-17T05:03:46.975516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997774+0000","last_change":"2024-09-17T05:03:46.994482+0000","last_active":"2024-09-17T05:03:46.997774+0000","last_peered":"2024-09-17T05:03:46.997774+0000","last_clean":"2024-09-17T05:03:46.997774+0000","last_became_active":"2024-09-17T05:03:46.993791+0000","last_became_peered":"2024-09-17T05:03:46.993791+0000","last_unstale":"2024-09-17T05:03:46.997774+0000","last_undegraded":"2024-09-17T05:03:46.997774+0000","last_fullsized":"2024-09-17T05:03:46.997774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:51.205489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986536+0000","last_change":"2024-09-17T05:03:46.986536+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986536+0000","last_undegraded":"2024-09-17T05:03:46.986536+0000","last_fullsized":"2024-09-17T05:03:46.986536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998376+0000","last_change":"2024-09-17T05:03:46.997619+0000","last_active":"2024-09-17T05:03:46.998376+0000","last_peered":"2024-09-17T05:03:46.998376+0000","last_clean":"2024-09-17T05:03:46.998376+0000","last_became_active":"2024-09-17T05:03:46.994779+0000","last_became_peered":"2024-09-17T05:03:46.994779+0000","last_unstale":"2024-09-17T05:03:46.998376+0000","last_undegraded":"2024-09-17T05:03:46.998376+0000","last_fullsized":"2024-09-17T05:03:46.998376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:52.009156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986807+0000","last_change":"2024-09-17T05:03:46.986807+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986807+0000","last_undegraded":"2024-09-17T05:03:46.986807+0000","last_fullsized":"2024-09-17T05:03:46.986807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000715+0000","last_change":"2024-09-17T05:03:46.994104+0000","last_active":"2024-09-17T05:03:47.000715+0000","last_peered":"2024-09-17T05:03:47.000715+0000","last_clean":"2024-09-17T05:03:47.000715+0000","last_became_active":"2024-09-17T05:03:46.993805+0000","last_became_peered":"2024-09-17T05:03:46.993805+0000","last_unstale":"2024-09-17T05:03:47.000715+0000","last_undegraded":"2024-09-17T05:03:47.000715+0000","last_fullsized":"2024-09-17T05:03:47.000715+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:39.264595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980649+0000","last_change":"2024-09-17T05:03:46.980649+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980649+0000","last_undegraded":"2024-09-17T05:03:46.980649+0000","last_fullsized":"2024-09-17T05:03:46.980649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.006223+0000","last_change":"2024-09-17T05:03:47.001970+0000","last_active":"2024-09-17T05:03:47.006223+0000","last_peered":"2024-09-17T05:03:47.006223+0000","last_clean":"2024-09-17T05:03:47.006223+0000","last_became_active":"2024-09-17T05:03:47.001531+0000","last_became_peered":"2024-09-17T05:03:47.001531+0000","last_unstale":"2024-09-17T05:03:47.006223+0000","last_undegraded":"2024-09-17T05:03:47.006223+0000","last_fullsized":"2024-09-17T05:03:47.006223+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:52.097905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001072+0000","last_change":"2024-09-17T05:03:48.030972+0000","last_active":"2024-09-17T05:03:49.001072+0000","last_peered":"2024-09-17T05:03:49.001072+0000","last_clean":"2024-09-17T05:03:49.001072+0000","last_became_active":"2024-09-17T05:03:48.030690+0000","last_became_peered":"2024-09-17T05:03:48.030690+0000","last_unstale":"2024-09-17T05:03:49.001072+0000","last_undegraded":"2024-09-17T05:03:49.001072+0000","last_fullsized":"2024-09-17T05:03:49.001072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000970+0000","last_change":"2024-09-17T05:03:46.985559+0000","last_active":"2024-09-17T05:03:49.000970+0000","last_peered":"2024-09-17T05:03:49.000970+0000","last_clean":"2024-09-17T05:03:49.000970+0000","last_became_active":"2024-09-17T05:03:46.985235+0000","last_became_peered":"2024-09-17T05:03:46.985235+0000","last_unstale":"2024-09-17T05:03:49.000970+0000","last_undegraded":"2024-09-17T05:03:49.000970+0000","last_fullsized":"2024-09-17T05:03:49.000970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:48.794376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991126+0000","last_change":"2024-09-17T05:03:46.991126+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991126+0000","last_undegraded":"2024-09-17T05:03:46.991126+0000","last_fullsized":"2024-09-17T05:03:46.991126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993187+0000","last_change":"2024-09-17T05:03:46.988419+0000","last_active":"2024-09-17T05:03:46.993187+0000","last_peered":"2024-09-17T05:03:46.993187+0000","last_clean":"2024-09-17T05:03:46.993187+0000","last_became_active":"2024-09-17T05:03:46.987659+0000","last_became_peered":"2024-09-17T05:03:46.987659+0000","last_unstale":"2024-09-17T05:03:46.993187+0000","last_undegraded":"2024-09-17T05:03:46.993187+0000","last_fullsized":"2024-09-17T05:03:46.993187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:07.848219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974798+0000","last_change":"2024-09-17T05:03:46.974798+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974798+0000","last_undegraded":"2024-09-17T05:03:46.974798+0000","last_fullsized":"2024-09-17T05:03:46.974798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000435+0000","last_change":"2024-09-17T05:03:46.994473+0000","last_active":"2024-09-17T05:03:49.000435+0000","last_peered":"2024-09-17T05:03:49.000435+0000","last_clean":"2024-09-17T05:03:49.000435+0000","last_became_active":"2024-09-17T05:03:46.994185+0000","last_became_peered":"2024-09-17T05:03:46.994185+0000","last_unstale":"2024-09-17T05:03:49.000435+0000","last_undegraded":"2024-09-17T05:03:49.000435+0000","last_fullsized":"2024-09-17T05:03:49.000435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.970693+0000","last_change":"2024-09-17T05:03:46.970693+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.970693+0000","last_undegraded":"2024-09-17T05:03:46.970693+0000","last_fullsized":"2024-09-17T05:03:46.970693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998045+0000","last_change":"2024-09-17T05:03:46.996248+0000","last_active":"2024-09-17T05:03:46.998045+0000","last_peered":"2024-09-17T05:03:46.998045+0000","last_clean":"2024-09-17T05:03:46.998045+0000","last_became_active":"2024-09-17T05:03:46.995891+0000","last_became_peered":"2024-09-17T05:03:46.995891+0000","last_unstale":"2024-09-17T05:03:46.998045+0000","last_undegraded":"2024-09-17T05:03:46.998045+0000","last_fullsized":"2024-09-17T05:03:46.998045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:00.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969366+0000","last_change":"2024-09-17T05:03:46.969366+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969366+0000","last_undegraded":"2024-09-17T05:03:46.969366+0000","last_fullsized":"2024-09-17T05:03:46.969366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997403+0000","last_change":"2024-09-17T05:03:46.990194+0000","last_active":"2024-09-17T05:03:46.997403+0000","last_peered":"2024-09-17T05:03:46.997403+0000","last_clean":"2024-09-17T05:03:46.997403+0000","last_became_active":"2024-09-17T05:03:46.989517+0000","last_became_peered":"2024-09-17T05:03:46.989517+0000","last_unstale":"2024-09-17T05:03:46.997403+0000","last_undegraded":"2024-09-17T05:03:46.997403+0000","last_fullsized":"2024-09-17T05:03:46.997403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:36.330582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978892+0000","last_change":"2024-09-17T05:03:46.978892+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978892+0000","last_undegraded":"2024-09-17T05:03:46.978892+0000","last_fullsized":"2024-09-17T05:03:46.978892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001218+0000","last_change":"2024-09-17T05:03:46.993382+0000","last_active":"2024-09-17T05:03:49.001218+0000","last_peered":"2024-09-17T05:03:49.001218+0000","last_clean":"2024-09-17T05:03:49.001218+0000","last_became_active":"2024-09-17T05:03:46.992226+0000","last_became_peered":"2024-09-17T05:03:46.992226+0000","last_unstale":"2024-09-17T05:03:49.001218+0000","last_undegraded":"2024-09-17T05:03:49.001218+0000","last_fullsized":"2024-09-17T05:03:49.001218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:56.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980152+0000","last_change":"2024-09-17T05:03:46.980152+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980152+0000","last_undegraded":"2024-09-17T05:03:46.980152+0000","last_fullsized":"2024-09-17T05:03:46.980152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000877+0000","last_change":"2024-09-17T05:03:46.995382+0000","last_active":"2024-09-17T05:03:47.000877+0000","last_peered":"2024-09-17T05:03:47.000877+0000","last_clean":"2024-09-17T05:03:47.000877+0000","last_became_active":"2024-09-17T05:03:46.995015+0000","last_became_peered":"2024-09-17T05:03:46.995015+0000","last_unstale":"2024-09-17T05:03:47.000877+0000","last_undegraded":"2024-09-17T05:03:47.000877+0000","last_fullsized":"2024-09-17T05:03:47.000877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:59.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974557+0000","last_change":"2024-09-17T05:03:46.974557+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974557+0000","last_undegraded":"2024-09-17T05:03:46.974557+0000","last_fullsized":"2024-09-17T05:03:46.974557+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001232+0000","last_change":"2024-09-17T05:03:46.993870+0000","last_active":"2024-09-17T05:03:47.001232+0000","last_peered":"2024-09-17T05:03:47.001232+0000","last_clean":"2024-09-17T05:03:47.001232+0000","last_became_active":"2024-09-17T05:03:46.993473+0000","last_became_peered":"2024-09-17T05:03:46.993473+0000","last_unstale":"2024-09-17T05:03:47.001232+0000","last_undegraded":"2024-09-17T05:03:47.001232+0000","last_fullsized":"2024-09-17T05:03:47.001232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:22.115429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986314+0000","last_change":"2024-09-17T05:03:46.986314+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986314+0000","last_undegraded":"2024-09-17T05:03:46.986314+0000","last_fullsized":"2024-09-17T05:03:46.986314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993909+0000","last_change":"2024-09-17T05:03:46.987076+0000","last_active":"2024-09-17T05:03:46.993909+0000","last_peered":"2024-09-17T05:03:46.993909+0000","last_clean":"2024-09-17T05:03:46.993909+0000","last_became_active":"2024-09-17T05:03:46.986715+0000","last_became_peered":"2024-09-17T05:03:46.986715+0000","last_unstale":"2024-09-17T05:03:46.993909+0000","last_undegraded":"2024-09-17T05:03:46.993909+0000","last_fullsized":"2024-09-17T05:03:46.993909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:14.999972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000704+0000","last_change":"2024-09-17T05:03:48.025679+0000","last_active":"2024-09-17T05:03:49.000704+0000","last_peered":"2024-09-17T05:03:49.000704+0000","last_clean":"2024-09-17T05:03:49.000704+0000","last_became_active":"2024-09-17T05:03:48.025319+0000","last_became_peered":"2024-09-17T05:03:48.025319+0000","last_unstale":"2024-09-17T05:03:49.000704+0000","last_undegraded":"2024-09-17T05:03:49.000704+0000","last_fullsized":"2024-09-17T05:03:49.000704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:33.812303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.995975+0000","last_change":"2024-09-17T05:03:46.991679+0000","last_active":"2024-09-17T05:03:46.995975+0000","last_peered":"2024-09-17T05:03:46.995975+0000","last_clean":"2024-09-17T05:03:46.995975+0000","last_became_active":"2024-09-17T05:03:46.990693+0000","last_became_peered":"2024-09-17T05:03:46.990693+0000","last_unstale":"2024-09-17T05:03:46.995975+0000","last_undegraded":"2024-09-17T05:03:46.995975+0000","last_fullsized":"2024-09-17T05:03:46.995975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:05.122749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969865+0000","last_change":"2024-09-17T05:03:46.969865+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969865+0000","last_undegraded":"2024-09-17T05:03:46.969865+0000","last_fullsized":"2024-09-17T05:03:46.969865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000628+0000","last_change":"2024-09-17T05:03:46.992882+0000","last_active":"2024-09-17T05:03:49.000628+0000","last_peered":"2024-09-17T05:03:49.000628+0000","last_clean":"2024-09-17T05:03:49.000628+0000","last_became_active":"2024-09-17T05:03:46.992558+0000","last_became_peered":"2024-09-17T05:03:46.992558+0000","last_unstale":"2024-09-17T05:03:49.000628+0000","last_undegraded":"2024-09-17T05:03:49.000628+0000","last_fullsized":"2024-09-17T05:03:49.000628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:35.489582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.973191+0000","last_change":"2024-09-17T05:03:46.973191+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.973191+0000","last_undegraded":"2024-09-17T05:03:46.973191+0000","last_fullsized":"2024-09-17T05:03:46.973191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000759+0000","last_change":"2024-09-17T05:03:46.982733+0000","last_active":"2024-09-17T05:03:49.000759+0000","last_peered":"2024-09-17T05:03:49.000759+0000","last_clean":"2024-09-17T05:03:49.000759+0000","last_became_active":"2024-09-17T05:03:46.982332+0000","last_became_peered":"2024-09-17T05:03:46.982332+0000","last_unstale":"2024-09-17T05:03:49.000759+0000","last_undegraded":"2024-09-17T05:03:49.000759+0000","last_fullsized":"2024-09-17T05:03:49.000759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:19.904729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":189,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.968891+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:46.968891+0000","last_peered":"2024-09-17T05:03:46.968891+0000","last_clean":"2024-09-17T05:03:46.968891+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:46.968891+0000","last_undegraded":"2024-09-17T05:03:46.968891+0000","last_fullsized":"2024-09-17T05:03:46.968891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":741376,"data_stored":329478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":741376,"data_stored":329478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":913179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1380352,"data_stored":935429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:52.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 -- 172.21.15.16:0/3470932839 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c15d600 msgr2=0x7f459c0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 --2- 172.21.15.16:0/3470932839 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c15d600 0x7f459c0cee50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f458c009900 tx=0x7f458c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 -- 172.21.15.16:0/3470932839 shutdown_connections 2024-09-17T05:03:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 --2- 172.21.15.16:0/3470932839 >> v2:172.21.15.104:3300/0 conn(0x7f459c0cf390 0x7f459c160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 --2- 172.21.15.16:0/3470932839 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c15d600 0x7f459c0cee50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 -- 172.21.15.16:0/3470932839 >> 172.21.15.16:0/3470932839 conn(0x7f459c06f900 msgr2=0x7f459c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 -- 172.21.15.16:0/3470932839 shutdown_connections 2024-09-17T05:03:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.405+0000 7f45a3000640 1 -- 172.21.15.16:0/3470932839 wait complete. 2024-09-17T05:03:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 Processor -- start 2024-09-17T05:03:52.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 -- start start 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 0x7f459c1f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f459c15d600 0x7f459c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c1606b0 0x7f459c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0b48b0 con 0x7f459c0cf390 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0b5280 con 0x7f459c15d600 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f459c0b4a30 con 0x7f459c1606b0 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 0x7f459c1f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 0x7f459c1f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33946/0 (socket says 172.21.15.16:33946) 2024-09-17T05:03:52.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 -- 172.21.15.16:0/1853494049 learned_addr learned my addr 172.21.15.16:0/1853494049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f4593e00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f459c15d600 0x7f459c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a1000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c1606b0 0x7f459c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 -- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c1606b0 msgr2=0x7f459c0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c1606b0 0x7f459c0cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 -- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f459c15d600 msgr2=0x7f459c155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f459c15d600 0x7f459c155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 -- 172.21.15.16:0/1853494049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0ca570 con 0x7f459c0cf390 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f4593e00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f459c15d600 0x7f459c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f459be00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 0x7f459c1f87b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f458c009850 tx=0x7f458c0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c0b2040 con 0x7f459c0cf390 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f458c0043b0 con 0x7f459c0cf390 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c005290 con 0x7f459c0cf390 2024-09-17T05:03:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c0ca850 con 0x7f459c0cf390 2024-09-17T05:03:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.409+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c1f8cf0 con 0x7f459c0cf390 2024-09-17T05:03:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.413+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c160170 con 0x7f459c0cf390 2024-09-17T05:03:52.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.413+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f458c005430 con 0x7f459c0cf390 2024-09-17T05:03:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.413+0000 7f4598c00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4578073230 0x7f4578075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.413+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f458c12dae0 con 0x7f459c0cf390 2024-09-17T05:03:52.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.413+0000 7f4593e00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4578073230 0x7f4578075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.413+0000 7f4593e00640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4578073230 0x7f4578075740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f459c15f160 tx=0x7f45840091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:52.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.417+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f458c0164d0 con 0x7f459c0cf390 2024-09-17T05:03:52.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.589+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f459c0ca320 con 0x7f4578073230 2024-09-17T05:03:52.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.593+0000 7f4598c00640 1 -- 172.21.15.16:0/1853494049 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247250 (secure 0 0 0) 0x7f459c0ca320 con 0x7f4578073230 2024-09-17T05:03:52.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:52.604 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:52.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4578073230 msgr2=0x7f4578075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4578073230 0x7f4578075740 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f459c15f160 tx=0x7f45840091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 msgr2=0x7f459c1f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 0x7f459c1f87b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f458c009850 tx=0x7f458c0a5710 comp rx=0 tx=0).stop 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 shutdown_connections 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4578073230 0x7f4578075740 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f459c1606b0 0x7f459c0cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f459c15d600 0x7f459c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 --2- 172.21.15.16:0/1853494049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f459c0cf390 0x7f459c1f87b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 >> 172.21.15.16:0/1853494049 conn(0x7f459c06f900 msgr2=0x7f459c15aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 shutdown_connections 2024-09-17T05:03:52.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.597+0000 7f45a3000640 1 -- 172.21.15.16:0/1853494049 wait complete. 2024-09-17T05:03:52.621 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-17T05:03:51.503375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":111,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244008,"kb_used_data":6936,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730712,"statfs":{"total":773094113280,"available":772844249088,"internally_reserved":0,"allocated":7102464,"data_stored":3798808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.541178"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978241+0000","last_change":"2024-09-17T05:03:46.978241+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978241+0000","last_undegraded":"2024-09-17T05:03:46.978241+0000","last_fullsized":"2024-09-17T05:03:46.978241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001275+0000","last_change":"2024-09-17T05:03:48.027266+0000","last_active":"2024-09-17T05:03:49.001275+0000","last_peered":"2024-09-17T05:03:49.001275+0000","last_clean":"2024-09-17T05:03:49.001275+0000","last_became_active":"2024-09-17T05:03:48.021805+0000","last_became_peered":"2024-09-17T05:03:48.021805+0000","last_unstale":"2024-09-17T05:03:49.001275+0000","last_undegraded":"2024-09-17T05:03:49.001275+0000","last_fullsized":"2024-09-17T05:03:49.001275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:19.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.984275+0000","last_change":"2024-09-17T05:03:46.984275+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.984275+0000","last_undegraded":"2024-09-17T05:03:46.984275+0000","last_fullsized":"2024-09-17T05:03:46.984275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977027+0000","last_change":"2024-09-17T05:03:46.977027+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977027+0000","last_undegraded":"2024-09-17T05:03:46.977027+0000","last_fullsized":"2024-09-17T05:03:46.977027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979040+0000","last_change":"2024-09-17T05:03:46.979040+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979040+0000","last_undegraded":"2024-09-17T05:03:46.979040+0000","last_fullsized":"2024-09-17T05:03:46.979040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000614+0000","last_change":"2024-09-17T05:03:48.031572+0000","last_active":"2024-09-17T05:03:49.000614+0000","last_peered":"2024-09-17T05:03:49.000614+0000","last_clean":"2024-09-17T05:03:49.000614+0000","last_became_active":"2024-09-17T05:03:48.031253+0000","last_became_peered":"2024-09-17T05:03:48.031253+0000","last_unstale":"2024-09-17T05:03:49.000614+0000","last_undegraded":"2024-09-17T05:03:49.000614+0000","last_fullsized":"2024-09-17T05:03:49.000614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:16.025505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978918+0000","last_change":"2024-09-17T05:03:46.978918+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978918+0000","last_undegraded":"2024-09-17T05:03:46.978918+0000","last_fullsized":"2024-09-17T05:03:46.978918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.983530+0000","last_change":"2024-09-17T05:03:46.983530+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.983530+0000","last_undegraded":"2024-09-17T05:03:46.983530+0000","last_fullsized":"2024-09-17T05:03:46.983530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979383+0000","last_change":"2024-09-17T05:03:46.979383+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979383+0000","last_undegraded":"2024-09-17T05:03:46.979383+0000","last_fullsized":"2024-09-17T05:03:46.979383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974260+0000","last_change":"2024-09-17T05:03:46.974260+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974260+0000","last_undegraded":"2024-09-17T05:03:46.974260+0000","last_fullsized":"2024-09-17T05:03:46.974260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978382+0000","last_change":"2024-09-17T05:03:46.978382+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978382+0000","last_undegraded":"2024-09-17T05:03:46.978382+0000","last_fullsized":"2024-09-17T05:03:46.978382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976518+0000","last_change":"2024-09-17T05:03:46.976518+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976518+0000","last_undegraded":"2024-09-17T05:03:46.976518+0000","last_fullsized":"2024-09-17T05:03:46.976518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986571+0000","last_change":"2024-09-17T05:03:46.986571+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986571+0000","last_undegraded":"2024-09-17T05:03:46.986571+0000","last_fullsized":"2024-09-17T05:03:46.986571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978501+0000","last_change":"2024-09-17T05:03:46.978501+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978501+0000","last_undegraded":"2024-09-17T05:03:46.978501+0000","last_fullsized":"2024-09-17T05:03:46.978501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976005+0000","last_change":"2024-09-17T05:03:46.976005+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976005+0000","last_undegraded":"2024-09-17T05:03:46.976005+0000","last_fullsized":"2024-09-17T05:03:46.976005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978421+0000","last_change":"2024-09-17T05:03:46.978421+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978421+0000","last_undegraded":"2024-09-17T05:03:46.978421+0000","last_fullsized":"2024-09-17T05:03:46.978421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969790+0000","last_change":"2024-09-17T05:03:46.969790+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969790+0000","last_undegraded":"2024-09-17T05:03:46.969790+0000","last_fullsized":"2024-09-17T05:03:46.969790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979878+0000","last_change":"2024-09-17T05:03:46.979878+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979878+0000","last_undegraded":"2024-09-17T05:03:46.979878+0000","last_fullsized":"2024-09-17T05:03:46.979878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977684+0000","last_change":"2024-09-17T05:03:46.977684+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977684+0000","last_undegraded":"2024-09-17T05:03:46.977684+0000","last_fullsized":"2024-09-17T05:03:46.977684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.971210+0000","last_change":"2024-09-17T05:03:46.971210+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.971210+0000","last_undegraded":"2024-09-17T05:03:46.971210+0000","last_fullsized":"2024-09-17T05:03:46.971210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982766+0000","last_change":"2024-09-17T05:03:46.982766+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982766+0000","last_undegraded":"2024-09-17T05:03:46.982766+0000","last_fullsized":"2024-09-17T05:03:46.982766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000912+0000","last_change":"2024-09-17T05:03:48.013364+0000","last_active":"2024-09-17T05:03:49.000912+0000","last_peered":"2024-09-17T05:03:49.000912+0000","last_clean":"2024-09-17T05:03:49.000912+0000","last_became_active":"2024-09-17T05:03:48.012273+0000","last_became_peered":"2024-09-17T05:03:48.012273+0000","last_unstale":"2024-09-17T05:03:49.000912+0000","last_undegraded":"2024-09-17T05:03:49.000912+0000","last_fullsized":"2024-09-17T05:03:49.000912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:46.993665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980950+0000","last_change":"2024-09-17T05:03:46.980950+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980950+0000","last_undegraded":"2024-09-17T05:03:46.980950+0000","last_fullsized":"2024-09-17T05:03:46.980950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982993+0000","last_change":"2024-09-17T05:03:46.982993+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982993+0000","last_undegraded":"2024-09-17T05:03:46.982993+0000","last_fullsized":"2024-09-17T05:03:46.982993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978589+0000","last_change":"2024-09-17T05:03:46.978589+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978589+0000","last_undegraded":"2024-09-17T05:03:46.978589+0000","last_fullsized":"2024-09-17T05:03:46.978589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979580+0000","last_change":"2024-09-17T05:03:46.979580+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979580+0000","last_undegraded":"2024-09-17T05:03:46.979580+0000","last_fullsized":"2024-09-17T05:03:46.979580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991002+0000","last_change":"2024-09-17T05:03:46.991002+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991002+0000","last_undegraded":"2024-09-17T05:03:46.991002+0000","last_fullsized":"2024-09-17T05:03:46.991002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980311+0000","last_change":"2024-09-17T05:03:46.980311+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980311+0000","last_undegraded":"2024-09-17T05:03:46.980311+0000","last_fullsized":"2024-09-17T05:03:46.980311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976442+0000","last_change":"2024-09-17T05:03:46.976442+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976442+0000","last_undegraded":"2024-09-17T05:03:46.976442+0000","last_fullsized":"2024-09-17T05:03:46.976442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975398+0000","last_change":"2024-09-17T05:03:46.975398+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975398+0000","last_undegraded":"2024-09-17T05:03:46.975398+0000","last_fullsized":"2024-09-17T05:03:46.975398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980342+0000","last_change":"2024-09-17T05:03:46.980342+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980342+0000","last_undegraded":"2024-09-17T05:03:46.980342+0000","last_fullsized":"2024-09-17T05:03:46.980342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977212+0000","last_change":"2024-09-17T05:03:46.977212+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977212+0000","last_undegraded":"2024-09-17T05:03:46.977212+0000","last_fullsized":"2024-09-17T05:03:46.977212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976032+0000","last_change":"2024-09-17T05:03:46.976032+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976032+0000","last_undegraded":"2024-09-17T05:03:46.976032+0000","last_fullsized":"2024-09-17T05:03:46.976032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993555+0000","last_change":"2024-09-17T05:03:46.987624+0000","last_active":"2024-09-17T05:03:46.993555+0000","last_peered":"2024-09-17T05:03:46.993555+0000","last_clean":"2024-09-17T05:03:46.993555+0000","last_became_active":"2024-09-17T05:03:46.987266+0000","last_became_peered":"2024-09-17T05:03:46.987266+0000","last_unstale":"2024-09-17T05:03:46.993555+0000","last_undegraded":"2024-09-17T05:03:46.993555+0000","last_fullsized":"2024-09-17T05:03:46.993555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:28.329239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.983834+0000","last_change":"2024-09-17T05:03:46.983834+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.983834+0000","last_undegraded":"2024-09-17T05:03:46.983834+0000","last_fullsized":"2024-09-17T05:03:46.983834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.996739+0000","last_change":"2024-09-17T05:03:46.993282+0000","last_active":"2024-09-17T05:03:46.996739+0000","last_peered":"2024-09-17T05:03:46.996739+0000","last_clean":"2024-09-17T05:03:46.996739+0000","last_became_active":"2024-09-17T05:03:46.992953+0000","last_became_peered":"2024-09-17T05:03:46.992953+0000","last_unstale":"2024-09-17T05:03:46.996739+0000","last_undegraded":"2024-09-17T05:03:46.996739+0000","last_fullsized":"2024-09-17T05:03:46.996739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:51.511991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001473+0000","last_change":"2024-09-17T05:03:48.023120+0000","last_active":"2024-09-17T05:03:49.001473+0000","last_peered":"2024-09-17T05:03:49.001473+0000","last_clean":"2024-09-17T05:03:49.001473+0000","last_became_active":"2024-09-17T05:03:48.022823+0000","last_became_peered":"2024-09-17T05:03:48.022823+0000","last_unstale":"2024-09-17T05:03:49.001473+0000","last_undegraded":"2024-09-17T05:03:49.001473+0000","last_fullsized":"2024-09-17T05:03:49.001473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:09.131399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.996709+0000","last_change":"2024-09-17T05:03:46.993381+0000","last_active":"2024-09-17T05:03:46.996709+0000","last_peered":"2024-09-17T05:03:46.996709+0000","last_clean":"2024-09-17T05:03:46.996709+0000","last_became_active":"2024-09-17T05:03:46.993086+0000","last_became_peered":"2024-09-17T05:03:46.993086+0000","last_unstale":"2024-09-17T05:03:46.996709+0000","last_undegraded":"2024-09-17T05:03:46.996709+0000","last_fullsized":"2024-09-17T05:03:46.996709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:00.606684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978829+0000","last_change":"2024-09-17T05:03:46.978829+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978829+0000","last_undegraded":"2024-09-17T05:03:46.978829+0000","last_fullsized":"2024-09-17T05:03:46.978829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001194+0000","last_change":"2024-09-17T05:03:46.989740+0000","last_active":"2024-09-17T05:03:49.001194+0000","last_peered":"2024-09-17T05:03:49.001194+0000","last_clean":"2024-09-17T05:03:49.001194+0000","last_became_active":"2024-09-17T05:03:46.988950+0000","last_became_peered":"2024-09-17T05:03:46.988950+0000","last_unstale":"2024-09-17T05:03:49.001194+0000","last_undegraded":"2024-09-17T05:03:49.001194+0000","last_fullsized":"2024-09-17T05:03:49.001194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:55.361318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969266+0000","last_change":"2024-09-17T05:03:46.969266+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969266+0000","last_undegraded":"2024-09-17T05:03:46.969266+0000","last_fullsized":"2024-09-17T05:03:46.969266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.989830+0000","last_change":"2024-09-17T05:03:46.984870+0000","last_active":"2024-09-17T05:03:46.989830+0000","last_peered":"2024-09-17T05:03:46.989830+0000","last_clean":"2024-09-17T05:03:46.989830+0000","last_became_active":"2024-09-17T05:03:46.984556+0000","last_became_peered":"2024-09-17T05:03:46.984556+0000","last_unstale":"2024-09-17T05:03:46.989830+0000","last_undegraded":"2024-09-17T05:03:46.989830+0000","last_fullsized":"2024-09-17T05:03:46.989830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:22.081970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978356+0000","last_change":"2024-09-17T05:03:46.978356+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978356+0000","last_undegraded":"2024-09-17T05:03:46.978356+0000","last_fullsized":"2024-09-17T05:03:46.978356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001024+0000","last_change":"2024-09-17T05:03:46.975859+0000","last_active":"2024-09-17T05:03:49.001024+0000","last_peered":"2024-09-17T05:03:49.001024+0000","last_clean":"2024-09-17T05:03:49.001024+0000","last_became_active":"2024-09-17T05:03:46.975513+0000","last_became_peered":"2024-09-17T05:03:46.975513+0000","last_unstale":"2024-09-17T05:03:49.001024+0000","last_undegraded":"2024-09-17T05:03:49.001024+0000","last_fullsized":"2024-09-17T05:03:49.001024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:24.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975768+0000","last_change":"2024-09-17T05:03:46.975768+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975768+0000","last_undegraded":"2024-09-17T05:03:46.975768+0000","last_fullsized":"2024-09-17T05:03:46.975768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001950+0000","last_change":"2024-09-17T05:03:46.993788+0000","last_active":"2024-09-17T05:03:47.001950+0000","last_peered":"2024-09-17T05:03:47.001950+0000","last_clean":"2024-09-17T05:03:47.001950+0000","last_became_active":"2024-09-17T05:03:46.993492+0000","last_became_peered":"2024-09-17T05:03:46.993492+0000","last_unstale":"2024-09-17T05:03:47.001950+0000","last_undegraded":"2024-09-17T05:03:47.001950+0000","last_fullsized":"2024-09-17T05:03:47.001950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:25.235304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.970441+0000","last_change":"2024-09-17T05:03:46.970441+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.970441+0000","last_undegraded":"2024-09-17T05:03:46.970441+0000","last_fullsized":"2024-09-17T05:03:46.970441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997457+0000","last_change":"2024-09-17T05:03:46.990873+0000","last_active":"2024-09-17T05:03:46.997457+0000","last_peered":"2024-09-17T05:03:46.997457+0000","last_clean":"2024-09-17T05:03:46.997457+0000","last_became_active":"2024-09-17T05:03:46.990380+0000","last_became_peered":"2024-09-17T05:03:46.990380+0000","last_unstale":"2024-09-17T05:03:46.997457+0000","last_undegraded":"2024-09-17T05:03:46.997457+0000","last_fullsized":"2024-09-17T05:03:46.997457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:06.392650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979741+0000","last_change":"2024-09-17T05:03:46.979741+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979741+0000","last_undegraded":"2024-09-17T05:03:46.979741+0000","last_fullsized":"2024-09-17T05:03:46.979741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997808+0000","last_change":"2024-09-17T05:03:46.994888+0000","last_active":"2024-09-17T05:03:46.997808+0000","last_peered":"2024-09-17T05:03:46.997808+0000","last_clean":"2024-09-17T05:03:46.997808+0000","last_became_active":"2024-09-17T05:03:46.994186+0000","last_became_peered":"2024-09-17T05:03:46.994186+0000","last_unstale":"2024-09-17T05:03:46.997808+0000","last_undegraded":"2024-09-17T05:03:46.997808+0000","last_fullsized":"2024-09-17T05:03:46.997808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:07.847464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969974+0000","last_change":"2024-09-17T05:03:46.969974+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969974+0000","last_undegraded":"2024-09-17T05:03:46.969974+0000","last_fullsized":"2024-09-17T05:03:46.969974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001298+0000","last_change":"2024-09-17T05:03:46.996818+0000","last_active":"2024-09-17T05:03:47.001298+0000","last_peered":"2024-09-17T05:03:47.001298+0000","last_clean":"2024-09-17T05:03:47.001298+0000","last_became_active":"2024-09-17T05:03:46.996508+0000","last_became_peered":"2024-09-17T05:03:46.996508+0000","last_unstale":"2024-09-17T05:03:47.001298+0000","last_undegraded":"2024-09-17T05:03:47.001298+0000","last_fullsized":"2024-09-17T05:03:47.001298+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:06.183013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991028+0000","last_change":"2024-09-17T05:03:46.991028+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991028+0000","last_undegraded":"2024-09-17T05:03:46.991028+0000","last_fullsized":"2024-09-17T05:03:46.991028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000750+0000","last_change":"2024-09-17T05:03:46.998029+0000","last_active":"2024-09-17T05:03:47.000750+0000","last_peered":"2024-09-17T05:03:47.000750+0000","last_clean":"2024-09-17T05:03:47.000750+0000","last_became_active":"2024-09-17T05:03:46.997688+0000","last_became_peered":"2024-09-17T05:03:46.997688+0000","last_unstale":"2024-09-17T05:03:47.000750+0000","last_undegraded":"2024-09-17T05:03:47.000750+0000","last_fullsized":"2024-09-17T05:03:47.000750+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:58.197722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982691+0000","last_change":"2024-09-17T05:03:46.982691+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982691+0000","last_undegraded":"2024-09-17T05:03:46.982691+0000","last_fullsized":"2024-09-17T05:03:46.982691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001031+0000","last_change":"2024-09-17T05:03:46.990156+0000","last_active":"2024-09-17T05:03:49.001031+0000","last_peered":"2024-09-17T05:03:49.001031+0000","last_clean":"2024-09-17T05:03:49.001031+0000","last_became_active":"2024-09-17T05:03:46.989449+0000","last_became_peered":"2024-09-17T05:03:46.989449+0000","last_unstale":"2024-09-17T05:03:49.001031+0000","last_undegraded":"2024-09-17T05:03:49.001031+0000","last_fullsized":"2024-09-17T05:03:49.001031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:45.812214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.973898+0000","last_change":"2024-09-17T05:03:46.973898+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.973898+0000","last_undegraded":"2024-09-17T05:03:46.973898+0000","last_fullsized":"2024-09-17T05:03:46.973898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.991309+0000","last_change":"2024-09-17T05:03:46.988141+0000","last_active":"2024-09-17T05:03:46.991309+0000","last_peered":"2024-09-17T05:03:46.991309+0000","last_clean":"2024-09-17T05:03:46.991309+0000","last_became_active":"2024-09-17T05:03:46.987762+0000","last_became_peered":"2024-09-17T05:03:46.987762+0000","last_unstale":"2024-09-17T05:03:46.991309+0000","last_undegraded":"2024-09-17T05:03:46.991309+0000","last_fullsized":"2024-09-17T05:03:46.991309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:20.658053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974975+0000","last_change":"2024-09-17T05:03:46.974975+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974975+0000","last_undegraded":"2024-09-17T05:03:46.974975+0000","last_fullsized":"2024-09-17T05:03:46.974975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.988663+0000","last_change":"2024-09-17T05:03:46.985782+0000","last_active":"2024-09-17T05:03:46.988663+0000","last_peered":"2024-09-17T05:03:46.988663+0000","last_clean":"2024-09-17T05:03:46.988663+0000","last_became_active":"2024-09-17T05:03:46.985426+0000","last_became_peered":"2024-09-17T05:03:46.985426+0000","last_unstale":"2024-09-17T05:03:46.988663+0000","last_undegraded":"2024-09-17T05:03:46.988663+0000","last_fullsized":"2024-09-17T05:03:46.988663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:15.869043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974547+0000","last_change":"2024-09-17T05:03:46.974547+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974547+0000","last_undegraded":"2024-09-17T05:03:46.974547+0000","last_fullsized":"2024-09-17T05:03:46.974547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000603+0000","last_change":"2024-09-17T05:03:46.992544+0000","last_active":"2024-09-17T05:03:47.000603+0000","last_peered":"2024-09-17T05:03:47.000603+0000","last_clean":"2024-09-17T05:03:47.000603+0000","last_became_active":"2024-09-17T05:03:46.992203+0000","last_became_peered":"2024-09-17T05:03:46.992203+0000","last_unstale":"2024-09-17T05:03:47.000603+0000","last_undegraded":"2024-09-17T05:03:47.000603+0000","last_fullsized":"2024-09-17T05:03:47.000603+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:07.389673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978996+0000","last_change":"2024-09-17T05:03:46.978996+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978996+0000","last_undegraded":"2024-09-17T05:03:46.978996+0000","last_fullsized":"2024-09-17T05:03:46.978996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998333+0000","last_change":"2024-09-17T05:03:46.989951+0000","last_active":"2024-09-17T05:03:46.998333+0000","last_peered":"2024-09-17T05:03:46.998333+0000","last_clean":"2024-09-17T05:03:46.998333+0000","last_became_active":"2024-09-17T05:03:46.989489+0000","last_became_peered":"2024-09-17T05:03:46.989489+0000","last_unstale":"2024-09-17T05:03:46.998333+0000","last_undegraded":"2024-09-17T05:03:46.998333+0000","last_fullsized":"2024-09-17T05:03:46.998333+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.475208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975516+0000","last_change":"2024-09-17T05:03:46.975516+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975516+0000","last_undegraded":"2024-09-17T05:03:46.975516+0000","last_fullsized":"2024-09-17T05:03:46.975516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997774+0000","last_change":"2024-09-17T05:03:46.994482+0000","last_active":"2024-09-17T05:03:46.997774+0000","last_peered":"2024-09-17T05:03:46.997774+0000","last_clean":"2024-09-17T05:03:46.997774+0000","last_became_active":"2024-09-17T05:03:46.993791+0000","last_became_peered":"2024-09-17T05:03:46.993791+0000","last_unstale":"2024-09-17T05:03:46.997774+0000","last_undegraded":"2024-09-17T05:03:46.997774+0000","last_fullsized":"2024-09-17T05:03:46.997774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:51.205489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986536+0000","last_change":"2024-09-17T05:03:46.986536+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986536+0000","last_undegraded":"2024-09-17T05:03:46.986536+0000","last_fullsized":"2024-09-17T05:03:46.986536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998376+0000","last_change":"2024-09-17T05:03:46.997619+0000","last_active":"2024-09-17T05:03:46.998376+0000","last_peered":"2024-09-17T05:03:46.998376+0000","last_clean":"2024-09-17T05:03:46.998376+0000","last_became_active":"2024-09-17T05:03:46.994779+0000","last_became_peered":"2024-09-17T05:03:46.994779+0000","last_unstale":"2024-09-17T05:03:46.998376+0000","last_undegraded":"2024-09-17T05:03:46.998376+0000","last_fullsized":"2024-09-17T05:03:46.998376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:52.009156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986807+0000","last_change":"2024-09-17T05:03:46.986807+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986807+0000","last_undegraded":"2024-09-17T05:03:46.986807+0000","last_fullsized":"2024-09-17T05:03:46.986807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000715+0000","last_change":"2024-09-17T05:03:46.994104+0000","last_active":"2024-09-17T05:03:47.000715+0000","last_peered":"2024-09-17T05:03:47.000715+0000","last_clean":"2024-09-17T05:03:47.000715+0000","last_became_active":"2024-09-17T05:03:46.993805+0000","last_became_peered":"2024-09-17T05:03:46.993805+0000","last_unstale":"2024-09-17T05:03:47.000715+0000","last_undegraded":"2024-09-17T05:03:47.000715+0000","last_fullsized":"2024-09-17T05:03:47.000715+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:39.264595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980649+0000","last_change":"2024-09-17T05:03:46.980649+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980649+0000","last_undegraded":"2024-09-17T05:03:46.980649+0000","last_fullsized":"2024-09-17T05:03:46.980649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.006223+0000","last_change":"2024-09-17T05:03:47.001970+0000","last_active":"2024-09-17T05:03:47.006223+0000","last_peered":"2024-09-17T05:03:47.006223+0000","last_clean":"2024-09-17T05:03:47.006223+0000","last_became_active":"2024-09-17T05:03:47.001531+0000","last_became_peered":"2024-09-17T05:03:47.001531+0000","last_unstale":"2024-09-17T05:03:47.006223+0000","last_undegraded":"2024-09-17T05:03:47.006223+0000","last_fullsized":"2024-09-17T05:03:47.006223+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:52.097905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001072+0000","last_change":"2024-09-17T05:03:48.030972+0000","last_active":"2024-09-17T05:03:49.001072+0000","last_peered":"2024-09-17T05:03:49.001072+0000","last_clean":"2024-09-17T05:03:49.001072+0000","last_became_active":"2024-09-17T05:03:48.030690+0000","last_became_peered":"2024-09-17T05:03:48.030690+0000","last_unstale":"2024-09-17T05:03:49.001072+0000","last_undegraded":"2024-09-17T05:03:49.001072+0000","last_fullsized":"2024-09-17T05:03:49.001072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000970+0000","last_change":"2024-09-17T05:03:46.985559+0000","last_active":"2024-09-17T05:03:49.000970+0000","last_peered":"2024-09-17T05:03:49.000970+0000","last_clean":"2024-09-17T05:03:49.000970+0000","last_became_active":"2024-09-17T05:03:46.985235+0000","last_became_peered":"2024-09-17T05:03:46.985235+0000","last_unstale":"2024-09-17T05:03:49.000970+0000","last_undegraded":"2024-09-17T05:03:49.000970+0000","last_fullsized":"2024-09-17T05:03:49.000970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:48.794376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991126+0000","last_change":"2024-09-17T05:03:46.991126+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991126+0000","last_undegraded":"2024-09-17T05:03:46.991126+0000","last_fullsized":"2024-09-17T05:03:46.991126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993187+0000","last_change":"2024-09-17T05:03:46.988419+0000","last_active":"2024-09-17T05:03:46.993187+0000","last_peered":"2024-09-17T05:03:46.993187+0000","last_clean":"2024-09-17T05:03:46.993187+0000","last_became_active":"2024-09-17T05:03:46.987659+0000","last_became_peered":"2024-09-17T05:03:46.987659+0000","last_unstale":"2024-09-17T05:03:46.993187+0000","last_undegraded":"2024-09-17T05:03:46.993187+0000","last_fullsized":"2024-09-17T05:03:46.993187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:07.848219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974798+0000","last_change":"2024-09-17T05:03:46.974798+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974798+0000","last_undegraded":"2024-09-17T05:03:46.974798+0000","last_fullsized":"2024-09-17T05:03:46.974798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000435+0000","last_change":"2024-09-17T05:03:46.994473+0000","last_active":"2024-09-17T05:03:49.000435+0000","last_peered":"2024-09-17T05:03:49.000435+0000","last_clean":"2024-09-17T05:03:49.000435+0000","last_became_active":"2024-09-17T05:03:46.994185+0000","last_became_peered":"2024-09-17T05:03:46.994185+0000","last_unstale":"2024-09-17T05:03:49.000435+0000","last_undegraded":"2024-09-17T05:03:49.000435+0000","last_fullsized":"2024-09-17T05:03:49.000435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.970693+0000","last_change":"2024-09-17T05:03:46.970693+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.970693+0000","last_undegraded":"2024-09-17T05:03:46.970693+0000","last_fullsized":"2024-09-17T05:03:46.970693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998045+0000","last_change":"2024-09-17T05:03:46.996248+0000","last_active":"2024-09-17T05:03:46.998045+0000","last_peered":"2024-09-17T05:03:46.998045+0000","last_clean":"2024-09-17T05:03:46.998045+0000","last_became_active":"2024-09-17T05:03:46.995891+0000","last_became_peered":"2024-09-17T05:03:46.995891+0000","last_unstale":"2024-09-17T05:03:46.998045+0000","last_undegraded":"2024-09-17T05:03:46.998045+0000","last_fullsized":"2024-09-17T05:03:46.998045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:00.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969366+0000","last_change":"2024-09-17T05:03:46.969366+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969366+0000","last_undegraded":"2024-09-17T05:03:46.969366+0000","last_fullsized":"2024-09-17T05:03:46.969366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997403+0000","last_change":"2024-09-17T05:03:46.990194+0000","last_active":"2024-09-17T05:03:46.997403+0000","last_peered":"2024-09-17T05:03:46.997403+0000","last_clean":"2024-09-17T05:03:46.997403+0000","last_became_active":"2024-09-17T05:03:46.989517+0000","last_became_peered":"2024-09-17T05:03:46.989517+0000","last_unstale":"2024-09-17T05:03:46.997403+0000","last_undegraded":"2024-09-17T05:03:46.997403+0000","last_fullsized":"2024-09-17T05:03:46.997403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:36.330582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978892+0000","last_change":"2024-09-17T05:03:46.978892+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978892+0000","last_undegraded":"2024-09-17T05:03:46.978892+0000","last_fullsized":"2024-09-17T05:03:46.978892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001218+0000","last_change":"2024-09-17T05:03:46.993382+0000","last_active":"2024-09-17T05:03:49.001218+0000","last_peered":"2024-09-17T05:03:49.001218+0000","last_clean":"2024-09-17T05:03:49.001218+0000","last_became_active":"2024-09-17T05:03:46.992226+0000","last_became_peered":"2024-09-17T05:03:46.992226+0000","last_unstale":"2024-09-17T05:03:49.001218+0000","last_undegraded":"2024-09-17T05:03:49.001218+0000","last_fullsized":"2024-09-17T05:03:49.001218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:56.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980152+0000","last_change":"2024-09-17T05:03:46.980152+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980152+0000","last_undegraded":"2024-09-17T05:03:46.980152+0000","last_fullsized":"2024-09-17T05:03:46.980152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000877+0000","last_change":"2024-09-17T05:03:46.995382+0000","last_active":"2024-09-17T05:03:47.000877+0000","last_peered":"2024-09-17T05:03:47.000877+0000","last_clean":"2024-09-17T05:03:47.000877+0000","last_became_active":"2024-09-17T05:03:46.995015+0000","last_became_peered":"2024-09-17T05:03:46.995015+0000","last_unstale":"2024-09-17T05:03:47.000877+0000","last_undegraded":"2024-09-17T05:03:47.000877+0000","last_fullsized":"2024-09-17T05:03:47.000877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:59.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974557+0000","last_change":"2024-09-17T05:03:46.974557+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974557+0000","last_undegraded":"2024-09-17T05:03:46.974557+0000","last_fullsized":"2024-09-17T05:03:46.974557+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001232+0000","last_change":"2024-09-17T05:03:46.993870+0000","last_active":"2024-09-17T05:03:47.001232+0000","last_peered":"2024-09-17T05:03:47.001232+0000","last_clean":"2024-09-17T05:03:47.001232+0000","last_became_active":"2024-09-17T05:03:46.993473+0000","last_became_peered":"2024-09-17T05:03:46.993473+0000","last_unstale":"2024-09-17T05:03:47.001232+0000","last_undegraded":"2024-09-17T05:03:47.001232+0000","last_fullsized":"2024-09-17T05:03:47.001232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:22.115429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986314+0000","last_change":"2024-09-17T05:03:46.986314+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986314+0000","last_undegraded":"2024-09-17T05:03:46.986314+0000","last_fullsized":"2024-09-17T05:03:46.986314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993909+0000","last_change":"2024-09-17T05:03:46.987076+0000","last_active":"2024-09-17T05:03:46.993909+0000","last_peered":"2024-09-17T05:03:46.993909+0000","last_clean":"2024-09-17T05:03:46.993909+0000","last_became_active":"2024-09-17T05:03:46.986715+0000","last_became_peered":"2024-09-17T05:03:46.986715+0000","last_unstale":"2024-09-17T05:03:46.993909+0000","last_undegraded":"2024-09-17T05:03:46.993909+0000","last_fullsized":"2024-09-17T05:03:46.993909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:14.999972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000704+0000","last_change":"2024-09-17T05:03:48.025679+0000","last_active":"2024-09-17T05:03:49.000704+0000","last_peered":"2024-09-17T05:03:49.000704+0000","last_clean":"2024-09-17T05:03:49.000704+0000","last_became_active":"2024-09-17T05:03:48.025319+0000","last_became_peered":"2024-09-17T05:03:48.025319+0000","last_unstale":"2024-09-17T05:03:49.000704+0000","last_undegraded":"2024-09-17T05:03:49.000704+0000","last_fullsized":"2024-09-17T05:03:49.000704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:33.812303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.995975+0000","last_change":"2024-09-17T05:03:46.991679+0000","last_active":"2024-09-17T05:03:46.995975+0000","last_peered":"2024-09-17T05:03:46.995975+0000","last_clean":"2024-09-17T05:03:46.995975+0000","last_became_active":"2024-09-17T05:03:46.990693+0000","last_became_peered":"2024-09-17T05:03:46.990693+0000","last_unstale":"2024-09-17T05:03:46.995975+0000","last_undegraded":"2024-09-17T05:03:46.995975+0000","last_fullsized":"2024-09-17T05:03:46.995975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:05.122749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969865+0000","last_change":"2024-09-17T05:03:46.969865+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969865+0000","last_undegraded":"2024-09-17T05:03:46.969865+0000","last_fullsized":"2024-09-17T05:03:46.969865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000628+0000","last_change":"2024-09-17T05:03:46.992882+0000","last_active":"2024-09-17T05:03:49.000628+0000","last_peered":"2024-09-17T05:03:49.000628+0000","last_clean":"2024-09-17T05:03:49.000628+0000","last_became_active":"2024-09-17T05:03:46.992558+0000","last_became_peered":"2024-09-17T05:03:46.992558+0000","last_unstale":"2024-09-17T05:03:49.000628+0000","last_undegraded":"2024-09-17T05:03:49.000628+0000","last_fullsized":"2024-09-17T05:03:49.000628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:35.489582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.973191+0000","last_change":"2024-09-17T05:03:46.973191+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.973191+0000","last_undegraded":"2024-09-17T05:03:46.973191+0000","last_fullsized":"2024-09-17T05:03:46.973191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000759+0000","last_change":"2024-09-17T05:03:46.982733+0000","last_active":"2024-09-17T05:03:49.000759+0000","last_peered":"2024-09-17T05:03:49.000759+0000","last_clean":"2024-09-17T05:03:49.000759+0000","last_became_active":"2024-09-17T05:03:46.982332+0000","last_became_peered":"2024-09-17T05:03:46.982332+0000","last_unstale":"2024-09-17T05:03:49.000759+0000","last_undegraded":"2024-09-17T05:03:49.000759+0000","last_fullsized":"2024-09-17T05:03:49.000759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:19.904729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":189,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.968891+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:46.968891+0000","last_peered":"2024-09-17T05:03:46.968891+0000","last_clean":"2024-09-17T05:03:46.968891+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:46.968891+0000","last_undegraded":"2024-09-17T05:03:46.968891+0000","last_fullsized":"2024-09-17T05:03:46.968891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":741376,"data_stored":329478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":741376,"data_stored":329478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":913179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1380352,"data_stored":935429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:52.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:52.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 -- 172.21.15.16:0/995108593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1815c270 msgr2=0x7f4c1815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 --2- 172.21.15.16:0/995108593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1815c270 0x7f4c1815c680 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009840 tx=0x7f4c0001c790 comp rx=0 tx=0).stop 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 -- 172.21.15.16:0/995108593 shutdown_connections 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 --2- 172.21.15.16:0/995108593 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4c18073240 0x7f4c18073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 --2- 172.21.15.16:0/995108593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c1815c270 0x7f4c1815c680 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 -- 172.21.15.16:0/995108593 >> 172.21.15.16:0/995108593 conn(0x7f4c1806f900 msgr2=0x7f4c180b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 -- 172.21.15.16:0/995108593 shutdown_connections 2024-09-17T05:03:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 -- 172.21.15.16:0/995108593 wait complete. 2024-09-17T05:03:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 Processor -- start 2024-09-17T05:03:52.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.821+0000 7f4c1d000640 1 -- start start 2024-09-17T05:03:52.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 0x7f4c181409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c180b2f50 0x7f4c180b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4c1815c270 0x7f4c180b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c18168cc0 con 0x7f4c18073240 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4c18168b40 con 0x7f4c180b2f50 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4c18168e40 con 0x7f4c1815c270 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 0x7f4c181409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 0x7f4c181409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33974/0 (socket says 172.21.15.16:33974) 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 -- 172.21.15.16:0/3792445865 learned_addr learned my addr 172.21.15.16:0/3792445865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c0fe00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c180b2f50 0x7f4c180b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c16800640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4c1815c270 0x7f4c180b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 -- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4c1815c270 msgr2=0x7f4c180b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4c1815c270 0x7f4c180b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 -- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c180b2f50 msgr2=0x7f4c180b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c180b2f50 0x7f4c180b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 -- 172.21.15.16:0/3792445865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c18140ee0 con 0x7f4c18073240 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c0fe00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c180b2f50 0x7f4c180b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c15e00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 0x7f4c181409a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4c08001ec0 tx=0x7f4c0800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08017070 con 0x7f4c18073240 2024-09-17T05:03:52.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c08003600 con 0x7f4c18073240 2024-09-17T05:03:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c18142110 con 0x7f4c18073240 2024-09-17T05:03:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08003a50 con 0x7f4c18073240 2024-09-17T05:03:52.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c181426d0 con 0x7f4c18073240 2024-09-17T05:03:52.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c18073650 con 0x7f4c18073240 2024-09-17T05:03:52.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.825+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4c0801a070 con 0x7f4c18073240 2024-09-17T05:03:52.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.829+0000 7f4c0e000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bf4073200 0x7f4bf4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:52.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.829+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f4c08093670 con 0x7f4c18073240 2024-09-17T05:03:52.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.829+0000 7f4c0fe00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bf4073200 0x7f4bf4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:52.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.829+0000 7f4c0fe00640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bf4073200 0x7f4bf4075710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c000af000 tx=0x7f4c0001afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:52.833+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4c0805e590 con 0x7f4c18073240 2024-09-17T05:03:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.001+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c180af970 con 0x7f4bf4073200 2024-09-17T05:03:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.005+0000 7f4c0e000640 1 -- 172.21.15.16:0/3792445865 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247250 (secure 0 0 0) 0x7f4c180af970 con 0x7f4bf4073200 2024-09-17T05:03:53.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bf4073200 msgr2=0x7f4bf4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bf4073200 0x7f4bf4075710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4c000af000 tx=0x7f4c0001afd0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 msgr2=0x7f4c181409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 0x7f4c181409a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4c08001ec0 tx=0x7f4c0800c890 comp rx=0 tx=0).stop 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 shutdown_connections 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bf4073200 0x7f4bf4075710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4c1815c270 0x7f4c180b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c180b2f50 0x7f4c180b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 --2- 172.21.15.16:0/3792445865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c18073240 0x7f4c181409a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 >> 172.21.15.16:0/3792445865 conn(0x7f4c1806f900 msgr2=0x7f4c1806e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 shutdown_connections 2024-09-17T05:03:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.009+0000 7f4c1d000640 1 -- 172.21.15.16:0/3792445865 wait complete. 2024-09-17T05:03:53.033 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-17T05:03:51.503375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":111,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":33,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244008,"kb_used_data":6936,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730712,"statfs":{"total":773094113280,"available":772844249088,"internally_reserved":0,"allocated":7102464,"data_stored":3798808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.541178"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978241+0000","last_change":"2024-09-17T05:03:46.978241+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978241+0000","last_undegraded":"2024-09-17T05:03:46.978241+0000","last_fullsized":"2024-09-17T05:03:46.978241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001275+0000","last_change":"2024-09-17T05:03:48.027266+0000","last_active":"2024-09-17T05:03:49.001275+0000","last_peered":"2024-09-17T05:03:49.001275+0000","last_clean":"2024-09-17T05:03:49.001275+0000","last_became_active":"2024-09-17T05:03:48.021805+0000","last_became_peered":"2024-09-17T05:03:48.021805+0000","last_unstale":"2024-09-17T05:03:49.001275+0000","last_undegraded":"2024-09-17T05:03:49.001275+0000","last_fullsized":"2024-09-17T05:03:49.001275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:19.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.984275+0000","last_change":"2024-09-17T05:03:46.984275+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.984275+0000","last_undegraded":"2024-09-17T05:03:46.984275+0000","last_fullsized":"2024-09-17T05:03:46.984275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977027+0000","last_change":"2024-09-17T05:03:46.977027+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977027+0000","last_undegraded":"2024-09-17T05:03:46.977027+0000","last_fullsized":"2024-09-17T05:03:46.977027+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979040+0000","last_change":"2024-09-17T05:03:46.979040+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979040+0000","last_undegraded":"2024-09-17T05:03:46.979040+0000","last_fullsized":"2024-09-17T05:03:46.979040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000614+0000","last_change":"2024-09-17T05:03:48.031572+0000","last_active":"2024-09-17T05:03:49.000614+0000","last_peered":"2024-09-17T05:03:49.000614+0000","last_clean":"2024-09-17T05:03:49.000614+0000","last_became_active":"2024-09-17T05:03:48.031253+0000","last_became_peered":"2024-09-17T05:03:48.031253+0000","last_unstale":"2024-09-17T05:03:49.000614+0000","last_undegraded":"2024-09-17T05:03:49.000614+0000","last_fullsized":"2024-09-17T05:03:49.000614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:16.025505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978918+0000","last_change":"2024-09-17T05:03:46.978918+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978918+0000","last_undegraded":"2024-09-17T05:03:46.978918+0000","last_fullsized":"2024-09-17T05:03:46.978918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.983530+0000","last_change":"2024-09-17T05:03:46.983530+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.983530+0000","last_undegraded":"2024-09-17T05:03:46.983530+0000","last_fullsized":"2024-09-17T05:03:46.983530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979383+0000","last_change":"2024-09-17T05:03:46.979383+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979383+0000","last_undegraded":"2024-09-17T05:03:46.979383+0000","last_fullsized":"2024-09-17T05:03:46.979383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974260+0000","last_change":"2024-09-17T05:03:46.974260+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974260+0000","last_undegraded":"2024-09-17T05:03:46.974260+0000","last_fullsized":"2024-09-17T05:03:46.974260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978382+0000","last_change":"2024-09-17T05:03:46.978382+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978382+0000","last_undegraded":"2024-09-17T05:03:46.978382+0000","last_fullsized":"2024-09-17T05:03:46.978382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976518+0000","last_change":"2024-09-17T05:03:46.976518+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976518+0000","last_undegraded":"2024-09-17T05:03:46.976518+0000","last_fullsized":"2024-09-17T05:03:46.976518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986571+0000","last_change":"2024-09-17T05:03:46.986571+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986571+0000","last_undegraded":"2024-09-17T05:03:46.986571+0000","last_fullsized":"2024-09-17T05:03:46.986571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978501+0000","last_change":"2024-09-17T05:03:46.978501+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978501+0000","last_undegraded":"2024-09-17T05:03:46.978501+0000","last_fullsized":"2024-09-17T05:03:46.978501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976005+0000","last_change":"2024-09-17T05:03:46.976005+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976005+0000","last_undegraded":"2024-09-17T05:03:46.976005+0000","last_fullsized":"2024-09-17T05:03:46.976005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978421+0000","last_change":"2024-09-17T05:03:46.978421+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978421+0000","last_undegraded":"2024-09-17T05:03:46.978421+0000","last_fullsized":"2024-09-17T05:03:46.978421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969790+0000","last_change":"2024-09-17T05:03:46.969790+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969790+0000","last_undegraded":"2024-09-17T05:03:46.969790+0000","last_fullsized":"2024-09-17T05:03:46.969790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979878+0000","last_change":"2024-09-17T05:03:46.979878+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979878+0000","last_undegraded":"2024-09-17T05:03:46.979878+0000","last_fullsized":"2024-09-17T05:03:46.979878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977684+0000","last_change":"2024-09-17T05:03:46.977684+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977684+0000","last_undegraded":"2024-09-17T05:03:46.977684+0000","last_fullsized":"2024-09-17T05:03:46.977684+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.971210+0000","last_change":"2024-09-17T05:03:46.971210+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.971210+0000","last_undegraded":"2024-09-17T05:03:46.971210+0000","last_fullsized":"2024-09-17T05:03:46.971210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982766+0000","last_change":"2024-09-17T05:03:46.982766+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982766+0000","last_undegraded":"2024-09-17T05:03:46.982766+0000","last_fullsized":"2024-09-17T05:03:46.982766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000912+0000","last_change":"2024-09-17T05:03:48.013364+0000","last_active":"2024-09-17T05:03:49.000912+0000","last_peered":"2024-09-17T05:03:49.000912+0000","last_clean":"2024-09-17T05:03:49.000912+0000","last_became_active":"2024-09-17T05:03:48.012273+0000","last_became_peered":"2024-09-17T05:03:48.012273+0000","last_unstale":"2024-09-17T05:03:49.000912+0000","last_undegraded":"2024-09-17T05:03:49.000912+0000","last_fullsized":"2024-09-17T05:03:49.000912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:46.993665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980950+0000","last_change":"2024-09-17T05:03:46.980950+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980950+0000","last_undegraded":"2024-09-17T05:03:46.980950+0000","last_fullsized":"2024-09-17T05:03:46.980950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982993+0000","last_change":"2024-09-17T05:03:46.982993+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982993+0000","last_undegraded":"2024-09-17T05:03:46.982993+0000","last_fullsized":"2024-09-17T05:03:46.982993+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978589+0000","last_change":"2024-09-17T05:03:46.978589+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978589+0000","last_undegraded":"2024-09-17T05:03:46.978589+0000","last_fullsized":"2024-09-17T05:03:46.978589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979580+0000","last_change":"2024-09-17T05:03:46.979580+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979580+0000","last_undegraded":"2024-09-17T05:03:46.979580+0000","last_fullsized":"2024-09-17T05:03:46.979580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991002+0000","last_change":"2024-09-17T05:03:46.991002+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991002+0000","last_undegraded":"2024-09-17T05:03:46.991002+0000","last_fullsized":"2024-09-17T05:03:46.991002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980311+0000","last_change":"2024-09-17T05:03:46.980311+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980311+0000","last_undegraded":"2024-09-17T05:03:46.980311+0000","last_fullsized":"2024-09-17T05:03:46.980311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976442+0000","last_change":"2024-09-17T05:03:46.976442+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976442+0000","last_undegraded":"2024-09-17T05:03:46.976442+0000","last_fullsized":"2024-09-17T05:03:46.976442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975398+0000","last_change":"2024-09-17T05:03:46.975398+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975398+0000","last_undegraded":"2024-09-17T05:03:46.975398+0000","last_fullsized":"2024-09-17T05:03:46.975398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980342+0000","last_change":"2024-09-17T05:03:46.980342+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980342+0000","last_undegraded":"2024-09-17T05:03:46.980342+0000","last_fullsized":"2024-09-17T05:03:46.980342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.977212+0000","last_change":"2024-09-17T05:03:46.977212+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.977212+0000","last_undegraded":"2024-09-17T05:03:46.977212+0000","last_fullsized":"2024-09-17T05:03:46.977212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.976032+0000","last_change":"2024-09-17T05:03:46.976032+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.976032+0000","last_undegraded":"2024-09-17T05:03:46.976032+0000","last_fullsized":"2024-09-17T05:03:46.976032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993555+0000","last_change":"2024-09-17T05:03:46.987624+0000","last_active":"2024-09-17T05:03:46.993555+0000","last_peered":"2024-09-17T05:03:46.993555+0000","last_clean":"2024-09-17T05:03:46.993555+0000","last_became_active":"2024-09-17T05:03:46.987266+0000","last_became_peered":"2024-09-17T05:03:46.987266+0000","last_unstale":"2024-09-17T05:03:46.993555+0000","last_undegraded":"2024-09-17T05:03:46.993555+0000","last_fullsized":"2024-09-17T05:03:46.993555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:28.329239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.983834+0000","last_change":"2024-09-17T05:03:46.983834+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.983834+0000","last_undegraded":"2024-09-17T05:03:46.983834+0000","last_fullsized":"2024-09-17T05:03:46.983834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.996739+0000","last_change":"2024-09-17T05:03:46.993282+0000","last_active":"2024-09-17T05:03:46.996739+0000","last_peered":"2024-09-17T05:03:46.996739+0000","last_clean":"2024-09-17T05:03:46.996739+0000","last_became_active":"2024-09-17T05:03:46.992953+0000","last_became_peered":"2024-09-17T05:03:46.992953+0000","last_unstale":"2024-09-17T05:03:46.996739+0000","last_undegraded":"2024-09-17T05:03:46.996739+0000","last_fullsized":"2024-09-17T05:03:46.996739+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:51.511991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001473+0000","last_change":"2024-09-17T05:03:48.023120+0000","last_active":"2024-09-17T05:03:49.001473+0000","last_peered":"2024-09-17T05:03:49.001473+0000","last_clean":"2024-09-17T05:03:49.001473+0000","last_became_active":"2024-09-17T05:03:48.022823+0000","last_became_peered":"2024-09-17T05:03:48.022823+0000","last_unstale":"2024-09-17T05:03:49.001473+0000","last_undegraded":"2024-09-17T05:03:49.001473+0000","last_fullsized":"2024-09-17T05:03:49.001473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:09.131399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.996709+0000","last_change":"2024-09-17T05:03:46.993381+0000","last_active":"2024-09-17T05:03:46.996709+0000","last_peered":"2024-09-17T05:03:46.996709+0000","last_clean":"2024-09-17T05:03:46.996709+0000","last_became_active":"2024-09-17T05:03:46.993086+0000","last_became_peered":"2024-09-17T05:03:46.993086+0000","last_unstale":"2024-09-17T05:03:46.996709+0000","last_undegraded":"2024-09-17T05:03:46.996709+0000","last_fullsized":"2024-09-17T05:03:46.996709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:00.606684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978829+0000","last_change":"2024-09-17T05:03:46.978829+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978829+0000","last_undegraded":"2024-09-17T05:03:46.978829+0000","last_fullsized":"2024-09-17T05:03:46.978829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001194+0000","last_change":"2024-09-17T05:03:46.989740+0000","last_active":"2024-09-17T05:03:49.001194+0000","last_peered":"2024-09-17T05:03:49.001194+0000","last_clean":"2024-09-17T05:03:49.001194+0000","last_became_active":"2024-09-17T05:03:46.988950+0000","last_became_peered":"2024-09-17T05:03:46.988950+0000","last_unstale":"2024-09-17T05:03:49.001194+0000","last_undegraded":"2024-09-17T05:03:49.001194+0000","last_fullsized":"2024-09-17T05:03:49.001194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:55.361318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969266+0000","last_change":"2024-09-17T05:03:46.969266+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969266+0000","last_undegraded":"2024-09-17T05:03:46.969266+0000","last_fullsized":"2024-09-17T05:03:46.969266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.989830+0000","last_change":"2024-09-17T05:03:46.984870+0000","last_active":"2024-09-17T05:03:46.989830+0000","last_peered":"2024-09-17T05:03:46.989830+0000","last_clean":"2024-09-17T05:03:46.989830+0000","last_became_active":"2024-09-17T05:03:46.984556+0000","last_became_peered":"2024-09-17T05:03:46.984556+0000","last_unstale":"2024-09-17T05:03:46.989830+0000","last_undegraded":"2024-09-17T05:03:46.989830+0000","last_fullsized":"2024-09-17T05:03:46.989830+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:22.081970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978356+0000","last_change":"2024-09-17T05:03:46.978356+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978356+0000","last_undegraded":"2024-09-17T05:03:46.978356+0000","last_fullsized":"2024-09-17T05:03:46.978356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001024+0000","last_change":"2024-09-17T05:03:46.975859+0000","last_active":"2024-09-17T05:03:49.001024+0000","last_peered":"2024-09-17T05:03:49.001024+0000","last_clean":"2024-09-17T05:03:49.001024+0000","last_became_active":"2024-09-17T05:03:46.975513+0000","last_became_peered":"2024-09-17T05:03:46.975513+0000","last_unstale":"2024-09-17T05:03:49.001024+0000","last_undegraded":"2024-09-17T05:03:49.001024+0000","last_fullsized":"2024-09-17T05:03:49.001024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:24.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975768+0000","last_change":"2024-09-17T05:03:46.975768+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975768+0000","last_undegraded":"2024-09-17T05:03:46.975768+0000","last_fullsized":"2024-09-17T05:03:46.975768+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001950+0000","last_change":"2024-09-17T05:03:46.993788+0000","last_active":"2024-09-17T05:03:47.001950+0000","last_peered":"2024-09-17T05:03:47.001950+0000","last_clean":"2024-09-17T05:03:47.001950+0000","last_became_active":"2024-09-17T05:03:46.993492+0000","last_became_peered":"2024-09-17T05:03:46.993492+0000","last_unstale":"2024-09-17T05:03:47.001950+0000","last_undegraded":"2024-09-17T05:03:47.001950+0000","last_fullsized":"2024-09-17T05:03:47.001950+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:25.235304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.970441+0000","last_change":"2024-09-17T05:03:46.970441+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.970441+0000","last_undegraded":"2024-09-17T05:03:46.970441+0000","last_fullsized":"2024-09-17T05:03:46.970441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997457+0000","last_change":"2024-09-17T05:03:46.990873+0000","last_active":"2024-09-17T05:03:46.997457+0000","last_peered":"2024-09-17T05:03:46.997457+0000","last_clean":"2024-09-17T05:03:46.997457+0000","last_became_active":"2024-09-17T05:03:46.990380+0000","last_became_peered":"2024-09-17T05:03:46.990380+0000","last_unstale":"2024-09-17T05:03:46.997457+0000","last_undegraded":"2024-09-17T05:03:46.997457+0000","last_fullsized":"2024-09-17T05:03:46.997457+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:06.392650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.979741+0000","last_change":"2024-09-17T05:03:46.979741+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.979741+0000","last_undegraded":"2024-09-17T05:03:46.979741+0000","last_fullsized":"2024-09-17T05:03:46.979741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997808+0000","last_change":"2024-09-17T05:03:46.994888+0000","last_active":"2024-09-17T05:03:46.997808+0000","last_peered":"2024-09-17T05:03:46.997808+0000","last_clean":"2024-09-17T05:03:46.997808+0000","last_became_active":"2024-09-17T05:03:46.994186+0000","last_became_peered":"2024-09-17T05:03:46.994186+0000","last_unstale":"2024-09-17T05:03:46.997808+0000","last_undegraded":"2024-09-17T05:03:46.997808+0000","last_fullsized":"2024-09-17T05:03:46.997808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:07.847464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969974+0000","last_change":"2024-09-17T05:03:46.969974+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969974+0000","last_undegraded":"2024-09-17T05:03:46.969974+0000","last_fullsized":"2024-09-17T05:03:46.969974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001298+0000","last_change":"2024-09-17T05:03:46.996818+0000","last_active":"2024-09-17T05:03:47.001298+0000","last_peered":"2024-09-17T05:03:47.001298+0000","last_clean":"2024-09-17T05:03:47.001298+0000","last_became_active":"2024-09-17T05:03:46.996508+0000","last_became_peered":"2024-09-17T05:03:46.996508+0000","last_unstale":"2024-09-17T05:03:47.001298+0000","last_undegraded":"2024-09-17T05:03:47.001298+0000","last_fullsized":"2024-09-17T05:03:47.001298+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:06.183013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991028+0000","last_change":"2024-09-17T05:03:46.991028+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991028+0000","last_undegraded":"2024-09-17T05:03:46.991028+0000","last_fullsized":"2024-09-17T05:03:46.991028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000750+0000","last_change":"2024-09-17T05:03:46.998029+0000","last_active":"2024-09-17T05:03:47.000750+0000","last_peered":"2024-09-17T05:03:47.000750+0000","last_clean":"2024-09-17T05:03:47.000750+0000","last_became_active":"2024-09-17T05:03:46.997688+0000","last_became_peered":"2024-09-17T05:03:46.997688+0000","last_unstale":"2024-09-17T05:03:47.000750+0000","last_undegraded":"2024-09-17T05:03:47.000750+0000","last_fullsized":"2024-09-17T05:03:47.000750+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:30:58.197722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.982691+0000","last_change":"2024-09-17T05:03:46.982691+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.982691+0000","last_undegraded":"2024-09-17T05:03:46.982691+0000","last_fullsized":"2024-09-17T05:03:46.982691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001031+0000","last_change":"2024-09-17T05:03:46.990156+0000","last_active":"2024-09-17T05:03:49.001031+0000","last_peered":"2024-09-17T05:03:49.001031+0000","last_clean":"2024-09-17T05:03:49.001031+0000","last_became_active":"2024-09-17T05:03:46.989449+0000","last_became_peered":"2024-09-17T05:03:46.989449+0000","last_unstale":"2024-09-17T05:03:49.001031+0000","last_undegraded":"2024-09-17T05:03:49.001031+0000","last_fullsized":"2024-09-17T05:03:49.001031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:45.812214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.973898+0000","last_change":"2024-09-17T05:03:46.973898+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.973898+0000","last_undegraded":"2024-09-17T05:03:46.973898+0000","last_fullsized":"2024-09-17T05:03:46.973898+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.991309+0000","last_change":"2024-09-17T05:03:46.988141+0000","last_active":"2024-09-17T05:03:46.991309+0000","last_peered":"2024-09-17T05:03:46.991309+0000","last_clean":"2024-09-17T05:03:46.991309+0000","last_became_active":"2024-09-17T05:03:46.987762+0000","last_became_peered":"2024-09-17T05:03:46.987762+0000","last_unstale":"2024-09-17T05:03:46.991309+0000","last_undegraded":"2024-09-17T05:03:46.991309+0000","last_fullsized":"2024-09-17T05:03:46.991309+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:20.658053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974975+0000","last_change":"2024-09-17T05:03:46.974975+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974975+0000","last_undegraded":"2024-09-17T05:03:46.974975+0000","last_fullsized":"2024-09-17T05:03:46.974975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.988663+0000","last_change":"2024-09-17T05:03:46.985782+0000","last_active":"2024-09-17T05:03:46.988663+0000","last_peered":"2024-09-17T05:03:46.988663+0000","last_clean":"2024-09-17T05:03:46.988663+0000","last_became_active":"2024-09-17T05:03:46.985426+0000","last_became_peered":"2024-09-17T05:03:46.985426+0000","last_unstale":"2024-09-17T05:03:46.988663+0000","last_undegraded":"2024-09-17T05:03:46.988663+0000","last_fullsized":"2024-09-17T05:03:46.988663+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:15.869043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974547+0000","last_change":"2024-09-17T05:03:46.974547+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974547+0000","last_undegraded":"2024-09-17T05:03:46.974547+0000","last_fullsized":"2024-09-17T05:03:46.974547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000603+0000","last_change":"2024-09-17T05:03:46.992544+0000","last_active":"2024-09-17T05:03:47.000603+0000","last_peered":"2024-09-17T05:03:47.000603+0000","last_clean":"2024-09-17T05:03:47.000603+0000","last_became_active":"2024-09-17T05:03:46.992203+0000","last_became_peered":"2024-09-17T05:03:46.992203+0000","last_unstale":"2024-09-17T05:03:47.000603+0000","last_undegraded":"2024-09-17T05:03:47.000603+0000","last_fullsized":"2024-09-17T05:03:47.000603+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:07.389673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978996+0000","last_change":"2024-09-17T05:03:46.978996+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978996+0000","last_undegraded":"2024-09-17T05:03:46.978996+0000","last_fullsized":"2024-09-17T05:03:46.978996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998333+0000","last_change":"2024-09-17T05:03:46.989951+0000","last_active":"2024-09-17T05:03:46.998333+0000","last_peered":"2024-09-17T05:03:46.998333+0000","last_clean":"2024-09-17T05:03:46.998333+0000","last_became_active":"2024-09-17T05:03:46.989489+0000","last_became_peered":"2024-09-17T05:03:46.989489+0000","last_unstale":"2024-09-17T05:03:46.998333+0000","last_undegraded":"2024-09-17T05:03:46.998333+0000","last_fullsized":"2024-09-17T05:03:46.998333+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.475208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.975516+0000","last_change":"2024-09-17T05:03:46.975516+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.975516+0000","last_undegraded":"2024-09-17T05:03:46.975516+0000","last_fullsized":"2024-09-17T05:03:46.975516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997774+0000","last_change":"2024-09-17T05:03:46.994482+0000","last_active":"2024-09-17T05:03:46.997774+0000","last_peered":"2024-09-17T05:03:46.997774+0000","last_clean":"2024-09-17T05:03:46.997774+0000","last_became_active":"2024-09-17T05:03:46.993791+0000","last_became_peered":"2024-09-17T05:03:46.993791+0000","last_unstale":"2024-09-17T05:03:46.997774+0000","last_undegraded":"2024-09-17T05:03:46.997774+0000","last_fullsized":"2024-09-17T05:03:46.997774+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:51.205489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986536+0000","last_change":"2024-09-17T05:03:46.986536+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986536+0000","last_undegraded":"2024-09-17T05:03:46.986536+0000","last_fullsized":"2024-09-17T05:03:46.986536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998376+0000","last_change":"2024-09-17T05:03:46.997619+0000","last_active":"2024-09-17T05:03:46.998376+0000","last_peered":"2024-09-17T05:03:46.998376+0000","last_clean":"2024-09-17T05:03:46.998376+0000","last_became_active":"2024-09-17T05:03:46.994779+0000","last_became_peered":"2024-09-17T05:03:46.994779+0000","last_unstale":"2024-09-17T05:03:46.998376+0000","last_undegraded":"2024-09-17T05:03:46.998376+0000","last_fullsized":"2024-09-17T05:03:46.998376+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:52.009156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986807+0000","last_change":"2024-09-17T05:03:46.986807+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986807+0000","last_undegraded":"2024-09-17T05:03:46.986807+0000","last_fullsized":"2024-09-17T05:03:46.986807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000715+0000","last_change":"2024-09-17T05:03:46.994104+0000","last_active":"2024-09-17T05:03:47.000715+0000","last_peered":"2024-09-17T05:03:47.000715+0000","last_clean":"2024-09-17T05:03:47.000715+0000","last_became_active":"2024-09-17T05:03:46.993805+0000","last_became_peered":"2024-09-17T05:03:46.993805+0000","last_unstale":"2024-09-17T05:03:47.000715+0000","last_undegraded":"2024-09-17T05:03:47.000715+0000","last_fullsized":"2024-09-17T05:03:47.000715+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:39.264595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980649+0000","last_change":"2024-09-17T05:03:46.980649+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980649+0000","last_undegraded":"2024-09-17T05:03:46.980649+0000","last_fullsized":"2024-09-17T05:03:46.980649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.006223+0000","last_change":"2024-09-17T05:03:47.001970+0000","last_active":"2024-09-17T05:03:47.006223+0000","last_peered":"2024-09-17T05:03:47.006223+0000","last_clean":"2024-09-17T05:03:47.006223+0000","last_became_active":"2024-09-17T05:03:47.001531+0000","last_became_peered":"2024-09-17T05:03:47.001531+0000","last_unstale":"2024-09-17T05:03:47.006223+0000","last_undegraded":"2024-09-17T05:03:47.006223+0000","last_fullsized":"2024-09-17T05:03:47.006223+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:52.097905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001072+0000","last_change":"2024-09-17T05:03:48.030972+0000","last_active":"2024-09-17T05:03:49.001072+0000","last_peered":"2024-09-17T05:03:49.001072+0000","last_clean":"2024-09-17T05:03:49.001072+0000","last_became_active":"2024-09-17T05:03:48.030690+0000","last_became_peered":"2024-09-17T05:03:48.030690+0000","last_unstale":"2024-09-17T05:03:49.001072+0000","last_undegraded":"2024-09-17T05:03:49.001072+0000","last_fullsized":"2024-09-17T05:03:49.001072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000970+0000","last_change":"2024-09-17T05:03:46.985559+0000","last_active":"2024-09-17T05:03:49.000970+0000","last_peered":"2024-09-17T05:03:49.000970+0000","last_clean":"2024-09-17T05:03:49.000970+0000","last_became_active":"2024-09-17T05:03:46.985235+0000","last_became_peered":"2024-09-17T05:03:46.985235+0000","last_unstale":"2024-09-17T05:03:49.000970+0000","last_undegraded":"2024-09-17T05:03:49.000970+0000","last_fullsized":"2024-09-17T05:03:49.000970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:48.794376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.991126+0000","last_change":"2024-09-17T05:03:46.991126+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.991126+0000","last_undegraded":"2024-09-17T05:03:46.991126+0000","last_fullsized":"2024-09-17T05:03:46.991126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993187+0000","last_change":"2024-09-17T05:03:46.988419+0000","last_active":"2024-09-17T05:03:46.993187+0000","last_peered":"2024-09-17T05:03:46.993187+0000","last_clean":"2024-09-17T05:03:46.993187+0000","last_became_active":"2024-09-17T05:03:46.987659+0000","last_became_peered":"2024-09-17T05:03:46.987659+0000","last_unstale":"2024-09-17T05:03:46.993187+0000","last_undegraded":"2024-09-17T05:03:46.993187+0000","last_fullsized":"2024-09-17T05:03:46.993187+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:07.848219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974798+0000","last_change":"2024-09-17T05:03:46.974798+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974798+0000","last_undegraded":"2024-09-17T05:03:46.974798+0000","last_fullsized":"2024-09-17T05:03:46.974798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000435+0000","last_change":"2024-09-17T05:03:46.994473+0000","last_active":"2024-09-17T05:03:49.000435+0000","last_peered":"2024-09-17T05:03:49.000435+0000","last_clean":"2024-09-17T05:03:49.000435+0000","last_became_active":"2024-09-17T05:03:46.994185+0000","last_became_peered":"2024-09-17T05:03:46.994185+0000","last_unstale":"2024-09-17T05:03:49.000435+0000","last_undegraded":"2024-09-17T05:03:49.000435+0000","last_fullsized":"2024-09-17T05:03:49.000435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.970693+0000","last_change":"2024-09-17T05:03:46.970693+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.970693+0000","last_undegraded":"2024-09-17T05:03:46.970693+0000","last_fullsized":"2024-09-17T05:03:46.970693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.998045+0000","last_change":"2024-09-17T05:03:46.996248+0000","last_active":"2024-09-17T05:03:46.998045+0000","last_peered":"2024-09-17T05:03:46.998045+0000","last_clean":"2024-09-17T05:03:46.998045+0000","last_became_active":"2024-09-17T05:03:46.995891+0000","last_became_peered":"2024-09-17T05:03:46.995891+0000","last_unstale":"2024-09-17T05:03:46.998045+0000","last_undegraded":"2024-09-17T05:03:46.998045+0000","last_fullsized":"2024-09-17T05:03:46.998045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:00.929936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969366+0000","last_change":"2024-09-17T05:03:46.969366+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969366+0000","last_undegraded":"2024-09-17T05:03:46.969366+0000","last_fullsized":"2024-09-17T05:03:46.969366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.997403+0000","last_change":"2024-09-17T05:03:46.990194+0000","last_active":"2024-09-17T05:03:46.997403+0000","last_peered":"2024-09-17T05:03:46.997403+0000","last_clean":"2024-09-17T05:03:46.997403+0000","last_became_active":"2024-09-17T05:03:46.989517+0000","last_became_peered":"2024-09-17T05:03:46.989517+0000","last_unstale":"2024-09-17T05:03:46.997403+0000","last_undegraded":"2024-09-17T05:03:46.997403+0000","last_fullsized":"2024-09-17T05:03:46.997403+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:36.330582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.978892+0000","last_change":"2024-09-17T05:03:46.978892+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.978892+0000","last_undegraded":"2024-09-17T05:03:46.978892+0000","last_fullsized":"2024-09-17T05:03:46.978892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001218+0000","last_change":"2024-09-17T05:03:46.993382+0000","last_active":"2024-09-17T05:03:49.001218+0000","last_peered":"2024-09-17T05:03:49.001218+0000","last_clean":"2024-09-17T05:03:49.001218+0000","last_became_active":"2024-09-17T05:03:46.992226+0000","last_became_peered":"2024-09-17T05:03:46.992226+0000","last_unstale":"2024-09-17T05:03:49.001218+0000","last_undegraded":"2024-09-17T05:03:49.001218+0000","last_fullsized":"2024-09-17T05:03:49.001218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:56.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.980152+0000","last_change":"2024-09-17T05:03:46.980152+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.980152+0000","last_undegraded":"2024-09-17T05:03:46.980152+0000","last_fullsized":"2024-09-17T05:03:46.980152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.000877+0000","last_change":"2024-09-17T05:03:46.995382+0000","last_active":"2024-09-17T05:03:47.000877+0000","last_peered":"2024-09-17T05:03:47.000877+0000","last_clean":"2024-09-17T05:03:47.000877+0000","last_became_active":"2024-09-17T05:03:46.995015+0000","last_became_peered":"2024-09-17T05:03:46.995015+0000","last_unstale":"2024-09-17T05:03:47.000877+0000","last_undegraded":"2024-09-17T05:03:47.000877+0000","last_fullsized":"2024-09-17T05:03:47.000877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:27:59.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.974557+0000","last_change":"2024-09-17T05:03:46.974557+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.974557+0000","last_undegraded":"2024-09-17T05:03:46.974557+0000","last_fullsized":"2024-09-17T05:03:46.974557+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:47.001232+0000","last_change":"2024-09-17T05:03:46.993870+0000","last_active":"2024-09-17T05:03:47.001232+0000","last_peered":"2024-09-17T05:03:47.001232+0000","last_clean":"2024-09-17T05:03:47.001232+0000","last_became_active":"2024-09-17T05:03:46.993473+0000","last_became_peered":"2024-09-17T05:03:46.993473+0000","last_unstale":"2024-09-17T05:03:47.001232+0000","last_undegraded":"2024-09-17T05:03:47.001232+0000","last_fullsized":"2024-09-17T05:03:47.001232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:07:22.115429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.986314+0000","last_change":"2024-09-17T05:03:46.986314+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.986314+0000","last_undegraded":"2024-09-17T05:03:46.986314+0000","last_fullsized":"2024-09-17T05:03:46.986314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.993909+0000","last_change":"2024-09-17T05:03:46.987076+0000","last_active":"2024-09-17T05:03:46.993909+0000","last_peered":"2024-09-17T05:03:46.993909+0000","last_clean":"2024-09-17T05:03:46.993909+0000","last_became_active":"2024-09-17T05:03:46.986715+0000","last_became_peered":"2024-09-17T05:03:46.986715+0000","last_unstale":"2024-09-17T05:03:46.993909+0000","last_undegraded":"2024-09-17T05:03:46.993909+0000","last_fullsized":"2024-09-17T05:03:46.993909+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:26:14.999972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000704+0000","last_change":"2024-09-17T05:03:48.025679+0000","last_active":"2024-09-17T05:03:49.000704+0000","last_peered":"2024-09-17T05:03:49.000704+0000","last_clean":"2024-09-17T05:03:49.000704+0000","last_became_active":"2024-09-17T05:03:48.025319+0000","last_became_peered":"2024-09-17T05:03:48.025319+0000","last_unstale":"2024-09-17T05:03:49.000704+0000","last_undegraded":"2024-09-17T05:03:49.000704+0000","last_fullsized":"2024-09-17T05:03:49.000704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:33.812303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.995975+0000","last_change":"2024-09-17T05:03:46.991679+0000","last_active":"2024-09-17T05:03:46.995975+0000","last_peered":"2024-09-17T05:03:46.995975+0000","last_clean":"2024-09-17T05:03:46.995975+0000","last_became_active":"2024-09-17T05:03:46.990693+0000","last_became_peered":"2024-09-17T05:03:46.990693+0000","last_unstale":"2024-09-17T05:03:46.995975+0000","last_undegraded":"2024-09-17T05:03:46.995975+0000","last_fullsized":"2024-09-17T05:03:46.995975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:05.122749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.969865+0000","last_change":"2024-09-17T05:03:46.969865+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.969865+0000","last_undegraded":"2024-09-17T05:03:46.969865+0000","last_fullsized":"2024-09-17T05:03:46.969865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000628+0000","last_change":"2024-09-17T05:03:46.992882+0000","last_active":"2024-09-17T05:03:49.000628+0000","last_peered":"2024-09-17T05:03:49.000628+0000","last_clean":"2024-09-17T05:03:49.000628+0000","last_became_active":"2024-09-17T05:03:46.992558+0000","last_became_peered":"2024-09-17T05:03:46.992558+0000","last_unstale":"2024-09-17T05:03:49.000628+0000","last_undegraded":"2024-09-17T05:03:49.000628+0000","last_fullsized":"2024-09-17T05:03:49.000628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:35.489582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":62,"state":"creating+peering","last_fresh":"2024-09-17T05:03:46.973191+0000","last_change":"2024-09-17T05:03:46.973191+0000","last_active":"2024-09-17T05:03:46.961444+0000","last_peered":"2024-09-17T05:03:46.961444+0000","last_clean":"2024-09-17T05:03:46.961444+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:03:46.973191+0000","last_undegraded":"2024-09-17T05:03:46.973191+0000","last_fullsized":"2024-09-17T05:03:46.973191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000759+0000","last_change":"2024-09-17T05:03:46.982733+0000","last_active":"2024-09-17T05:03:49.000759+0000","last_peered":"2024-09-17T05:03:49.000759+0000","last_clean":"2024-09-17T05:03:49.000759+0000","last_became_active":"2024-09-17T05:03:46.982332+0000","last_became_peered":"2024-09-17T05:03:46.982332+0000","last_unstale":"2024-09-17T05:03:49.000759+0000","last_undegraded":"2024-09-17T05:03:49.000759+0000","last_fullsized":"2024-09-17T05:03:49.000759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:19.904729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":189,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-17T05:03:46.968891+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:46.968891+0000","last_peered":"2024-09-17T05:03:46.968891+0000","last_clean":"2024-09-17T05:03:46.968891+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:46.968891+0000","last_undegraded":"2024-09-17T05:03:46.968891+0000","last_fullsized":"2024-09-17T05:03:46.968891+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509479,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":741376,"data_stored":329478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509477,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509476,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":741376,"data_stored":329478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":913179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509478,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":322811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1380352,"data_stored":935429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:53.036 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3f is not active+clean 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978241+0000', 'last_change': '2024-09-17T05:03:46.978241+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978241+0000', 'last_undegraded': '2024-09-17T05:03:46.978241+0000', 'last_fullsized': '2024-09-17T05:03:46.978241+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3d is not active+clean 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.984275+0000', 'last_change': '2024-09-17T05:03:46.984275+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.984275+0000', 'last_undegraded': '2024-09-17T05:03:46.984275+0000', 'last_fullsized': '2024-09-17T05:03:46.984275+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3c is not active+clean 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.977027+0000', 'last_change': '2024-09-17T05:03:46.977027+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.977027+0000', 'last_undegraded': '2024-09-17T05:03:46.977027+0000', 'last_fullsized': '2024-09-17T05:03:46.977027+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3b is not active+clean 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.979040+0000', 'last_change': '2024-09-17T05:03:46.979040+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.979040+0000', 'last_undegraded': '2024-09-17T05:03:46.979040+0000', 'last_fullsized': '2024-09-17T05:03:46.979040+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:PG 10.39 is not active+clean 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978918+0000', 'last_change': '2024-09-17T05:03:46.978918+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978918+0000', 'last_undegraded': '2024-09-17T05:03:46.978918+0000', 'last_fullsized': '2024-09-17T05:03:46.978918+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.037 INFO:tasks.ceph.ceph_manager.ceph:PG 10.38 is not active+clean 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.983530+0000', 'last_change': '2024-09-17T05:03:46.983530+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.983530+0000', 'last_undegraded': '2024-09-17T05:03:46.983530+0000', 'last_fullsized': '2024-09-17T05:03:46.983530+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:PG 10.37 is not active+clean 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.979383+0000', 'last_change': '2024-09-17T05:03:46.979383+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.979383+0000', 'last_undegraded': '2024-09-17T05:03:46.979383+0000', 'last_fullsized': '2024-09-17T05:03:46.979383+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:PG 10.36 is not active+clean 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.974260+0000', 'last_change': '2024-09-17T05:03:46.974260+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.974260+0000', 'last_undegraded': '2024-09-17T05:03:46.974260+0000', 'last_fullsized': '2024-09-17T05:03:46.974260+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:PG 10.35 is not active+clean 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978382+0000', 'last_change': '2024-09-17T05:03:46.978382+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978382+0000', 'last_undegraded': '2024-09-17T05:03:46.978382+0000', 'last_fullsized': '2024-09-17T05:03:46.978382+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:PG 10.34 is not active+clean 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.976518+0000', 'last_change': '2024-09-17T05:03:46.976518+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.976518+0000', 'last_undegraded': '2024-09-17T05:03:46.976518+0000', 'last_fullsized': '2024-09-17T05:03:46.976518+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.038 INFO:tasks.ceph.ceph_manager.ceph:PG 10.33 is not active+clean 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.986571+0000', 'last_change': '2024-09-17T05:03:46.986571+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.986571+0000', 'last_undegraded': '2024-09-17T05:03:46.986571+0000', 'last_fullsized': '2024-09-17T05:03:46.986571+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:PG 10.32 is not active+clean 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978501+0000', 'last_change': '2024-09-17T05:03:46.978501+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978501+0000', 'last_undegraded': '2024-09-17T05:03:46.978501+0000', 'last_fullsized': '2024-09-17T05:03:46.978501+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:PG 10.31 is not active+clean 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.976005+0000', 'last_change': '2024-09-17T05:03:46.976005+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.976005+0000', 'last_undegraded': '2024-09-17T05:03:46.976005+0000', 'last_fullsized': '2024-09-17T05:03:46.976005+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:PG 10.30 is not active+clean 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978421+0000', 'last_change': '2024-09-17T05:03:46.978421+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978421+0000', 'last_undegraded': '2024-09-17T05:03:46.978421+0000', 'last_fullsized': '2024-09-17T05:03:46.978421+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2f is not active+clean 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.969790+0000', 'last_change': '2024-09-17T05:03:46.969790+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.969790+0000', 'last_undegraded': '2024-09-17T05:03:46.969790+0000', 'last_fullsized': '2024-09-17T05:03:46.969790+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2e is not active+clean 2024-09-17T05:03:53.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.979878+0000', 'last_change': '2024-09-17T05:03:46.979878+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.979878+0000', 'last_undegraded': '2024-09-17T05:03:46.979878+0000', 'last_fullsized': '2024-09-17T05:03:46.979878+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2d is not active+clean 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.977684+0000', 'last_change': '2024-09-17T05:03:46.977684+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.977684+0000', 'last_undegraded': '2024-09-17T05:03:46.977684+0000', 'last_fullsized': '2024-09-17T05:03:46.977684+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2c is not active+clean 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.971210+0000', 'last_change': '2024-09-17T05:03:46.971210+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.971210+0000', 'last_undegraded': '2024-09-17T05:03:46.971210+0000', 'last_fullsized': '2024-09-17T05:03:46.971210+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2b is not active+clean 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.982766+0000', 'last_change': '2024-09-17T05:03:46.982766+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.982766+0000', 'last_undegraded': '2024-09-17T05:03:46.982766+0000', 'last_fullsized': '2024-09-17T05:03:46.982766+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:PG 10.29 is not active+clean 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.980950+0000', 'last_change': '2024-09-17T05:03:46.980950+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.980950+0000', 'last_undegraded': '2024-09-17T05:03:46.980950+0000', 'last_fullsized': '2024-09-17T05:03:46.980950+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:PG 10.28 is not active+clean 2024-09-17T05:03:53.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.982993+0000', 'last_change': '2024-09-17T05:03:46.982993+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.982993+0000', 'last_undegraded': '2024-09-17T05:03:46.982993+0000', 'last_fullsized': '2024-09-17T05:03:46.982993+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:PG 10.27 is not active+clean 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978589+0000', 'last_change': '2024-09-17T05:03:46.978589+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978589+0000', 'last_undegraded': '2024-09-17T05:03:46.978589+0000', 'last_fullsized': '2024-09-17T05:03:46.978589+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:PG 10.26 is not active+clean 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.979580+0000', 'last_change': '2024-09-17T05:03:46.979580+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.979580+0000', 'last_undegraded': '2024-09-17T05:03:46.979580+0000', 'last_fullsized': '2024-09-17T05:03:46.979580+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:PG 10.25 is not active+clean 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.991002+0000', 'last_change': '2024-09-17T05:03:46.991002+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.991002+0000', 'last_undegraded': '2024-09-17T05:03:46.991002+0000', 'last_fullsized': '2024-09-17T05:03:46.991002+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:PG 10.24 is not active+clean 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.980311+0000', 'last_change': '2024-09-17T05:03:46.980311+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.980311+0000', 'last_undegraded': '2024-09-17T05:03:46.980311+0000', 'last_fullsized': '2024-09-17T05:03:46.980311+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:PG 10.23 is not active+clean 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.976442+0000', 'last_change': '2024-09-17T05:03:46.976442+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.976442+0000', 'last_undegraded': '2024-09-17T05:03:46.976442+0000', 'last_fullsized': '2024-09-17T05:03:46.976442+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.041 INFO:tasks.ceph.ceph_manager.ceph:PG 10.22 is not active+clean 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.975398+0000', 'last_change': '2024-09-17T05:03:46.975398+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.975398+0000', 'last_undegraded': '2024-09-17T05:03:46.975398+0000', 'last_fullsized': '2024-09-17T05:03:46.975398+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:PG 10.21 is not active+clean 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.980342+0000', 'last_change': '2024-09-17T05:03:46.980342+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.980342+0000', 'last_undegraded': '2024-09-17T05:03:46.980342+0000', 'last_fullsized': '2024-09-17T05:03:46.980342+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:PG 10.20 is not active+clean 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.977212+0000', 'last_change': '2024-09-17T05:03:46.977212+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.977212+0000', 'last_undegraded': '2024-09-17T05:03:46.977212+0000', 'last_fullsized': '2024-09-17T05:03:46.977212+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1c is not active+clean 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.976032+0000', 'last_change': '2024-09-17T05:03:46.976032+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.976032+0000', 'last_undegraded': '2024-09-17T05:03:46.976032+0000', 'last_fullsized': '2024-09-17T05:03:46.976032+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1d is not active+clean 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.983834+0000', 'last_change': '2024-09-17T05:03:46.983834+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.983834+0000', 'last_undegraded': '2024-09-17T05:03:46.983834+0000', 'last_fullsized': '2024-09-17T05:03:46.983834+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1f is not active+clean 2024-09-17T05:03:53.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978829+0000', 'last_change': '2024-09-17T05:03:46.978829+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978829+0000', 'last_undegraded': '2024-09-17T05:03:46.978829+0000', 'last_fullsized': '2024-09-17T05:03:46.978829+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:PG 10.18 is not active+clean 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.969266+0000', 'last_change': '2024-09-17T05:03:46.969266+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.969266+0000', 'last_undegraded': '2024-09-17T05:03:46.969266+0000', 'last_fullsized': '2024-09-17T05:03:46.969266+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978356+0000', 'last_change': '2024-09-17T05:03:46.978356+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978356+0000', 'last_undegraded': '2024-09-17T05:03:46.978356+0000', 'last_fullsized': '2024-09-17T05:03:46.978356+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1a is not active+clean 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.975768+0000', 'last_change': '2024-09-17T05:03:46.975768+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.975768+0000', 'last_undegraded': '2024-09-17T05:03:46.975768+0000', 'last_fullsized': '2024-09-17T05:03:46.975768+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.043 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.970441+0000', 'last_change': '2024-09-17T05:03:46.970441+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.970441+0000', 'last_undegraded': '2024-09-17T05:03:46.970441+0000', 'last_fullsized': '2024-09-17T05:03:46.970441+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.979741+0000', 'last_change': '2024-09-17T05:03:46.979741+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.979741+0000', 'last_undegraded': '2024-09-17T05:03:46.979741+0000', 'last_fullsized': '2024-09-17T05:03:46.979741+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.969974+0000', 'last_change': '2024-09-17T05:03:46.969974+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.969974+0000', 'last_undegraded': '2024-09-17T05:03:46.969974+0000', 'last_fullsized': '2024-09-17T05:03:46.969974+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:PG 10.16 is not active+clean 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.991028+0000', 'last_change': '2024-09-17T05:03:46.991028+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.991028+0000', 'last_undegraded': '2024-09-17T05:03:46.991028+0000', 'last_fullsized': '2024-09-17T05:03:46.991028+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:PG 10.17 is not active+clean 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.982691+0000', 'last_change': '2024-09-17T05:03:46.982691+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.982691+0000', 'last_undegraded': '2024-09-17T05:03:46.982691+0000', 'last_fullsized': '2024-09-17T05:03:46.982691+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.044 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.973898+0000', 'last_change': '2024-09-17T05:03:46.973898+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.973898+0000', 'last_undegraded': '2024-09-17T05:03:46.973898+0000', 'last_fullsized': '2024-09-17T05:03:46.973898+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.974975+0000', 'last_change': '2024-09-17T05:03:46.974975+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.974975+0000', 'last_undegraded': '2024-09-17T05:03:46.974975+0000', 'last_fullsized': '2024-09-17T05:03:46.974975+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:PG 10.12 is not active+clean 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.974547+0000', 'last_change': '2024-09-17T05:03:46.974547+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.974547+0000', 'last_undegraded': '2024-09-17T05:03:46.974547+0000', 'last_fullsized': '2024-09-17T05:03:46.974547+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978996+0000', 'last_change': '2024-09-17T05:03:46.978996+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978996+0000', 'last_undegraded': '2024-09-17T05:03:46.978996+0000', 'last_fullsized': '2024-09-17T05:03:46.978996+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:03:53.045 INFO:tasks.ceph.ceph_manager.ceph:PG 10.c is not active+clean 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.975516+0000', 'last_change': '2024-09-17T05:03:46.975516+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.975516+0000', 'last_undegraded': '2024-09-17T05:03:46.975516+0000', 'last_fullsized': '2024-09-17T05:03:46.975516+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:PG 10.d is not active+clean 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.986536+0000', 'last_change': '2024-09-17T05:03:46.986536+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.986536+0000', 'last_undegraded': '2024-09-17T05:03:46.986536+0000', 'last_fullsized': '2024-09-17T05:03:46.986536+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:PG 10.e is not active+clean 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.986807+0000', 'last_change': '2024-09-17T05:03:46.986807+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.986807+0000', 'last_undegraded': '2024-09-17T05:03:46.986807+0000', 'last_fullsized': '2024-09-17T05:03:46.986807+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:PG 10.f is not active+clean 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.980649+0000', 'last_change': '2024-09-17T05:03:46.980649+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.980649+0000', 'last_undegraded': '2024-09-17T05:03:46.980649+0000', 'last_fullsized': '2024-09-17T05:03:46.980649+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.991126+0000', 'last_change': '2024-09-17T05:03:46.991126+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.991126+0000', 'last_undegraded': '2024-09-17T05:03:46.991126+0000', 'last_fullsized': '2024-09-17T05:03:46.991126+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:PG 10.a is not active+clean 2024-09-17T05:03:53.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.974798+0000', 'last_change': '2024-09-17T05:03:46.974798+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.974798+0000', 'last_undegraded': '2024-09-17T05:03:46.974798+0000', 'last_fullsized': '2024-09-17T05:03:46.974798+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.970693+0000', 'last_change': '2024-09-17T05:03:46.970693+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.970693+0000', 'last_undegraded': '2024-09-17T05:03:46.970693+0000', 'last_fullsized': '2024-09-17T05:03:46.970693+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:PG 10.5 is not active+clean 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.969366+0000', 'last_change': '2024-09-17T05:03:46.969366+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.969366+0000', 'last_undegraded': '2024-09-17T05:03:46.969366+0000', 'last_fullsized': '2024-09-17T05:03:46.969366+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.978892+0000', 'last_change': '2024-09-17T05:03:46.978892+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.978892+0000', 'last_undegraded': '2024-09-17T05:03:46.978892+0000', 'last_fullsized': '2024-09-17T05:03:46.978892+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:PG 10.7 is not active+clean 2024-09-17T05:03:53.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.980152+0000', 'last_change': '2024-09-17T05:03:46.980152+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.980152+0000', 'last_undegraded': '2024-09-17T05:03:46.980152+0000', 'last_fullsized': '2024-09-17T05:03:46.980152+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:PG 10.0 is not active+clean 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.974557+0000', 'last_change': '2024-09-17T05:03:46.974557+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.974557+0000', 'last_undegraded': '2024-09-17T05:03:46.974557+0000', 'last_fullsized': '2024-09-17T05:03:46.974557+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1 is not active+clean 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.986314+0000', 'last_change': '2024-09-17T05:03:46.986314+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.986314+0000', 'last_undegraded': '2024-09-17T05:03:46.986314+0000', 'last_fullsized': '2024-09-17T05:03:46.986314+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3 is not active+clean 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.969865+0000', 'last_change': '2024-09-17T05:03:46.969865+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.969865+0000', 'last_undegraded': '2024-09-17T05:03:46.969865+0000', 'last_fullsized': '2024-09-17T05:03:46.969865+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:PG 10.b is not active+clean 2024-09-17T05:03:53.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 62, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:03:46.973191+0000', 'last_change': '2024-09-17T05:03:46.973191+0000', 'last_active': '2024-09-17T05:03:46.961444+0000', 'last_peered': '2024-09-17T05:03:46.961444+0000', 'last_clean': '2024-09-17T05:03:46.961444+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:03:46.973191+0000', 'last_undegraded': '2024-09-17T05:03:46.973191+0000', 'last_fullsized': '2024-09-17T05:03:46.973191+0000', 'mapping_epoch': 62, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 62, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'last_clean_scrub_stamp': '2024-09-17T05:03:46.961444+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:03:53.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:03:53.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 -- 172.21.15.16:0/2905182203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 msgr2=0x7ff29815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 --2- 172.21.15.16:0/2905182203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff29815bb60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff29806e960 tx=0x7ff28801c830 comp rx=0 tx=0).stop 2024-09-17T05:03:53.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 -- 172.21.15.16:0/2905182203 shutdown_connections 2024-09-17T05:03:53.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 --2- 172.21.15.16:0/2905182203 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff29815f150 0x7ff29815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 --2- 172.21.15.16:0/2905182203 >> v2:172.21.15.104:3300/0 conn(0x7ff29815c0a0 0x7ff29815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 --2- 172.21.15.16:0/2905182203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff29815bb60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 -- 172.21.15.16:0/2905182203 >> 172.21.15.16:0/2905182203 conn(0x7ff29806f900 msgr2=0x7ff2980b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:53.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 -- 172.21.15.16:0/2905182203 shutdown_connections 2024-09-17T05:03:53.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 -- 172.21.15.16:0/2905182203 wait complete. 2024-09-17T05:03:53.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 Processor -- start 2024-09-17T05:03:53.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.229+0000 7ff29d400640 1 -- start start 2024-09-17T05:03:53.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff2981f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff29815c0a0 0x7ff29813e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff29815f150 0x7ff2981f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff29806e3d0 con 0x7ff2980b2f50 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff29806e250 con 0x7ff29815f150 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff29806e550 con 0x7ff29815c0a0 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff2981f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff2981f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33988/0 (socket says 172.21.15.16:33988) 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 -- 172.21.15.16:0/2050024231 learned_addr learned my addr 172.21.15.16:0/2050024231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff297200640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff29815f150 0x7ff2981f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff28fe00640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff29815c0a0 0x7ff29813e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 -- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff29815c0a0 msgr2=0x7ff29813e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff29815c0a0 0x7ff29813e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 -- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff29815f150 msgr2=0x7ff2981f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff29815f150 0x7ff2981f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 -- 172.21.15.16:0/2050024231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2981f9a70 con 0x7ff2980b2f50 2024-09-17T05:03:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff297200640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff29815f150 0x7ff2981f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff296800640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff2981f9530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff2880b0040 tx=0x7ff2880a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880b2040 con 0x7ff2980b2f50 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2880042c0 con 0x7ff2980b2f50 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2980caac0 con 0x7ff2980b2f50 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2880052d0 con 0x7ff2980b2f50 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2980cb080 con 0x7ff2980b2f50 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.233+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29815c4b0 con 0x7ff2980b2f50 2024-09-17T05:03:53.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.237+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff288005470 con 0x7ff2980b2f50 2024-09-17T05:03:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.237+0000 7ff28ea00640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff274073200 0x7ff274075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.237+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7ff28812ddd0 con 0x7ff2980b2f50 2024-09-17T05:03:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.237+0000 7ff28fe00640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff274073200 0x7ff274075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.237+0000 7ff28fe00640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff274073200 0x7ff274075710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff280005fd0 tx=0x7ff280005d80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:53.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.241+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff2981f9a70 con 0x7ff2980b2f50 2024-09-17T05:03:53.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff2980cc320 con 0x7ff2980b2f50 2024-09-17T05:03:53.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff28ea00640 1 -- 172.21.15.16:0/2050024231 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1301 (secure 0 0 0) 0x7ff2880164b0 con 0x7ff2980b2f50 2024-09-17T05:03:53.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:53.483 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":191,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":64,"num_osds":8,"num_up_osds":8,"osd_up_since":1726549257,"num_in_osds":8,"osd_in_since":1726549246,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":58},{"state_name":"active+clean","count":39}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":591980,"bytes_used":249864192,"bytes_avail":772844249088,"bytes_total":773094113280,"inactive_pgs_ratio":0.59793812036514282,"write_bytes_sec":554,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5775}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T05:03:01.491151+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff274073200 msgr2=0x7ff274075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff274073200 0x7ff274075710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff280005fd0 tx=0x7ff280005d80 comp rx=0 tx=0).stop 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 msgr2=0x7ff2981f9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff2981f9530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff2880b0040 tx=0x7ff2880a5650 comp rx=0 tx=0).stop 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 shutdown_connections 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff274073200 0x7ff274075710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff29815f150 0x7ff2981f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff29815c0a0 0x7ff29813e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 --2- 172.21.15.16:0/2050024231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2980b2f50 0x7ff2981f9530 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 >> 172.21.15.16:0/2050024231 conn(0x7ff29806f900 msgr2=0x7ff298151be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 shutdown_connections 2024-09-17T05:03:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.481+0000 7ff29d400640 1 -- 172.21.15.16:0/2050024231 wait complete. 2024-09-17T05:03:53.504 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:03:53.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 -- 172.21.15.16:0/1625192119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97440b4230 msgr2=0x7f97440b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 --2- 172.21.15.16:0/1625192119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97440b4230 0x7f97440b4640 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f973c009bc0 tx=0x7f973c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 -- 172.21.15.16:0/1625192119 shutdown_connections 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 --2- 172.21.15.16:0/1625192119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97440b4230 0x7f97440b4640 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 -- 172.21.15.16:0/1625192119 >> 172.21.15.16:0/1625192119 conn(0x7f97440d0860 msgr2=0x7f97440d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 -- 172.21.15.16:0/1625192119 shutdown_connections 2024-09-17T05:03:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 -- 172.21.15.16:0/1625192119 wait complete. 2024-09-17T05:03:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 Processor -- start 2024-09-17T05:03:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.661+0000 7f974a600640 1 -- start start 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 0x7f97441518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97440b4230 0x7f974413a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97440cfa30 0x7f9744154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9744169d80 con 0x7f974406ce90 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9744169c00 con 0x7f97440b4230 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9744169f00 con 0x7f97440cfa30 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 0x7f97441518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 0x7f97441518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34004/0 (socket says 172.21.15.16:34004) 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 -- 172.21.15.16:0/2517262830 learned_addr learned my addr 172.21.15.16:0/2517262830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f973be00640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97440b4230 0x7f974413a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743e00640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97440cfa30 0x7f9744154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 -- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97440cfa30 msgr2=0x7f9744154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97440cfa30 0x7f9744154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 -- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97440b4230 msgr2=0x7f974413a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97440b4230 0x7f974413a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 -- 172.21.15.16:0/2517262830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974414bac0 con 0x7f974406ce90 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f9743400640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 0x7f97441518c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f973400c760 tx=0x7f973400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734016020 con 0x7f974406ce90 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f973400a950 con 0x7f974406ce90 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974414bda0 con 0x7f974406ce90 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400ada0 con 0x7f974406ce90 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9744151140 con 0x7f974406ce90 2024-09-17T05:03:53.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.665+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97440b4640 con 0x7f974406ce90 2024-09-17T05:03:53.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.669+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9734019020 con 0x7f974406ce90 2024-09-17T05:03:53.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.669+0000 7f973b400640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9720073200 0x7f9720075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:53.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.669+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f9734094260 con 0x7f974406ce90 2024-09-17T05:03:53.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.669+0000 7f973be00640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9720073200 0x7f9720075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:53.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.669+0000 7f973be00640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9720073200 0x7f9720075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97440afbe0 tx=0x7f972c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:53.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.673+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9734020020 con 0x7f974406ce90 2024-09-17T05:03:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.845+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9744069690 con 0x7f9720073200 2024-09-17T05:03:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.849+0000 7f973b400640 1 -- 172.21.15.16:0/2517262830 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252013 (secure 0 0 0) 0x7f9744069690 con 0x7f9720073200 2024-09-17T05:03:53.855 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9720073200 msgr2=0x7f9720075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9720073200 0x7f9720075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97440afbe0 tx=0x7f972c009040 comp rx=0 tx=0).stop 2024-09-17T05:03:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 msgr2=0x7f97441518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 0x7f97441518c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f973400c760 tx=0x7f973400cc20 comp rx=0 tx=0).stop 2024-09-17T05:03:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 shutdown_connections 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9720073200 0x7f9720075710 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97440cfa30 0x7f9744154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97440b4230 0x7f974413a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 --2- 172.21.15.16:0/2517262830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f974406ce90 0x7f97441518c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 >> 172.21.15.16:0/2517262830 conn(0x7f97440d0860 msgr2=0x7f97440ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 shutdown_connections 2024-09-17T05:03:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:53.853+0000 7f974a600640 1 -- 172.21.15.16:0/2517262830 wait complete. 2024-09-17T05:03:53.876 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-17T05:03:53.503984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244272,"kb_used_data":7208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730448,"statfs":{"total":773094113280,"available":772843978752,"internally_reserved":0,"allocated":7380992,"data_stored":3932252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.537942"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034962+0000","last_change":"2024-09-17T05:03:48.013215+0000","last_active":"2024-09-17T05:03:49.034962+0000","last_peered":"2024-09-17T05:03:49.034962+0000","last_clean":"2024-09-17T05:03:49.034962+0000","last_became_active":"2024-09-17T05:03:48.012876+0000","last_became_peered":"2024-09-17T05:03:48.012876+0000","last_unstale":"2024-09-17T05:03:49.034962+0000","last_undegraded":"2024-09-17T05:03:49.034962+0000","last_fullsized":"2024-09-17T05:03:49.034962+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:18.639295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001275+0000","last_change":"2024-09-17T05:03:48.027266+0000","last_active":"2024-09-17T05:03:49.001275+0000","last_peered":"2024-09-17T05:03:49.001275+0000","last_clean":"2024-09-17T05:03:49.001275+0000","last_became_active":"2024-09-17T05:03:48.021805+0000","last_became_peered":"2024-09-17T05:03:48.021805+0000","last_unstale":"2024-09-17T05:03:49.001275+0000","last_undegraded":"2024-09-17T05:03:49.001275+0000","last_fullsized":"2024-09-17T05:03:49.001275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:19.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000408+0000","last_change":"2024-09-17T05:03:48.001997+0000","last_active":"2024-09-17T05:03:49.000408+0000","last_peered":"2024-09-17T05:03:49.000408+0000","last_clean":"2024-09-17T05:03:49.000408+0000","last_became_active":"2024-09-17T05:03:48.001682+0000","last_became_peered":"2024-09-17T05:03:48.001682+0000","last_unstale":"2024-09-17T05:03:49.000408+0000","last_undegraded":"2024-09-17T05:03:49.000408+0000","last_fullsized":"2024-09-17T05:03:49.000408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:18.507396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014446+0000","last_change":"2024-09-17T05:03:48.024186+0000","last_active":"2024-09-17T05:03:49.014446+0000","last_peered":"2024-09-17T05:03:49.014446+0000","last_clean":"2024-09-17T05:03:49.014446+0000","last_became_active":"2024-09-17T05:03:48.019376+0000","last_became_peered":"2024-09-17T05:03:48.019376+0000","last_unstale":"2024-09-17T05:03:49.014446+0000","last_undegraded":"2024-09-17T05:03:49.014446+0000","last_fullsized":"2024-09-17T05:03:49.014446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:56.564278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999956+0000","last_change":"2024-09-17T05:03:48.013905+0000","last_active":"2024-09-17T05:03:48.999956+0000","last_peered":"2024-09-17T05:03:48.999956+0000","last_clean":"2024-09-17T05:03:48.999956+0000","last_became_active":"2024-09-17T05:03:48.013541+0000","last_became_peered":"2024-09-17T05:03:48.013541+0000","last_unstale":"2024-09-17T05:03:48.999956+0000","last_undegraded":"2024-09-17T05:03:48.999956+0000","last_fullsized":"2024-09-17T05:03:48.999956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:28.901368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000614+0000","last_change":"2024-09-17T05:03:48.031572+0000","last_active":"2024-09-17T05:03:49.000614+0000","last_peered":"2024-09-17T05:03:49.000614+0000","last_clean":"2024-09-17T05:03:49.000614+0000","last_became_active":"2024-09-17T05:03:48.031253+0000","last_became_peered":"2024-09-17T05:03:48.031253+0000","last_unstale":"2024-09-17T05:03:49.000614+0000","last_undegraded":"2024-09-17T05:03:49.000614+0000","last_fullsized":"2024-09-17T05:03:49.000614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:16.025505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034440+0000","last_change":"2024-09-17T05:03:48.017856+0000","last_active":"2024-09-17T05:03:49.034440+0000","last_peered":"2024-09-17T05:03:49.034440+0000","last_clean":"2024-09-17T05:03:49.034440+0000","last_became_active":"2024-09-17T05:03:48.017494+0000","last_became_peered":"2024-09-17T05:03:48.017494+0000","last_unstale":"2024-09-17T05:03:49.034440+0000","last_undegraded":"2024-09-17T05:03:49.034440+0000","last_fullsized":"2024-09-17T05:03:49.034440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:44.663764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999262+0000","last_change":"2024-09-17T05:03:48.013895+0000","last_active":"2024-09-17T05:03:48.999262+0000","last_peered":"2024-09-17T05:03:48.999262+0000","last_clean":"2024-09-17T05:03:48.999262+0000","last_became_active":"2024-09-17T05:03:48.013581+0000","last_became_peered":"2024-09-17T05:03:48.013581+0000","last_unstale":"2024-09-17T05:03:48.999262+0000","last_undegraded":"2024-09-17T05:03:48.999262+0000","last_fullsized":"2024-09-17T05:03:48.999262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:15.892857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014615+0000","last_change":"2024-09-17T05:03:48.012221+0000","last_active":"2024-09-17T05:03:49.014615+0000","last_peered":"2024-09-17T05:03:49.014615+0000","last_clean":"2024-09-17T05:03:49.014615+0000","last_became_active":"2024-09-17T05:03:48.011887+0000","last_became_peered":"2024-09-17T05:03:48.011887+0000","last_unstale":"2024-09-17T05:03:49.014615+0000","last_undegraded":"2024-09-17T05:03:49.014615+0000","last_fullsized":"2024-09-17T05:03:49.014615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:07.996382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065328+0000","last_change":"2024-09-17T05:03:48.023192+0000","last_active":"2024-09-17T05:03:49.065328+0000","last_peered":"2024-09-17T05:03:49.065328+0000","last_clean":"2024-09-17T05:03:49.065328+0000","last_became_active":"2024-09-17T05:03:48.022306+0000","last_became_peered":"2024-09-17T05:03:48.022306+0000","last_unstale":"2024-09-17T05:03:49.065328+0000","last_undegraded":"2024-09-17T05:03:49.065328+0000","last_fullsized":"2024-09-17T05:03:49.065328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:06.697471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014540+0000","last_change":"2024-09-17T05:03:47.996215+0000","last_active":"2024-09-17T05:03:49.014540+0000","last_peered":"2024-09-17T05:03:49.014540+0000","last_clean":"2024-09-17T05:03:49.014540+0000","last_became_active":"2024-09-17T05:03:47.995860+0000","last_became_peered":"2024-09-17T05:03:47.995860+0000","last_unstale":"2024-09-17T05:03:49.014540+0000","last_undegraded":"2024-09-17T05:03:49.014540+0000","last_fullsized":"2024-09-17T05:03:49.014540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:23.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014515+0000","last_change":"2024-09-17T05:03:48.000279+0000","last_active":"2024-09-17T05:03:49.014515+0000","last_peered":"2024-09-17T05:03:49.014515+0000","last_clean":"2024-09-17T05:03:49.014515+0000","last_became_active":"2024-09-17T05:03:47.999157+0000","last_became_peered":"2024-09-17T05:03:47.999157+0000","last_unstale":"2024-09-17T05:03:49.014515+0000","last_undegraded":"2024-09-17T05:03:49.014515+0000","last_fullsized":"2024-09-17T05:03:49.014515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:59.675067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001367+0000","last_change":"2024-09-17T05:03:48.014910+0000","last_active":"2024-09-17T05:03:49.001367+0000","last_peered":"2024-09-17T05:03:49.001367+0000","last_clean":"2024-09-17T05:03:49.001367+0000","last_became_active":"2024-09-17T05:03:48.014603+0000","last_became_peered":"2024-09-17T05:03:48.014603+0000","last_unstale":"2024-09-17T05:03:49.001367+0000","last_undegraded":"2024-09-17T05:03:49.001367+0000","last_fullsized":"2024-09-17T05:03:49.001367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:32.692661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001342+0000","last_change":"2024-09-17T05:03:48.022902+0000","last_active":"2024-09-17T05:03:49.001342+0000","last_peered":"2024-09-17T05:03:49.001342+0000","last_clean":"2024-09-17T05:03:49.001342+0000","last_became_active":"2024-09-17T05:03:48.022535+0000","last_became_peered":"2024-09-17T05:03:48.022535+0000","last_unstale":"2024-09-17T05:03:49.001342+0000","last_undegraded":"2024-09-17T05:03:49.001342+0000","last_fullsized":"2024-09-17T05:03:49.001342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:24.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064878+0000","last_change":"2024-09-17T05:03:48.021758+0000","last_active":"2024-09-17T05:03:49.064878+0000","last_peered":"2024-09-17T05:03:49.064878+0000","last_clean":"2024-09-17T05:03:49.064878+0000","last_became_active":"2024-09-17T05:03:48.020251+0000","last_became_peered":"2024-09-17T05:03:48.020251+0000","last_unstale":"2024-09-17T05:03:49.064878+0000","last_undegraded":"2024-09-17T05:03:49.064878+0000","last_fullsized":"2024-09-17T05:03:49.064878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:34.420552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017103+0000","last_change":"2024-09-17T05:03:48.009149+0000","last_active":"2024-09-17T05:03:49.017103+0000","last_peered":"2024-09-17T05:03:49.017103+0000","last_clean":"2024-09-17T05:03:49.017103+0000","last_became_active":"2024-09-17T05:03:48.008880+0000","last_became_peered":"2024-09-17T05:03:48.008880+0000","last_unstale":"2024-09-17T05:03:49.017103+0000","last_undegraded":"2024-09-17T05:03:49.017103+0000","last_fullsized":"2024-09-17T05:03:49.017103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:01.937327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065711+0000","last_change":"2024-09-17T05:03:48.014727+0000","last_active":"2024-09-17T05:03:49.065711+0000","last_peered":"2024-09-17T05:03:49.065711+0000","last_clean":"2024-09-17T05:03:49.065711+0000","last_became_active":"2024-09-17T05:03:48.014318+0000","last_became_peered":"2024-09-17T05:03:48.014318+0000","last_unstale":"2024-09-17T05:03:49.065711+0000","last_undegraded":"2024-09-17T05:03:49.065711+0000","last_fullsized":"2024-09-17T05:03:49.065711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:12.164843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014491+0000","last_change":"2024-09-17T05:03:48.024868+0000","last_active":"2024-09-17T05:03:49.014491+0000","last_peered":"2024-09-17T05:03:49.014491+0000","last_clean":"2024-09-17T05:03:49.014491+0000","last_became_active":"2024-09-17T05:03:48.024555+0000","last_became_peered":"2024-09-17T05:03:48.024555+0000","last_unstale":"2024-09-17T05:03:49.014491+0000","last_undegraded":"2024-09-17T05:03:49.014491+0000","last_fullsized":"2024-09-17T05:03:49.014491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:02.442859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014801+0000","last_change":"2024-09-17T05:03:47.994233+0000","last_active":"2024-09-17T05:03:49.014801+0000","last_peered":"2024-09-17T05:03:49.014801+0000","last_clean":"2024-09-17T05:03:49.014801+0000","last_became_active":"2024-09-17T05:03:47.993250+0000","last_became_peered":"2024-09-17T05:03:47.993250+0000","last_unstale":"2024-09-17T05:03:49.014801+0000","last_undegraded":"2024-09-17T05:03:49.014801+0000","last_fullsized":"2024-09-17T05:03:49.014801+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.200220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065248+0000","last_change":"2024-09-17T05:03:48.015536+0000","last_active":"2024-09-17T05:03:49.065248+0000","last_peered":"2024-09-17T05:03:49.065248+0000","last_clean":"2024-09-17T05:03:49.065248+0000","last_became_active":"2024-09-17T05:03:48.015214+0000","last_became_peered":"2024-09-17T05:03:48.015214+0000","last_unstale":"2024-09-17T05:03:49.065248+0000","last_undegraded":"2024-09-17T05:03:49.065248+0000","last_fullsized":"2024-09-17T05:03:49.065248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:52.284850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001645+0000","last_change":"2024-09-17T05:03:48.016724+0000","last_active":"2024-09-17T05:03:49.001645+0000","last_peered":"2024-09-17T05:03:49.001645+0000","last_clean":"2024-09-17T05:03:49.001645+0000","last_became_active":"2024-09-17T05:03:48.016150+0000","last_became_peered":"2024-09-17T05:03:48.016150+0000","last_unstale":"2024-09-17T05:03:49.001645+0000","last_undegraded":"2024-09-17T05:03:49.001645+0000","last_fullsized":"2024-09-17T05:03:49.001645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:17.868708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000912+0000","last_change":"2024-09-17T05:03:48.013364+0000","last_active":"2024-09-17T05:03:49.000912+0000","last_peered":"2024-09-17T05:03:49.000912+0000","last_clean":"2024-09-17T05:03:49.000912+0000","last_became_active":"2024-09-17T05:03:48.012273+0000","last_became_peered":"2024-09-17T05:03:48.012273+0000","last_unstale":"2024-09-17T05:03:49.000912+0000","last_undegraded":"2024-09-17T05:03:49.000912+0000","last_fullsized":"2024-09-17T05:03:49.000912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:46.993665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017206+0000","last_change":"2024-09-17T05:03:48.008799+0000","last_active":"2024-09-17T05:03:49.017206+0000","last_peered":"2024-09-17T05:03:49.017206+0000","last_clean":"2024-09-17T05:03:49.017206+0000","last_became_active":"2024-09-17T05:03:48.005133+0000","last_became_peered":"2024-09-17T05:03:48.005133+0000","last_unstale":"2024-09-17T05:03:49.017206+0000","last_undegraded":"2024-09-17T05:03:49.017206+0000","last_fullsized":"2024-09-17T05:03:49.017206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:53.016547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999261+0000","last_change":"2024-09-17T05:03:48.015909+0000","last_active":"2024-09-17T05:03:48.999261+0000","last_peered":"2024-09-17T05:03:48.999261+0000","last_clean":"2024-09-17T05:03:48.999261+0000","last_became_active":"2024-09-17T05:03:48.015487+0000","last_became_peered":"2024-09-17T05:03:48.015487+0000","last_unstale":"2024-09-17T05:03:48.999261+0000","last_undegraded":"2024-09-17T05:03:48.999261+0000","last_fullsized":"2024-09-17T05:03:48.999261+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:24.466123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002381+0000","last_change":"2024-09-17T05:03:48.025091+0000","last_active":"2024-09-17T05:03:49.002381+0000","last_peered":"2024-09-17T05:03:49.002381+0000","last_clean":"2024-09-17T05:03:49.002381+0000","last_became_active":"2024-09-17T05:03:48.024635+0000","last_became_peered":"2024-09-17T05:03:48.024635+0000","last_unstale":"2024-09-17T05:03:49.002381+0000","last_undegraded":"2024-09-17T05:03:49.002381+0000","last_fullsized":"2024-09-17T05:03:49.002381+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:35.215246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000419+0000","last_change":"2024-09-17T05:03:48.015424+0000","last_active":"2024-09-17T05:03:49.000419+0000","last_peered":"2024-09-17T05:03:49.000419+0000","last_clean":"2024-09-17T05:03:49.000419+0000","last_became_active":"2024-09-17T05:03:48.014801+0000","last_became_peered":"2024-09-17T05:03:48.014801+0000","last_unstale":"2024-09-17T05:03:49.000419+0000","last_undegraded":"2024-09-17T05:03:49.000419+0000","last_fullsized":"2024-09-17T05:03:49.000419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:18.567770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001889+0000","last_change":"2024-09-17T05:03:48.013166+0000","last_active":"2024-09-17T05:03:49.001889+0000","last_peered":"2024-09-17T05:03:49.001889+0000","last_clean":"2024-09-17T05:03:49.001889+0000","last_became_active":"2024-09-17T05:03:48.012840+0000","last_became_peered":"2024-09-17T05:03:48.012840+0000","last_unstale":"2024-09-17T05:03:49.001889+0000","last_undegraded":"2024-09-17T05:03:49.001889+0000","last_fullsized":"2024-09-17T05:03:49.001889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:52.365194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034692+0000","last_change":"2024-09-17T05:03:47.997180+0000","last_active":"2024-09-17T05:03:49.034692+0000","last_peered":"2024-09-17T05:03:49.034692+0000","last_clean":"2024-09-17T05:03:49.034692+0000","last_became_active":"2024-09-17T05:03:47.996702+0000","last_became_peered":"2024-09-17T05:03:47.996702+0000","last_unstale":"2024-09-17T05:03:49.034692+0000","last_undegraded":"2024-09-17T05:03:49.034692+0000","last_fullsized":"2024-09-17T05:03:49.034692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:39.166350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014387+0000","last_change":"2024-09-17T05:03:48.009380+0000","last_active":"2024-09-17T05:03:49.014387+0000","last_peered":"2024-09-17T05:03:49.014387+0000","last_clean":"2024-09-17T05:03:49.014387+0000","last_became_active":"2024-09-17T05:03:48.009044+0000","last_became_peered":"2024-09-17T05:03:48.009044+0000","last_unstale":"2024-09-17T05:03:49.014387+0000","last_undegraded":"2024-09-17T05:03:49.014387+0000","last_fullsized":"2024-09-17T05:03:49.014387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:59.262724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014169+0000","last_change":"2024-09-17T05:03:48.009029+0000","last_active":"2024-09-17T05:03:49.014169+0000","last_peered":"2024-09-17T05:03:49.014169+0000","last_clean":"2024-09-17T05:03:49.014169+0000","last_became_active":"2024-09-17T05:03:48.008009+0000","last_became_peered":"2024-09-17T05:03:48.008009+0000","last_unstale":"2024-09-17T05:03:49.014169+0000","last_undegraded":"2024-09-17T05:03:49.014169+0000","last_fullsized":"2024-09-17T05:03:49.014169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:14.891647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999432+0000","last_change":"2024-09-17T05:03:48.007439+0000","last_active":"2024-09-17T05:03:48.999432+0000","last_peered":"2024-09-17T05:03:48.999432+0000","last_clean":"2024-09-17T05:03:48.999432+0000","last_became_active":"2024-09-17T05:03:48.007060+0000","last_became_peered":"2024-09-17T05:03:48.007060+0000","last_unstale":"2024-09-17T05:03:48.999432+0000","last_undegraded":"2024-09-17T05:03:48.999432+0000","last_fullsized":"2024-09-17T05:03:48.999432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:36.792619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034382+0000","last_change":"2024-09-17T05:03:48.023898+0000","last_active":"2024-09-17T05:03:49.034382+0000","last_peered":"2024-09-17T05:03:49.034382+0000","last_clean":"2024-09-17T05:03:49.034382+0000","last_became_active":"2024-09-17T05:03:48.023573+0000","last_became_peered":"2024-09-17T05:03:48.023573+0000","last_unstale":"2024-09-17T05:03:49.034382+0000","last_undegraded":"2024-09-17T05:03:49.034382+0000","last_fullsized":"2024-09-17T05:03:49.034382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:37.827085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014702+0000","last_change":"2024-09-17T05:03:47.995954+0000","last_active":"2024-09-17T05:03:49.014702+0000","last_peered":"2024-09-17T05:03:49.014702+0000","last_clean":"2024-09-17T05:03:49.014702+0000","last_became_active":"2024-09-17T05:03:47.995596+0000","last_became_peered":"2024-09-17T05:03:47.995596+0000","last_unstale":"2024-09-17T05:03:49.014702+0000","last_undegraded":"2024-09-17T05:03:49.014702+0000","last_fullsized":"2024-09-17T05:03:49.014702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:40.784043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034957+0000","last_change":"2024-09-17T05:03:46.987624+0000","last_active":"2024-09-17T05:03:49.034957+0000","last_peered":"2024-09-17T05:03:49.034957+0000","last_clean":"2024-09-17T05:03:49.034957+0000","last_became_active":"2024-09-17T05:03:46.987266+0000","last_became_peered":"2024-09-17T05:03:46.987266+0000","last_unstale":"2024-09-17T05:03:49.034957+0000","last_undegraded":"2024-09-17T05:03:49.034957+0000","last_fullsized":"2024-09-17T05:03:49.034957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:23.319826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000668+0000","last_change":"2024-09-17T05:03:48.009158+0000","last_active":"2024-09-17T05:03:49.000668+0000","last_peered":"2024-09-17T05:03:49.000668+0000","last_clean":"2024-09-17T05:03:49.000668+0000","last_became_active":"2024-09-17T05:03:48.008860+0000","last_became_peered":"2024-09-17T05:03:48.008860+0000","last_unstale":"2024-09-17T05:03:49.000668+0000","last_undegraded":"2024-09-17T05:03:49.000668+0000","last_fullsized":"2024-09-17T05:03:49.000668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:56.291803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034865+0000","last_change":"2024-09-17T05:03:46.993282+0000","last_active":"2024-09-17T05:03:49.034865+0000","last_peered":"2024-09-17T05:03:49.034865+0000","last_clean":"2024-09-17T05:03:49.034865+0000","last_became_active":"2024-09-17T05:03:46.992953+0000","last_became_peered":"2024-09-17T05:03:46.992953+0000","last_unstale":"2024-09-17T05:03:49.034865+0000","last_undegraded":"2024-09-17T05:03:49.034865+0000","last_fullsized":"2024-09-17T05:03:49.034865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:33.413162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001473+0000","last_change":"2024-09-17T05:03:48.023120+0000","last_active":"2024-09-17T05:03:49.001473+0000","last_peered":"2024-09-17T05:03:49.001473+0000","last_clean":"2024-09-17T05:03:49.001473+0000","last_became_active":"2024-09-17T05:03:48.022823+0000","last_became_peered":"2024-09-17T05:03:48.022823+0000","last_unstale":"2024-09-17T05:03:49.001473+0000","last_undegraded":"2024-09-17T05:03:49.001473+0000","last_fullsized":"2024-09-17T05:03:49.001473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:09.131399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034886+0000","last_change":"2024-09-17T05:03:46.993381+0000","last_active":"2024-09-17T05:03:49.034886+0000","last_peered":"2024-09-17T05:03:49.034886+0000","last_clean":"2024-09-17T05:03:49.034886+0000","last_became_active":"2024-09-17T05:03:46.993086+0000","last_became_peered":"2024-09-17T05:03:46.993086+0000","last_unstale":"2024-09-17T05:03:49.034886+0000","last_undegraded":"2024-09-17T05:03:49.034886+0000","last_fullsized":"2024-09-17T05:03:49.034886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.685271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014852+0000","last_change":"2024-09-17T05:03:48.017975+0000","last_active":"2024-09-17T05:03:49.014852+0000","last_peered":"2024-09-17T05:03:49.014852+0000","last_clean":"2024-09-17T05:03:49.014852+0000","last_became_active":"2024-09-17T05:03:48.017654+0000","last_became_peered":"2024-09-17T05:03:48.017654+0000","last_unstale":"2024-09-17T05:03:49.014852+0000","last_undegraded":"2024-09-17T05:03:49.014852+0000","last_fullsized":"2024-09-17T05:03:49.014852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:56.907433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001194+0000","last_change":"2024-09-17T05:03:46.989740+0000","last_active":"2024-09-17T05:03:49.001194+0000","last_peered":"2024-09-17T05:03:49.001194+0000","last_clean":"2024-09-17T05:03:49.001194+0000","last_became_active":"2024-09-17T05:03:46.988950+0000","last_became_peered":"2024-09-17T05:03:46.988950+0000","last_unstale":"2024-09-17T05:03:49.001194+0000","last_undegraded":"2024-09-17T05:03:49.001194+0000","last_fullsized":"2024-09-17T05:03:49.001194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:55.361318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064851+0000","last_change":"2024-09-17T05:03:48.011028+0000","last_active":"2024-09-17T05:03:49.064851+0000","last_peered":"2024-09-17T05:03:49.064851+0000","last_clean":"2024-09-17T05:03:49.064851+0000","last_became_active":"2024-09-17T05:03:48.010691+0000","last_became_peered":"2024-09-17T05:03:48.010691+0000","last_unstale":"2024-09-17T05:03:49.064851+0000","last_undegraded":"2024-09-17T05:03:49.064851+0000","last_fullsized":"2024-09-17T05:03:49.064851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:48.136851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065615+0000","last_change":"2024-09-17T05:03:46.984870+0000","last_active":"2024-09-17T05:03:49.065615+0000","last_peered":"2024-09-17T05:03:49.065615+0000","last_clean":"2024-09-17T05:03:49.065615+0000","last_became_active":"2024-09-17T05:03:46.984556+0000","last_became_peered":"2024-09-17T05:03:46.984556+0000","last_unstale":"2024-09-17T05:03:49.065615+0000","last_undegraded":"2024-09-17T05:03:49.065615+0000","last_fullsized":"2024-09-17T05:03:49.065615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:58.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034423+0000","last_change":"2024-09-17T05:03:48.010175+0000","last_active":"2024-09-17T05:03:49.034423+0000","last_peered":"2024-09-17T05:03:49.034423+0000","last_clean":"2024-09-17T05:03:49.034423+0000","last_became_active":"2024-09-17T05:03:48.009839+0000","last_became_peered":"2024-09-17T05:03:48.009839+0000","last_unstale":"2024-09-17T05:03:49.034423+0000","last_undegraded":"2024-09-17T05:03:49.034423+0000","last_fullsized":"2024-09-17T05:03:49.034423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:05.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001024+0000","last_change":"2024-09-17T05:03:46.975859+0000","last_active":"2024-09-17T05:03:49.001024+0000","last_peered":"2024-09-17T05:03:49.001024+0000","last_clean":"2024-09-17T05:03:49.001024+0000","last_became_active":"2024-09-17T05:03:46.975513+0000","last_became_peered":"2024-09-17T05:03:46.975513+0000","last_unstale":"2024-09-17T05:03:49.001024+0000","last_undegraded":"2024-09-17T05:03:49.001024+0000","last_fullsized":"2024-09-17T05:03:49.001024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:24.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065092+0000","last_change":"2024-09-17T05:03:48.017788+0000","last_active":"2024-09-17T05:03:49.065092+0000","last_peered":"2024-09-17T05:03:49.065092+0000","last_clean":"2024-09-17T05:03:49.065092+0000","last_became_active":"2024-09-17T05:03:48.010898+0000","last_became_peered":"2024-09-17T05:03:48.010898+0000","last_unstale":"2024-09-17T05:03:49.065092+0000","last_undegraded":"2024-09-17T05:03:49.065092+0000","last_fullsized":"2024-09-17T05:03:49.065092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:32.692661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034646+0000","last_change":"2024-09-17T05:03:46.993788+0000","last_active":"2024-09-17T05:03:49.034646+0000","last_peered":"2024-09-17T05:03:49.034646+0000","last_clean":"2024-09-17T05:03:49.034646+0000","last_became_active":"2024-09-17T05:03:46.993492+0000","last_became_peered":"2024-09-17T05:03:46.993492+0000","last_unstale":"2024-09-17T05:03:49.034646+0000","last_undegraded":"2024-09-17T05:03:49.034646+0000","last_fullsized":"2024-09-17T05:03:49.034646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:16.861318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065389+0000","last_change":"2024-09-17T05:03:48.015839+0000","last_active":"2024-09-17T05:03:49.065389+0000","last_peered":"2024-09-17T05:03:49.065389+0000","last_clean":"2024-09-17T05:03:49.065389+0000","last_became_active":"2024-09-17T05:03:48.015470+0000","last_became_peered":"2024-09-17T05:03:48.015470+0000","last_unstale":"2024-09-17T05:03:49.065389+0000","last_undegraded":"2024-09-17T05:03:49.065389+0000","last_fullsized":"2024-09-17T05:03:49.065389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:39.323732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014043+0000","last_change":"2024-09-17T05:03:46.990873+0000","last_active":"2024-09-17T05:03:49.014043+0000","last_peered":"2024-09-17T05:03:49.014043+0000","last_clean":"2024-09-17T05:03:49.014043+0000","last_became_active":"2024-09-17T05:03:46.990380+0000","last_became_peered":"2024-09-17T05:03:46.990380+0000","last_unstale":"2024-09-17T05:03:49.014043+0000","last_undegraded":"2024-09-17T05:03:49.014043+0000","last_fullsized":"2024-09-17T05:03:49.014043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:57.251806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034721+0000","last_change":"2024-09-17T05:03:48.006779+0000","last_active":"2024-09-17T05:03:49.034721+0000","last_peered":"2024-09-17T05:03:49.034721+0000","last_clean":"2024-09-17T05:03:49.034721+0000","last_became_active":"2024-09-17T05:03:48.006281+0000","last_became_peered":"2024-09-17T05:03:48.006281+0000","last_unstale":"2024-09-17T05:03:49.034721+0000","last_undegraded":"2024-09-17T05:03:49.034721+0000","last_fullsized":"2024-09-17T05:03:49.034721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:32.646519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014821+0000","last_change":"2024-09-17T05:03:46.994888+0000","last_active":"2024-09-17T05:03:49.014821+0000","last_peered":"2024-09-17T05:03:49.014821+0000","last_clean":"2024-09-17T05:03:49.014821+0000","last_became_active":"2024-09-17T05:03:46.994186+0000","last_became_peered":"2024-09-17T05:03:46.994186+0000","last_unstale":"2024-09-17T05:03:49.014821+0000","last_undegraded":"2024-09-17T05:03:49.014821+0000","last_fullsized":"2024-09-17T05:03:49.014821+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:03.328151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065375+0000","last_change":"2024-09-17T05:03:48.025192+0000","last_active":"2024-09-17T05:03:49.065375+0000","last_peered":"2024-09-17T05:03:49.065375+0000","last_clean":"2024-09-17T05:03:49.065375+0000","last_became_active":"2024-09-17T05:03:48.024641+0000","last_became_peered":"2024-09-17T05:03:48.024641+0000","last_unstale":"2024-09-17T05:03:49.065375+0000","last_undegraded":"2024-09-17T05:03:49.065375+0000","last_fullsized":"2024-09-17T05:03:49.065375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:14.601287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002195+0000","last_change":"2024-09-17T05:03:46.996818+0000","last_active":"2024-09-17T05:03:49.002195+0000","last_peered":"2024-09-17T05:03:49.002195+0000","last_clean":"2024-09-17T05:03:49.002195+0000","last_became_active":"2024-09-17T05:03:46.996508+0000","last_became_peered":"2024-09-17T05:03:46.996508+0000","last_unstale":"2024-09-17T05:03:49.002195+0000","last_undegraded":"2024-09-17T05:03:49.002195+0000","last_fullsized":"2024-09-17T05:03:49.002195+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.475208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002020+0000","last_change":"2024-09-17T05:03:48.019657+0000","last_active":"2024-09-17T05:03:49.002020+0000","last_peered":"2024-09-17T05:03:49.002020+0000","last_clean":"2024-09-17T05:03:49.002020+0000","last_became_active":"2024-09-17T05:03:48.019250+0000","last_became_peered":"2024-09-17T05:03:48.019250+0000","last_unstale":"2024-09-17T05:03:49.002020+0000","last_undegraded":"2024-09-17T05:03:49.002020+0000","last_fullsized":"2024-09-17T05:03:49.002020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:40.693477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000846+0000","last_change":"2024-09-17T05:03:46.998029+0000","last_active":"2024-09-17T05:03:49.000846+0000","last_peered":"2024-09-17T05:03:49.000846+0000","last_clean":"2024-09-17T05:03:49.000846+0000","last_became_active":"2024-09-17T05:03:46.997688+0000","last_became_peered":"2024-09-17T05:03:46.997688+0000","last_unstale":"2024-09-17T05:03:49.000846+0000","last_undegraded":"2024-09-17T05:03:49.000846+0000","last_fullsized":"2024-09-17T05:03:49.000846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:10.739196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001330+0000","last_change":"2024-09-17T05:03:48.015866+0000","last_active":"2024-09-17T05:03:49.001330+0000","last_peered":"2024-09-17T05:03:49.001330+0000","last_clean":"2024-09-17T05:03:49.001330+0000","last_became_active":"2024-09-17T05:03:48.015481+0000","last_became_peered":"2024-09-17T05:03:48.015481+0000","last_unstale":"2024-09-17T05:03:49.001330+0000","last_undegraded":"2024-09-17T05:03:49.001330+0000","last_fullsized":"2024-09-17T05:03:49.001330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:32.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001031+0000","last_change":"2024-09-17T05:03:46.990156+0000","last_active":"2024-09-17T05:03:49.001031+0000","last_peered":"2024-09-17T05:03:49.001031+0000","last_clean":"2024-09-17T05:03:49.001031+0000","last_became_active":"2024-09-17T05:03:46.989449+0000","last_became_peered":"2024-09-17T05:03:46.989449+0000","last_unstale":"2024-09-17T05:03:49.001031+0000","last_undegraded":"2024-09-17T05:03:49.001031+0000","last_fullsized":"2024-09-17T05:03:49.001031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:45.812214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064816+0000","last_change":"2024-09-17T05:03:47.995726+0000","last_active":"2024-09-17T05:03:49.064816+0000","last_peered":"2024-09-17T05:03:49.064816+0000","last_clean":"2024-09-17T05:03:49.064816+0000","last_became_active":"2024-09-17T05:03:47.995344+0000","last_became_peered":"2024-09-17T05:03:47.995344+0000","last_unstale":"2024-09-17T05:03:49.064816+0000","last_undegraded":"2024-09-17T05:03:49.064816+0000","last_fullsized":"2024-09-17T05:03:49.064816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:56.121299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014404+0000","last_change":"2024-09-17T05:03:46.988141+0000","last_active":"2024-09-17T05:03:49.014404+0000","last_peered":"2024-09-17T05:03:49.014404+0000","last_clean":"2024-09-17T05:03:49.014404+0000","last_became_active":"2024-09-17T05:03:46.987762+0000","last_became_peered":"2024-09-17T05:03:46.987762+0000","last_unstale":"2024-09-17T05:03:49.014404+0000","last_undegraded":"2024-09-17T05:03:49.014404+0000","last_fullsized":"2024-09-17T05:03:49.014404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:05.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065166+0000","last_change":"2024-09-17T05:03:48.021731+0000","last_active":"2024-09-17T05:03:49.065166+0000","last_peered":"2024-09-17T05:03:49.065166+0000","last_clean":"2024-09-17T05:03:49.065166+0000","last_became_active":"2024-09-17T05:03:48.020473+0000","last_became_peered":"2024-09-17T05:03:48.020473+0000","last_unstale":"2024-09-17T05:03:49.065166+0000","last_undegraded":"2024-09-17T05:03:49.065166+0000","last_fullsized":"2024-09-17T05:03:49.065166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:17.868708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064908+0000","last_change":"2024-09-17T05:03:46.985782+0000","last_active":"2024-09-17T05:03:49.064908+0000","last_peered":"2024-09-17T05:03:49.064908+0000","last_clean":"2024-09-17T05:03:49.064908+0000","last_became_active":"2024-09-17T05:03:46.985426+0000","last_became_peered":"2024-09-17T05:03:46.985426+0000","last_unstale":"2024-09-17T05:03:49.064908+0000","last_undegraded":"2024-09-17T05:03:49.064908+0000","last_fullsized":"2024-09-17T05:03:49.064908+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:53.284564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064895+0000","last_change":"2024-09-17T05:03:48.025717+0000","last_active":"2024-09-17T05:03:49.064895+0000","last_peered":"2024-09-17T05:03:49.064895+0000","last_clean":"2024-09-17T05:03:49.064895+0000","last_became_active":"2024-09-17T05:03:48.025318+0000","last_became_peered":"2024-09-17T05:03:48.025318+0000","last_unstale":"2024-09-17T05:03:49.064895+0000","last_undegraded":"2024-09-17T05:03:49.064895+0000","last_fullsized":"2024-09-17T05:03:49.064895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:13.523946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999630+0000","last_change":"2024-09-17T05:03:46.992544+0000","last_active":"2024-09-17T05:03:48.999630+0000","last_peered":"2024-09-17T05:03:48.999630+0000","last_clean":"2024-09-17T05:03:48.999630+0000","last_became_active":"2024-09-17T05:03:46.992203+0000","last_became_peered":"2024-09-17T05:03:46.992203+0000","last_unstale":"2024-09-17T05:03:48.999630+0000","last_undegraded":"2024-09-17T05:03:48.999630+0000","last_fullsized":"2024-09-17T05:03:48.999630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:08.479610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034526+0000","last_change":"2024-09-17T05:03:48.000482+0000","last_active":"2024-09-17T05:03:49.034526+0000","last_peered":"2024-09-17T05:03:49.034526+0000","last_clean":"2024-09-17T05:03:49.034526+0000","last_became_active":"2024-09-17T05:03:48.000159+0000","last_became_peered":"2024-09-17T05:03:48.000159+0000","last_unstale":"2024-09-17T05:03:49.034526+0000","last_undegraded":"2024-09-17T05:03:49.034526+0000","last_fullsized":"2024-09-17T05:03:49.034526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:06.345556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":23,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064752+0000","last_change":"2024-09-17T05:03:46.989951+0000","last_active":"2024-09-17T05:03:49.064752+0000","last_peered":"2024-09-17T05:03:49.064752+0000","last_clean":"2024-09-17T05:03:49.064752+0000","last_became_active":"2024-09-17T05:03:46.989489+0000","last_became_peered":"2024-09-17T05:03:46.989489+0000","last_unstale":"2024-09-17T05:03:49.064752+0000","last_undegraded":"2024-09-17T05:03:49.064752+0000","last_fullsized":"2024-09-17T05:03:49.064752+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:34.074842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014755+0000","last_change":"2024-09-17T05:03:48.004942+0000","last_active":"2024-09-17T05:03:49.014755+0000","last_peered":"2024-09-17T05:03:49.014755+0000","last_clean":"2024-09-17T05:03:49.014755+0000","last_became_active":"2024-09-17T05:03:47.998747+0000","last_became_peered":"2024-09-17T05:03:47.998747+0000","last_unstale":"2024-09-17T05:03:49.014755+0000","last_undegraded":"2024-09-17T05:03:49.014755+0000","last_fullsized":"2024-09-17T05:03:49.014755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:28.871133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014933+0000","last_change":"2024-09-17T05:03:46.994482+0000","last_active":"2024-09-17T05:03:49.014933+0000","last_peered":"2024-09-17T05:03:49.014933+0000","last_clean":"2024-09-17T05:03:49.014933+0000","last_became_active":"2024-09-17T05:03:46.993791+0000","last_became_peered":"2024-09-17T05:03:46.993791+0000","last_unstale":"2024-09-17T05:03:49.014933+0000","last_undegraded":"2024-09-17T05:03:49.014933+0000","last_fullsized":"2024-09-17T05:03:49.014933+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:45.213971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001933+0000","last_change":"2024-09-17T05:03:48.021767+0000","last_active":"2024-09-17T05:03:49.001933+0000","last_peered":"2024-09-17T05:03:49.001933+0000","last_clean":"2024-09-17T05:03:49.001933+0000","last_became_active":"2024-09-17T05:03:48.019939+0000","last_became_peered":"2024-09-17T05:03:48.019939+0000","last_unstale":"2024-09-17T05:03:49.001933+0000","last_undegraded":"2024-09-17T05:03:49.001933+0000","last_fullsized":"2024-09-17T05:03:49.001933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:06.697471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014517+0000","last_change":"2024-09-17T05:03:46.997619+0000","last_active":"2024-09-17T05:03:49.014517+0000","last_peered":"2024-09-17T05:03:49.014517+0000","last_clean":"2024-09-17T05:03:49.014517+0000","last_became_active":"2024-09-17T05:03:46.994779+0000","last_became_peered":"2024-09-17T05:03:46.994779+0000","last_unstale":"2024-09-17T05:03:49.014517+0000","last_undegraded":"2024-09-17T05:03:49.014517+0000","last_fullsized":"2024-09-17T05:03:49.014517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:03.090867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002058+0000","last_change":"2024-09-17T05:03:48.019725+0000","last_active":"2024-09-17T05:03:49.002058+0000","last_peered":"2024-09-17T05:03:49.002058+0000","last_clean":"2024-09-17T05:03:49.002058+0000","last_became_active":"2024-09-17T05:03:48.019340+0000","last_became_peered":"2024-09-17T05:03:48.019340+0000","last_unstale":"2024-09-17T05:03:49.002058+0000","last_undegraded":"2024-09-17T05:03:49.002058+0000","last_fullsized":"2024-09-17T05:03:49.002058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:50.072233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000977+0000","last_change":"2024-09-17T05:03:46.994104+0000","last_active":"2024-09-17T05:03:49.000977+0000","last_peered":"2024-09-17T05:03:49.000977+0000","last_clean":"2024-09-17T05:03:49.000977+0000","last_became_active":"2024-09-17T05:03:46.993805+0000","last_became_peered":"2024-09-17T05:03:46.993805+0000","last_unstale":"2024-09-17T05:03:49.000977+0000","last_undegraded":"2024-09-17T05:03:49.000977+0000","last_fullsized":"2024-09-17T05:03:49.000977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:13.810376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001407+0000","last_change":"2024-09-17T05:03:48.020901+0000","last_active":"2024-09-17T05:03:49.001407+0000","last_peered":"2024-09-17T05:03:49.001407+0000","last_clean":"2024-09-17T05:03:49.001407+0000","last_became_active":"2024-09-17T05:03:48.018455+0000","last_became_peered":"2024-09-17T05:03:48.018455+0000","last_unstale":"2024-09-17T05:03:49.001407+0000","last_undegraded":"2024-09-17T05:03:49.001407+0000","last_fullsized":"2024-09-17T05:03:49.001407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:38.886684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017391+0000","last_change":"2024-09-17T05:03:47.001970+0000","last_active":"2024-09-17T05:03:49.017391+0000","last_peered":"2024-09-17T05:03:49.017391+0000","last_clean":"2024-09-17T05:03:49.017391+0000","last_became_active":"2024-09-17T05:03:47.001531+0000","last_became_peered":"2024-09-17T05:03:47.001531+0000","last_unstale":"2024-09-17T05:03:49.017391+0000","last_undegraded":"2024-09-17T05:03:49.017391+0000","last_fullsized":"2024-09-17T05:03:49.017391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:53.284564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001072+0000","last_change":"2024-09-17T05:03:48.030972+0000","last_active":"2024-09-17T05:03:49.001072+0000","last_peered":"2024-09-17T05:03:49.001072+0000","last_clean":"2024-09-17T05:03:49.001072+0000","last_became_active":"2024-09-17T05:03:48.030690+0000","last_became_peered":"2024-09-17T05:03:48.030690+0000","last_unstale":"2024-09-17T05:03:49.001072+0000","last_undegraded":"2024-09-17T05:03:49.001072+0000","last_fullsized":"2024-09-17T05:03:49.001072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000970+0000","last_change":"2024-09-17T05:03:46.985559+0000","last_active":"2024-09-17T05:03:49.000970+0000","last_peered":"2024-09-17T05:03:49.000970+0000","last_clean":"2024-09-17T05:03:49.000970+0000","last_became_active":"2024-09-17T05:03:46.985235+0000","last_became_peered":"2024-09-17T05:03:46.985235+0000","last_unstale":"2024-09-17T05:03:49.000970+0000","last_undegraded":"2024-09-17T05:03:49.000970+0000","last_fullsized":"2024-09-17T05:03:49.000970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:48.794376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000326+0000","last_change":"2024-09-17T05:03:48.033089+0000","last_active":"2024-09-17T05:03:49.000326+0000","last_peered":"2024-09-17T05:03:49.000326+0000","last_clean":"2024-09-17T05:03:49.000326+0000","last_became_active":"2024-09-17T05:03:48.027734+0000","last_became_peered":"2024-09-17T05:03:48.027734+0000","last_unstale":"2024-09-17T05:03:49.000326+0000","last_undegraded":"2024-09-17T05:03:49.000326+0000","last_fullsized":"2024-09-17T05:03:49.000326+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:56.121299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065081+0000","last_change":"2024-09-17T05:03:46.988419+0000","last_active":"2024-09-17T05:03:49.065081+0000","last_peered":"2024-09-17T05:03:49.065081+0000","last_clean":"2024-09-17T05:03:49.065081+0000","last_became_active":"2024-09-17T05:03:46.987659+0000","last_became_peered":"2024-09-17T05:03:46.987659+0000","last_unstale":"2024-09-17T05:03:49.065081+0000","last_undegraded":"2024-09-17T05:03:49.065081+0000","last_fullsized":"2024-09-17T05:03:49.065081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:54.033231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014179+0000","last_change":"2024-09-17T05:03:48.018319+0000","last_active":"2024-09-17T05:03:49.014179+0000","last_peered":"2024-09-17T05:03:49.014179+0000","last_clean":"2024-09-17T05:03:49.014179+0000","last_became_active":"2024-09-17T05:03:48.018040+0000","last_became_peered":"2024-09-17T05:03:48.018040+0000","last_unstale":"2024-09-17T05:03:49.014179+0000","last_undegraded":"2024-09-17T05:03:49.014179+0000","last_fullsized":"2024-09-17T05:03:49.014179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:43.830018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000435+0000","last_change":"2024-09-17T05:03:46.994473+0000","last_active":"2024-09-17T05:03:49.000435+0000","last_peered":"2024-09-17T05:03:49.000435+0000","last_clean":"2024-09-17T05:03:49.000435+0000","last_became_active":"2024-09-17T05:03:46.994185+0000","last_became_peered":"2024-09-17T05:03:46.994185+0000","last_unstale":"2024-09-17T05:03:49.000435+0000","last_undegraded":"2024-09-17T05:03:49.000435+0000","last_fullsized":"2024-09-17T05:03:49.000435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065338+0000","last_change":"2024-09-17T05:03:48.024957+0000","last_active":"2024-09-17T05:03:49.065338+0000","last_peered":"2024-09-17T05:03:49.065338+0000","last_clean":"2024-09-17T05:03:49.065338+0000","last_became_active":"2024-09-17T05:03:48.024664+0000","last_became_peered":"2024-09-17T05:03:48.024664+0000","last_unstale":"2024-09-17T05:03:49.065338+0000","last_undegraded":"2024-09-17T05:03:49.065338+0000","last_fullsized":"2024-09-17T05:03:49.065338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:27.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.015015+0000","last_change":"2024-09-17T05:03:46.996248+0000","last_active":"2024-09-17T05:03:49.015015+0000","last_peered":"2024-09-17T05:03:49.015015+0000","last_clean":"2024-09-17T05:03:49.015015+0000","last_became_active":"2024-09-17T05:03:46.995891+0000","last_became_peered":"2024-09-17T05:03:46.995891+0000","last_unstale":"2024-09-17T05:03:49.015015+0000","last_undegraded":"2024-09-17T05:03:49.015015+0000","last_fullsized":"2024-09-17T05:03:49.015015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:42.206425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065268+0000","last_change":"2024-09-17T05:03:48.005512+0000","last_active":"2024-09-17T05:03:49.065268+0000","last_peered":"2024-09-17T05:03:49.065268+0000","last_clean":"2024-09-17T05:03:49.065268+0000","last_became_active":"2024-09-17T05:03:48.004278+0000","last_became_peered":"2024-09-17T05:03:48.004278+0000","last_unstale":"2024-09-17T05:03:49.065268+0000","last_undegraded":"2024-09-17T05:03:49.065268+0000","last_fullsized":"2024-09-17T05:03:49.065268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:02.091990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014768+0000","last_change":"2024-09-17T05:03:46.990194+0000","last_active":"2024-09-17T05:03:49.014768+0000","last_peered":"2024-09-17T05:03:49.014768+0000","last_clean":"2024-09-17T05:03:49.014768+0000","last_became_active":"2024-09-17T05:03:46.989517+0000","last_became_peered":"2024-09-17T05:03:46.989517+0000","last_unstale":"2024-09-17T05:03:49.014768+0000","last_undegraded":"2024-09-17T05:03:49.014768+0000","last_fullsized":"2024-09-17T05:03:49.014768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:12.516555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000755+0000","last_change":"2024-09-17T05:03:48.007503+0000","last_active":"2024-09-17T05:03:49.000755+0000","last_peered":"2024-09-17T05:03:49.000755+0000","last_clean":"2024-09-17T05:03:49.000755+0000","last_became_active":"2024-09-17T05:03:48.007186+0000","last_became_peered":"2024-09-17T05:03:48.007186+0000","last_unstale":"2024-09-17T05:03:49.000755+0000","last_undegraded":"2024-09-17T05:03:49.000755+0000","last_fullsized":"2024-09-17T05:03:49.000755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:46.891773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001218+0000","last_change":"2024-09-17T05:03:46.993382+0000","last_active":"2024-09-17T05:03:49.001218+0000","last_peered":"2024-09-17T05:03:49.001218+0000","last_clean":"2024-09-17T05:03:49.001218+0000","last_became_active":"2024-09-17T05:03:46.992226+0000","last_became_peered":"2024-09-17T05:03:46.992226+0000","last_unstale":"2024-09-17T05:03:49.001218+0000","last_undegraded":"2024-09-17T05:03:49.001218+0000","last_fullsized":"2024-09-17T05:03:49.001218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:56.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001577+0000","last_change":"2024-09-17T05:03:48.018745+0000","last_active":"2024-09-17T05:03:49.001577+0000","last_peered":"2024-09-17T05:03:49.001577+0000","last_clean":"2024-09-17T05:03:49.001577+0000","last_became_active":"2024-09-17T05:03:48.018402+0000","last_became_peered":"2024-09-17T05:03:48.018402+0000","last_unstale":"2024-09-17T05:03:49.001577+0000","last_undegraded":"2024-09-17T05:03:49.001577+0000","last_fullsized":"2024-09-17T05:03:49.001577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:32:21.958089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017434+0000","last_change":"2024-09-17T05:03:46.995382+0000","last_active":"2024-09-17T05:03:49.017434+0000","last_peered":"2024-09-17T05:03:49.017434+0000","last_clean":"2024-09-17T05:03:49.017434+0000","last_became_active":"2024-09-17T05:03:46.995015+0000","last_became_peered":"2024-09-17T05:03:46.995015+0000","last_unstale":"2024-09-17T05:03:49.017434+0000","last_undegraded":"2024-09-17T05:03:49.017434+0000","last_fullsized":"2024-09-17T05:03:49.017434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:54.033231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014082+0000","last_change":"2024-09-17T05:03:48.024780+0000","last_active":"2024-09-17T05:03:49.014082+0000","last_peered":"2024-09-17T05:03:49.014082+0000","last_clean":"2024-09-17T05:03:49.014082+0000","last_became_active":"2024-09-17T05:03:48.024360+0000","last_became_peered":"2024-09-17T05:03:48.024360+0000","last_unstale":"2024-09-17T05:03:49.014082+0000","last_undegraded":"2024-09-17T05:03:49.014082+0000","last_fullsized":"2024-09-17T05:03:49.014082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:38.426268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001806+0000","last_change":"2024-09-17T05:03:46.993870+0000","last_active":"2024-09-17T05:03:49.001806+0000","last_peered":"2024-09-17T05:03:49.001806+0000","last_clean":"2024-09-17T05:03:49.001806+0000","last_became_active":"2024-09-17T05:03:46.993473+0000","last_became_peered":"2024-09-17T05:03:46.993473+0000","last_unstale":"2024-09-17T05:03:49.001806+0000","last_undegraded":"2024-09-17T05:03:49.001806+0000","last_fullsized":"2024-09-17T05:03:49.001806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:07.848219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000575+0000","last_change":"2024-09-17T05:03:48.015161+0000","last_active":"2024-09-17T05:03:49.000575+0000","last_peered":"2024-09-17T05:03:49.000575+0000","last_clean":"2024-09-17T05:03:49.000575+0000","last_became_active":"2024-09-17T05:03:48.014839+0000","last_became_peered":"2024-09-17T05:03:48.014839+0000","last_unstale":"2024-09-17T05:03:49.000575+0000","last_undegraded":"2024-09-17T05:03:49.000575+0000","last_fullsized":"2024-09-17T05:03:49.000575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:34.420552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034430+0000","last_change":"2024-09-17T05:03:46.987076+0000","last_active":"2024-09-17T05:03:49.034430+0000","last_peered":"2024-09-17T05:03:49.034430+0000","last_clean":"2024-09-17T05:03:49.034430+0000","last_became_active":"2024-09-17T05:03:46.986715+0000","last_became_peered":"2024-09-17T05:03:46.986715+0000","last_unstale":"2024-09-17T05:03:49.034430+0000","last_undegraded":"2024-09-17T05:03:49.034430+0000","last_fullsized":"2024-09-17T05:03:49.034430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:55.113909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000704+0000","last_change":"2024-09-17T05:03:48.025679+0000","last_active":"2024-09-17T05:03:49.000704+0000","last_peered":"2024-09-17T05:03:49.000704+0000","last_clean":"2024-09-17T05:03:49.000704+0000","last_became_active":"2024-09-17T05:03:48.025319+0000","last_became_peered":"2024-09-17T05:03:48.025319+0000","last_unstale":"2024-09-17T05:03:49.000704+0000","last_undegraded":"2024-09-17T05:03:49.000704+0000","last_fullsized":"2024-09-17T05:03:49.000704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:33.812303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017145+0000","last_change":"2024-09-17T05:03:46.991679+0000","last_active":"2024-09-17T05:03:49.017145+0000","last_peered":"2024-09-17T05:03:49.017145+0000","last_clean":"2024-09-17T05:03:49.017145+0000","last_became_active":"2024-09-17T05:03:46.990693+0000","last_became_peered":"2024-09-17T05:03:46.990693+0000","last_unstale":"2024-09-17T05:03:49.017145+0000","last_undegraded":"2024-09-17T05:03:49.017145+0000","last_fullsized":"2024-09-17T05:03:49.017145+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:16.961347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065666+0000","last_change":"2024-09-17T05:03:48.015551+0000","last_active":"2024-09-17T05:03:49.065666+0000","last_peered":"2024-09-17T05:03:49.065666+0000","last_clean":"2024-09-17T05:03:49.065666+0000","last_became_active":"2024-09-17T05:03:48.015082+0000","last_became_peered":"2024-09-17T05:03:48.015082+0000","last_unstale":"2024-09-17T05:03:49.065666+0000","last_undegraded":"2024-09-17T05:03:49.065666+0000","last_fullsized":"2024-09-17T05:03:49.065666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:20.780383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000628+0000","last_change":"2024-09-17T05:03:46.992882+0000","last_active":"2024-09-17T05:03:49.000628+0000","last_peered":"2024-09-17T05:03:49.000628+0000","last_clean":"2024-09-17T05:03:49.000628+0000","last_became_active":"2024-09-17T05:03:46.992558+0000","last_became_peered":"2024-09-17T05:03:46.992558+0000","last_unstale":"2024-09-17T05:03:49.000628+0000","last_undegraded":"2024-09-17T05:03:49.000628+0000","last_fullsized":"2024-09-17T05:03:49.000628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:35.489582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065178+0000","last_change":"2024-09-17T05:03:48.017909+0000","last_active":"2024-09-17T05:03:49.065178+0000","last_peered":"2024-09-17T05:03:49.065178+0000","last_clean":"2024-09-17T05:03:49.065178+0000","last_became_active":"2024-09-17T05:03:48.017603+0000","last_became_peered":"2024-09-17T05:03:48.017603+0000","last_unstale":"2024-09-17T05:03:49.065178+0000","last_undegraded":"2024-09-17T05:03:49.065178+0000","last_fullsized":"2024-09-17T05:03:49.065178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:43.213816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000759+0000","last_change":"2024-09-17T05:03:46.982733+0000","last_active":"2024-09-17T05:03:49.000759+0000","last_peered":"2024-09-17T05:03:49.000759+0000","last_clean":"2024-09-17T05:03:49.000759+0000","last_became_active":"2024-09-17T05:03:46.982332+0000","last_became_peered":"2024-09-17T05:03:46.982332+0000","last_unstale":"2024-09-17T05:03:49.000759+0000","last_undegraded":"2024-09-17T05:03:49.000759+0000","last_fullsized":"2024-09-17T05:03:49.000759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:19.904729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064906+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:49.064906+0000","last_peered":"2024-09-17T05:03:49.064906+0000","last_clean":"2024-09-17T05:03:49.064906+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:49.064906+0000","last_undegraded":"2024-09-17T05:03:49.064906+0000","last_fullsized":"2024-09-17T05:03:49.064906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":765952,"data_stored":343337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":786432,"data_stored":345601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":770048,"data_stored":343919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":343497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":343393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":933739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":765952,"data_stored":343337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1380352,"data_stored":935429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:56.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:03:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 -- 172.21.15.16:0/1500756847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140cc8c0 msgr2=0x7f351415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 --2- 172.21.15.16:0/1500756847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140cc8c0 0x7f351415cab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3504009840 tx=0x7f350401c820 comp rx=0 tx=0).stop 2024-09-17T05:03:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 -- 172.21.15.16:0/1500756847 shutdown_connections 2024-09-17T05:03:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 --2- 172.21.15.16:0/1500756847 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f35140c9440 0x7f35140c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 --2- 172.21.15.16:0/1500756847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140cc8c0 0x7f351415cab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 -- 172.21.15.16:0/1500756847 >> 172.21.15.16:0/1500756847 conn(0x7f35140d2270 msgr2=0x7f35140d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:57.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 -- 172.21.15.16:0/1500756847 shutdown_connections 2024-09-17T05:03:57.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 -- 172.21.15.16:0/1500756847 wait complete. 2024-09-17T05:03:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 Processor -- start 2024-09-17T05:03:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.053+0000 7f351aa00640 1 -- start start 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f35140c9440 0x7f35141537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 0x7f3514071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140d1580 0x7f35141fa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3514169fa0 con 0x7f35140cc8c0 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3514169e20 con 0x7f35140d1580 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f351416a120 con 0x7f35140c9440 2024-09-17T05:03:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 0x7f3514071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 0x7f3514071a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35098/0 (socket says 172.21.15.16:35098) 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 -- 172.21.15.16:0/163026533 learned_addr learned my addr 172.21.15.16:0/163026533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f3518a00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140d1580 0x7f35141fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f3513e00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f35140c9440 0x7f35141537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 -- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f35140c9440 msgr2=0x7f35141537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f35140c9440 0x7f35141537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 -- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140d1580 msgr2=0x7f35141fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140d1580 0x7f35141fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 -- 172.21.15.16:0/163026533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351414fd60 con 0x7f35140cc8c0 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f3518a00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140d1580 0x7f35141fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f350be00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 0x7f3514071a20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c390 tx=0x7f34fc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc011790 con 0x7f35140cc8c0 2024-09-17T05:03:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34fc017070 con 0x7f35140cc8c0 2024-09-17T05:03:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3514150040 con 0x7f35140cc8c0 2024-09-17T05:03:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc011cc0 con 0x7f35140cc8c0 2024-09-17T05:03:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.057+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3514071500 con 0x7f35140cc8c0 2024-09-17T05:03:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.061+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35140c9850 con 0x7f35140cc8c0 2024-09-17T05:03:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.061+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f34fc01c460 con 0x7f35140cc8c0 2024-09-17T05:03:57.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.061+0000 7f3510c00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34f0073200 0x7f34f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.061+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f34fc0941c0 con 0x7f35140cc8c0 2024-09-17T05:03:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.061+0000 7f3513e00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34f0073200 0x7f34f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.061+0000 7f3513e00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34f0073200 0x7f34f0075710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35040af000 tx=0x7f350401b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.065+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34fc05efc0 con 0x7f35140cc8c0 2024-09-17T05:03:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.237+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3514071f60 con 0x7f34f0073200 2024-09-17T05:03:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.241+0000 7f3510c00640 1 -- 172.21.15.16:0/163026533 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252013 (secure 0 0 0) 0x7f3514071f60 con 0x7f34f0073200 2024-09-17T05:03:57.247 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:03:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34f0073200 msgr2=0x7f34f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34f0073200 0x7f34f0075710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35040af000 tx=0x7f350401b060 comp rx=0 tx=0).stop 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 msgr2=0x7f3514071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 0x7f3514071a20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c390 tx=0x7f34fc00c850 comp rx=0 tx=0).stop 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 shutdown_connections 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34f0073200 0x7f34f0075710 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35140d1580 0x7f35141fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140cc8c0 0x7f3514071a20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 --2- 172.21.15.16:0/163026533 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f35140c9440 0x7f35141537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 >> 172.21.15.16:0/163026533 conn(0x7f35140d2270 msgr2=0x7f3514151690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 shutdown_connections 2024-09-17T05:03:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.245+0000 7f351aa00640 1 -- 172.21.15.16:0/163026533 wait complete. 2024-09-17T05:03:57.270 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-17T05:03:55.504828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244272,"kb_used_data":7208,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730448,"statfs":{"total":773094113280,"available":772843978752,"internally_reserved":0,"allocated":7380992,"data_stored":3932252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002364"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034962+0000","last_change":"2024-09-17T05:03:48.013215+0000","last_active":"2024-09-17T05:03:49.034962+0000","last_peered":"2024-09-17T05:03:49.034962+0000","last_clean":"2024-09-17T05:03:49.034962+0000","last_became_active":"2024-09-17T05:03:48.012876+0000","last_became_peered":"2024-09-17T05:03:48.012876+0000","last_unstale":"2024-09-17T05:03:49.034962+0000","last_undegraded":"2024-09-17T05:03:49.034962+0000","last_fullsized":"2024-09-17T05:03:49.034962+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:18.639295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001275+0000","last_change":"2024-09-17T05:03:48.027266+0000","last_active":"2024-09-17T05:03:49.001275+0000","last_peered":"2024-09-17T05:03:49.001275+0000","last_clean":"2024-09-17T05:03:49.001275+0000","last_became_active":"2024-09-17T05:03:48.021805+0000","last_became_peered":"2024-09-17T05:03:48.021805+0000","last_unstale":"2024-09-17T05:03:49.001275+0000","last_undegraded":"2024-09-17T05:03:49.001275+0000","last_fullsized":"2024-09-17T05:03:49.001275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:19.712294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000408+0000","last_change":"2024-09-17T05:03:48.001997+0000","last_active":"2024-09-17T05:03:49.000408+0000","last_peered":"2024-09-17T05:03:49.000408+0000","last_clean":"2024-09-17T05:03:49.000408+0000","last_became_active":"2024-09-17T05:03:48.001682+0000","last_became_peered":"2024-09-17T05:03:48.001682+0000","last_unstale":"2024-09-17T05:03:49.000408+0000","last_undegraded":"2024-09-17T05:03:49.000408+0000","last_fullsized":"2024-09-17T05:03:49.000408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:18.507396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014446+0000","last_change":"2024-09-17T05:03:48.024186+0000","last_active":"2024-09-17T05:03:49.014446+0000","last_peered":"2024-09-17T05:03:49.014446+0000","last_clean":"2024-09-17T05:03:49.014446+0000","last_became_active":"2024-09-17T05:03:48.019376+0000","last_became_peered":"2024-09-17T05:03:48.019376+0000","last_unstale":"2024-09-17T05:03:49.014446+0000","last_undegraded":"2024-09-17T05:03:49.014446+0000","last_fullsized":"2024-09-17T05:03:49.014446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:56.564278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999956+0000","last_change":"2024-09-17T05:03:48.013905+0000","last_active":"2024-09-17T05:03:48.999956+0000","last_peered":"2024-09-17T05:03:48.999956+0000","last_clean":"2024-09-17T05:03:48.999956+0000","last_became_active":"2024-09-17T05:03:48.013541+0000","last_became_peered":"2024-09-17T05:03:48.013541+0000","last_unstale":"2024-09-17T05:03:48.999956+0000","last_undegraded":"2024-09-17T05:03:48.999956+0000","last_fullsized":"2024-09-17T05:03:48.999956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:28.901368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000614+0000","last_change":"2024-09-17T05:03:48.031572+0000","last_active":"2024-09-17T05:03:49.000614+0000","last_peered":"2024-09-17T05:03:49.000614+0000","last_clean":"2024-09-17T05:03:49.000614+0000","last_became_active":"2024-09-17T05:03:48.031253+0000","last_became_peered":"2024-09-17T05:03:48.031253+0000","last_unstale":"2024-09-17T05:03:49.000614+0000","last_undegraded":"2024-09-17T05:03:49.000614+0000","last_fullsized":"2024-09-17T05:03:49.000614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:16.025505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034440+0000","last_change":"2024-09-17T05:03:48.017856+0000","last_active":"2024-09-17T05:03:49.034440+0000","last_peered":"2024-09-17T05:03:49.034440+0000","last_clean":"2024-09-17T05:03:49.034440+0000","last_became_active":"2024-09-17T05:03:48.017494+0000","last_became_peered":"2024-09-17T05:03:48.017494+0000","last_unstale":"2024-09-17T05:03:49.034440+0000","last_undegraded":"2024-09-17T05:03:49.034440+0000","last_fullsized":"2024-09-17T05:03:49.034440+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:44.663764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999262+0000","last_change":"2024-09-17T05:03:48.013895+0000","last_active":"2024-09-17T05:03:48.999262+0000","last_peered":"2024-09-17T05:03:48.999262+0000","last_clean":"2024-09-17T05:03:48.999262+0000","last_became_active":"2024-09-17T05:03:48.013581+0000","last_became_peered":"2024-09-17T05:03:48.013581+0000","last_unstale":"2024-09-17T05:03:48.999262+0000","last_undegraded":"2024-09-17T05:03:48.999262+0000","last_fullsized":"2024-09-17T05:03:48.999262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:15.892857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014615+0000","last_change":"2024-09-17T05:03:48.012221+0000","last_active":"2024-09-17T05:03:49.014615+0000","last_peered":"2024-09-17T05:03:49.014615+0000","last_clean":"2024-09-17T05:03:49.014615+0000","last_became_active":"2024-09-17T05:03:48.011887+0000","last_became_peered":"2024-09-17T05:03:48.011887+0000","last_unstale":"2024-09-17T05:03:49.014615+0000","last_undegraded":"2024-09-17T05:03:49.014615+0000","last_fullsized":"2024-09-17T05:03:49.014615+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:07.996382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065328+0000","last_change":"2024-09-17T05:03:48.023192+0000","last_active":"2024-09-17T05:03:49.065328+0000","last_peered":"2024-09-17T05:03:49.065328+0000","last_clean":"2024-09-17T05:03:49.065328+0000","last_became_active":"2024-09-17T05:03:48.022306+0000","last_became_peered":"2024-09-17T05:03:48.022306+0000","last_unstale":"2024-09-17T05:03:49.065328+0000","last_undegraded":"2024-09-17T05:03:49.065328+0000","last_fullsized":"2024-09-17T05:03:49.065328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:06.697471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014540+0000","last_change":"2024-09-17T05:03:47.996215+0000","last_active":"2024-09-17T05:03:49.014540+0000","last_peered":"2024-09-17T05:03:49.014540+0000","last_clean":"2024-09-17T05:03:49.014540+0000","last_became_active":"2024-09-17T05:03:47.995860+0000","last_became_peered":"2024-09-17T05:03:47.995860+0000","last_unstale":"2024-09-17T05:03:49.014540+0000","last_undegraded":"2024-09-17T05:03:49.014540+0000","last_fullsized":"2024-09-17T05:03:49.014540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:23.089360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014515+0000","last_change":"2024-09-17T05:03:48.000279+0000","last_active":"2024-09-17T05:03:49.014515+0000","last_peered":"2024-09-17T05:03:49.014515+0000","last_clean":"2024-09-17T05:03:49.014515+0000","last_became_active":"2024-09-17T05:03:47.999157+0000","last_became_peered":"2024-09-17T05:03:47.999157+0000","last_unstale":"2024-09-17T05:03:49.014515+0000","last_undegraded":"2024-09-17T05:03:49.014515+0000","last_fullsized":"2024-09-17T05:03:49.014515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:41:59.675067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001367+0000","last_change":"2024-09-17T05:03:48.014910+0000","last_active":"2024-09-17T05:03:49.001367+0000","last_peered":"2024-09-17T05:03:49.001367+0000","last_clean":"2024-09-17T05:03:49.001367+0000","last_became_active":"2024-09-17T05:03:48.014603+0000","last_became_peered":"2024-09-17T05:03:48.014603+0000","last_unstale":"2024-09-17T05:03:49.001367+0000","last_undegraded":"2024-09-17T05:03:49.001367+0000","last_fullsized":"2024-09-17T05:03:49.001367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:32.692661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001342+0000","last_change":"2024-09-17T05:03:48.022902+0000","last_active":"2024-09-17T05:03:49.001342+0000","last_peered":"2024-09-17T05:03:49.001342+0000","last_clean":"2024-09-17T05:03:49.001342+0000","last_became_active":"2024-09-17T05:03:48.022535+0000","last_became_peered":"2024-09-17T05:03:48.022535+0000","last_unstale":"2024-09-17T05:03:49.001342+0000","last_undegraded":"2024-09-17T05:03:49.001342+0000","last_fullsized":"2024-09-17T05:03:49.001342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:24.327217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064878+0000","last_change":"2024-09-17T05:03:48.021758+0000","last_active":"2024-09-17T05:03:49.064878+0000","last_peered":"2024-09-17T05:03:49.064878+0000","last_clean":"2024-09-17T05:03:49.064878+0000","last_became_active":"2024-09-17T05:03:48.020251+0000","last_became_peered":"2024-09-17T05:03:48.020251+0000","last_unstale":"2024-09-17T05:03:49.064878+0000","last_undegraded":"2024-09-17T05:03:49.064878+0000","last_fullsized":"2024-09-17T05:03:49.064878+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:34.420552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017103+0000","last_change":"2024-09-17T05:03:48.009149+0000","last_active":"2024-09-17T05:03:49.017103+0000","last_peered":"2024-09-17T05:03:49.017103+0000","last_clean":"2024-09-17T05:03:49.017103+0000","last_became_active":"2024-09-17T05:03:48.008880+0000","last_became_peered":"2024-09-17T05:03:48.008880+0000","last_unstale":"2024-09-17T05:03:49.017103+0000","last_undegraded":"2024-09-17T05:03:49.017103+0000","last_fullsized":"2024-09-17T05:03:49.017103+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:01.937327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065711+0000","last_change":"2024-09-17T05:03:48.014727+0000","last_active":"2024-09-17T05:03:49.065711+0000","last_peered":"2024-09-17T05:03:49.065711+0000","last_clean":"2024-09-17T05:03:49.065711+0000","last_became_active":"2024-09-17T05:03:48.014318+0000","last_became_peered":"2024-09-17T05:03:48.014318+0000","last_unstale":"2024-09-17T05:03:49.065711+0000","last_undegraded":"2024-09-17T05:03:49.065711+0000","last_fullsized":"2024-09-17T05:03:49.065711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:12.164843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014491+0000","last_change":"2024-09-17T05:03:48.024868+0000","last_active":"2024-09-17T05:03:49.014491+0000","last_peered":"2024-09-17T05:03:49.014491+0000","last_clean":"2024-09-17T05:03:49.014491+0000","last_became_active":"2024-09-17T05:03:48.024555+0000","last_became_peered":"2024-09-17T05:03:48.024555+0000","last_unstale":"2024-09-17T05:03:49.014491+0000","last_undegraded":"2024-09-17T05:03:49.014491+0000","last_fullsized":"2024-09-17T05:03:49.014491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:02.442859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014801+0000","last_change":"2024-09-17T05:03:47.994233+0000","last_active":"2024-09-17T05:03:49.014801+0000","last_peered":"2024-09-17T05:03:49.014801+0000","last_clean":"2024-09-17T05:03:49.014801+0000","last_became_active":"2024-09-17T05:03:47.993250+0000","last_became_peered":"2024-09-17T05:03:47.993250+0000","last_unstale":"2024-09-17T05:03:49.014801+0000","last_undegraded":"2024-09-17T05:03:49.014801+0000","last_fullsized":"2024-09-17T05:03:49.014801+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.200220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065248+0000","last_change":"2024-09-17T05:03:48.015536+0000","last_active":"2024-09-17T05:03:49.065248+0000","last_peered":"2024-09-17T05:03:49.065248+0000","last_clean":"2024-09-17T05:03:49.065248+0000","last_became_active":"2024-09-17T05:03:48.015214+0000","last_became_peered":"2024-09-17T05:03:48.015214+0000","last_unstale":"2024-09-17T05:03:49.065248+0000","last_undegraded":"2024-09-17T05:03:49.065248+0000","last_fullsized":"2024-09-17T05:03:49.065248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:52.284850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001645+0000","last_change":"2024-09-17T05:03:48.016724+0000","last_active":"2024-09-17T05:03:49.001645+0000","last_peered":"2024-09-17T05:03:49.001645+0000","last_clean":"2024-09-17T05:03:49.001645+0000","last_became_active":"2024-09-17T05:03:48.016150+0000","last_became_peered":"2024-09-17T05:03:48.016150+0000","last_unstale":"2024-09-17T05:03:49.001645+0000","last_undegraded":"2024-09-17T05:03:49.001645+0000","last_fullsized":"2024-09-17T05:03:49.001645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:17.868708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000912+0000","last_change":"2024-09-17T05:03:48.013364+0000","last_active":"2024-09-17T05:03:49.000912+0000","last_peered":"2024-09-17T05:03:49.000912+0000","last_clean":"2024-09-17T05:03:49.000912+0000","last_became_active":"2024-09-17T05:03:48.012273+0000","last_became_peered":"2024-09-17T05:03:48.012273+0000","last_unstale":"2024-09-17T05:03:49.000912+0000","last_undegraded":"2024-09-17T05:03:49.000912+0000","last_fullsized":"2024-09-17T05:03:49.000912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:46.993665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017206+0000","last_change":"2024-09-17T05:03:48.008799+0000","last_active":"2024-09-17T05:03:49.017206+0000","last_peered":"2024-09-17T05:03:49.017206+0000","last_clean":"2024-09-17T05:03:49.017206+0000","last_became_active":"2024-09-17T05:03:48.005133+0000","last_became_peered":"2024-09-17T05:03:48.005133+0000","last_unstale":"2024-09-17T05:03:49.017206+0000","last_undegraded":"2024-09-17T05:03:49.017206+0000","last_fullsized":"2024-09-17T05:03:49.017206+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:53.016547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999261+0000","last_change":"2024-09-17T05:03:48.015909+0000","last_active":"2024-09-17T05:03:48.999261+0000","last_peered":"2024-09-17T05:03:48.999261+0000","last_clean":"2024-09-17T05:03:48.999261+0000","last_became_active":"2024-09-17T05:03:48.015487+0000","last_became_peered":"2024-09-17T05:03:48.015487+0000","last_unstale":"2024-09-17T05:03:48.999261+0000","last_undegraded":"2024-09-17T05:03:48.999261+0000","last_fullsized":"2024-09-17T05:03:48.999261+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:24.466123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002381+0000","last_change":"2024-09-17T05:03:48.025091+0000","last_active":"2024-09-17T05:03:49.002381+0000","last_peered":"2024-09-17T05:03:49.002381+0000","last_clean":"2024-09-17T05:03:49.002381+0000","last_became_active":"2024-09-17T05:03:48.024635+0000","last_became_peered":"2024-09-17T05:03:48.024635+0000","last_unstale":"2024-09-17T05:03:49.002381+0000","last_undegraded":"2024-09-17T05:03:49.002381+0000","last_fullsized":"2024-09-17T05:03:49.002381+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:35.215246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000419+0000","last_change":"2024-09-17T05:03:48.015424+0000","last_active":"2024-09-17T05:03:49.000419+0000","last_peered":"2024-09-17T05:03:49.000419+0000","last_clean":"2024-09-17T05:03:49.000419+0000","last_became_active":"2024-09-17T05:03:48.014801+0000","last_became_peered":"2024-09-17T05:03:48.014801+0000","last_unstale":"2024-09-17T05:03:49.000419+0000","last_undegraded":"2024-09-17T05:03:49.000419+0000","last_fullsized":"2024-09-17T05:03:49.000419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:18.567770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001889+0000","last_change":"2024-09-17T05:03:48.013166+0000","last_active":"2024-09-17T05:03:49.001889+0000","last_peered":"2024-09-17T05:03:49.001889+0000","last_clean":"2024-09-17T05:03:49.001889+0000","last_became_active":"2024-09-17T05:03:48.012840+0000","last_became_peered":"2024-09-17T05:03:48.012840+0000","last_unstale":"2024-09-17T05:03:49.001889+0000","last_undegraded":"2024-09-17T05:03:49.001889+0000","last_fullsized":"2024-09-17T05:03:49.001889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:52.365194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034692+0000","last_change":"2024-09-17T05:03:47.997180+0000","last_active":"2024-09-17T05:03:49.034692+0000","last_peered":"2024-09-17T05:03:49.034692+0000","last_clean":"2024-09-17T05:03:49.034692+0000","last_became_active":"2024-09-17T05:03:47.996702+0000","last_became_peered":"2024-09-17T05:03:47.996702+0000","last_unstale":"2024-09-17T05:03:49.034692+0000","last_undegraded":"2024-09-17T05:03:49.034692+0000","last_fullsized":"2024-09-17T05:03:49.034692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:39.166350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014387+0000","last_change":"2024-09-17T05:03:48.009380+0000","last_active":"2024-09-17T05:03:49.014387+0000","last_peered":"2024-09-17T05:03:49.014387+0000","last_clean":"2024-09-17T05:03:49.014387+0000","last_became_active":"2024-09-17T05:03:48.009044+0000","last_became_peered":"2024-09-17T05:03:48.009044+0000","last_unstale":"2024-09-17T05:03:49.014387+0000","last_undegraded":"2024-09-17T05:03:49.014387+0000","last_fullsized":"2024-09-17T05:03:49.014387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:59.262724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014169+0000","last_change":"2024-09-17T05:03:48.009029+0000","last_active":"2024-09-17T05:03:49.014169+0000","last_peered":"2024-09-17T05:03:49.014169+0000","last_clean":"2024-09-17T05:03:49.014169+0000","last_became_active":"2024-09-17T05:03:48.008009+0000","last_became_peered":"2024-09-17T05:03:48.008009+0000","last_unstale":"2024-09-17T05:03:49.014169+0000","last_undegraded":"2024-09-17T05:03:49.014169+0000","last_fullsized":"2024-09-17T05:03:49.014169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:14.891647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999432+0000","last_change":"2024-09-17T05:03:48.007439+0000","last_active":"2024-09-17T05:03:48.999432+0000","last_peered":"2024-09-17T05:03:48.999432+0000","last_clean":"2024-09-17T05:03:48.999432+0000","last_became_active":"2024-09-17T05:03:48.007060+0000","last_became_peered":"2024-09-17T05:03:48.007060+0000","last_unstale":"2024-09-17T05:03:48.999432+0000","last_undegraded":"2024-09-17T05:03:48.999432+0000","last_fullsized":"2024-09-17T05:03:48.999432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:36.792619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034382+0000","last_change":"2024-09-17T05:03:48.023898+0000","last_active":"2024-09-17T05:03:49.034382+0000","last_peered":"2024-09-17T05:03:49.034382+0000","last_clean":"2024-09-17T05:03:49.034382+0000","last_became_active":"2024-09-17T05:03:48.023573+0000","last_became_peered":"2024-09-17T05:03:48.023573+0000","last_unstale":"2024-09-17T05:03:49.034382+0000","last_undegraded":"2024-09-17T05:03:49.034382+0000","last_fullsized":"2024-09-17T05:03:49.034382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:37.827085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014702+0000","last_change":"2024-09-17T05:03:47.995954+0000","last_active":"2024-09-17T05:03:49.014702+0000","last_peered":"2024-09-17T05:03:49.014702+0000","last_clean":"2024-09-17T05:03:49.014702+0000","last_became_active":"2024-09-17T05:03:47.995596+0000","last_became_peered":"2024-09-17T05:03:47.995596+0000","last_unstale":"2024-09-17T05:03:49.014702+0000","last_undegraded":"2024-09-17T05:03:49.014702+0000","last_fullsized":"2024-09-17T05:03:49.014702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:40.784043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"63'4","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034957+0000","last_change":"2024-09-17T05:03:46.987624+0000","last_active":"2024-09-17T05:03:49.034957+0000","last_peered":"2024-09-17T05:03:49.034957+0000","last_clean":"2024-09-17T05:03:49.034957+0000","last_became_active":"2024-09-17T05:03:46.987266+0000","last_became_peered":"2024-09-17T05:03:46.987266+0000","last_unstale":"2024-09-17T05:03:49.034957+0000","last_undegraded":"2024-09-17T05:03:49.034957+0000","last_fullsized":"2024-09-17T05:03:49.034957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:50:23.319826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000668+0000","last_change":"2024-09-17T05:03:48.009158+0000","last_active":"2024-09-17T05:03:49.000668+0000","last_peered":"2024-09-17T05:03:49.000668+0000","last_clean":"2024-09-17T05:03:49.000668+0000","last_became_active":"2024-09-17T05:03:48.008860+0000","last_became_peered":"2024-09-17T05:03:48.008860+0000","last_unstale":"2024-09-17T05:03:49.000668+0000","last_undegraded":"2024-09-17T05:03:49.000668+0000","last_fullsized":"2024-09-17T05:03:49.000668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:56.291803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034865+0000","last_change":"2024-09-17T05:03:46.993282+0000","last_active":"2024-09-17T05:03:49.034865+0000","last_peered":"2024-09-17T05:03:49.034865+0000","last_clean":"2024-09-17T05:03:49.034865+0000","last_became_active":"2024-09-17T05:03:46.992953+0000","last_became_peered":"2024-09-17T05:03:46.992953+0000","last_unstale":"2024-09-17T05:03:49.034865+0000","last_undegraded":"2024-09-17T05:03:49.034865+0000","last_fullsized":"2024-09-17T05:03:49.034865+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:33.413162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001473+0000","last_change":"2024-09-17T05:03:48.023120+0000","last_active":"2024-09-17T05:03:49.001473+0000","last_peered":"2024-09-17T05:03:49.001473+0000","last_clean":"2024-09-17T05:03:49.001473+0000","last_became_active":"2024-09-17T05:03:48.022823+0000","last_became_peered":"2024-09-17T05:03:48.022823+0000","last_unstale":"2024-09-17T05:03:49.001473+0000","last_undegraded":"2024-09-17T05:03:49.001473+0000","last_fullsized":"2024-09-17T05:03:49.001473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:29:09.131399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034886+0000","last_change":"2024-09-17T05:03:46.993381+0000","last_active":"2024-09-17T05:03:49.034886+0000","last_peered":"2024-09-17T05:03:49.034886+0000","last_clean":"2024-09-17T05:03:49.034886+0000","last_became_active":"2024-09-17T05:03:46.993086+0000","last_became_peered":"2024-09-17T05:03:46.993086+0000","last_unstale":"2024-09-17T05:03:49.034886+0000","last_undegraded":"2024-09-17T05:03:49.034886+0000","last_fullsized":"2024-09-17T05:03:49.034886+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:31.685271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014852+0000","last_change":"2024-09-17T05:03:48.017975+0000","last_active":"2024-09-17T05:03:49.014852+0000","last_peered":"2024-09-17T05:03:49.014852+0000","last_clean":"2024-09-17T05:03:49.014852+0000","last_became_active":"2024-09-17T05:03:48.017654+0000","last_became_peered":"2024-09-17T05:03:48.017654+0000","last_unstale":"2024-09-17T05:03:49.014852+0000","last_undegraded":"2024-09-17T05:03:49.014852+0000","last_fullsized":"2024-09-17T05:03:49.014852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:56.907433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001194+0000","last_change":"2024-09-17T05:03:46.989740+0000","last_active":"2024-09-17T05:03:49.001194+0000","last_peered":"2024-09-17T05:03:49.001194+0000","last_clean":"2024-09-17T05:03:49.001194+0000","last_became_active":"2024-09-17T05:03:46.988950+0000","last_became_peered":"2024-09-17T05:03:46.988950+0000","last_unstale":"2024-09-17T05:03:49.001194+0000","last_undegraded":"2024-09-17T05:03:49.001194+0000","last_fullsized":"2024-09-17T05:03:49.001194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:55.361318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064851+0000","last_change":"2024-09-17T05:03:48.011028+0000","last_active":"2024-09-17T05:03:49.064851+0000","last_peered":"2024-09-17T05:03:49.064851+0000","last_clean":"2024-09-17T05:03:49.064851+0000","last_became_active":"2024-09-17T05:03:48.010691+0000","last_became_peered":"2024-09-17T05:03:48.010691+0000","last_unstale":"2024-09-17T05:03:49.064851+0000","last_undegraded":"2024-09-17T05:03:49.064851+0000","last_fullsized":"2024-09-17T05:03:49.064851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:48.136851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065615+0000","last_change":"2024-09-17T05:03:46.984870+0000","last_active":"2024-09-17T05:03:49.065615+0000","last_peered":"2024-09-17T05:03:49.065615+0000","last_clean":"2024-09-17T05:03:49.065615+0000","last_became_active":"2024-09-17T05:03:46.984556+0000","last_became_peered":"2024-09-17T05:03:46.984556+0000","last_unstale":"2024-09-17T05:03:49.065615+0000","last_undegraded":"2024-09-17T05:03:49.065615+0000","last_fullsized":"2024-09-17T05:03:49.065615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:58.255333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034423+0000","last_change":"2024-09-17T05:03:48.010175+0000","last_active":"2024-09-17T05:03:49.034423+0000","last_peered":"2024-09-17T05:03:49.034423+0000","last_clean":"2024-09-17T05:03:49.034423+0000","last_became_active":"2024-09-17T05:03:48.009839+0000","last_became_peered":"2024-09-17T05:03:48.009839+0000","last_unstale":"2024-09-17T05:03:49.034423+0000","last_undegraded":"2024-09-17T05:03:49.034423+0000","last_fullsized":"2024-09-17T05:03:49.034423+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:05.382514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001024+0000","last_change":"2024-09-17T05:03:46.975859+0000","last_active":"2024-09-17T05:03:49.001024+0000","last_peered":"2024-09-17T05:03:49.001024+0000","last_clean":"2024-09-17T05:03:49.001024+0000","last_became_active":"2024-09-17T05:03:46.975513+0000","last_became_peered":"2024-09-17T05:03:46.975513+0000","last_unstale":"2024-09-17T05:03:49.001024+0000","last_undegraded":"2024-09-17T05:03:49.001024+0000","last_fullsized":"2024-09-17T05:03:49.001024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:24.210531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065092+0000","last_change":"2024-09-17T05:03:48.017788+0000","last_active":"2024-09-17T05:03:49.065092+0000","last_peered":"2024-09-17T05:03:49.065092+0000","last_clean":"2024-09-17T05:03:49.065092+0000","last_became_active":"2024-09-17T05:03:48.010898+0000","last_became_peered":"2024-09-17T05:03:48.010898+0000","last_unstale":"2024-09-17T05:03:49.065092+0000","last_undegraded":"2024-09-17T05:03:49.065092+0000","last_fullsized":"2024-09-17T05:03:49.065092+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:32.692661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034646+0000","last_change":"2024-09-17T05:03:46.993788+0000","last_active":"2024-09-17T05:03:49.034646+0000","last_peered":"2024-09-17T05:03:49.034646+0000","last_clean":"2024-09-17T05:03:49.034646+0000","last_became_active":"2024-09-17T05:03:46.993492+0000","last_became_peered":"2024-09-17T05:03:46.993492+0000","last_unstale":"2024-09-17T05:03:49.034646+0000","last_undegraded":"2024-09-17T05:03:49.034646+0000","last_fullsized":"2024-09-17T05:03:49.034646+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:16.861318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065389+0000","last_change":"2024-09-17T05:03:48.015839+0000","last_active":"2024-09-17T05:03:49.065389+0000","last_peered":"2024-09-17T05:03:49.065389+0000","last_clean":"2024-09-17T05:03:49.065389+0000","last_became_active":"2024-09-17T05:03:48.015470+0000","last_became_peered":"2024-09-17T05:03:48.015470+0000","last_unstale":"2024-09-17T05:03:49.065389+0000","last_undegraded":"2024-09-17T05:03:49.065389+0000","last_fullsized":"2024-09-17T05:03:49.065389+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:39.323732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014043+0000","last_change":"2024-09-17T05:03:46.990873+0000","last_active":"2024-09-17T05:03:49.014043+0000","last_peered":"2024-09-17T05:03:49.014043+0000","last_clean":"2024-09-17T05:03:49.014043+0000","last_became_active":"2024-09-17T05:03:46.990380+0000","last_became_peered":"2024-09-17T05:03:46.990380+0000","last_unstale":"2024-09-17T05:03:49.014043+0000","last_undegraded":"2024-09-17T05:03:49.014043+0000","last_fullsized":"2024-09-17T05:03:49.014043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:57.251806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034721+0000","last_change":"2024-09-17T05:03:48.006779+0000","last_active":"2024-09-17T05:03:49.034721+0000","last_peered":"2024-09-17T05:03:49.034721+0000","last_clean":"2024-09-17T05:03:49.034721+0000","last_became_active":"2024-09-17T05:03:48.006281+0000","last_became_peered":"2024-09-17T05:03:48.006281+0000","last_unstale":"2024-09-17T05:03:49.034721+0000","last_undegraded":"2024-09-17T05:03:49.034721+0000","last_fullsized":"2024-09-17T05:03:49.034721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:32.646519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014821+0000","last_change":"2024-09-17T05:03:46.994888+0000","last_active":"2024-09-17T05:03:49.014821+0000","last_peered":"2024-09-17T05:03:49.014821+0000","last_clean":"2024-09-17T05:03:49.014821+0000","last_became_active":"2024-09-17T05:03:46.994186+0000","last_became_peered":"2024-09-17T05:03:46.994186+0000","last_unstale":"2024-09-17T05:03:49.014821+0000","last_undegraded":"2024-09-17T05:03:49.014821+0000","last_fullsized":"2024-09-17T05:03:49.014821+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:03.328151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065375+0000","last_change":"2024-09-17T05:03:48.025192+0000","last_active":"2024-09-17T05:03:49.065375+0000","last_peered":"2024-09-17T05:03:49.065375+0000","last_clean":"2024-09-17T05:03:49.065375+0000","last_became_active":"2024-09-17T05:03:48.024641+0000","last_became_peered":"2024-09-17T05:03:48.024641+0000","last_unstale":"2024-09-17T05:03:49.065375+0000","last_undegraded":"2024-09-17T05:03:49.065375+0000","last_fullsized":"2024-09-17T05:03:49.065375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:14.601287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002195+0000","last_change":"2024-09-17T05:03:46.996818+0000","last_active":"2024-09-17T05:03:49.002195+0000","last_peered":"2024-09-17T05:03:49.002195+0000","last_clean":"2024-09-17T05:03:49.002195+0000","last_became_active":"2024-09-17T05:03:46.996508+0000","last_became_peered":"2024-09-17T05:03:46.996508+0000","last_unstale":"2024-09-17T05:03:49.002195+0000","last_undegraded":"2024-09-17T05:03:49.002195+0000","last_fullsized":"2024-09-17T05:03:49.002195+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:56.475208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002020+0000","last_change":"2024-09-17T05:03:48.019657+0000","last_active":"2024-09-17T05:03:49.002020+0000","last_peered":"2024-09-17T05:03:49.002020+0000","last_clean":"2024-09-17T05:03:49.002020+0000","last_became_active":"2024-09-17T05:03:48.019250+0000","last_became_peered":"2024-09-17T05:03:48.019250+0000","last_unstale":"2024-09-17T05:03:49.002020+0000","last_undegraded":"2024-09-17T05:03:49.002020+0000","last_fullsized":"2024-09-17T05:03:49.002020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:40.693477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"63'3","reported_seq":18,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000846+0000","last_change":"2024-09-17T05:03:46.998029+0000","last_active":"2024-09-17T05:03:49.000846+0000","last_peered":"2024-09-17T05:03:49.000846+0000","last_clean":"2024-09-17T05:03:49.000846+0000","last_became_active":"2024-09-17T05:03:46.997688+0000","last_became_peered":"2024-09-17T05:03:46.997688+0000","last_unstale":"2024-09-17T05:03:49.000846+0000","last_undegraded":"2024-09-17T05:03:49.000846+0000","last_fullsized":"2024-09-17T05:03:49.000846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:10.739196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001330+0000","last_change":"2024-09-17T05:03:48.015866+0000","last_active":"2024-09-17T05:03:49.001330+0000","last_peered":"2024-09-17T05:03:49.001330+0000","last_clean":"2024-09-17T05:03:49.001330+0000","last_became_active":"2024-09-17T05:03:48.015481+0000","last_became_peered":"2024-09-17T05:03:48.015481+0000","last_unstale":"2024-09-17T05:03:49.001330+0000","last_undegraded":"2024-09-17T05:03:49.001330+0000","last_fullsized":"2024-09-17T05:03:49.001330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:32.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"63'6","reported_seq":21,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001031+0000","last_change":"2024-09-17T05:03:46.990156+0000","last_active":"2024-09-17T05:03:49.001031+0000","last_peered":"2024-09-17T05:03:49.001031+0000","last_clean":"2024-09-17T05:03:49.001031+0000","last_became_active":"2024-09-17T05:03:46.989449+0000","last_became_peered":"2024-09-17T05:03:46.989449+0000","last_unstale":"2024-09-17T05:03:49.001031+0000","last_undegraded":"2024-09-17T05:03:49.001031+0000","last_fullsized":"2024-09-17T05:03:49.001031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:45.812214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064816+0000","last_change":"2024-09-17T05:03:47.995726+0000","last_active":"2024-09-17T05:03:49.064816+0000","last_peered":"2024-09-17T05:03:49.064816+0000","last_clean":"2024-09-17T05:03:49.064816+0000","last_became_active":"2024-09-17T05:03:47.995344+0000","last_became_peered":"2024-09-17T05:03:47.995344+0000","last_unstale":"2024-09-17T05:03:49.064816+0000","last_undegraded":"2024-09-17T05:03:49.064816+0000","last_fullsized":"2024-09-17T05:03:49.064816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:56.121299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014404+0000","last_change":"2024-09-17T05:03:46.988141+0000","last_active":"2024-09-17T05:03:49.014404+0000","last_peered":"2024-09-17T05:03:49.014404+0000","last_clean":"2024-09-17T05:03:49.014404+0000","last_became_active":"2024-09-17T05:03:46.987762+0000","last_became_peered":"2024-09-17T05:03:46.987762+0000","last_unstale":"2024-09-17T05:03:49.014404+0000","last_undegraded":"2024-09-17T05:03:49.014404+0000","last_fullsized":"2024-09-17T05:03:49.014404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:05.312689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065166+0000","last_change":"2024-09-17T05:03:48.021731+0000","last_active":"2024-09-17T05:03:49.065166+0000","last_peered":"2024-09-17T05:03:49.065166+0000","last_clean":"2024-09-17T05:03:49.065166+0000","last_became_active":"2024-09-17T05:03:48.020473+0000","last_became_peered":"2024-09-17T05:03:48.020473+0000","last_unstale":"2024-09-17T05:03:49.065166+0000","last_undegraded":"2024-09-17T05:03:49.065166+0000","last_fullsized":"2024-09-17T05:03:49.065166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:45:17.868708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064908+0000","last_change":"2024-09-17T05:03:46.985782+0000","last_active":"2024-09-17T05:03:49.064908+0000","last_peered":"2024-09-17T05:03:49.064908+0000","last_clean":"2024-09-17T05:03:49.064908+0000","last_became_active":"2024-09-17T05:03:46.985426+0000","last_became_peered":"2024-09-17T05:03:46.985426+0000","last_unstale":"2024-09-17T05:03:49.064908+0000","last_undegraded":"2024-09-17T05:03:49.064908+0000","last_fullsized":"2024-09-17T05:03:49.064908+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:53.284564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064895+0000","last_change":"2024-09-17T05:03:48.025717+0000","last_active":"2024-09-17T05:03:49.064895+0000","last_peered":"2024-09-17T05:03:49.064895+0000","last_clean":"2024-09-17T05:03:49.064895+0000","last_became_active":"2024-09-17T05:03:48.025318+0000","last_became_peered":"2024-09-17T05:03:48.025318+0000","last_unstale":"2024-09-17T05:03:49.064895+0000","last_undegraded":"2024-09-17T05:03:49.064895+0000","last_fullsized":"2024-09-17T05:03:49.064895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:13.523946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:48.999630+0000","last_change":"2024-09-17T05:03:46.992544+0000","last_active":"2024-09-17T05:03:48.999630+0000","last_peered":"2024-09-17T05:03:48.999630+0000","last_clean":"2024-09-17T05:03:48.999630+0000","last_became_active":"2024-09-17T05:03:46.992203+0000","last_became_peered":"2024-09-17T05:03:46.992203+0000","last_unstale":"2024-09-17T05:03:48.999630+0000","last_undegraded":"2024-09-17T05:03:48.999630+0000","last_fullsized":"2024-09-17T05:03:48.999630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:08.479610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034526+0000","last_change":"2024-09-17T05:03:48.000482+0000","last_active":"2024-09-17T05:03:49.034526+0000","last_peered":"2024-09-17T05:03:49.034526+0000","last_clean":"2024-09-17T05:03:49.034526+0000","last_became_active":"2024-09-17T05:03:48.000159+0000","last_became_peered":"2024-09-17T05:03:48.000159+0000","last_unstale":"2024-09-17T05:03:49.034526+0000","last_undegraded":"2024-09-17T05:03:49.034526+0000","last_fullsized":"2024-09-17T05:03:49.034526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:06.345556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"63'8","reported_seq":23,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064752+0000","last_change":"2024-09-17T05:03:46.989951+0000","last_active":"2024-09-17T05:03:49.064752+0000","last_peered":"2024-09-17T05:03:49.064752+0000","last_clean":"2024-09-17T05:03:49.064752+0000","last_became_active":"2024-09-17T05:03:46.989489+0000","last_became_peered":"2024-09-17T05:03:46.989489+0000","last_unstale":"2024-09-17T05:03:49.064752+0000","last_undegraded":"2024-09-17T05:03:49.064752+0000","last_fullsized":"2024-09-17T05:03:49.064752+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:34.074842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014755+0000","last_change":"2024-09-17T05:03:48.004942+0000","last_active":"2024-09-17T05:03:49.014755+0000","last_peered":"2024-09-17T05:03:49.014755+0000","last_clean":"2024-09-17T05:03:49.014755+0000","last_became_active":"2024-09-17T05:03:47.998747+0000","last_became_peered":"2024-09-17T05:03:47.998747+0000","last_unstale":"2024-09-17T05:03:49.014755+0000","last_undegraded":"2024-09-17T05:03:49.014755+0000","last_fullsized":"2024-09-17T05:03:49.014755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:28.871133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014933+0000","last_change":"2024-09-17T05:03:46.994482+0000","last_active":"2024-09-17T05:03:49.014933+0000","last_peered":"2024-09-17T05:03:49.014933+0000","last_clean":"2024-09-17T05:03:49.014933+0000","last_became_active":"2024-09-17T05:03:46.993791+0000","last_became_peered":"2024-09-17T05:03:46.993791+0000","last_unstale":"2024-09-17T05:03:49.014933+0000","last_undegraded":"2024-09-17T05:03:49.014933+0000","last_fullsized":"2024-09-17T05:03:49.014933+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:45.213971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001933+0000","last_change":"2024-09-17T05:03:48.021767+0000","last_active":"2024-09-17T05:03:49.001933+0000","last_peered":"2024-09-17T05:03:49.001933+0000","last_clean":"2024-09-17T05:03:49.001933+0000","last_became_active":"2024-09-17T05:03:48.019939+0000","last_became_peered":"2024-09-17T05:03:48.019939+0000","last_unstale":"2024-09-17T05:03:49.001933+0000","last_undegraded":"2024-09-17T05:03:49.001933+0000","last_fullsized":"2024-09-17T05:03:49.001933+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:06.697471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014517+0000","last_change":"2024-09-17T05:03:46.997619+0000","last_active":"2024-09-17T05:03:49.014517+0000","last_peered":"2024-09-17T05:03:49.014517+0000","last_clean":"2024-09-17T05:03:49.014517+0000","last_became_active":"2024-09-17T05:03:46.994779+0000","last_became_peered":"2024-09-17T05:03:46.994779+0000","last_unstale":"2024-09-17T05:03:49.014517+0000","last_undegraded":"2024-09-17T05:03:49.014517+0000","last_fullsized":"2024-09-17T05:03:49.014517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:03.090867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.002058+0000","last_change":"2024-09-17T05:03:48.019725+0000","last_active":"2024-09-17T05:03:49.002058+0000","last_peered":"2024-09-17T05:03:49.002058+0000","last_clean":"2024-09-17T05:03:49.002058+0000","last_became_active":"2024-09-17T05:03:48.019340+0000","last_became_peered":"2024-09-17T05:03:48.019340+0000","last_unstale":"2024-09-17T05:03:49.002058+0000","last_undegraded":"2024-09-17T05:03:49.002058+0000","last_fullsized":"2024-09-17T05:03:49.002058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:50.072233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"63'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000977+0000","last_change":"2024-09-17T05:03:46.994104+0000","last_active":"2024-09-17T05:03:49.000977+0000","last_peered":"2024-09-17T05:03:49.000977+0000","last_clean":"2024-09-17T05:03:49.000977+0000","last_became_active":"2024-09-17T05:03:46.993805+0000","last_became_peered":"2024-09-17T05:03:46.993805+0000","last_unstale":"2024-09-17T05:03:49.000977+0000","last_undegraded":"2024-09-17T05:03:49.000977+0000","last_fullsized":"2024-09-17T05:03:49.000977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:13.810376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001407+0000","last_change":"2024-09-17T05:03:48.020901+0000","last_active":"2024-09-17T05:03:49.001407+0000","last_peered":"2024-09-17T05:03:49.001407+0000","last_clean":"2024-09-17T05:03:49.001407+0000","last_became_active":"2024-09-17T05:03:48.018455+0000","last_became_peered":"2024-09-17T05:03:48.018455+0000","last_unstale":"2024-09-17T05:03:49.001407+0000","last_undegraded":"2024-09-17T05:03:49.001407+0000","last_fullsized":"2024-09-17T05:03:49.001407+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:29:38.886684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017391+0000","last_change":"2024-09-17T05:03:47.001970+0000","last_active":"2024-09-17T05:03:49.017391+0000","last_peered":"2024-09-17T05:03:49.017391+0000","last_clean":"2024-09-17T05:03:49.017391+0000","last_became_active":"2024-09-17T05:03:47.001531+0000","last_became_peered":"2024-09-17T05:03:47.001531+0000","last_unstale":"2024-09-17T05:03:49.017391+0000","last_undegraded":"2024-09-17T05:03:49.017391+0000","last_fullsized":"2024-09-17T05:03:49.017391+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:53.284564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001072+0000","last_change":"2024-09-17T05:03:48.030972+0000","last_active":"2024-09-17T05:03:49.001072+0000","last_peered":"2024-09-17T05:03:49.001072+0000","last_clean":"2024-09-17T05:03:49.001072+0000","last_became_active":"2024-09-17T05:03:48.030690+0000","last_became_peered":"2024-09-17T05:03:48.030690+0000","last_unstale":"2024-09-17T05:03:49.001072+0000","last_undegraded":"2024-09-17T05:03:49.001072+0000","last_fullsized":"2024-09-17T05:03:49.001072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:57.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000970+0000","last_change":"2024-09-17T05:03:46.985559+0000","last_active":"2024-09-17T05:03:49.000970+0000","last_peered":"2024-09-17T05:03:49.000970+0000","last_clean":"2024-09-17T05:03:49.000970+0000","last_became_active":"2024-09-17T05:03:46.985235+0000","last_became_peered":"2024-09-17T05:03:46.985235+0000","last_unstale":"2024-09-17T05:03:49.000970+0000","last_undegraded":"2024-09-17T05:03:49.000970+0000","last_fullsized":"2024-09-17T05:03:49.000970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:48.794376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000326+0000","last_change":"2024-09-17T05:03:48.033089+0000","last_active":"2024-09-17T05:03:49.000326+0000","last_peered":"2024-09-17T05:03:49.000326+0000","last_clean":"2024-09-17T05:03:49.000326+0000","last_became_active":"2024-09-17T05:03:48.027734+0000","last_became_peered":"2024-09-17T05:03:48.027734+0000","last_unstale":"2024-09-17T05:03:49.000326+0000","last_undegraded":"2024-09-17T05:03:49.000326+0000","last_fullsized":"2024-09-17T05:03:49.000326+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:56.121299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065081+0000","last_change":"2024-09-17T05:03:46.988419+0000","last_active":"2024-09-17T05:03:49.065081+0000","last_peered":"2024-09-17T05:03:49.065081+0000","last_clean":"2024-09-17T05:03:49.065081+0000","last_became_active":"2024-09-17T05:03:46.987659+0000","last_became_peered":"2024-09-17T05:03:46.987659+0000","last_unstale":"2024-09-17T05:03:49.065081+0000","last_undegraded":"2024-09-17T05:03:49.065081+0000","last_fullsized":"2024-09-17T05:03:49.065081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:54.033231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014179+0000","last_change":"2024-09-17T05:03:48.018319+0000","last_active":"2024-09-17T05:03:49.014179+0000","last_peered":"2024-09-17T05:03:49.014179+0000","last_clean":"2024-09-17T05:03:49.014179+0000","last_became_active":"2024-09-17T05:03:48.018040+0000","last_became_peered":"2024-09-17T05:03:48.018040+0000","last_unstale":"2024-09-17T05:03:49.014179+0000","last_undegraded":"2024-09-17T05:03:49.014179+0000","last_fullsized":"2024-09-17T05:03:49.014179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:43.830018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000435+0000","last_change":"2024-09-17T05:03:46.994473+0000","last_active":"2024-09-17T05:03:49.000435+0000","last_peered":"2024-09-17T05:03:49.000435+0000","last_clean":"2024-09-17T05:03:49.000435+0000","last_became_active":"2024-09-17T05:03:46.994185+0000","last_became_peered":"2024-09-17T05:03:46.994185+0000","last_unstale":"2024-09-17T05:03:49.000435+0000","last_undegraded":"2024-09-17T05:03:49.000435+0000","last_fullsized":"2024-09-17T05:03:49.000435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:23:27.738482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065338+0000","last_change":"2024-09-17T05:03:48.024957+0000","last_active":"2024-09-17T05:03:49.065338+0000","last_peered":"2024-09-17T05:03:49.065338+0000","last_clean":"2024-09-17T05:03:49.065338+0000","last_became_active":"2024-09-17T05:03:48.024664+0000","last_became_peered":"2024-09-17T05:03:48.024664+0000","last_unstale":"2024-09-17T05:03:49.065338+0000","last_undegraded":"2024-09-17T05:03:49.065338+0000","last_fullsized":"2024-09-17T05:03:49.065338+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:27.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"63'2","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.015015+0000","last_change":"2024-09-17T05:03:46.996248+0000","last_active":"2024-09-17T05:03:49.015015+0000","last_peered":"2024-09-17T05:03:49.015015+0000","last_clean":"2024-09-17T05:03:49.015015+0000","last_became_active":"2024-09-17T05:03:46.995891+0000","last_became_peered":"2024-09-17T05:03:46.995891+0000","last_unstale":"2024-09-17T05:03:49.015015+0000","last_undegraded":"2024-09-17T05:03:49.015015+0000","last_fullsized":"2024-09-17T05:03:49.015015+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:42.206425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065268+0000","last_change":"2024-09-17T05:03:48.005512+0000","last_active":"2024-09-17T05:03:49.065268+0000","last_peered":"2024-09-17T05:03:49.065268+0000","last_clean":"2024-09-17T05:03:49.065268+0000","last_became_active":"2024-09-17T05:03:48.004278+0000","last_became_peered":"2024-09-17T05:03:48.004278+0000","last_unstale":"2024-09-17T05:03:49.065268+0000","last_undegraded":"2024-09-17T05:03:49.065268+0000","last_fullsized":"2024-09-17T05:03:49.065268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:02.091990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"64'1","reported_seq":16,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014768+0000","last_change":"2024-09-17T05:03:46.990194+0000","last_active":"2024-09-17T05:03:49.014768+0000","last_peered":"2024-09-17T05:03:49.014768+0000","last_clean":"2024-09-17T05:03:49.014768+0000","last_became_active":"2024-09-17T05:03:46.989517+0000","last_became_peered":"2024-09-17T05:03:46.989517+0000","last_unstale":"2024-09-17T05:03:49.014768+0000","last_undegraded":"2024-09-17T05:03:49.014768+0000","last_fullsized":"2024-09-17T05:03:49.014768+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:12.516555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000755+0000","last_change":"2024-09-17T05:03:48.007503+0000","last_active":"2024-09-17T05:03:49.000755+0000","last_peered":"2024-09-17T05:03:49.000755+0000","last_clean":"2024-09-17T05:03:49.000755+0000","last_became_active":"2024-09-17T05:03:48.007186+0000","last_became_peered":"2024-09-17T05:03:48.007186+0000","last_unstale":"2024-09-17T05:03:49.000755+0000","last_undegraded":"2024-09-17T05:03:49.000755+0000","last_fullsized":"2024-09-17T05:03:49.000755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:46.891773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001218+0000","last_change":"2024-09-17T05:03:46.993382+0000","last_active":"2024-09-17T05:03:49.001218+0000","last_peered":"2024-09-17T05:03:49.001218+0000","last_clean":"2024-09-17T05:03:49.001218+0000","last_became_active":"2024-09-17T05:03:46.992226+0000","last_became_peered":"2024-09-17T05:03:46.992226+0000","last_unstale":"2024-09-17T05:03:49.001218+0000","last_undegraded":"2024-09-17T05:03:49.001218+0000","last_fullsized":"2024-09-17T05:03:49.001218+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:56.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001577+0000","last_change":"2024-09-17T05:03:48.018745+0000","last_active":"2024-09-17T05:03:49.001577+0000","last_peered":"2024-09-17T05:03:49.001577+0000","last_clean":"2024-09-17T05:03:49.001577+0000","last_became_active":"2024-09-17T05:03:48.018402+0000","last_became_peered":"2024-09-17T05:03:48.018402+0000","last_unstale":"2024-09-17T05:03:49.001577+0000","last_undegraded":"2024-09-17T05:03:49.001577+0000","last_fullsized":"2024-09-17T05:03:49.001577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:32:21.958089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017434+0000","last_change":"2024-09-17T05:03:46.995382+0000","last_active":"2024-09-17T05:03:49.017434+0000","last_peered":"2024-09-17T05:03:49.017434+0000","last_clean":"2024-09-17T05:03:49.017434+0000","last_became_active":"2024-09-17T05:03:46.995015+0000","last_became_peered":"2024-09-17T05:03:46.995015+0000","last_unstale":"2024-09-17T05:03:49.017434+0000","last_undegraded":"2024-09-17T05:03:49.017434+0000","last_fullsized":"2024-09-17T05:03:49.017434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:54.033231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.014082+0000","last_change":"2024-09-17T05:03:48.024780+0000","last_active":"2024-09-17T05:03:49.014082+0000","last_peered":"2024-09-17T05:03:49.014082+0000","last_clean":"2024-09-17T05:03:49.014082+0000","last_became_active":"2024-09-17T05:03:48.024360+0000","last_became_peered":"2024-09-17T05:03:48.024360+0000","last_unstale":"2024-09-17T05:03:49.014082+0000","last_undegraded":"2024-09-17T05:03:49.014082+0000","last_fullsized":"2024-09-17T05:03:49.014082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:38.426268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.001806+0000","last_change":"2024-09-17T05:03:46.993870+0000","last_active":"2024-09-17T05:03:49.001806+0000","last_peered":"2024-09-17T05:03:49.001806+0000","last_clean":"2024-09-17T05:03:49.001806+0000","last_became_active":"2024-09-17T05:03:46.993473+0000","last_became_peered":"2024-09-17T05:03:46.993473+0000","last_unstale":"2024-09-17T05:03:49.001806+0000","last_undegraded":"2024-09-17T05:03:49.001806+0000","last_fullsized":"2024-09-17T05:03:49.001806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:07.848219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000575+0000","last_change":"2024-09-17T05:03:48.015161+0000","last_active":"2024-09-17T05:03:49.000575+0000","last_peered":"2024-09-17T05:03:49.000575+0000","last_clean":"2024-09-17T05:03:49.000575+0000","last_became_active":"2024-09-17T05:03:48.014839+0000","last_became_peered":"2024-09-17T05:03:48.014839+0000","last_unstale":"2024-09-17T05:03:49.000575+0000","last_undegraded":"2024-09-17T05:03:49.000575+0000","last_fullsized":"2024-09-17T05:03:49.000575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:34.420552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.034430+0000","last_change":"2024-09-17T05:03:46.987076+0000","last_active":"2024-09-17T05:03:49.034430+0000","last_peered":"2024-09-17T05:03:49.034430+0000","last_clean":"2024-09-17T05:03:49.034430+0000","last_became_active":"2024-09-17T05:03:46.986715+0000","last_became_peered":"2024-09-17T05:03:46.986715+0000","last_unstale":"2024-09-17T05:03:49.034430+0000","last_undegraded":"2024-09-17T05:03:49.034430+0000","last_fullsized":"2024-09-17T05:03:49.034430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:50:55.113909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000704+0000","last_change":"2024-09-17T05:03:48.025679+0000","last_active":"2024-09-17T05:03:49.000704+0000","last_peered":"2024-09-17T05:03:49.000704+0000","last_clean":"2024-09-17T05:03:49.000704+0000","last_became_active":"2024-09-17T05:03:48.025319+0000","last_became_peered":"2024-09-17T05:03:48.025319+0000","last_unstale":"2024-09-17T05:03:49.000704+0000","last_undegraded":"2024-09-17T05:03:49.000704+0000","last_fullsized":"2024-09-17T05:03:49.000704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:33.812303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.017145+0000","last_change":"2024-09-17T05:03:46.991679+0000","last_active":"2024-09-17T05:03:49.017145+0000","last_peered":"2024-09-17T05:03:49.017145+0000","last_clean":"2024-09-17T05:03:49.017145+0000","last_became_active":"2024-09-17T05:03:46.990693+0000","last_became_peered":"2024-09-17T05:03:46.990693+0000","last_unstale":"2024-09-17T05:03:49.017145+0000","last_undegraded":"2024-09-17T05:03:49.017145+0000","last_fullsized":"2024-09-17T05:03:49.017145+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:44:16.961347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065666+0000","last_change":"2024-09-17T05:03:48.015551+0000","last_active":"2024-09-17T05:03:49.065666+0000","last_peered":"2024-09-17T05:03:49.065666+0000","last_clean":"2024-09-17T05:03:49.065666+0000","last_became_active":"2024-09-17T05:03:48.015082+0000","last_became_peered":"2024-09-17T05:03:48.015082+0000","last_unstale":"2024-09-17T05:03:49.065666+0000","last_undegraded":"2024-09-17T05:03:49.065666+0000","last_fullsized":"2024-09-17T05:03:49.065666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:20.780383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000628+0000","last_change":"2024-09-17T05:03:46.992882+0000","last_active":"2024-09-17T05:03:49.000628+0000","last_peered":"2024-09-17T05:03:49.000628+0000","last_clean":"2024-09-17T05:03:49.000628+0000","last_became_active":"2024-09-17T05:03:46.992558+0000","last_became_peered":"2024-09-17T05:03:46.992558+0000","last_unstale":"2024-09-17T05:03:49.000628+0000","last_undegraded":"2024-09-17T05:03:49.000628+0000","last_fullsized":"2024-09-17T05:03:49.000628+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:35.489582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.065178+0000","last_change":"2024-09-17T05:03:48.017909+0000","last_active":"2024-09-17T05:03:49.065178+0000","last_peered":"2024-09-17T05:03:49.065178+0000","last_clean":"2024-09-17T05:03:49.065178+0000","last_became_active":"2024-09-17T05:03:48.017603+0000","last_became_peered":"2024-09-17T05:03:48.017603+0000","last_unstale":"2024-09-17T05:03:49.065178+0000","last_undegraded":"2024-09-17T05:03:49.065178+0000","last_fullsized":"2024-09-17T05:03:49.065178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:46.961444+0000","last_clean_scrub_stamp":"2024-09-17T05:03:46.961444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:43.213816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.000759+0000","last_change":"2024-09-17T05:03:46.982733+0000","last_active":"2024-09-17T05:03:49.000759+0000","last_peered":"2024-09-17T05:03:49.000759+0000","last_clean":"2024-09-17T05:03:49.000759+0000","last_became_active":"2024-09-17T05:03:46.982332+0000","last_became_peered":"2024-09-17T05:03:46.982332+0000","last_unstale":"2024-09-17T05:03:49.000759+0000","last_undegraded":"2024-09-17T05:03:49.000759+0000","last_fullsized":"2024-09-17T05:03:49.000759+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:03:45.954053+0000","last_clean_scrub_stamp":"2024-09-17T05:03:45.954053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:39:19.904729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-17T05:03:49.064906+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:03:49.064906+0000","last_peered":"2024-09-17T05:03:49.064906+0000","last_clean":"2024-09-17T05:03:49.064906+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:03:49.064906+0000","last_undegraded":"2024-09-17T05:03:49.064906+0000","last_fullsized":"2024-09-17T05:03:49.064906+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":765952,"data_stored":343337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":786432,"data_stored":345601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509478,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":770048,"data_stored":343919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":774144,"data_stored":343497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":343393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":933739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":765952,"data_stored":343337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":1380352,"data_stored":935429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:03:57.274 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:03:57.274 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a41000640 1 -- 172.21.15.16:0/230823048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 msgr2=0x7f6a20005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a41000640 1 --2- 172.21.15.16:0/230823048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a20005160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0698d0 tx=0x7f6a3401d2e0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a3a800640 1 -- 172.21.15.16:0/230823048 reap_dead start 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a41000640 1 -- 172.21.15.16:0/230823048 shutdown_connections 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a41000640 1 -- 172.21.15.16:0/230823048 >> 172.21.15.16:0/230823048 conn(0x7f6a3c0d0860 msgr2=0x7f6a3c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a41000640 1 -- 172.21.15.16:0/230823048 shutdown_connections 2024-09-17T05:03:57.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.477+0000 7f6a41000640 1 -- 172.21.15.16:0/230823048 wait complete. 2024-09-17T05:03:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 Processor -- start 2024-09-17T05:03:57.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 -- start start 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a3c0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a3c156360 0x7f6a3c1503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a3c06da20 0x7f6a3c06de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c0cfdf0 con 0x7f6a20004d50 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c0cfc70 con 0x7f6a3c06da20 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3c0b5870 con 0x7f6a3c156360 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a3c0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a3c0c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35102/0 (socket says 172.21.15.16:35102) 2024-09-17T05:03:57.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 -- 172.21.15.16:0/3814044805 learned_addr learned my addr 172.21.15.16:0/3814044805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 -- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a3c156360 msgr2=0x7f6a3c1503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a3c156360 0x7f6a3c1503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 -- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a3c06da20 msgr2=0x7f6a3c06de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a3c06da20 0x7f6a3c06de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 -- 172.21.15.16:0/3814044805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c06e370 con 0x7f6a20004d50 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a39e00640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a3c0c5780 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c011ab0 tx=0x7f6a2c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00fe50 con 0x7f6a20004d50 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a2c016070 con 0x7f6a20004d50 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0127e0 con 0x7f6a20004d50 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c150920 con 0x7f6a20004d50 2024-09-17T05:03:57.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.481+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1f8b90 con 0x7f6a20004d50 2024-09-17T05:03:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.485+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a3c15f1b0 con 0x7f6a20004d50 2024-09-17T05:03:57.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.485+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6a2c018030 con 0x7f6a20004d50 2024-09-17T05:03:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.485+0000 7f6a32000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6a18073200 0x7f6a18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.485+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f6a2c094e80 con 0x7f6a20004d50 2024-09-17T05:03:57.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.485+0000 7f6a33e00640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6a18073200 0x7f6a18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.485+0000 7f6a33e00640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6a18073200 0x7f6a18075710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0b4240 tx=0x7f6a24009650 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.489+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6a3c06e370 con 0x7f6a20004d50 2024-09-17T05:03:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.725+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6a3c0b44c0 con 0x7f6a20004d50 2024-09-17T05:03:57.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.729+0000 7f6a32000640 1 -- 172.21.15.16:0/3814044805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) ==== 190+0+147 (secure 0 0 0) 0x7f6a2c05fc70 con 0x7f6a20004d50 2024-09-17T05:03:57.731 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:03:57.732 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:03:57.732 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:03:57.732 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:03:57.732 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:03:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.729+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6a18073200 msgr2=0x7f6a18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.729+0000 7f6a41000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6a18073200 0x7f6a18075710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0b4240 tx=0x7f6a24009650 comp rx=0 tx=0).stop 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.729+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 msgr2=0x7f6a3c0c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.729+0000 7f6a41000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a3c0c5780 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c011ab0 tx=0x7f6a2c011f70 comp rx=0 tx=0).stop 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 shutdown_connections 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6a18073200 0x7f6a18075710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a3c06da20 0x7f6a3c06de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6a3c156360 0x7f6a3c1503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 --2- 172.21.15.16:0/3814044805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a20004d50 0x7f6a3c0c5780 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 >> 172.21.15.16:0/3814044805 conn(0x7f6a3c0d0860 msgr2=0x7f6a3c0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 shutdown_connections 2024-09-17T05:03:57.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.733+0000 7f6a41000640 1 -- 172.21.15.16:0/3814044805 wait complete. 2024-09-17T05:03:57.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 -- 172.21.15.16:0/1073875603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76806efb0 msgr2=0x7ff76806f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 --2- 172.21.15.16:0/1073875603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76806efb0 0x7ff76806f3c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff7680b62f0 tx=0x7ff75401c6c0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 -- 172.21.15.16:0/1073875603 shutdown_connections 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 --2- 172.21.15.16:0/1073875603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff76806efb0 0x7ff76806f3c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 --2- 172.21.15.16:0/1073875603 >> v2:172.21.15.104:3300/0 conn(0x7ff768071380 0x7ff768071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 -- 172.21.15.16:0/1073875603 >> 172.21.15.16:0/1073875603 conn(0x7ff768073ca0 msgr2=0x7ff7680740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 -- 172.21.15.16:0/1073875603 shutdown_connections 2024-09-17T05:03:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.909+0000 7ff76e400640 1 -- 172.21.15.16:0/1073875603 wait complete. 2024-09-17T05:03:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 Processor -- start 2024-09-17T05:03:57.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 -- start start 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff76806efb0 0x7ff7681f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff76806ffb0 0x7ff768158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 0x7ff76814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff768168c10 con 0x7ff768071380 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff768168a90 con 0x7ff76806efb0 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76e400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff768168d90 con 0x7ff76806ffb0 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 0x7ff76814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff76806efb0 0x7ff7681f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 0x7ff76814d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35130/0 (socket says 172.21.15.16:35130) 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 -- 172.21.15.16:0/3057428478 learned_addr learned my addr 172.21.15.16:0/3057428478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:57.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff767e00640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff76806ffb0 0x7ff768158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 -- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff76806ffb0 msgr2=0x7ff768158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff76806ffb0 0x7ff768158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 -- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff76806efb0 msgr2=0x7ff7681f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff76806efb0 0x7ff7681f8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.913+0000 7ff76da00640 1 -- 172.21.15.16:0/3057428478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7681fc0b0 con 0x7ff768071380 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff76da00640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 0x7ff76814d4a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff76000b9e0 tx=0x7ff76000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760003b80 con 0x7ff768071380 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff760003e10 con 0x7ff768071380 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76000cb80 con 0x7ff768071380 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7681f9030 con 0x7ff768071380 2024-09-17T05:03:57.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7681f95f0 con 0x7ff768071380 2024-09-17T05:03:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff76806f3c0 con 0x7ff768071380 2024-09-17T05:03:57.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff760006d10 con 0x7ff768071380 2024-09-17T05:03:57.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff764c00640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff748073200 0x7ff748075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:57.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7ff760093ea0 con 0x7ff768071380 2024-09-17T05:03:57.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.917+0000 7ff76d000640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff748073200 0x7ff748075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.921+0000 7ff76d000640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff748073200 0x7ff748075710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00bb50 tx=0x7ff75c00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:57.921+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff76005e6d0 con 0x7ff768071380 2024-09-17T05:03:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7680b5990 con 0x7ff768071380 2024-09-17T05:03:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff764c00640 1 -- 172.21.15.16:0/3057428478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5230 (secure 0 0 0) 0x7ff760063100 con 0x7ff768071380 2024-09-17T05:03:58.135 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:03:58.135 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:47.404183+0000","modified":"2024-09-17T05:03:50.438145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/3720312961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3720312961},{"type":"v1","addr":"172.21.15.104:6836","nonce":3720312961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5775,"qdb_cluster":[5775]},"id":4}]} 2024-09-17T05:03:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-17T05:03:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff748073200 msgr2=0x7ff748075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff748073200 0x7ff748075710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00bb50 tx=0x7ff75c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 msgr2=0x7ff76814d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 0x7ff76814d4a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff76000b9e0 tx=0x7ff76000bea0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 shutdown_connections 2024-09-17T05:03:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff748073200 0x7ff748075710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff768071380 0x7ff76814d4a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff76806ffb0 0x7ff768158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 --2- 172.21.15.16:0/3057428478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff76806efb0 0x7ff7681f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 >> 172.21.15.16:0/3057428478 conn(0x7ff768073ca0 msgr2=0x7ff76814deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 shutdown_connections 2024-09-17T05:03:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.133+0000 7ff76e400640 1 -- 172.21.15.16:0/3057428478 wait complete. 2024-09-17T05:03:58.560 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:03:47.404183+0000', 'modified': '2024-09-17T05:03:50.438145+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5775}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5775': {'gid': 5775, 'name': 'd', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6836/3720312961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 3720312961}, {'type': 'v1', 'addr': '172.21.15.104:6836', 'nonce': 3720312961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5775, 'qdb_cluster': [5775]} 2024-09-17T05:03:58.560 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:03:58.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T05:03:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e9e00640 1 -- 172.21.15.16:0/1853885579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8004810 msgr2=0x7f98c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e9e00640 1 --2- 172.21.15.16:0/1853885579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8004810 0x7f98c8004c20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009b40 tx=0x7f98cc01d270 comp rx=0 tx=0).stop 2024-09-17T05:03:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e3400640 1 -- 172.21.15.16:0/1853885579 reap_dead start 2024-09-17T05:03:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e9e00640 1 -- 172.21.15.16:0/1853885579 shutdown_connections 2024-09-17T05:03:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e9e00640 1 -- 172.21.15.16:0/1853885579 >> 172.21.15.16:0/1853885579 conn(0x7f98e40b2250 msgr2=0x7f98e40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e9e00640 1 -- 172.21.15.16:0/1853885579 shutdown_connections 2024-09-17T05:03:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.717+0000 7f98e9e00640 1 -- 172.21.15.16:0/1853885579 wait complete. 2024-09-17T05:03:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 Processor -- start 2024-09-17T05:03:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 -- start start 2024-09-17T05:03:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98c8004810 0x7f98e4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 0x7f98e41e9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f98e41e9740 0x7f98e41f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40ce960 con 0x7f98e40b4780 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40ce7e0 con 0x7f98c8004810 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e9e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f98e40cf220 con 0x7f98e41e9740 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 0x7f98e41e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 0x7f98e41e9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35146/0 (socket says 172.21.15.16:35146) 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 -- 172.21.15.16:0/3649409648 learned_addr learned my addr 172.21.15.16:0/3649409648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e2a00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98c8004810 0x7f98e4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 -- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f98e41e9740 msgr2=0x7f98e41f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e3400640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f98e41e9740 0x7f98e41f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f98e41e9740 0x7f98e41f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 -- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98c8004810 msgr2=0x7f98e4154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98c8004810 0x7f98e4154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98da000640 1 -- 172.21.15.16:0/3649409648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e41e9b50 con 0x7f98e40b4780 2024-09-17T05:03:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.721+0000 7f98e3400640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f98e41e9740 0x7f98e41f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:03:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98da000640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 0x7f98e41e9200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0ae040 tx=0x7f98cc0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98dbe00640 1 -- 172.21.15.16:0/3649409648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0bb030 con 0x7f98e40b4780 2024-09-17T05:03:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98dbe00640 1 -- 172.21.15.16:0/3649409648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98cc004a50 con 0x7f98e40b4780 2024-09-17T05:03:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98dbe00640 1 -- 172.21.15.16:0/3649409648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc004e60 con 0x7f98e40b4780 2024-09-17T05:03:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e40b4550 con 0x7f98e40b4780 2024-09-17T05:03:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41f5d60 con 0x7f98e40b4780 2024-09-17T05:03:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.725+0000 7f98dbe00640 1 -- 172.21.15.16:0/3649409648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f98cc0be030 con 0x7f98e40b4780 2024-09-17T05:03:58.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98dbe00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f98c0073200 0x7f98c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98dbe00640 1 -- 172.21.15.16:0/3649409648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f98cc12fce0 con 0x7f98e40b4780 2024-09-17T05:03:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98e2a00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f98c0073200 0x7f98c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 Processor -- start 2024-09-17T05:03:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 -- start start 2024-09-17T05:03:58.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98e2a00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f98c0073200 0x7f98c0075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006fd0 tx=0x7f98d4006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4445ac460 0x55a444aed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 0x55a444aedbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a444985ca0 0x55a444aee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4449c2410 con 0x55a44499c950 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a4448ffb40 con 0x55a4445ac460 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55a4448eef70 con 0x55a444985ca0 2024-09-17T05:03:58.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 0x55a444aedbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 0x55a444aedbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35148/0 (socket says 172.21.15.16:35148) 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 -- 172.21.15.16:0/393425151 learned_addr learned my addr 172.21.15.16:0/393425151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98e2a00640 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4445ac460 0x55a444aed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98e3400640 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a444985ca0 0x55a444aee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 -- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a444985ca0 msgr2=0x55a444aee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a444985ca0 0x55a444aee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 -- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4445ac460 msgr2=0x55a444aed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4445ac460 0x55a444aed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 -- 172.21.15.16:0/393425151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4448b65b0 con 0x55a44499c950 2024-09-17T05:03:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98da000640 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 0x55a444aedbe0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f98cc01d750 tx=0x7f98cc01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ba02a3e3200be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98d3400640 1 -- 172.21.15.16:0/393425151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0c5040 con 0x55a44499c950 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98d3400640 1 -- 172.21.15.16:0/393425151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98cc0feeb0 con 0x55a44499c950 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.729+0000 7f98d3400640 1 -- 172.21.15.16:0/393425151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0fa870 con 0x55a44499c950 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- 172.21.15.16:0/393425151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 msgr2=0x55a444aedbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 0x55a444aedbe0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f98cc01d750 tx=0x7f98cc01bab0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- 172.21.15.16:0/393425151 shutdown_connections 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a444985ca0 0x55a444aee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a44499c950 0x55a444aedbe0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- 172.21.15.16:0/393425151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a4445ac460 0x55a444aed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- 172.21.15.16:0/393425151 >> 172.21.15.16:0/393425151 conn(0x55a444a72d90 msgr2=0x55a444aa2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- 172.21.15.16:0/393425151 shutdown_connections 2024-09-17T05:03:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- 172.21.15.16:0/393425151 wait complete. 2024-09-17T05:03:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- start start 2024-09-17T05:03:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 Processor -- start 2024-09-17T05:03:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444af4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444af4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444af53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4448895f0 con 0x55a4445ac460 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a4448f2d80 con 0x55a444985ca0 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55a444895520 con 0x55a44499c950 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98f14d9000 10 client.0 _ll_register_callbacks cb 0x55a444899830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444af4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444af4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35156/0 (socket says 172.21.15.16:35156) 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98e2a00640 1 -- 172.21.15.16:0/2497307314 learned_addr learned my addr 172.21.15.16:0/2497307314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98da000640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444af4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.733+0000 7f98e3400640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444af53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e2a00640 1 -- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 msgr2=0x55a444af53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e2a00640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444af53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e2a00640 1 -- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 msgr2=0x55a444af4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e2a00640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444af4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e2a00640 1 -- 172.21.15.16:0/2497307314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a44488d190 con 0x55a4445ac460 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e3400640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444af53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98e2a00640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444af4920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f98d4003880 tx=0x7f98d4001370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90cb6ed5f4c2a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 1 -- 172.21.15.16:0/2497307314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40138c0 con 0x55a4445ac460 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 1 -- 172.21.15.16:0/2497307314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f98d4019aa0 con 0x55a4445ac460 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a444887110 con 0x55a4445ac460 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 1 -- 172.21.15.16:0/2497307314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f98d4030050 con 0x55a4445ac460 2024-09-17T05:03:58.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 10 client.5925 fetch_fsmap learned FSMap version 35 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55a444973f10 con 0x55a4445ac460 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 1 -- 172.21.15.16:0/2497307314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a444887110 con 0x55a4445ac460 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 1 -- 172.21.15.16:0/2497307314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f98d4033020 con 0x55a4445ac460 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 10 client.5925 fetch_fsmap finished waiting for FSMap version 35 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 10 client.5925 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 20 client.5925 populate_metadata read hostname 'smithi016' 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a4445917d0 0x55a444afed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 4 client.5925 mds_command: new command op to 5775 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 --> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a444901a90 con 0x55a4445917d0 2024-09-17T05:03:58.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98da000640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a4445917d0 0x55a444afed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98da000640 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a4445917d0 0x55a444afed00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19cc9a1727feed69 server_cookie=c14eadf6a3c6c387 in_seq=0 out_seq=0 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 10 client.5925 ms_handle_connect on v2:172.21.15.104:6834/3720312961 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 1 -- 172.21.15.16:0/2497307314 <== mds.0 v2:172.21.15.104:6834/3720312961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a444901a90 con 0x55a4445917d0 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98c7e00640 10 client.5925 handle_command_reply: tid=0 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 client.5925 shutdown 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 msgr2=0x55a444af4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444af4920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f98d4003880 tx=0x7f98d4001370 comp rx=0 tx=0).stop 2024-09-17T05:03:58.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 shutdown_connections 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.737+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a4445917d0 0x55a444afed00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444af53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444af4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98c7e00640 0 client.5925 ms_handle_reset on v2:172.21.15.104:6834/3720312961 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2497307314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444af4920 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 >> 172.21.15.16:0/2497307314 conn(0x55a444a72d90 msgr2=0x55a444aa9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 shutdown_connections 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 1 -- 172.21.15.16:0/2497307314 wait complete. 2024-09-17T05:03:58.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.741+0000 7f98f14d9000 20 client.5925 trim_cache size 0 max 16384 2024-09-17T05:03:58.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.757+0000 7f98f14d9000 1 Processor -- start 2024-09-17T05:03:58.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.757+0000 7f98f14d9000 1 -- start start 2024-09-17T05:03:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a4445ac460 0x55a444b5f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 0x55a444b5fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a44499c950 0x55a444b600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4203140 con 0x55a444985ca0 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4202160 con 0x55a44499c950 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f98e4201520 con 0x55a4445ac460 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 0x55a444b5fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 0x55a444b5fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35162/0 (socket says 172.21.15.16:35162) 2024-09-17T05:03:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 -- 172.21.15.16:0/2256324381 learned_addr learned my addr 172.21.15.16:0/2256324381 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98e2a00640 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a4445ac460 0x55a444b5f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98e3400640 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a44499c950 0x55a444b600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 -- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a4445ac460 msgr2=0x55a444b5f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a4445ac460 0x55a444b5f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 -- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a44499c950 msgr2=0x55a444b600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a44499c950 0x55a444b600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 -- 172.21.15.16:0/2256324381 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a44494adb0 con 0x55a444985ca0 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98e3400640 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a44499c950 0x55a444b600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98da000640 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 0x55a444b5fb60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0fa310 tx=0x7f98cc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43675d6560acf9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98d2a00640 1 -- 172.21.15.16:0/2256324381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0c5040 con 0x55a444985ca0 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98d2a00640 1 -- 172.21.15.16:0/2256324381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98cc0feeb0 con 0x55a444985ca0 2024-09-17T05:03:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98d2a00640 1 -- 172.21.15.16:0/2256324381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc114040 con 0x55a444985ca0 2024-09-17T05:03:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- 172.21.15.16:0/2256324381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 msgr2=0x55a444b5fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 0x55a444b5fb60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0fa310 tx=0x7f98cc001e80 comp rx=0 tx=0).stop 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- 172.21.15.16:0/2256324381 shutdown_connections 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a44499c950 0x55a444b600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a444985ca0 0x55a444b5fb60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 --2- 172.21.15.16:0/2256324381 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a4445ac460 0x55a444b5f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- 172.21.15.16:0/2256324381 >> 172.21.15.16:0/2256324381 conn(0x55a444a72d90 msgr2=0x55a444aeb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- 172.21.15.16:0/2256324381 shutdown_connections 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.761+0000 7f98f14d9000 1 -- 172.21.15.16:0/2256324381 wait complete. 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- start start 2024-09-17T05:03:58.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 Processor -- start 2024-09-17T05:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444b63390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444b638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444b63e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4448fe910 con 0x55a4445ac460 2024-09-17T05:03:58.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a4448ffb40 con 0x55a444985ca0 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55a4448baee0 con 0x55a44499c950 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 10 client.0 _ll_register_callbacks cb 0x7f98d40062c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444b63390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444b63390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35166/0 (socket says 172.21.15.16:35166) 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 -- 172.21.15.16:0/3468611180 learned_addr learned my addr 172.21.15.16:0/3468611180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e3400640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444b63e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98da000640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444b638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 -- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 msgr2=0x55a444b63e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444b63e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 -- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 msgr2=0x55a444b638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444b638d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 -- 172.21.15.16:0/3468611180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4448b5ab0 con 0x55a4445ac460 2024-09-17T05:03:58.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98da000640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444b638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98e2a00640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444b63390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005fd0 tx=0x7f98d4014c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fefb715e5fd9df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98c7e00640 1 -- 172.21.15.16:0/3468611180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d40194e0 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98c7e00640 1 -- 172.21.15.16:0/3468611180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f98d4019680 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98cc0feeb0 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98c7e00640 1 -- 172.21.15.16:0/3468611180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f98d4044050 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 10 client.5931 fetch_fsmap learned FSMap version 35 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55a444894070 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98c7e00640 1 -- 172.21.15.16:0/3468611180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98cc0feeb0 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98c7e00640 1 -- 172.21.15.16:0/3468611180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f98d4047020 con 0x55a4445ac460 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 10 client.5931 fetch_fsmap finished waiting for FSMap version 35 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 10 client.5931 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T05:03:58.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 20 client.5931 populate_metadata read hostname 'smithi016' 2024-09-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a44457f830 0x55a444b1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 4 client.5931 mds_command: new command op to 5775 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.765+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 --> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] -- command(tid 0: {"prefix": "status"}) -- 0x55a4448fd540 con 0x55a44457f830 2024-09-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98da000640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a44457f830 0x55a444b1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98da000640 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a44457f830 0x55a444b1fa40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecee9f22186c2aff server_cookie=796f5a30155866ae in_seq=0 out_seq=0 2024-09-17T05:03:58.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98c7e00640 10 client.5931 ms_handle_connect on v2:172.21.15.104:6834/3720312961 2024-09-17T05:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98c7e00640 1 -- 172.21.15.16:0/3468611180 <== mds.0 v2:172.21.15.104:6834/3720312961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55a4448fd540 con 0x55a44457f830 2024-09-17T05:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98c7e00640 10 client.5931 handle_command_reply: tid=0 2024-09-17T05:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 client.5931 shutdown 2024-09-17T05:03:58.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 msgr2=0x55a444b63390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444b63390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f98d4005fd0 tx=0x7f98d4014c80 comp rx=0 tx=0).stop 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 shutdown_connections 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:6834/3720312961,v1:172.21.15.104:6836/3720312961] conn(0x55a44457f830 0x55a444b1fa40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55a44499c950 0x55a444b63e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a444985ca0 0x55a444b638d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 --2- 172.21.15.16:0/3468611180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4445ac460 0x55a444b63390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98c7e00640 0 client.5931 ms_handle_reset on v2:172.21.15.104:6834/3720312961 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 >> 172.21.15.16:0/3468611180 conn(0x55a444a72d90 msgr2=0x55a444aebfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 shutdown_connections 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 1 -- 172.21.15.16:0/3468611180 wait complete. 2024-09-17T05:03:58.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98f14d9000 20 client.5931 trim_cache size 0 max 16384 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5775, 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.773906364, 2024-09-17T05:03:58.776 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 35, 2024-09-17T05:03:58.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 64, 2024-09-17T05:03:58.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 64, 2024-09-17T05:03:58.777 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.166049477 2024-09-17T05:03:58.777 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f98c0073200 msgr2=0x7f98c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f98c0073200 0x7f98c0075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98d4006fd0 tx=0x7f98d4006b20 comp rx=0 tx=0).stop 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 msgr2=0x7f98e41e9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 0x7f98e41e9200 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0ae040 tx=0x7f98cc0050b0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 shutdown_connections 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f98c0073200 0x7f98c0075710 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f98e41e9740 0x7f98e41f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e40b4780 0x7f98e41e9200 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 --2- 172.21.15.16:0/3649409648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98c8004810 0x7f98e4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 >> 172.21.15.16:0/3649409648 conn(0x7f98e40b2250 msgr2=0x7f98e41598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 shutdown_connections 2024-09-17T05:03:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.769+0000 7f98e9e00640 1 -- 172.21.15.16:0/3649409648 wait complete. 2024-09-17T05:03:58.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:03:58.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 -- 172.21.15.16:0/248704677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b20c0 msgr2=0x7fdaec160530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 --2- 172.21.15.16:0/248704677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b20c0 0x7fdaec160530 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdaec073b00 tx=0x7fdad801c800 comp rx=0 tx=0).stop 2024-09-17T05:03:58.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 -- 172.21.15.16:0/248704677 shutdown_connections 2024-09-17T05:03:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 --2- 172.21.15.16:0/248704677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b20c0 0x7fdaec160530 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 --2- 172.21.15.16:0/248704677 >> v2:172.21.15.104:3300/0 conn(0x7fdaec0b1cb0 0x7fdaec15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 -- 172.21.15.16:0/248704677 >> 172.21.15.16:0/248704677 conn(0x7fdaec0b4600 msgr2=0x7fdaec0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 -- 172.21.15.16:0/248704677 shutdown_connections 2024-09-17T05:03:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 -- 172.21.15.16:0/248704677 wait complete. 2024-09-17T05:03:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 Processor -- start 2024-09-17T05:03:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.949+0000 7fdaf4200640 1 -- start start 2024-09-17T05:03:58.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdaec0b1cb0 0x7fdaec144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaec0b20c0 0x7fdaec1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 0x7fdaec15f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec169840 con 0x7fdaec0b6a90 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec1696c0 con 0x7fdaec0b20c0 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdaec1699c0 con 0x7fdaec0b1cb0 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 0x7fdaec15f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 0x7fdaec15f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35188/0 (socket says 172.21.15.16:35188) 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 -- 172.21.15.16:0/133927266 learned_addr learned my addr 172.21.15.16:0/133927266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf1800640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdaec0b1cb0 0x7fdaec144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf0e00640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaec0b20c0 0x7fdaec1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 -- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdaec0b1cb0 msgr2=0x7fdaec144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdaec0b1cb0 0x7fdaec144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 -- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaec0b20c0 msgr2=0x7fdaec1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaec0b20c0 0x7fdaec1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:58.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 -- 172.21.15.16:0/133927266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec145750 con 0x7fdaec0b6a90 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf2200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 0x7fdaec15f2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c730 tx=0x7fdae400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4003b80 con 0x7fdaec0b6a90 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdae4003d20 con 0x7fdaec0b6a90 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec145a30 con 0x7fdaec0b6a90 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae400aa50 con 0x7fdaec0b6a90 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec145ff0 con 0x7fdaec0b6a90 2024-09-17T05:03:58.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.953+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec0b6ea0 con 0x7fdaec0b6a90 2024-09-17T05:03:58.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.957+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdae400abf0 con 0x7fdaec0b6a90 2024-09-17T05:03:58.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.957+0000 7fdae9600640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdacc073230 0x7fdacc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:03:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.957+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fdae40949d0 con 0x7fdaec0b6a90 2024-09-17T05:03:58.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.957+0000 7fdaf1800640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdacc073230 0x7fdacc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:03:58.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.957+0000 7fdaf1800640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdacc073230 0x7fdacc075740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009b40 tx=0x7fdae00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:03:58.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:58.961+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdaec0cee60 con 0x7fdaec0b6a90 2024-09-17T05:03:59.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdaec069300 con 0x7fdaec0b6a90 2024-09-17T05:03:59.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdae9600640 1 -- 172.21.15.16:0/133927266 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7fdae405f380 con 0x7fdaec0b6a90 2024-09-17T05:03:59.175 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:03:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdacc073230 msgr2=0x7fdacc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdacc073230 0x7fdacc075740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009b40 tx=0x7fdae00091e0 comp rx=0 tx=0).stop 2024-09-17T05:03:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 msgr2=0x7fdaec15f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:03:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 0x7fdaec15f2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdae400c730 tx=0x7fdae400cbf0 comp rx=0 tx=0).stop 2024-09-17T05:03:59.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 shutdown_connections 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdacc073230 0x7fdacc075740 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaec0b6a90 0x7fdaec15f2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdaec0b20c0 0x7fdaec1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 --2- 172.21.15.16:0/133927266 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdaec0b1cb0 0x7fdaec144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 >> 172.21.15.16:0/133927266 conn(0x7fdaec0b4600 msgr2=0x7fdaec146620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 shutdown_connections 2024-09-17T05:03:59.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59.173+0000 7fdaf4200640 1 -- 172.21.15.16:0/133927266 wait complete. 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:03:59.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:03:59.198 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:03:59.198 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:03:59.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.198 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:03:59.207 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:03:59.207 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:03:59.207 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:03:59.207 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.207 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42254sec preferred_lft 42254sec 2024-09-17T05:03:59.208 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:03:59.209 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:03:59.210 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:03:59.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.211 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:03:59.271 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:03:59.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.273 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:03:59.331 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:03:59.331 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:03:59.331 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:03:59.331 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:03:59.331 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:03:59.331 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:03:59.331 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:03:59.332 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:03:59.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:03:59.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:03:59.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:03:59.332 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:03:59.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:03:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:59.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.469 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:03:59.479 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:03:59.479 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:03:59.479 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:03:59.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:03:59.541 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:03:59.541 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:59.597 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:03:59.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:03:59.598 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:59.649 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:03:59.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:59.750 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:59.750 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:03:59.750 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:03:59.768 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:59.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:03:59.769 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:03:59.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:03:59.769 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:03:59.779 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:03:59.782 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:03:59.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:03:59.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:03:59.803 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:03:59.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:03:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:03:59.915 INFO:teuthology.orchestra.run.smithi016.stdout:client5369 2024-09-17T05:03:59.916 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:03:59.916 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:03:59.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 -- 172.21.15.16:0/113225063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91700b2f50 msgr2=0x7f917015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 --2- 172.21.15.16:0/113225063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91700b2f50 0x7f917015bb60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9160009840 tx=0x7f916001c790 comp rx=0 tx=0).stop 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 -- 172.21.15.16:0/113225063 shutdown_connections 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 --2- 172.21.15.16:0/113225063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91700b2f50 0x7f917015bb60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 -- 172.21.15.16:0/113225063 >> 172.21.15.16:0/113225063 conn(0x7f917006f900 msgr2=0x7f91700b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 -- 172.21.15.16:0/113225063 shutdown_connections 2024-09-17T05:04:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.129+0000 7f9175000640 1 -- 172.21.15.16:0/113225063 wait complete. 2024-09-17T05:04:00.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 Processor -- start 2024-09-17T05:04:00.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 -- start start 2024-09-17T05:04:00.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 0x7f9170142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:00.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91700b2f50 0x7f9170159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917015c0a0 0x7f917015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9170169c90 con 0x7f917015c0a0 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9170169b10 con 0x7f9170073240 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f9175000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9170169e10 con 0x7f91700b2f50 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.133+0000 7f916e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917015c0a0 0x7f917015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f916de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 0x7f9170142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91700b2f50 0x7f9170159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f916de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 0x7f9170142950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:34350/0 (socket says 172.21.15.16:34350) 2024-09-17T05:04:00.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f916de00640 1 -- 172.21.15.16:0/3597112431 learned_addr learned my addr 172.21.15.16:0/3597112431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:00.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 -- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 msgr2=0x7f9170142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:00.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 0x7f9170142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 -- 172.21.15.16:0/3597112431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917015c0a0 msgr2=0x7f917015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:00.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917015c0a0 0x7f917015a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 -- 172.21.15.16:0/3597112431 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9170165da0 con 0x7f91700b2f50 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f916de00640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 0x7f9170142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f916e800640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917015c0a0 0x7f917015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9167e00640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91700b2f50 0x7f9170159de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f915800c420 tx=0x7f9158001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158017070 con 0x7f91700b2f50 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91580053e0 con 0x7f91700b2f50 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9170166080 con 0x7f91700b2f50 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158003860 con 0x7f91700b2f50 2024-09-17T05:04:00.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9170166640 con 0x7f91700b2f50 2024-09-17T05:04:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9170165b10 con 0x7f91700b2f50 2024-09-17T05:04:00.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.137+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f915801a070 con 0x7f91700b2f50 2024-09-17T05:04:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.141+0000 7f9166000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f914c073200 0x7f914c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.141+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f9158093aa0 con 0x7f91700b2f50 2024-09-17T05:04:00.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.141+0000 7f916de00640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f914c073200 0x7f914c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:00.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.141+0000 7f916de00640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f914c073200 0x7f914c075710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f91600193a0 tx=0x7f916001afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:00.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.145+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9158021020 con 0x7f91700b2f50 2024-09-17T05:04:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f91700b1f80 con 0x7f91700b2f50 2024-09-17T05:04:00.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9166000640 1 -- 172.21.15.16:0/3597112431 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f915805e410 con 0x7f91700b2f50 2024-09-17T05:04:00.359 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:04:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f914c073200 msgr2=0x7f914c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f914c073200 0x7f914c075710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f91600193a0 tx=0x7f916001afd0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91700b2f50 msgr2=0x7f9170159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91700b2f50 0x7f9170159de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f915800c420 tx=0x7f9158001e80 comp rx=0 tx=0).stop 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 shutdown_connections 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f914c073200 0x7f914c075710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917015c0a0 0x7f917015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91700b2f50 0x7f9170159de0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 --2- 172.21.15.16:0/3597112431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9170073240 0x7f9170142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 >> 172.21.15.16:0/3597112431 conn(0x7f917006f900 msgr2=0x7f91700cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 shutdown_connections 2024-09-17T05:04:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:00.357+0000 7f9175000640 1 -- 172.21.15.16:0/3597112431 wait complete. 2024-09-17T05:04:00.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:04:00.370 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:04:00.383 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:04:00.383 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:04:00.384 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:04:00.384 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:04:00.384 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:00.384 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:04:00.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:04:00.384 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:04:00.385 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:04:00.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:00.385 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:00.427 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42247sec preferred_lft 42247sec 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:00.428 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:04:00.429 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:04:00.432 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:04:00.433 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:00.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:00.433 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:04:00.497 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:04:00.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:00.499 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:04:00.596 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:04:00.596 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:04:00.596 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:04:00.596 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:04:00.597 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:04:00.597 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:04:00.597 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:04:00.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:04:00.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:04:00.598 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:04:00.598 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:04:00.687 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:04:00.728 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:00.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:00.734 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:04:00.784 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:04:00.784 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:04:00.785 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:04:00.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:00.785 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:04:00.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:04:00.848 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:00.905 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:04:00.905 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:04:00.906 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:00.957 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:04:00.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:00.958 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:01.065 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:01.065 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:04:01.065 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:04:01.084 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:01.084 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:04:01.085 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:04:01.085 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:04:01.085 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:04:01.094 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:04:01.095 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:04:01.097 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:04:01.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:01.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:01.117 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:04:01.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:04:01.214 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:01.220 INFO:teuthology.orchestra.run.smithi104.stdout:client5949 2024-09-17T05:04:01.221 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:01.221 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:01.230 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:04:01.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:01.325 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:01.423 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:04:01.423 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:01.423 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:01.423 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:01.434 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:01.434 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:01.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 -- 172.21.15.16:0/3861353467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd015d680 msgr2=0x7f1dd0160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/3861353467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd015d680 0x7f1dd0160130 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009900 tx=0x7f1db801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 -- 172.21.15.16:0/3861353467 shutdown_connections 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/3861353467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd015d680 0x7f1dd0160130 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/3861353467 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1dd00733e0 0x7f1dd015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 -- 172.21.15.16:0/3861353467 >> 172.21.15.16:0/3861353467 conn(0x7f1dd00b2250 msgr2=0x7f1dd00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 -- 172.21.15.16:0/3861353467 shutdown_connections 2024-09-17T05:04:01.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.697+0000 7f1dd7a00640 1 -- 172.21.15.16:0/3861353467 wait complete. 2024-09-17T05:04:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 Processor -- start 2024-09-17T05:04:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 -- start start 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 0x7f1dd00b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1dd015d680 0x7f1dd0159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd0160670 0x7f1dd00b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd016a2d0 con 0x7f1dd00733e0 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd00cae90 con 0x7f1dd0160670 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd7a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd016a450 con 0x7f1dd015d680 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 0x7f1dd00b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:01.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 0x7f1dd00b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35242/0 (socket says 172.21.15.16:35242) 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dd5000640 1 -- 172.21.15.16:0/1018293241 learned_addr learned my addr 172.21.15.16:0/1018293241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.701+0000 7f1dc7e00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1dd015d680 0x7f1dd0159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd0160670 0x7f1dd00b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5000640 1 -- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1dd015d680 msgr2=0x7f1dd0159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5000640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1dd015d680 0x7f1dd0159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5000640 1 -- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd0160670 msgr2=0x7f1dd00b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5000640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd0160670 0x7f1dd00b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5000640 1 -- 172.21.15.16:0/1018293241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd01f79a0 con 0x7f1dd00733e0 2024-09-17T05:04:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd5000640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 0x7f1dd00b4350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e7c0 tx=0x7f1dc000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000ccc0 con 0x7f1dd00733e0 2024-09-17T05:04:01.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dc000ce80 con 0x7f1dd00733e0 2024-09-17T05:04:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0005280 con 0x7f1dd00733e0 2024-09-17T05:04:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd01f7100 con 0x7f1dd00733e0 2024-09-17T05:04:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01f76c0 con 0x7f1dd00733e0 2024-09-17T05:04:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd015d140 con 0x7f1dd00733e0 2024-09-17T05:04:01.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.705+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1dc0005420 con 0x7f1dd00733e0 2024-09-17T05:04:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.709+0000 7f1dcd600640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1dac073200 0x7f1dac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.709+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f1dc0094c60 con 0x7f1dd00733e0 2024-09-17T05:04:01.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.709+0000 7f1dc7e00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1dac073200 0x7f1dac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:01.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.709+0000 7f1dc7e00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1dac073200 0x7f1dac075710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1db80af000 tx=0x7f1db801b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.713+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1dc005f490 con 0x7f1dd00733e0 2024-09-17T05:04:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1dd015fcb0 con 0x7f1dd00733e0 2024-09-17T05:04:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dcd600640 1 -- 172.21.15.16:0/1018293241 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) ==== 80+0+0 (secure 0 0 0) 0x7f1dc0068020 con 0x7f1dd00733e0 2024-09-17T05:04:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:04:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1dac073200 msgr2=0x7f1dac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1dac073200 0x7f1dac075710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1db80af000 tx=0x7f1db801b0f0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 msgr2=0x7f1dd00b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 0x7f1dd00b4350 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e7c0 tx=0x7f1dc000ec80 comp rx=0 tx=0).stop 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 shutdown_connections 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1dac073200 0x7f1dac075710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd0160670 0x7f1dd00b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1dd015d680 0x7f1dd0159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 --2- 172.21.15.16:0/1018293241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dd00733e0 0x7f1dd00b4350 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 >> 172.21.15.16:0/1018293241 conn(0x7f1dd00b2250 msgr2=0x7f1dd01561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 shutdown_connections 2024-09-17T05:04:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:01.901+0000 7f1dd7a00640 1 -- 172.21.15.16:0/1018293241 wait complete. 2024-09-17T05:04:01.926 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:04:01.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:01.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:04:01.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:04:01.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:01.954 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:04:02.006 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:02.006 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:02.006 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:02.015 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:02.015 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:02.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:04:02.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5d5e00640 1 -- 172.21.15.16:0/1856347502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 msgr2=0x7fb5b4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/1856347502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5b4005230 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009e90 tx=0x7fb5c801d5d0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5cf400640 1 -- 172.21.15.16:0/1856347502 reap_dead start 2024-09-17T05:04:02.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5d5e00640 1 -- 172.21.15.16:0/1856347502 shutdown_connections 2024-09-17T05:04:02.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5d5e00640 1 -- 172.21.15.16:0/1856347502 >> 172.21.15.16:0/1856347502 conn(0x7fb5d00b2250 msgr2=0x7fb5d00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:02.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5d5e00640 1 -- 172.21.15.16:0/1856347502 shutdown_connections 2024-09-17T05:04:02.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.221+0000 7fb5d5e00640 1 -- 172.21.15.16:0/1856347502 wait complete. 2024-09-17T05:04:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 Processor -- start 2024-09-17T05:04:02.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 -- start start 2024-09-17T05:04:02.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5d00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5d0151e00 0x7fb5d00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5d0152210 0x7fb5d00b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d00cf070 con 0x7fb5b4004da0 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d00ceef0 con 0x7fb5d0151e00 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5d5e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d00cf1f0 con 0x7fb5d0152210 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5d00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5cea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5d00b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35264/0 (socket says 172.21.15.16:35264) 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5cea00640 1 -- 172.21.15.16:0/2239818024 learned_addr learned my addr 172.21.15.16:0/2239818024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5cf400640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5d0152210 0x7fb5d00b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.225+0000 7fb5c7e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5d0151e00 0x7fb5d00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5cea00640 1 -- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5d0152210 msgr2=0x7fb5d00b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5cea00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5d0152210 0x7fb5d00b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5cea00640 1 -- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5d0151e00 msgr2=0x7fb5d00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5cea00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5d0151e00 0x7fb5d00b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5cea00640 1 -- 172.21.15.16:0/2239818024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d00c8930 con 0x7fb5b4004da0 2024-09-17T05:04:02.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5cea00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5d00b3880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0012610 tx=0x7fb5c0012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0013920 con 0x7fb5b4004da0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5c0013ac0 con 0x7fb5b4004da0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d0152970 con 0x7fb5b4004da0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0010480 con 0x7fb5b4004da0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d00c7b70 con 0x7fb5b4004da0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d015efe0 con 0x7fb5b4004da0 2024-09-17T05:04:02.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.229+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb5c0010620 con 0x7fb5b4004da0 2024-09-17T05:04:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.233+0000 7fb5c7400640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb5ac073230 0x7fb5ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.233+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fb5c0098180 con 0x7fb5b4004da0 2024-09-17T05:04:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.233+0000 7fb5c7e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb5ac073230 0x7fb5ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.233+0000 7fb5c7e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb5ac073230 0x7fb5ac075740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb5d01eb200 tx=0x7fb5b8006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:02.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.237+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb5c00629b0 con 0x7fb5b4004da0 2024-09-17T05:04:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.421+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5d0074350 con 0x7fb5b4004da0 2024-09-17T05:04:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.421+0000 7fb5c7400640 1 -- 172.21.15.16:0/2239818024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v64) ==== 80+0+0 (secure 0 0 0) 0x7fb5c00673e0 con 0x7fb5b4004da0 2024-09-17T05:04:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb5ac073230 msgr2=0x7fb5ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb5ac073230 0x7fb5ac075740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb5d01eb200 tx=0x7fb5b8006b20 comp rx=0 tx=0).stop 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 msgr2=0x7fb5d00b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5d00b3880 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0012610 tx=0x7fb5c0012a10 comp rx=0 tx=0).stop 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 shutdown_connections 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb5ac073230 0x7fb5ac075740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5d0152210 0x7fb5d00b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5d0151e00 0x7fb5d00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 --2- 172.21.15.16:0/2239818024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b4004da0 0x7fb5d00b3880 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 >> 172.21.15.16:0/2239818024 conn(0x7fb5d00b2250 msgr2=0x7fb5d01eb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 shutdown_connections 2024-09-17T05:04:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.425+0000 7fb5d5e00640 1 -- 172.21.15.16:0/2239818024 wait complete. 2024-09-17T05:04:02.447 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:04:02.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:02.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:02.474 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:04:02.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:02.474 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:04:02.525 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:02.525 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:02.526 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:02.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:04:02.535 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:02.535 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:02.535 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:02.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:04:02.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 -- 172.21.15.16:0/1022293425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f500b2f50 msgr2=0x7f0f5015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 --2- 172.21.15.16:0/1022293425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f500b2f50 0x7f0f5015bb60 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0f5015cc70 tx=0x7f0f4001c830 comp rx=0 tx=0).stop 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 -- 172.21.15.16:0/1022293425 shutdown_connections 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 --2- 172.21.15.16:0/1022293425 >> v2:172.21.15.104:3300/0 conn(0x7f0f5015f150 0x7f0f5015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 --2- 172.21.15.16:0/1022293425 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f5015c0a0 0x7f0f5015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 --2- 172.21.15.16:0/1022293425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f500b2f50 0x7f0f5015bb60 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 -- 172.21.15.16:0/1022293425 >> 172.21.15.16:0/1022293425 conn(0x7f0f5006f900 msgr2=0x7f0f500b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 -- 172.21.15.16:0/1022293425 shutdown_connections 2024-09-17T05:04:02.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 -- 172.21.15.16:0/1022293425 wait complete. 2024-09-17T05:04:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 Processor -- start 2024-09-17T05:04:02.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.737+0000 7f0f56800640 1 -- start start 2024-09-17T05:04:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f500b2f50 0x7f0f5015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 0x7f0f5014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f5015f150 0x7f0f500cf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5006e430 con 0x7f0f5015c0a0 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5006e2b0 con 0x7f0f5015f150 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5006e5b0 con 0x7f0f500b2f50 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 0x7f0f5014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 0x7f0f5014d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35278/0 (socket says 172.21.15.16:35278) 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 -- 172.21.15.16:0/1710127423 learned_addr learned my addr 172.21.15.16:0/1710127423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f4fe00640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f5015f150 0x7f0f500cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f4f400640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f500b2f50 0x7f0f5015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 -- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f500b2f50 msgr2=0x7f0f5015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f500b2f50 0x7f0f5015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 -- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f5015f150 msgr2=0x7f0f500cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f5015f150 0x7f0f500cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 -- 172.21.15.16:0/1710127423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5015ea40 con 0x7f0f5015c0a0 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47e00640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 0x7f0f5014d0d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e6b0 tx=0x7f0f3800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38005220 con 0x7f0f5015c0a0 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f380053c0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800c850 con 0x7f0f5015c0a0 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5015cef0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5015d4b0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.741+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5015c4b0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.745+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0f3800c9f0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.745+0000 7f0f47400640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f2c073200 0x7f0f2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.745+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f0f38094cc0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.745+0000 7f0f4f400640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f2c073200 0x7f0f2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.745+0000 7f0f4f400640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f2c073200 0x7f0f2c075710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0f400af000 tx=0x7f0f400a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.749+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0f3809b080 con 0x7f0f5015c0a0 2024-09-17T05:04:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f500ce2c0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f47400640 1 -- 172.21.15.16:0/1710127423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5230 (secure 0 0 0) 0x7f0f3805f4f0 con 0x7f0f5015c0a0 2024-09-17T05:04:02.958 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:02.959 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:47.404183+0000","modified":"2024-09-17T05:03:50.438145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/3720312961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3720312961},{"type":"v1","addr":"172.21.15.104:6836","nonce":3720312961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5775,"qdb_cluster":[5775]},"id":4}]} 2024-09-17T05:04:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f2c073200 msgr2=0x7f0f2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f2c073200 0x7f0f2c075710 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0f400af000 tx=0x7f0f400a7040 comp rx=0 tx=0).stop 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 msgr2=0x7f0f5014d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 0x7f0f5014d0d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e6b0 tx=0x7f0f3800eb70 comp rx=0 tx=0).stop 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 shutdown_connections 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0f2c073200 0x7f0f2c075710 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f5015f150 0x7f0f500cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f5015c0a0 0x7f0f5014d0d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 --2- 172.21.15.16:0/1710127423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0f500b2f50 0x7f0f5015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:02.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 >> 172.21.15.16:0/1710127423 conn(0x7f0f5006f900 msgr2=0x7f0f500ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 shutdown_connections 2024-09-17T05:04:02.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:02.957+0000 7f0f56800640 1 -- 172.21.15.16:0/1710127423 wait complete. 2024-09-17T05:04:02.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 -- 172.21.15.16:0/3349137306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728160aa0 msgr2=0x7f87280b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 --2- 172.21.15.16:0/3349137306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728160aa0 0x7f87280b20a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8710009840 tx=0x7f871001c700 comp rx=0 tx=0).stop 2024-09-17T05:04:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 -- 172.21.15.16:0/3349137306 shutdown_connections 2024-09-17T05:04:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 --2- 172.21.15.16:0/3349137306 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f87280b25e0 0x7f87280b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 --2- 172.21.15.16:0/3349137306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8728160aa0 0x7f87280b20a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 -- 172.21.15.16:0/3349137306 >> 172.21.15.16:0/3349137306 conn(0x7f87280cf380 msgr2=0x7f87280d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 -- 172.21.15.16:0/3349137306 shutdown_connections 2024-09-17T05:04:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 -- 172.21.15.16:0/3349137306 wait complete. 2024-09-17T05:04:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 Processor -- start 2024-09-17T05:04:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.141+0000 7f872d000640 1 -- start start 2024-09-17T05:04:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87280b25e0 0x7f8728149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 0x7f8728071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f872814b7a0 0x7f8728071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f872816a170 con 0x7f872815c6f0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8728169ff0 con 0x7f87280b25e0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f872816a2f0 con 0x7f872814b7a0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 0x7f8728071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 0x7f8728071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35288/0 (socket says 172.21.15.16:35288) 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 -- 172.21.15.16:0/1330328259 learned_addr learned my addr 172.21.15.16:0/1330328259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f8726800640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f872814b7a0 0x7f8728071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f8725e00640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87280b25e0 0x7f8728149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 -- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f872814b7a0 msgr2=0x7f8728071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f872814b7a0 0x7f8728071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 -- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87280b25e0 msgr2=0x7f8728149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87280b25e0 0x7f8728149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 -- 172.21.15.16:0/1330328259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87280b4b60 con 0x7f872815c6f0 2024-09-17T05:04:03.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f8725e00640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87280b25e0 0x7f8728149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871d400640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 0x7f8728071330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f871001cbe0 tx=0x7f87100a8430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100be070 con 0x7f872815c6f0 2024-09-17T05:04:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87100a85a0 con 0x7f872815c6f0 2024-09-17T05:04:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872814b350 con 0x7f872815c6f0 2024-09-17T05:04:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100a8cd0 con 0x7f872815c6f0 2024-09-17T05:04:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8728159ee0 con 0x7f872815c6f0 2024-09-17T05:04:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872814b0c0 con 0x7f872815c6f0 2024-09-17T05:04:03.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.145+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f87100c1070 con 0x7f872815c6f0 2024-09-17T05:04:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.149+0000 7f871ea00640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8704073200 0x7f8704075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.149+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f871012e430 con 0x7f872815c6f0 2024-09-17T05:04:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.149+0000 7f8725e00640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8704073200 0x7f8704075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.149+0000 7f8725e00640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8704073200 0x7f8704075710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8718000f70 tx=0x7f8718009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.153+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f87100ac030 con 0x7f872815c6f0 2024-09-17T05:04:03.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.365+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87280b29f0 con 0x7f872815c6f0 2024-09-17T05:04:03.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.365+0000 7f871ea00640 1 -- 172.21.15.16:0/1330328259 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5230 (secure 0 0 0) 0x7f8710016410 con 0x7f872815c6f0 2024-09-17T05:04:03.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:03.368 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:47.404183+0000","modified":"2024-09-17T05:03:50.438145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/3720312961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3720312961},{"type":"v1","addr":"172.21.15.104:6836","nonce":3720312961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5775,"qdb_cluster":[5775]},"id":4}]} 2024-09-17T05:04:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-17T05:04:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8704073200 msgr2=0x7f8704075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8704073200 0x7f8704075710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8718000f70 tx=0x7f8718009040 comp rx=0 tx=0).stop 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 msgr2=0x7f8728071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 0x7f8728071330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f871001cbe0 tx=0x7f87100a8430 comp rx=0 tx=0).stop 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 shutdown_connections 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8704073200 0x7f8704075710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f872814b7a0 0x7f8728071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f872815c6f0 0x7f8728071330 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 --2- 172.21.15.16:0/1330328259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87280b25e0 0x7f8728149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 >> 172.21.15.16:0/1330328259 conn(0x7f87280cf380 msgr2=0x7f87280d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 shutdown_connections 2024-09-17T05:04:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.369+0000 7f872d000640 1 -- 172.21.15.16:0/1330328259 wait complete. 2024-09-17T05:04:03.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.545+0000 7f9a1d000640 1 -- 172.21.15.16:0/1985325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180cc8c0 msgr2=0x7f9a1815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.545+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1985325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180cc8c0 0x7f9a1815cab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009840 tx=0x7f9a0801c740 comp rx=0 tx=0).stop 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.545+0000 7f9a1d000640 1 -- 172.21.15.16:0/1985325222 shutdown_connections 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.545+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1985325222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180cc8c0 0x7f9a1815cab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.545+0000 7f9a1d000640 1 -- 172.21.15.16:0/1985325222 >> 172.21.15.16:0/1985325222 conn(0x7f9a180d2270 msgr2=0x7f9a180d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 -- 172.21.15.16:0/1985325222 shutdown_connections 2024-09-17T05:04:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 -- 172.21.15.16:0/1985325222 wait complete. 2024-09-17T05:04:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 Processor -- start 2024-09-17T05:04:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 -- start start 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180c9440 0x7f9a181560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 0x7f9a181fc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9a180d1580 0x7f9a181fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1816af60 con 0x7f9a180cc8c0 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1816ade0 con 0x7f9a180c9440 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.549+0000 7f9a1d000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1816b0e0 con 0x7f9a180d1580 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 0x7f9a181fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 0x7f9a181fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35292/0 (socket says 172.21.15.16:35292) 2024-09-17T05:04:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 -- 172.21.15.16:0/1068596318 learned_addr learned my addr 172.21.15.16:0/1068596318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a15e00640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180c9440 0x7f9a181560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a16800640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9a180d1580 0x7f9a181fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 -- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9a180d1580 msgr2=0x7f9a181fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9a180d1580 0x7f9a181fc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 -- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180c9440 msgr2=0x7f9a181560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180c9440 0x7f9a181560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 -- 172.21.15.16:0/1068596318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1815e290 con 0x7f9a180cc8c0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0d400640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 0x7f9a181fc060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c7b0 tx=0x7f9a0000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00003b80 con 0x7f9a180cc8c0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a00003d20 con 0x7f9a180cc8c0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000aab0 con 0x7f9a180cc8c0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a181576c0 con 0x7f9a180cc8c0 2024-09-17T05:04:03.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a18167530 con 0x7f9a180cc8c0 2024-09-17T05:04:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1815cab0 con 0x7f9a180cc8c0 2024-09-17T05:04:03.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.553+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a0000ac50 con 0x7f9a180cc8c0 2024-09-17T05:04:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.557+0000 7f9a0ea00640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99f4073230 0x7f99f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.557+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f9a00094c70 con 0x7f9a180cc8c0 2024-09-17T05:04:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.557+0000 7f9a15e00640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99f4073230 0x7f99f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.557+0000 7f9a15e00640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99f4073230 0x7f99f4075740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a080af000 tx=0x7f9a080b2040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:03.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.557+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9a1815e290 con 0x7f9a180cc8c0 2024-09-17T05:04:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.737+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a180cce30 con 0x7f9a180cc8c0 2024-09-17T05:04:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.737+0000 7f9a0ea00640 1 -- 172.21.15.16:0/1068596318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16980 (secure 0 0 0) 0x7f9a0005f490 con 0x7f9a180cc8c0 2024-09-17T05:04:03.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:03.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:48.995211+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:45.146869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:46.365938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99f4073230 msgr2=0x7f99f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99f4073230 0x7f99f4075740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a080af000 tx=0x7f9a080b2040 comp rx=0 tx=0).stop 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 msgr2=0x7f9a181fc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 0x7f9a181fc060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c7b0 tx=0x7f9a0000cc70 comp rx=0 tx=0).stop 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 shutdown_connections 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f99f4073230 0x7f99f4075740 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9a180d1580 0x7f9a181fc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a180cc8c0 0x7f9a181fc060 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 --2- 172.21.15.16:0/1068596318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9a180c9440 0x7f9a181560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 >> 172.21.15.16:0/1068596318 conn(0x7f9a180d2270 msgr2=0x7f9a180cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 shutdown_connections 2024-09-17T05:04:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.741+0000 7f9a1d000640 1 -- 172.21.15.16:0/1068596318 wait complete. 2024-09-17T05:04:03.763 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:04:03.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 -- 172.21.15.16:0/566118790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3960073240 msgr2=0x7f3960073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 --2- 172.21.15.16:0/566118790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3960073240 0x7f3960073650 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3958009c80 tx=0x7f395801cd10 comp rx=0 tx=0).stop 2024-09-17T05:04:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 -- 172.21.15.16:0/566118790 shutdown_connections 2024-09-17T05:04:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 --2- 172.21.15.16:0/566118790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3960073240 0x7f3960073650 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 --2- 172.21.15.16:0/566118790 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39600b2f50 0x7f396015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 -- 172.21.15.16:0/566118790 >> 172.21.15.16:0/566118790 conn(0x7f396006f900 msgr2=0x7f39600b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 -- 172.21.15.16:0/566118790 shutdown_connections 2024-09-17T05:04:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 -- 172.21.15.16:0/566118790 wait complete. 2024-09-17T05:04:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 Processor -- start 2024-09-17T05:04:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.921+0000 7f3967c00640 1 -- start start 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3960073240 0x7f396015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39600b2f50 0x7f3960150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396015c0a0 0x7f3960150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3960168e00 con 0x7f396015c0a0 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3960168c80 con 0x7f3960073240 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3960168f80 con 0x7f39600b2f50 2024-09-17T05:04:03.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3960073240 0x7f396015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396015c0a0 0x7f3960150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3960073240 0x7f396015a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:34450/0 (socket says 172.21.15.16:34450) 2024-09-17T05:04:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f395fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39600b2f50 0x7f3960150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965200640 1 -- 172.21.15.16:0/2148589250 learned_addr learned my addr 172.21.15.16:0/2148589250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:03.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 -- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39600b2f50 msgr2=0x7f3960150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39600b2f50 0x7f3960150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 -- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3960073240 msgr2=0x7f396015a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3960073240 0x7f396015a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 -- 172.21.15.16:0/2148589250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39600b1520 con 0x7f396015c0a0 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3965c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396015c0a0 0x7f3960150fa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f39580ac000 tx=0x7f3958009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580bc070 con 0x7f396015c0a0 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39580ad8e0 con 0x7f396015c0a0 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580b2040 con 0x7f396015c0a0 2024-09-17T05:04:03.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39601fd1e0 con 0x7f396015c0a0 2024-09-17T05:04:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601fcc20 con 0x7f396015c0a0 2024-09-17T05:04:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.925+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3960073650 con 0x7f396015c0a0 2024-09-17T05:04:03.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.929+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f39580adcf0 con 0x7f396015c0a0 2024-09-17T05:04:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.929+0000 7f395cc00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3940073230 0x7f3940075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.929+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f395812e0b0 con 0x7f396015c0a0 2024-09-17T05:04:03.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.929+0000 7f3965200640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3940073230 0x7f3940075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:03.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.929+0000 7f3965200640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3940073230 0x7f3940075740 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3954005fd0 tx=0x7f395400b410 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:03.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:03.933+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39580f2db0 con 0x7f396015c0a0 2024-09-17T05:04:04.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.149+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39600b1dd0 con 0x7f396015c0a0 2024-09-17T05:04:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.149+0000 7f395cc00640 1 -- 172.21.15.16:0/2148589250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5230 (secure 0 0 0) 0x7f3958016990 con 0x7f396015c0a0 2024-09-17T05:04:04.154 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:04.154 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:47.404183+0000","modified":"2024-09-17T05:03:50.438145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/3720312961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3720312961},{"type":"v1","addr":"172.21.15.104:6836","nonce":3720312961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5775,"qdb_cluster":[5775]},"id":4}]} 2024-09-17T05:04:04.154 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-17T05:04:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3940073230 msgr2=0x7f3940075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3940073230 0x7f3940075740 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3954005fd0 tx=0x7f395400b410 comp rx=0 tx=0).stop 2024-09-17T05:04:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396015c0a0 msgr2=0x7f3960150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396015c0a0 0x7f3960150fa0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f39580ac000 tx=0x7f3958009350 comp rx=0 tx=0).stop 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 shutdown_connections 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3940073230 0x7f3940075740 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396015c0a0 0x7f3960150fa0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39600b2f50 0x7f3960150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 --2- 172.21.15.16:0/2148589250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3960073240 0x7f396015a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 >> 172.21.15.16:0/2148589250 conn(0x7f396006f900 msgr2=0x7f396015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 shutdown_connections 2024-09-17T05:04:04.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.153+0000 7f3967c00640 1 -- 172.21.15.16:0/2148589250 wait complete. 2024-09-17T05:04:04.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 -- 172.21.15.16:0/675511817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd006ebb0 msgr2=0x7f6cd00b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 --2- 172.21.15.16:0/675511817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd006ebb0 0x7f6cd00b5a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0073690 tx=0x7f6cc801caf0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 -- 172.21.15.16:0/675511817 shutdown_connections 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 --2- 172.21.15.16:0/675511817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd006ebb0 0x7f6cd00b5a20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 --2- 172.21.15.16:0/675511817 >> v2:172.21.15.104:3300/0 conn(0x7f6cd0152540 0x7f6cd015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 -- 172.21.15.16:0/675511817 >> 172.21.15.16:0/675511817 conn(0x7f6cd00d10c0 msgr2=0x7f6cd00d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 -- 172.21.15.16:0/675511817 shutdown_connections 2024-09-17T05:04:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.333+0000 7f6cd7400640 1 -- 172.21.15.16:0/675511817 wait complete. 2024-09-17T05:04:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 Processor -- start 2024-09-17T05:04:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 -- start start 2024-09-17T05:04:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd006e7a0 0x7f6cd0145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6cd006ebb0 0x7f6cd0159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 0x7f6cd015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0168e00 con 0x7f6cd0152540 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0168c80 con 0x7f6cd006e7a0 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd7400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd0168f80 con 0x7f6cd006ebb0 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 0x7f6cd015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 0x7f6cd015a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35318/0 (socket says 172.21.15.16:35318) 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd5400640 1 -- 172.21.15.16:0/3429972468 learned_addr learned my addr 172.21.15.16:0/3429972468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cd4a00640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd006e7a0 0x7f6cd0145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.337+0000 7f6cc7e00640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6cd006ebb0 0x7f6cd0159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd5400640 1 -- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6cd006ebb0 msgr2=0x7f6cd0159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd5400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6cd006ebb0 0x7f6cd0159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd5400640 1 -- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd006e7a0 msgr2=0x7f6cd0145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd5400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd006e7a0 0x7f6cd0145c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd5400640 1 -- 172.21.15.16:0/3429972468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd0156010 con 0x7f6cd0152540 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cc7e00640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6cd006ebb0 0x7f6cd0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd5400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 0x7f6cd015a250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8019700 tx=0x7f6cc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80bb070 con 0x7f6cd0152540 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cc8004d40 con 0x7f6cd0152540 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80b2040 con 0x7f6cd0152540 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd0156510 con 0x7f6cd0152540 2024-09-17T05:04:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd0158ba0 con 0x7f6cd0152540 2024-09-17T05:04:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd0156280 con 0x7f6cd0152540 2024-09-17T05:04:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.341+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6cc80be070 con 0x7f6cd0152540 2024-09-17T05:04:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.345+0000 7f6ccd600640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6cac073200 0x7f6cac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.345+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f6cc812e160 con 0x7f6cd0152540 2024-09-17T05:04:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.345+0000 7f6cd4a00640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6cac073200 0x7f6cac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.345+0000 7f6cd4a00640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6cac073200 0x7f6cac075710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009b40 tx=0x7f6cc00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.349+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6cc80f2db0 con 0x7f6cd0152540 2024-09-17T05:04:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.557+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cd00b5a20 con 0x7f6cd0152540 2024-09-17T05:04:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.557+0000 7f6ccd600640 1 -- 172.21.15.16:0/3429972468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5230 (secure 0 0 0) 0x7f6cc8016770 con 0x7f6cd0152540 2024-09-17T05:04:04.560 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:04.560 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"btime":"2024-09-17T05:03:51:015974+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:03:47.404183+0000","modified":"2024-09-17T05:03:50.438145+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6836/3720312961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3720312961},{"type":"v1","addr":"172.21.15.104:6836","nonce":3720312961}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5775,"qdb_cluster":[5775]},"id":4}]} 2024-09-17T05:04:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 35 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6cac073200 msgr2=0x7f6cac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6cac073200 0x7f6cac075710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0009b40 tx=0x7f6cc00091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 msgr2=0x7f6cd015a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 0x7f6cd015a250 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8019700 tx=0x7f6cc8009260 comp rx=0 tx=0).stop 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 shutdown_connections 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6cac073200 0x7f6cac075710 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd0152540 0x7f6cd015a250 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6cd006ebb0 0x7f6cd0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 --2- 172.21.15.16:0/3429972468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cd006e7a0 0x7f6cd0145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 >> 172.21.15.16:0/3429972468 conn(0x7f6cd00d10c0 msgr2=0x7f6cd015aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 shutdown_connections 2024-09-17T05:04:04.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.561+0000 7f6cd7400640 1 -- 172.21.15.16:0/3429972468 wait complete. 2024-09-17T05:04:04.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 -- 172.21.15.16:0/4144046245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 msgr2=0x7f3e9c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 --2- 172.21.15.16:0/4144046245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3e9c003ed0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009b40 tx=0x7f3ea801d250 comp rx=0 tx=0).stop 2024-09-17T05:04:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 -- 172.21.15.16:0/4144046245 shutdown_connections 2024-09-17T05:04:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 --2- 172.21.15.16:0/4144046245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3e9c003ed0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 -- 172.21.15.16:0/4144046245 >> 172.21.15.16:0/4144046245 conn(0x7f3eb406f900 msgr2=0x7f3eb40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:04.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 -- 172.21.15.16:0/4144046245 shutdown_connections 2024-09-17T05:04:04.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.737+0000 7f3ebb600640 1 -- 172.21.15.16:0/4144046245 wait complete. 2024-09-17T05:04:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 Processor -- start 2024-09-17T05:04:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 -- start start 2024-09-17T05:04:04.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3eb41f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb40b0270 0x7f3eb41fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3eb414d1b0 0x7f3eb4154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb40743e0 con 0x7f3e9c003a60 2024-09-17T05:04:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4074260 con 0x7f3eb40b0270 2024-09-17T05:04:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3ebb600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb415f150 con 0x7f3eb414d1b0 2024-09-17T05:04:04.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3eb8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3eb41f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3eb8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3eb41f9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35336/0 (socket says 172.21.15.16:35336) 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.741+0000 7f3eb8c00640 1 -- 172.21.15.16:0/2409307678 learned_addr learned my addr 172.21.15.16:0/2409307678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb3e00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb40b0270 0x7f3eb41fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb9600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3eb414d1b0 0x7f3eb4154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb8c00640 1 -- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3eb414d1b0 msgr2=0x7f3eb4154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb8c00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3eb414d1b0 0x7f3eb4154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb8c00640 1 -- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb40b0270 msgr2=0x7f3eb41fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb8c00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb40b0270 0x7f3eb41fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb8c00640 1 -- 172.21.15.16:0/2409307678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb41fc6b0 con 0x7f3e9c003a60 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb8c00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3eb41f9ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009a60 tx=0x7f3ea8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:04.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80bb030 con 0x7f3e9c003a60 2024-09-17T05:04:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ea80046d0 con 0x7f3e9c003a60 2024-09-17T05:04:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8004b20 con 0x7f3e9c003a60 2024-09-17T05:04:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb41ea0d0 con 0x7f3e9c003a60 2024-09-17T05:04:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41f8e40 con 0x7f3e9c003a60 2024-09-17T05:04:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb415fe70 con 0x7f3e9c003a60 2024-09-17T05:04:04.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.745+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ea80be030 con 0x7f3e9c003a60 2024-09-17T05:04:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.749+0000 7f3eb0c00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e94073230 0x7f3e94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.749+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f3ea812f660 con 0x7f3e9c003a60 2024-09-17T05:04:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.749+0000 7f3eb3e00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e94073230 0x7f3e94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:04.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.749+0000 7f3eb3e00640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e94073230 0x7f3e94075740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0001d70 tx=0x7f3ea0001d00 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:04.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.753+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ea8016f60 con 0x7f3e9c003a60 2024-09-17T05:04:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.925+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eb40ce100 con 0x7f3e9c003a60 2024-09-17T05:04:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.925+0000 7f3eb0c00640 1 -- 172.21.15.16:0/2409307678 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+16980 (secure 0 0 0) 0x7f3ea80f9e00 con 0x7f3e9c003a60 2024-09-17T05:04:04.930 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:04.930 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":64,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:03:48.995211+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:03:45.146869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T05:03:46.365938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e94073230 msgr2=0x7f3e94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e94073230 0x7f3e94075740 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0001d70 tx=0x7f3ea0001d00 comp rx=0 tx=0).stop 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 msgr2=0x7f3eb41f9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3eb41f9ac0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009a60 tx=0x7f3ea8005850 comp rx=0 tx=0).stop 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 shutdown_connections 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e94073230 0x7f3e94075740 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3eb414d1b0 0x7f3eb4154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3eb40b0270 0x7f3eb41fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 --2- 172.21.15.16:0/2409307678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e9c003a60 0x7f3eb41f9ac0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 >> 172.21.15.16:0/2409307678 conn(0x7f3eb406f900 msgr2=0x7f3eb4152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 shutdown_connections 2024-09-17T05:04:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:04.929+0000 7f3ebb600640 1 -- 172.21.15.16:0/2409307678 wait complete. 2024-09-17T05:04:04.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:04:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2717231632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f380733e0 msgr2=0x7f8f3815d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2717231632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f380733e0 0x7f8f3815d140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009900 tx=0x7f8f2801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2717231632 shutdown_connections 2024-09-17T05:04:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2717231632 >> v2:172.21.15.104:3300/0 conn(0x7f8f38160840 0x7f8f381672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2717231632 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f380733e0 0x7f8f3815d140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2717231632 >> 172.21.15.16:0/2717231632 conn(0x7f8f380b2250 msgr2=0x7f8f380d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2717231632 shutdown_connections 2024-09-17T05:04:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.109+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2717231632 wait complete. 2024-09-17T05:04:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 Processor -- start 2024-09-17T05:04:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 -- start start 2024-09-17T05:04:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 0x7f8f38149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f3815d680 0x7f8f380b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f38160840 0x7f8f3814ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3816a010 con 0x7f8f380733e0 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f380cae90 con 0x7f8f3815d680 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f3dc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3816a190 con 0x7f8f38160840 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f36a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 0x7f8f38149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f36a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 0x7f8f38149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35342/0 (socket says 172.21.15.16:35342) 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f36a00640 1 -- 172.21.15.16:0/2143385109 learned_addr learned my addr 172.21.15.16:0/2143385109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f37400640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f38160840 0x7f8f3814ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.113+0000 7f8f2fe00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f3815d680 0x7f8f380b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f36a00640 1 -- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f38160840 msgr2=0x7f8f3814ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f36a00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f38160840 0x7f8f3814ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f36a00640 1 -- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f3815d680 msgr2=0x7f8f380b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f36a00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f3815d680 0x7f8f380b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f36a00640 1 -- 172.21.15.16:0/2143385109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3814a190 con 0x7f8f380733e0 2024-09-17T05:04:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f36a00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 0x7f8f38149c50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8f280b0040 tx=0x7f8f28005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280b2040 con 0x7f8f380733e0 2024-09-17T05:04:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f28005320 con 0x7f8f380733e0 2024-09-17T05:04:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280046d0 con 0x7f8f380733e0 2024-09-17T05:04:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f380b3490 con 0x7f8f380733e0 2024-09-17T05:04:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3814b2d0 con 0x7f8f380733e0 2024-09-17T05:04:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3815d140 con 0x7f8f380733e0 2024-09-17T05:04:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.117+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8f280be060 con 0x7f8f380733e0 2024-09-17T05:04:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.121+0000 7f8f2ea00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8f14073200 0x7f8f14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.121+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f8f2812dfc0 con 0x7f8f380733e0 2024-09-17T05:04:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.121+0000 7f8f2fe00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8f14073200 0x7f8f14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.121+0000 7f8f2fe00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8f14073200 0x7f8f14075710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8f380c3cb0 tx=0x7f8f20009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:05.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.125+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8f280164d0 con 0x7f8f380733e0 2024-09-17T05:04:05.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.333+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8f3806e170 con 0x7f8f380733e0 2024-09-17T05:04:05.337 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:04:05.333+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:04:05.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.337+0000 7f8f2ea00640 1 -- 172.21.15.16:0/2143385109 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f8f280f8e40 con 0x7f8f380733e0 2024-09-17T05:04:05.339 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8f14073200 msgr2=0x7f8f14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8f14073200 0x7f8f14075710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8f380c3cb0 tx=0x7f8f20009040 comp rx=0 tx=0).stop 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 msgr2=0x7f8f38149c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 0x7f8f38149c50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8f280b0040 tx=0x7f8f28005de0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 shutdown_connections 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8f14073200 0x7f8f14075710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8f38160840 0x7f8f3814ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f3815d680 0x7f8f380b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 --2- 172.21.15.16:0/2143385109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f380733e0 0x7f8f38149c50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 >> 172.21.15.16:0/2143385109 conn(0x7f8f380b2250 msgr2=0x7f8f380cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:05.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 shutdown_connections 2024-09-17T05:04:05.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.341+0000 7f8f3dc00640 1 -- 172.21.15.16:0/2143385109 wait complete. 2024-09-17T05:04:05.350 INFO:tasks.ceph.mds.d.smithi104.stderr: -2330> 2024-09-17T05:03:49.958+0000 7f9a06e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:04:05.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:04:05.465 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:04:05.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 -- 172.21.15.16:0/3827839280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f844415f520 msgr2=0x7f844415c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 --2- 172.21.15.16:0/3827839280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f844415f520 0x7f844415c160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f843c009c80 tx=0x7f843c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 -- 172.21.15.16:0/3827839280 shutdown_connections 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 --2- 172.21.15.16:0/3827839280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f844415f520 0x7f844415c160 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 --2- 172.21.15.16:0/3827839280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8444074a20 0x7f844415efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 -- 172.21.15.16:0/3827839280 >> 172.21.15.16:0/3827839280 conn(0x7f84440b2250 msgr2=0x7f84440d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 -- 172.21.15.16:0/3827839280 shutdown_connections 2024-09-17T05:04:05.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 -- 172.21.15.16:0/3827839280 wait complete. 2024-09-17T05:04:05.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 Processor -- start 2024-09-17T05:04:05.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.521+0000 7f844c400640 1 -- start start 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 0x7f8444151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8444074e30 0x7f844415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f844415f520 0x7f844415a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8444168f30 con 0x7f8444074a20 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8444168db0 con 0x7f8444074e30 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f84441690b0 con 0x7f844415f520 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 0x7f8444151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 0x7f8444151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35354/0 (socket says 172.21.15.16:35354) 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 -- 172.21.15.16:0/2498694862 learned_addr learned my addr 172.21.15.16:0/2498694862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844a400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f844415f520 0x7f844415a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449000640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8444074e30 0x7f844415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 -- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f844415f520 msgr2=0x7f844415a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f844415f520 0x7f844415a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 -- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8444074e30 msgr2=0x7f844415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8444074e30 0x7f844415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 -- 172.21.15.16:0/2498694862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441532b0 con 0x7f8444074a20 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449000640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8444074e30 0x7f844415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8449a00640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 0x7f8444151b30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f843800e7a0 tx=0x7f843800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843800ccc0 con 0x7f8444074a20 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8438010430 con 0x7f8444074a20 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84440b53f0 con 0x7f8444074a20 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438010860 con 0x7f8444074a20 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.525+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8444153540 con 0x7f8444074a20 2024-09-17T05:04:05.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.529+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844415efe0 con 0x7f8444074a20 2024-09-17T05:04:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.529+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8438010a00 con 0x7f8444074a20 2024-09-17T05:04:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.529+0000 7f8441600640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8424073230 0x7f8424075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.529+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6187+0+0 (secure 0 0 0) 0x7f8438094da0 con 0x7f8444074a20 2024-09-17T05:04:05.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.529+0000 7f8449000640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8424073230 0x7f8424075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:05.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.529+0000 7f8449000640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8424073230 0x7f8424075740 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f84440b3ca0 tx=0x7f8430009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:05.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.533+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f843805f610 con 0x7f8444074a20 2024-09-17T05:04:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:05.749+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f844414ff20 con 0x7f8444074a20 2024-09-17T05:04:06.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f8441600640 1 -- 172.21.15.16:0/2498694862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7f8438064040 con 0x7f8444074a20 2024-09-17T05:04:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8424073230 msgr2=0x7f8424075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8424073230 0x7f8424075740 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f84440b3ca0 tx=0x7f8430009040 comp rx=0 tx=0).stop 2024-09-17T05:04:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 msgr2=0x7f8444151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 0x7f8444151b30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f843800e7a0 tx=0x7f843800ec60 comp rx=0 tx=0).stop 2024-09-17T05:04:06.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 shutdown_connections 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8424073230 0x7f8424075740 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f844415f520 0x7f844415a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8444074e30 0x7f844415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 --2- 172.21.15.16:0/2498694862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8444074a20 0x7f8444151b30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 >> 172.21.15.16:0/2498694862 conn(0x7f84440b2250 msgr2=0x7f84440cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 shutdown_connections 2024-09-17T05:04:06.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.349+0000 7f844c400640 1 -- 172.21.15.16:0/2498694862 wait complete. 2024-09-17T05:04:06.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:04:06.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 -- 172.21.15.16:0/4225411018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa540cdff0 msgr2=0x7ffa540b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 --2- 172.21.15.16:0/4225411018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa540cdff0 0x7ffa540b4d30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 -- 172.21.15.16:0/4225411018 shutdown_connections 2024-09-17T05:04:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 --2- 172.21.15.16:0/4225411018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa540cdff0 0x7ffa540b4d30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 -- 172.21.15.16:0/4225411018 >> 172.21.15.16:0/4225411018 conn(0x7ffa54073ea0 msgr2=0x7ffa540742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 -- 172.21.15.16:0/4225411018 shutdown_connections 2024-09-17T05:04:06.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.537+0000 7ffa5b600640 1 -- 172.21.15.16:0/4225411018 wait complete. 2024-09-17T05:04:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 Processor -- start 2024-09-17T05:04:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 -- start start 2024-09-17T05:04:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 0x7ffa541532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa540cdff0 0x7ffa5415f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa540ce400 0x7ffa5414bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5416a0f0 con 0x7ffa54071600 2024-09-17T05:04:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54169f70 con 0x7ffa540cdff0 2024-09-17T05:04:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5416a270 con 0x7ffa540ce400 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 0x7ffa541532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 0x7ffa541532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 -- 172.21.15.16:0/3473534983 learned_addr learned my addr 172.21.15.16:0/3473534983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 -- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa540ce400 msgr2=0x7ffa5414bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa540ce400 0x7ffa5414bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 -- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa540cdff0 msgr2=0x7ffa5415f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa540cdff0 0x7ffa5415f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 -- 172.21.15.16:0/3473534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa541582e0 con 0x7ffa54071600 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa58c00640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 0x7ffa541532b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b0040 tx=0x7ffa440040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440b2040 con 0x7ffa54071600 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa44003600 con 0x7ffa54071600 2024-09-17T05:04:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44003a10 con 0x7ffa54071600 2024-09-17T05:04:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa541585c0 con 0x7ffa54071600 2024-09-17T05:04:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.541+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa54158b80 con 0x7ffa54071600 2024-09-17T05:04:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.545+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa540b4d30 con 0x7ffa54071600 2024-09-17T05:04:06.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.545+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa440bf060 con 0x7ffa54071600 2024-09-17T05:04:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.545+0000 7ffa51600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa30073200 0x7ffa30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.545+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6187+0+0 (secure 0 0 0) 0x7ffa4412e0c0 con 0x7ffa54071600 2024-09-17T05:04:06.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.545+0000 7ffa4be00640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa30073200 0x7ffa30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.549+0000 7ffa4be00640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa30073200 0x7ffa30075710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000f70 tx=0x7ffa3c0079c0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.549+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffa440164d0 con 0x7ffa54071600 2024-09-17T05:04:06.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:06.725+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffa540b4280 con 0x7ffa54071600 2024-09-17T05:04:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.349+0000 7ffa51600640 1 -- 172.21.15.16:0/3473534983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) ==== 181+0+0 (secure 0 0 0) 0x7ffa440f8f20 con 0x7ffa54071600 2024-09-17T05:04:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:04:07.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa30073200 msgr2=0x7ffa30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:07.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa30073200 0x7ffa30075710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000f70 tx=0x7ffa3c0079c0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 msgr2=0x7ffa541532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:07.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 0x7ffa541532b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffa440b0040 tx=0x7ffa440040c0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 shutdown_connections 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa30073200 0x7ffa30075710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa540ce400 0x7ffa5414bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa540cdff0 0x7ffa5415f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 --2- 172.21.15.16:0/3473534983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa54071600 0x7ffa541532b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 >> 172.21.15.16:0/3473534983 conn(0x7ffa54073ea0 msgr2=0x7ffa540b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:07.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 shutdown_connections 2024-09-17T05:04:07.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.353+0000 7ffa5b600640 1 -- 172.21.15.16:0/3473534983 wait complete. 2024-09-17T05:04:07.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:04:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 -- 172.21.15.16:0/3491782100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 msgr2=0x7f721415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 --2- 172.21.15.16:0/3491782100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721415b470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f72140cc920 tx=0x7f720c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 -- 172.21.15.16:0/3491782100 shutdown_connections 2024-09-17T05:04:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 --2- 172.21.15.16:0/3491782100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721415b470 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 --2- 172.21.15.16:0/3491782100 >> v2:172.21.15.104:3300/0 conn(0x7f721406ce90 0x7f721406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 -- 172.21.15.16:0/3491782100 >> 172.21.15.16:0/3491782100 conn(0x7f72140d0050 msgr2=0x7f72140d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 -- 172.21.15.16:0/3491782100 shutdown_connections 2024-09-17T05:04:07.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.545+0000 7f721ba00640 1 -- 172.21.15.16:0/3491782100 wait complete. 2024-09-17T05:04:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 Processor -- start 2024-09-17T05:04:07.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 -- start start 2024-09-17T05:04:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721406ce90 0x7f721415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:07.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f72140cc0f0 0x7f721413e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721413eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7214165d20 con 0x7f721415b060 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7214165ba0 con 0x7f721406ce90 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f721ba00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7214165ea0 con 0x7f72140cc0f0 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721413eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721413eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38434/0 (socket says 172.21.15.16:38434) 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 -- 172.21.15.16:0/845445400 learned_addr learned my addr 172.21.15.16:0/845445400 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7213e00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f72140cc0f0 0x7f721413e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219000640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721406ce90 0x7f721415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 -- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f72140cc0f0 msgr2=0x7f721413e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f72140cc0f0 0x7f721413e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 -- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721406ce90 msgr2=0x7f721415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:07.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721406ce90 0x7f721415e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219a00640 1 -- 172.21.15.16:0/845445400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721415e8d0 con 0x7f721415b060 2024-09-17T05:04:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.549+0000 7f7219000640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721406ce90 0x7f721415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f7219a00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721413eb80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f720c019900 tx=0x7f720c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:07.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c0bc070 con 0x7f721415b060 2024-09-17T05:04:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f720c004130 con 0x7f721415b060 2024-09-17T05:04:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72140cdab0 con 0x7f721415b060 2024-09-17T05:04:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c0b2040 con 0x7f721415b060 2024-09-17T05:04:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721415ee90 con 0x7f721415b060 2024-09-17T05:04:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7214166390 con 0x7f721415b060 2024-09-17T05:04:07.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.553+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f720c0bf070 con 0x7f721415b060 2024-09-17T05:04:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.557+0000 7f7210c00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71f4073200 0x7f71f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.557+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5749+0+0 (secure 0 0 0) 0x7f720c12df40 con 0x7f721415b060 2024-09-17T05:04:07.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.557+0000 7f7219000640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71f4073200 0x7f71f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:07.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.557+0000 7f7219000640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71f4073200 0x7f71f4075710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f72080016b0 tx=0x7f7208009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:07.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.561+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f720c016970 con 0x7f721415b060 2024-09-17T05:04:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:07.737+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f721415e640 con 0x7f721415b060 2024-09-17T05:04:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.357+0000 7f7210c00640 1 -- 172.21.15.16:0/845445400 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) ==== 169+0+0 (secure 0 0 0) 0x7f720c0f8840 con 0x7f721415b060 2024-09-17T05:04:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:04:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71f4073200 msgr2=0x7f71f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71f4073200 0x7f71f4075710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f72080016b0 tx=0x7f7208009040 comp rx=0 tx=0).stop 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 msgr2=0x7f721413eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721413eb80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f720c019900 tx=0x7f720c005ae0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 shutdown_connections 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71f4073200 0x7f71f4075710 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f721415b060 0x7f721413eb80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f72140cc0f0 0x7f721413e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 --2- 172.21.15.16:0/845445400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f721406ce90 0x7f721415e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 >> 172.21.15.16:0/845445400 conn(0x7f72140d0050 msgr2=0x7f721414e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 shutdown_connections 2024-09-17T05:04:08.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.361+0000 7f721ba00640 1 -- 172.21.15.16:0/845445400 wait complete. 2024-09-17T05:04:08.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 -- 172.21.15.16:0/4207499176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074a20 msgr2=0x7f0ed815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 --2- 172.21.15.16:0/4207499176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074a20 0x7f0ed815efe0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0ed80ce2c0 tx=0x7f0ec801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 -- 172.21.15.16:0/4207499176 shutdown_connections 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 --2- 172.21.15.16:0/4207499176 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0ed815f520 0x7f0ed815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 --2- 172.21.15.16:0/4207499176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074a20 0x7f0ed815efe0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 -- 172.21.15.16:0/4207499176 >> 172.21.15.16:0/4207499176 conn(0x7f0ed80b2250 msgr2=0x7f0ed80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 -- 172.21.15.16:0/4207499176 shutdown_connections 2024-09-17T05:04:08.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.549+0000 7f0edd800640 1 -- 172.21.15.16:0/4207499176 wait complete. 2024-09-17T05:04:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 Processor -- start 2024-09-17T05:04:08.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 -- start start 2024-09-17T05:04:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ed8074a20 0x7f0ed80c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 0x7f0ed8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0ed815f520 0x7f0ed815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8168ed0 con 0x7f0ed8074e30 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed8168d50 con 0x7f0ed8074a20 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0edd800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed8169050 con 0x7f0ed815f520 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 0x7f0ed8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 0x7f0ed8159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38462/0 (socket says 172.21.15.16:38462) 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 -- 172.21.15.16:0/536872746 learned_addr learned my addr 172.21.15.16:0/536872746 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ed6a00640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ed8074a20 0x7f0ed80c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ed7400640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0ed815f520 0x7f0ed815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 -- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0ed815f520 msgr2=0x7f0ed815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0ed815f520 0x7f0ed815a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 -- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ed8074a20 msgr2=0x7f0ed80c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ed8074a20 0x7f0ed80c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.553+0000 7f0ece000640 1 -- 172.21.15.16:0/536872746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed81542e0 con 0x7f0ed8074e30 2024-09-17T05:04:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ece000640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 0x7f0ed8159dc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000b3e0 tx=0x7f0ec000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0015020 con 0x7f0ed8074e30 2024-09-17T05:04:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ec0011940 con 0x7f0ed8074e30 2024-09-17T05:04:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed81545c0 con 0x7f0ed8074e30 2024-09-17T05:04:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0011d90 con 0x7f0ed8074e30 2024-09-17T05:04:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed8154b80 con 0x7f0ed8074e30 2024-09-17T05:04:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed815efe0 con 0x7f0ed8074e30 2024-09-17T05:04:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0ec0018020 con 0x7f0ed8074e30 2024-09-17T05:04:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ecf400640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eb4073230 0x7f0eb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.557+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f0ec0093fe0 con 0x7f0ed8074e30 2024-09-17T05:04:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.561+0000 7f0ed6a00640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eb4073230 0x7f0eb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.561+0000 7f0ed6a00640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eb4073230 0x7f0eb4075740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80af000 tx=0x7f0ec801b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.565+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ec001f020 con 0x7f0ed8074e30 2024-09-17T05:04:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.737+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0ed814f1d0 con 0x7f0ed8074e30 2024-09-17T05:04:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.741+0000 7f0ecf400640 1 -- 172.21.15.16:0/536872746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7f0ec005ea20 con 0x7f0ed8074e30 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eb4073230 msgr2=0x7f0eb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eb4073230 0x7f0eb4075740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80af000 tx=0x7f0ec801b100 comp rx=0 tx=0).stop 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 msgr2=0x7f0ed8159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 0x7f0ed8159dc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000b3e0 tx=0x7f0ec000b8a0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 shutdown_connections 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0eb4073230 0x7f0eb4075740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0ed815f520 0x7f0ed815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed8074e30 0x7f0ed8159dc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 --2- 172.21.15.16:0/536872746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ed8074a20 0x7f0ed80c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 >> 172.21.15.16:0/536872746 conn(0x7f0ed80b2250 msgr2=0x7f0ed814f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 shutdown_connections 2024-09-17T05:04:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.745+0000 7f0edd800640 1 -- 172.21.15.16:0/536872746 wait complete. 2024-09-17T05:04:08.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.921+0000 7f53ae800640 1 -- 172.21.15.16:0/614423719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a815cab0 msgr2=0x7f53a80c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.921+0000 7f53ae800640 1 --2- 172.21.15.16:0/614423719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a815cab0 0x7f53a80c9b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009c80 tx=0x7f53a001cd70 comp rx=0 tx=0).stop 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.921+0000 7f53ae800640 1 -- 172.21.15.16:0/614423719 shutdown_connections 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.921+0000 7f53ae800640 1 --2- 172.21.15.16:0/614423719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a815cab0 0x7f53a80c9b50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.921+0000 7f53ae800640 1 -- 172.21.15.16:0/614423719 >> 172.21.15.16:0/614423719 conn(0x7f53a80d2270 msgr2=0x7f53a80d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 -- 172.21.15.16:0/614423719 shutdown_connections 2024-09-17T05:04:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 -- 172.21.15.16:0/614423719 wait complete. 2024-09-17T05:04:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 Processor -- start 2024-09-17T05:04:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 -- start start 2024-09-17T05:04:08.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 0x7f53a8155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53a815cab0 0x7f53a81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a81568c0 0x7f53a8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53a816aec0 con 0x7f53a80cc8c0 2024-09-17T05:04:08.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53a816ad40 con 0x7f53a815cab0 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.925+0000 7f53ae800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f53a816b040 con 0x7f53a81568c0 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 0x7f53a8155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 0x7f53a8155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38468/0 (socket says 172.21.15.16:38468) 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 -- 172.21.15.16:0/722405718 learned_addr learned my addr 172.21.15.16:0/722405718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f539ea00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53a815cab0 0x7f53a81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7e00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a81568c0 0x7f53a8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 -- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a81568c0 msgr2=0x7f53a8157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a81568c0 0x7f53a8157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 -- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53a815cab0 msgr2=0x7f53a81574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:08.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53a815cab0 0x7f53a81574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 -- 172.21.15.16:0/722405718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a815a3f0 con 0x7f53a80cc8c0 2024-09-17T05:04:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7e00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a81568c0 0x7f53a8157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f539ea00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53a815cab0 0x7f53a81574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:08.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53a7400640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 0x7f53a8155bb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f539800c460 tx=0x7f539800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:08.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53980052d0 con 0x7f53a80cc8c0 2024-09-17T05:04:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5398005470 con 0x7f53a80cc8c0 2024-09-17T05:04:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398012870 con 0x7f53a80cc8c0 2024-09-17T05:04:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a8158250 con 0x7f53a80cc8c0 2024-09-17T05:04:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a806ec60 con 0x7f53a80cc8c0 2024-09-17T05:04:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a80d1580 con 0x7f53a80cc8c0 2024-09-17T05:04:08.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.929+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5398003600 con 0x7f53a80cc8c0 2024-09-17T05:04:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.933+0000 7f539fe00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073230 0x7f5384075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.933+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f5398093c00 con 0x7f53a80cc8c0 2024-09-17T05:04:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.933+0000 7f539ea00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073230 0x7f5384075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:08.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.933+0000 7f539ea00640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073230 0x7f5384075740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53a806a6e0 tx=0x7f5390009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:08.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:08.937+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f539805ed80 con 0x7f53a80cc8c0 2024-09-17T05:04:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.185+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f53a81560f0 con 0x7f53a80cc8c0 2024-09-17T05:04:09.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.357+0000 7f539fe00640 1 -- 172.21.15.16:0/722405718 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v174) ==== 150+0+0 (secure 0 0 0) 0x7f539805e680 con 0x7f53a80cc8c0 2024-09-17T05:04:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073230 msgr2=0x7f5384075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073230 0x7f5384075740 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53a806a6e0 tx=0x7f5390009040 comp rx=0 tx=0).stop 2024-09-17T05:04:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 msgr2=0x7f53a8155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 0x7f53a8155bb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f539800c460 tx=0x7f539800c920 comp rx=0 tx=0).stop 2024-09-17T05:04:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 shutdown_connections 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073230 0x7f5384075740 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a81568c0 0x7f53a8157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53a815cab0 0x7f53a81574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 --2- 172.21.15.16:0/722405718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a80cc8c0 0x7f53a8155bb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 >> 172.21.15.16:0/722405718 conn(0x7f53a80d2270 msgr2=0x7f53a80b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 shutdown_connections 2024-09-17T05:04:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.361+0000 7f53ae800640 1 -- 172.21.15.16:0/722405718 wait complete. 2024-09-17T05:04:09.383 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... skipped 'Require FUSE client to inject client release failure' 2024-09-17T05:04:09.383 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) That the MDS will not let a client sit above mds_max_caps_per_client caps. 2024-09-17T05:04:09.383 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:04:09.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f643dc00640 1 -- 172.21.15.16:0/7138583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 msgr2=0x7f641c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f643dc00640 1 --2- 172.21.15.16:0/7138583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f641c003e80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f642800c290 tx=0x7f642801cfa0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f6437400640 1 -- 172.21.15.16:0/7138583 reap_dead start 2024-09-17T05:04:09.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f643dc00640 1 -- 172.21.15.16:0/7138583 shutdown_connections 2024-09-17T05:04:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f643dc00640 1 -- 172.21.15.16:0/7138583 >> 172.21.15.16:0/7138583 conn(0x7f643806f900 msgr2=0x7f64380b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f643dc00640 1 -- 172.21.15.16:0/7138583 shutdown_connections 2024-09-17T05:04:09.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.541+0000 7f643dc00640 1 -- 172.21.15.16:0/7138583 wait complete. 2024-09-17T05:04:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 Processor -- start 2024-09-17T05:04:09.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 -- start start 2024-09-17T05:04:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f6438144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6438153040 0x7f6438153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6438153990 0x7f64381f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6438166b80 con 0x7f641c003a10 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6438166a00 con 0x7f6438153040 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6438166d00 con 0x7f6438153990 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f6438144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f6438144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38490/0 (socket says 172.21.15.16:38490) 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 -- 172.21.15.16:0/1031031153 learned_addr learned my addr 172.21.15.16:0/1031031153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f642e000640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6438153040 0x7f6438153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6437400640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6438153990 0x7f64381f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 -- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6438153990 msgr2=0x7f64381f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6438153990 0x7f64381f9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 -- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6438153040 msgr2=0x7f6438153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6438153040 0x7f6438153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 -- 172.21.15.16:0/1031031153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6438144eb0 con 0x7f641c003a10 2024-09-17T05:04:09.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f6436a00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f6438144970 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f642801ca60 tx=0x7f6428006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:09.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428006440 con 0x7f641c003a10 2024-09-17T05:04:09.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6428003600 con 0x7f641c003a10 2024-09-17T05:04:09.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428003a10 con 0x7f641c003a10 2024-09-17T05:04:09.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643815b580 con 0x7f641c003a10 2024-09-17T05:04:09.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.545+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6438155a60 con 0x7f641c003a10 2024-09-17T05:04:09.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.549+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6438152c80 con 0x7f641c003a10 2024-09-17T05:04:09.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.549+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6428003bb0 con 0x7f641c003a10 2024-09-17T05:04:09.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.549+0000 7f642f400640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6414073200 0x7f6414075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.549+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f642812ea30 con 0x7f641c003a10 2024-09-17T05:04:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.549+0000 7f642e000640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6414073200 0x7f6414075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.553+0000 7f642e000640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6414073200 0x7f6414075710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f64381581d0 tx=0x7f64200091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:09.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.553+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6428016d50 con 0x7f641c003a10 2024-09-17T05:04:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.729+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6438073650 con 0x7f641c003a10 2024-09-17T05:04:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.729+0000 7f642f400640 1 -- 172.21.15.16:0/1031031153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+13296 (secure 0 0 0) 0x7f64280f9730 con 0x7f641c003a10 2024-09-17T05:04:09.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:09.733 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":67,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:08.354421+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6836/3720312961":"2024-09-18T05:04:05.338689+0000","172.21.15.104:6834/3720312961":"2024-09-18T05:04:05.338689+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6414073200 msgr2=0x7f6414075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6414073200 0x7f6414075710 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f64381581d0 tx=0x7f64200091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 msgr2=0x7f6438144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f6438144970 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f642801ca60 tx=0x7f6428006d60 comp rx=0 tx=0).stop 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 shutdown_connections 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6414073200 0x7f6414075710 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6438153990 0x7f64381f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6438153040 0x7f6438153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 --2- 172.21.15.16:0/1031031153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c003a10 0x7f6438144970 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 >> 172.21.15.16:0/1031031153 conn(0x7f643806f900 msgr2=0x7f64381517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 shutdown_connections 2024-09-17T05:04:09.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.733+0000 7f643dc00640 1 -- 172.21.15.16:0/1031031153 wait complete. 2024-09-17T05:04:09.755 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:04:09.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:04:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 -- 172.21.15.16:0/3816381316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c06e1d0 msgr2=0x7f6d9c0d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 --2- 172.21.15.16:0/3816381316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c0d0770 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009840 tx=0x7f6d8c01c790 comp rx=0 tx=0).stop 2024-09-17T05:04:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 -- 172.21.15.16:0/3816381316 shutdown_connections 2024-09-17T05:04:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 --2- 172.21.15.16:0/3816381316 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d9c0ce490 0x7f6d9c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 --2- 172.21.15.16:0/3816381316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c0d0770 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 -- 172.21.15.16:0/3816381316 >> 172.21.15.16:0/3816381316 conn(0x7f6d9c0b2250 msgr2=0x7f6d9c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 -- 172.21.15.16:0/3816381316 shutdown_connections 2024-09-17T05:04:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.913+0000 7f6da3200640 1 -- 172.21.15.16:0/3816381316 wait complete. 2024-09-17T05:04:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 Processor -- start 2024-09-17T05:04:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 -- start start 2024-09-17T05:04:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d9c0ce490 0x7f6d9c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 0x7f6d9c0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c168cb0 con 0x7f6d9c15e780 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c168b30 con 0x7f6d9c06e1d0 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da3200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9c168e30 con 0x7f6d9c0ce490 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 0x7f6d9c0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 0x7f6d9c0c6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38504/0 (socket says 172.21.15.16:38504) 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6da1200640 1 -- 172.21.15.16:0/1205178745 learned_addr learned my addr 172.21.15.16:0/1205178745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6d9be00640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.917+0000 7f6d93e00640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d9c0ce490 0x7f6d9c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da1200640 1 -- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d9c0ce490 msgr2=0x7f6d9c0c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da1200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d9c0ce490 0x7f6d9c0c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da1200640 1 -- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c06e1d0 msgr2=0x7f6d9c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da1200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da1200640 1 -- 172.21.15.16:0/1205178745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c154640 con 0x7f6d9c15e780 2024-09-17T05:04:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6d9be00640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da1200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 0x7f6d9c0c6290 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400cce0 tx=0x7f6d9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400f450 con 0x7f6d9c15e780 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d9400f6e0 con 0x7f6d9c15e780 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9c154920 con 0x7f6d9c15e780 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400fb30 con 0x7f6d9c15e780 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9c0b2ff0 con 0x7f6d9c15e780 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9c0d0770 con 0x7f6d9c15e780 2024-09-17T05:04:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.921+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d9400fcd0 con 0x7f6d9c15e780 2024-09-17T05:04:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.925+0000 7f6d98c00640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d78073200 0x7f6d78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.925+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f6d94094d10 con 0x7f6d9c15e780 2024-09-17T05:04:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.925+0000 7f6d9be00640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d78073200 0x7f6d78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.925+0000 7f6d9be00640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d78073200 0x7f6d78075710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0af000 tx=0x7f6d8c01afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:09.929+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d9409a020 con 0x7f6d9c15e780 2024-09-17T05:04:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d9c15c460 con 0x7f6d9c15e780 2024-09-17T05:04:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6d98c00640 1 -- 172.21.15.16:0/1205178745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f6d94014030 con 0x7f6d9c15e780 2024-09-17T05:04:10.111 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d78073200 msgr2=0x7f6d78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d78073200 0x7f6d78075710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0af000 tx=0x7f6d8c01afd0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 msgr2=0x7f6d9c0c6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 0x7f6d9c0c6290 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400cce0 tx=0x7f6d9400a620 comp rx=0 tx=0).stop 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 shutdown_connections 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d78073200 0x7f6d78075710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9c15e780 0x7f6d9c0c6290 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d9c0ce490 0x7f6d9c0c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 --2- 172.21.15.16:0/1205178745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d9c06e1d0 0x7f6d9c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 >> 172.21.15.16:0/1205178745 conn(0x7f6d9c0b2250 msgr2=0x7f6d9c0b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 shutdown_connections 2024-09-17T05:04:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.109+0000 7f6da3200640 1 -- 172.21.15.16:0/1205178745 wait complete. 2024-09-17T05:04:10.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-09-17T05:04:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 -- 172.21.15.16:0/3490408328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 msgr2=0x7f6ba00d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 --2- 172.21.15.16:0/3490408328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba00d06d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009bc0 tx=0x7f6b9801cac0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 -- 172.21.15.16:0/3490408328 shutdown_connections 2024-09-17T05:04:10.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 --2- 172.21.15.16:0/3490408328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba00d06d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 --2- 172.21.15.16:0/3490408328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ba015d630 0x7f6ba00cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 -- 172.21.15.16:0/3490408328 >> 172.21.15.16:0/3490408328 conn(0x7f6ba00cf070 msgr2=0x7f6ba00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 -- 172.21.15.16:0/3490408328 shutdown_connections 2024-09-17T05:04:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.289+0000 7f6ba7600640 1 -- 172.21.15.16:0/3490408328 wait complete. 2024-09-17T05:04:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 Processor -- start 2024-09-17T05:04:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 -- start start 2024-09-17T05:04:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ba015d630 0x7f6ba015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ba0160bb0 0x7f6ba00c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01699c0 con 0x7f6ba00ccf90 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0169840 con 0x7f6ba015d630 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba7600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba0169b40 con 0x7f6ba0160bb0 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba015a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38510/0 (socket says 172.21.15.16:38510) 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 -- 172.21.15.16:0/2680587721 learned_addr learned my addr 172.21.15.16:0/2680587721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6b9fe00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ba015d630 0x7f6ba015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba5600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ba0160bb0 0x7f6ba00c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 -- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ba0160bb0 msgr2=0x7f6ba00c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ba0160bb0 0x7f6ba00c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 -- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ba015d630 msgr2=0x7f6ba015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ba015d630 0x7f6ba015a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.293+0000 7f6ba4c00640 1 -- 172.21.15.16:0/2680587721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba00cac40 con 0x7f6ba00ccf90 2024-09-17T05:04:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6ba4c00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba015a200 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c3a0 tx=0x7f6b9400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94017070 con 0x7f6ba00ccf90 2024-09-17T05:04:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b94003600 con 0x7f6ba00ccf90 2024-09-17T05:04:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba00caf20 con 0x7f6ba00ccf90 2024-09-17T05:04:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94003a50 con 0x7f6ba00ccf90 2024-09-17T05:04:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba00cb4e0 con 0x7f6ba00ccf90 2024-09-17T05:04:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba00d06d0 con 0x7f6ba00ccf90 2024-09-17T05:04:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b9401a070 con 0x7f6ba00ccf90 2024-09-17T05:04:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6b9cc00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b80073230 0x7f6b80075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.297+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f6b940933c0 con 0x7f6ba00ccf90 2024-09-17T05:04:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.301+0000 7f6b9fe00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b80073230 0x7f6b80075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.301+0000 7f6b9fe00640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b80073230 0x7f6b80075740 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00c9b60 tx=0x7f6b8c00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.301+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6ba00cd3a0 con 0x7f6ba00ccf90 2024-09-17T05:04:10.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:10.541+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f6ba00c5a30 con 0x7f6ba00ccf90 2024-09-17T05:04:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.365+0000 7f6b9cc00640 1 -- 172.21.15.16:0/2680587721 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v176) ==== 151+0+0 (secure 0 0 0) 0x7f6b9405de60 con 0x7f6ba00ccf90 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b80073230 msgr2=0x7f6b80075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b80073230 0x7f6b80075740 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00c9b60 tx=0x7f6b8c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 msgr2=0x7f6ba015a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba015a200 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c3a0 tx=0x7f6b9400c860 comp rx=0 tx=0).stop 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 shutdown_connections 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b80073230 0x7f6b80075740 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ba0160bb0 0x7f6ba00c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ba015d630 0x7f6ba015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 --2- 172.21.15.16:0/2680587721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ba00ccf90 0x7f6ba015a200 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 >> 172.21.15.16:0/2680587721 conn(0x7f6ba00cf070 msgr2=0x7f6ba00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 shutdown_connections 2024-09-17T05:04:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.369+0000 7f6ba7600640 1 -- 172.21.15.16:0/2680587721 wait complete. 2024-09-17T05:04:11.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:04:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 -- 172.21.15.16:0/1913177506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938003a60 msgr2=0x7f4938003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 --2- 172.21.15.16:0/1913177506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938003a60 0x7f4938003ed0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4944009b40 tx=0x7f494401d250 comp rx=0 tx=0).stop 2024-09-17T05:04:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 -- 172.21.15.16:0/1913177506 shutdown_connections 2024-09-17T05:04:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 --2- 172.21.15.16:0/1913177506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4938003a60 0x7f4938003ed0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 -- 172.21.15.16:0/1913177506 >> 172.21.15.16:0/1913177506 conn(0x7f495406f900 msgr2=0x7f49540b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 -- 172.21.15.16:0/1913177506 shutdown_connections 2024-09-17T05:04:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.557+0000 7f4958a00640 1 -- 172.21.15.16:0/1913177506 wait complete. 2024-09-17T05:04:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 Processor -- start 2024-09-17T05:04:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 -- start start 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4938003a60 0x7f49541f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 0x7f495414ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f495414f150 0x7f4954155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4954074430 con 0x7f4954156220 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49540742b0 con 0x7f4938003a60 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4958a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f495406d720 con 0x7f495414f150 2024-09-17T05:04:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 0x7f495414ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 0x7f495414ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38532/0 (socket says 172.21.15.16:38532) 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 -- 172.21.15.16:0/556081457 learned_addr learned my addr 172.21.15.16:0/556081457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4951e00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4938003a60 0x7f49541f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f4952800640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f495414f150 0x7f4954155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 -- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f495414f150 msgr2=0x7f4954155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f495414f150 0x7f4954155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 -- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4938003a60 msgr2=0x7f49541f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4938003a60 0x7f49541f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 -- 172.21.15.16:0/556081457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495414f560 con 0x7f4954156220 2024-09-17T05:04:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.561+0000 7f494be00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 0x7f495414ec10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f493c000fc0 tx=0x7f493c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c00de90 con 0x7f4954156220 2024-09-17T05:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f493c00b830 con 0x7f4954156220 2024-09-17T05:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495414de50 con 0x7f4954156220 2024-09-17T05:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c00bc00 con 0x7f4954156220 2024-09-17T05:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495414e410 con 0x7f4954156220 2024-09-17T05:04:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495406dc80 con 0x7f4954156220 2024-09-17T05:04:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f493c003600 con 0x7f4954156220 2024-09-17T05:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f494a000640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f49300733a0 0x7f49300758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.565+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f493c0944a0 con 0x7f4954156220 2024-09-17T05:04:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.569+0000 7f4951e00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f49300733a0 0x7f49300758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.569+0000 7f4951e00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f49300733a0 0x7f49300758b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49440ad000 tx=0x7f4944019ff0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.569+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f495414f560 con 0x7f4954156220 2024-09-17T05:04:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.745+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4954156840 con 0x7f4954156220 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.745+0000 7f494a000640 1 -- 172.21.15.16:0/556081457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7f493c05efd0 con 0x7f4954156220 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:04:11.749 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:04:11.750 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:04:11.751 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:04:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f49300733a0 msgr2=0x7f49300758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f49300733a0 0x7f49300758b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49440ad000 tx=0x7f4944019ff0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 msgr2=0x7f495414ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 0x7f495414ec10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f493c000fc0 tx=0x7f493c00dc40 comp rx=0 tx=0).stop 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 shutdown_connections 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f49300733a0 0x7f49300758b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f495414f150 0x7f4954155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4954156220 0x7f495414ec10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 --2- 172.21.15.16:0/556081457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4938003a60 0x7f49541f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 >> 172.21.15.16:0/556081457 conn(0x7f495406f900 msgr2=0x7f49540b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 shutdown_connections 2024-09-17T05:04:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.749+0000 7f4958a00640 1 -- 172.21.15.16:0/556081457 wait complete. 2024-09-17T05:04:11.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 -- 172.21.15.16:0/2411606807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 msgr2=0x7f3d7015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 --2- 172.21.15.16:0/2411606807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d7015efe0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3d700cca90 tx=0x7f3d6001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 -- 172.21.15.16:0/2411606807 shutdown_connections 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 --2- 172.21.15.16:0/2411606807 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d7015ba50 0x7f3d7015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 --2- 172.21.15.16:0/2411606807 >> v2:172.21.15.104:3300/0 conn(0x7f3d70074e30 0x7f3d7015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 --2- 172.21.15.16:0/2411606807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d7015efe0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 -- 172.21.15.16:0/2411606807 >> 172.21.15.16:0/2411606807 conn(0x7f3d700b2250 msgr2=0x7f3d700d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 -- 172.21.15.16:0/2411606807 shutdown_connections 2024-09-17T05:04:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.929+0000 7f3d76c00640 1 -- 172.21.15.16:0/2411606807 wait complete. 2024-09-17T05:04:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 Processor -- start 2024-09-17T05:04:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 -- start start 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d701542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d70074e30 0x7f3d7014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d7015ba50 0x7f3d701506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d700cf360 con 0x7f3d70074a20 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3d700cf1e0 con 0x7f3d70074e30 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d76c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3d701681d0 con 0x7f3d7015ba50 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d701542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d701542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38548/0 (socket says 172.21.15.16:38548) 2024-09-17T05:04:11.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 -- 172.21.15.16:0/1787771122 learned_addr learned my addr 172.21.15.16:0/1787771122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d74c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d7015ba50 0x7f3d701506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d67e00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d70074e30 0x7f3d7014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 -- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d7015ba50 msgr2=0x7f3d701506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d7015ba50 0x7f3d701506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 -- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d70074e30 msgr2=0x7f3d7014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d70074e30 0x7f3d7014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 -- 172.21.15.16:0/1787771122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d701f71a0 con 0x7f3d70074a20 2024-09-17T05:04:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6fe00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d701542b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3d6001af70 tx=0x7f3d600a5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.933+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600bb070 con 0x7f3d70074a20 2024-09-17T05:04:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d600043b0 con 0x7f3d70074a20 2024-09-17T05:04:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d701f7480 con 0x7f3d70074a20 2024-09-17T05:04:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600b2040 con 0x7f3d70074a20 2024-09-17T05:04:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d701f7a40 con 0x7f3d70074a20 2024-09-17T05:04:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7015be60 con 0x7f3d70074a20 2024-09-17T05:04:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d600be070 con 0x7f3d70074a20 2024-09-17T05:04:11.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d6cc00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3d4c073200 0x7f3d4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f3d6012d850 con 0x7f3d70074a20 2024-09-17T05:04:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.937+0000 7f3d67e00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3d4c073200 0x7f3d4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.941+0000 7f3d67e00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3d4c073200 0x7f3d4c075710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005fd0 tx=0x7f3d58005d80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:11.941+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3d600164b0 con 0x7f3d70074a20 2024-09-17T05:04:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.117+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3d70150070 con 0x7f3d70074a20 2024-09-17T05:04:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.121+0000 7f3d6cc00640 1 -- 172.21.15.16:0/1787771122 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f3d600f8a60 con 0x7f3d70074a20 2024-09-17T05:04:12.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3d4c073200 msgr2=0x7f3d4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3d4c073200 0x7f3d4c075710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005fd0 tx=0x7f3d58005d80 comp rx=0 tx=0).stop 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 msgr2=0x7f3d701542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d701542b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3d6001af70 tx=0x7f3d600a5540 comp rx=0 tx=0).stop 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 shutdown_connections 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3d4c073200 0x7f3d4c075710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3d7015ba50 0x7f3d701506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d70074e30 0x7f3d7014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 --2- 172.21.15.16:0/1787771122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d70074a20 0x7f3d701542b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 >> 172.21.15.16:0/1787771122 conn(0x7f3d700b2250 msgr2=0x7f3d7015cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 shutdown_connections 2024-09-17T05:04:12.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.125+0000 7f3d76c00640 1 -- 172.21.15.16:0/1787771122 wait complete. 2024-09-17T05:04:12.148 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:04:12.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:12.244 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:12.335 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:04:12.335 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:12.335 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:12.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:04:12.344 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:12.344 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:12.353 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:04:12.354 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:12.354 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:12.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:04:12.394 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:12.394 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:12.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:04:12.404 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:04:12.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:12.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 -- 172.21.15.16:0/1397680250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4815c270 msgr2=0x7f1f4815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 --2- 172.21.15.16:0/1397680250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4815c270 0x7f1f4815c680 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f4815cc70 tx=0x7f1f3001c830 comp rx=0 tx=0).stop 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 -- 172.21.15.16:0/1397680250 shutdown_connections 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 --2- 172.21.15.16:0/1397680250 >> v2:172.21.15.104:3300/0 conn(0x7f1f48073240 0x7f1f48073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 --2- 172.21.15.16:0/1397680250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f4815c270 0x7f1f4815c680 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 -- 172.21.15.16:0/1397680250 >> 172.21.15.16:0/1397680250 conn(0x7f1f4806f900 msgr2=0x7f1f480b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 -- 172.21.15.16:0/1397680250 shutdown_connections 2024-09-17T05:04:12.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 -- 172.21.15.16:0/1397680250 wait complete. 2024-09-17T05:04:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.593+0000 7f1f4da00640 1 Processor -- start 2024-09-17T05:04:12.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 -- start start 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 0x7f1f48157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f480b2f50 0x7f1f4815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1f4815c270 0x7f1f48160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48168c70 con 0x7f1f48073240 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f48168af0 con 0x7f1f480b2f50 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1f48168df0 con 0x7f1f4815c270 2024-09-17T05:04:12.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 0x7f1f48157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 0x7f1f48157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38568/0 (socket says 172.21.15.16:38568) 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 -- 172.21.15.16:0/152109205 learned_addr learned my addr 172.21.15.16:0/152109205 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f47400640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1f4815c270 0x7f1f48160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f3e000640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f480b2f50 0x7f1f4815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 -- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1f4815c270 msgr2=0x7f1f48160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1f4815c270 0x7f1f48160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 -- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f480b2f50 msgr2=0x7f1f4815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f480b2f50 0x7f1f4815a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 -- 172.21.15.16:0/152109205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f480b4290 con 0x7f1f48073240 2024-09-17T05:04:12.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f46a00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 0x7f1f48157270 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800c3c0 tx=0x7f1f3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:12.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38017070 con 0x7f1f48073240 2024-09-17T05:04:12.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f380053c0 con 0x7f1f48073240 2024-09-17T05:04:12.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38003860 con 0x7f1f48073240 2024-09-17T05:04:12.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f48157be0 con 0x7f1f48073240 2024-09-17T05:04:12.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.597+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f481581a0 con 0x7f1f48073240 2024-09-17T05:04:12.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.601+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f48073650 con 0x7f1f48073240 2024-09-17T05:04:12.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.601+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f3801a070 con 0x7f1f48073240 2024-09-17T05:04:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.601+0000 7f1f3f400640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1f24073200 0x7f1f24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:12.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.601+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7f1f38093400 con 0x7f1f48073240 2024-09-17T05:04:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.601+0000 7f1f3e000640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1f24073200 0x7f1f24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:12.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.605+0000 7f1f3e000640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1f24073200 0x7f1f24075710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f300af000 tx=0x7f1f3001b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:12.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.605+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f3805dea0 con 0x7f1f48073240 2024-09-17T05:04:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.813+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f4815b4a0 con 0x7f1f48073240 2024-09-17T05:04:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.813+0000 7f1f3f400640 1 -- 172.21.15.16:0/152109205 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3917 (secure 0 0 0) 0x7f1f38091e20 con 0x7f1f48073240 2024-09-17T05:04:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:12.817 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":37,"btime":"2024-09-17T05:04:06:341424+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/4168180154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":4168180154},{"type":"v1","addr":"172.21.15.104:6837","nonce":4168180154}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1069473520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1069473520},{"type":"v1","addr":"172.21.15.16:6833","nonce":1069473520}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5772,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1160394947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1160394947},{"type":"v1","addr":"172.21.15.16:6835","nonce":1160394947}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/3777166339","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3777166339},{"type":"v1","addr":"172.21.15.104:6836","nonce":3777166339}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-09-17T05:04:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 37 2024-09-17T05:04:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1f24073200 msgr2=0x7f1f24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1f24073200 0x7f1f24075710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f300af000 tx=0x7f1f3001b070 comp rx=0 tx=0).stop 2024-09-17T05:04:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 msgr2=0x7f1f48157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 0x7f1f48157270 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800c3c0 tx=0x7f1f3800c880 comp rx=0 tx=0).stop 2024-09-17T05:04:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 shutdown_connections 2024-09-17T05:04:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1f24073200 0x7f1f24075710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1f4815c270 0x7f1f48160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f480b2f50 0x7f1f4815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 --2- 172.21.15.16:0/152109205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f48073240 0x7f1f48157270 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 >> 172.21.15.16:0/152109205 conn(0x7f1f4806f900 msgr2=0x7f1f4815e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 shutdown_connections 2024-09-17T05:04:12.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:12.817+0000 7f1f4da00640 1 -- 172.21.15.16:0/152109205 wait complete. 2024-09-17T05:04:12.840 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:04:12.840 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:04:12.840 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:04:12.840 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:12.840 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:04:12.840 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:04:12.841 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:04:12.841 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:12.841 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:04:12.841 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:04:12.841 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:04:12.841 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:12.841 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:04:12.841 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:04:12.842 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:04:12.842 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:12.857 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:04:12.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:04:12.860 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:04:12.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:04:12.861 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:04:12.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:04:12.863 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:04:12.863 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:04:12.904 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:04:12.904 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:12.905 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:04:12.906 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:12.906 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:04:12.907 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:12.909 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:04:12.909 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:12.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:04:13.008 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:04:13.008 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:04:13.009 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:04:13.009 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:04:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 -- 172.21.15.16:0/3088423238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 msgr2=0x7fa2f40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 --2- 172.21.15.16:0/3088423238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f40ce5d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40ce2c0 tx=0x7fa2e001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 -- 172.21.15.16:0/3088423238 shutdown_connections 2024-09-17T05:04:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 --2- 172.21.15.16:0/3088423238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2f415f6f0 0x7fa2f40ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 --2- 172.21.15.16:0/3088423238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f40ce5d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 -- 172.21.15.16:0/3088423238 >> 172.21.15.16:0/3088423238 conn(0x7fa2f40b2250 msgr2=0x7fa2f40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 -- 172.21.15.16:0/3088423238 shutdown_connections 2024-09-17T05:04:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.073+0000 7fa2fb400640 1 -- 172.21.15.16:0/3088423238 wait complete. 2024-09-17T05:04:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 Processor -- start 2024-09-17T05:04:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 -- start start 2024-09-17T05:04:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f4074a20 0x7fa2f4158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f41541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2f415f6f0 0x7fa2f41f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4168e80 con 0x7fa2f4074e30 2024-09-17T05:04:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f4168d00 con 0x7fa2f4074a20 2024-09-17T05:04:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f4169000 con 0x7fa2f415f6f0 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f41541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f41541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38610/0 (socket says 172.21.15.16:38610) 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 -- 172.21.15.16:0/1147573466 learned_addr learned my addr 172.21.15.16:0/1147573466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f8a00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f4074a20 0x7fa2f4158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f9400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2f415f6f0 0x7fa2f41f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 -- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2f415f6f0 msgr2=0x7fa2f41f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2f415f6f0 0x7fa2f41f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 -- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f4074a20 msgr2=0x7fa2f4158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f4074a20 0x7fa2f4158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 -- 172.21.15.16:0/1147573466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f40ca320 con 0x7fa2f4074e30 2024-09-17T05:04:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f3e00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f41541e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa2e001a510 tx=0x7fa2e0005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00b2040 con 0x7fa2f4074e30 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2e0004330 con 0x7fa2f4074e30 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2f4154720 con 0x7fa2f4074e30 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0005290 con 0x7fa2f4074e30 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.077+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2f40ca040 con 0x7fa2f4074e30 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.081+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2f415efe0 con 0x7fa2f4074e30 2024-09-17T05:04:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.081+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa2e0005430 con 0x7fa2f4074e30 2024-09-17T05:04:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.081+0000 7fa2f0c00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2d4073200 0x7fa2d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.081+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5335+0+0 (secure 0 0 0) 0x7fa2e012dad0 con 0x7fa2f4074e30 2024-09-17T05:04:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.081+0000 7fa2f8a00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2d4073200 0x7fa2d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.085+0000 7fa2f8a00640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2d4073200 0x7fa2d4075710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8007fd0 tx=0x7fa2e8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:13.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.085+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa2e00164d0 con 0x7fa2f4074e30 2024-09-17T05:04:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.273+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa2f415dba0 con 0x7fa2f4074e30 2024-09-17T05:04:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2f0c00640 1 -- 172.21.15.16:0/1147573466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) ==== 97+0+0 (secure 0 0 0) 0x7fa2e00f8580 con 0x7fa2f4074e30 2024-09-17T05:04:13.386 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2d4073200 msgr2=0x7fa2d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2d4073200 0x7fa2d4075710 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8007fd0 tx=0x7fa2e8009040 comp rx=0 tx=0).stop 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 msgr2=0x7fa2f41541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f41541e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa2e001a510 tx=0x7fa2e0005e00 comp rx=0 tx=0).stop 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 shutdown_connections 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2d4073200 0x7fa2d4075710 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa2f415f6f0 0x7fa2f41f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2f4074e30 0x7fa2f41541e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 --2- 172.21.15.16:0/1147573466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f4074a20 0x7fa2f4158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 >> 172.21.15.16:0/1147573466 conn(0x7fa2f40b2250 msgr2=0x7fa2f415e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:13.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 shutdown_connections 2024-09-17T05:04:13.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.385+0000 7fa2fb400640 1 -- 172.21.15.16:0/1147573466 wait complete. 2024-09-17T05:04:13.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:04:13.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 -- 172.21.15.16:0/1866832363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406e7a0 msgr2=0x7fa8b40b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1866832363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406e7a0 0x7fa8b40b40d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009840 tx=0x7fa89c01c790 comp rx=0 tx=0).stop 2024-09-17T05:04:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 -- 172.21.15.16:0/1866832363 shutdown_connections 2024-09-17T05:04:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1866832363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406e7a0 0x7fa8b40b40d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 -- 172.21.15.16:0/1866832363 >> 172.21.15.16:0/1866832363 conn(0x7fa8b40d10c0 msgr2=0x7fa8b40d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 -- 172.21.15.16:0/1866832363 shutdown_connections 2024-09-17T05:04:13.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 -- 172.21.15.16:0/1866832363 wait complete. 2024-09-17T05:04:13.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 Processor -- start 2024-09-17T05:04:13.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.569+0000 7fa8b9000640 1 -- start start 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b406e7a0 0x7fa8b41504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406ebb0 0x7fa8b41509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 0x7fa8b4148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4169b00 con 0x7fa8b4152540 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b40b49f0 con 0x7fa8b406ebb0 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b4169c80 con 0x7fa8b406e7a0 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 0x7fa8b4148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 0x7fa8b4148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38622/0 (socket says 172.21.15.16:38622) 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b1e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b406e7a0 0x7fa8b41504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 -- 172.21.15.16:0/1326113215 learned_addr learned my addr 172.21.15.16:0/1326113215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8a9400640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406ebb0 0x7fa8b41509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 -- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b406e7a0 msgr2=0x7fa8b41504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b406e7a0 0x7fa8b41504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 -- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406ebb0 msgr2=0x7fa8b41509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406ebb0 0x7fa8b41509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 -- 172.21.15.16:0/1326113215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b41f9940 con 0x7fa8b4152540 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8a9400640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406ebb0 0x7fa8b41509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:13.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b2800640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 0x7fa8b4148ae0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00cb10 tx=0x7fa8ac00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac016020 con 0x7fa8b4152540 2024-09-17T05:04:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ac00ce50 con 0x7fa8b4152540 2024-09-17T05:04:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b415a020 con 0x7fa8b4152540 2024-09-17T05:04:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac00ad80 con 0x7fa8b4152540 2024-09-17T05:04:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b4149740 con 0x7fa8b4152540 2024-09-17T05:04:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.573+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b40b40d0 con 0x7fa8b4152540 2024-09-17T05:04:13.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.577+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8ac019020 con 0x7fa8b4152540 2024-09-17T05:04:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.577+0000 7fa8aaa00640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.577+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7fa8ac093e50 con 0x7fa8b4152540 2024-09-17T05:04:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.577+0000 7fa8b1e00640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:13.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.577+0000 7fa8b1e00640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4007fd0 tx=0x7fa8a4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:13.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.581+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8ac020030 con 0x7fa8b4152540 2024-09-17T05:04:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.817+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa8b4149970 con 0x7fa8b4152540 2024-09-17T05:04:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.817+0000 7fa8aaa00640 1 -- 172.21.15.16:0/1326113215 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) ==== 80+0+5304 (secure 0 0 0) 0x7fa8ac05e9d0 con 0x7fa8b4152540 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.822 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.823 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.824 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.825 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:04:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.827 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.828 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:13.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.830 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:04:13.832 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.833 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:13.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:04:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:04:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:13.835 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:04:13.835 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 msgr2=0x7fa890075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4007fd0 tx=0x7fa8a4009040 comp rx=0 tx=0).stop 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 msgr2=0x7fa8b4148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 0x7fa8b4148ae0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00cb10 tx=0x7fa8ac00cc00 comp rx=0 tx=0).stop 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 shutdown_connections 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b4152540 0x7fa8b4148ae0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b406ebb0 0x7fa8b41509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 --2- 172.21.15.16:0/1326113215 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b406e7a0 0x7fa8b41504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 >> 172.21.15.16:0/1326113215 conn(0x7fa8b40d10c0 msgr2=0x7fa8b406d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 shutdown_connections 2024-09-17T05:04:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.821+0000 7fa8b9000640 1 -- 172.21.15.16:0/1326113215 wait complete. 2024-09-17T05:04:13.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:04:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 -- 172.21.15.16:0/2051923368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc15d1c0 msgr2=0x7f80cc15dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 --2- 172.21.15.16:0/2051923368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc15d1c0 0x7f80cc15dbc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f80cc073690 tx=0x7f80c401cb80 comp rx=0 tx=0).stop 2024-09-17T05:04:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 -- 172.21.15.16:0/2051923368 shutdown_connections 2024-09-17T05:04:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 --2- 172.21.15.16:0/2051923368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc15d1c0 0x7f80cc15dbc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 --2- 172.21.15.16:0/2051923368 >> v2:172.21.15.104:3300/0 conn(0x7f80cc0ce630 0x7f80cc073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 -- 172.21.15.16:0/2051923368 >> 172.21.15.16:0/2051923368 conn(0x7f80cc0d10c0 msgr2=0x7f80cc0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 -- 172.21.15.16:0/2051923368 shutdown_connections 2024-09-17T05:04:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:13.997+0000 7f80d3600640 1 -- 172.21.15.16:0/2051923368 wait complete. 2024-09-17T05:04:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 Processor -- start 2024-09-17T05:04:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 -- start start 2024-09-17T05:04:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80cc074400 0x7f80cc1fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 0x7f80cc0af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80cc15d1c0 0x7f80cc0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc169dc0 con 0x7f80cc0ce630 2024-09-17T05:04:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc169c40 con 0x7f80cc15d1c0 2024-09-17T05:04:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d3600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f80cc169f40 con 0x7f80cc074400 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 0x7f80cc0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d0c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80cc074400 0x7f80cc1fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 0x7f80cc0af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38646/0 (socket says 172.21.15.16:38646) 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 -- 172.21.15.16:0/2808737842 learned_addr learned my addr 172.21.15.16:0/2808737842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80d1600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80cc15d1c0 0x7f80cc0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 -- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80cc074400 msgr2=0x7f80cc1fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80cc074400 0x7f80cc1fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 -- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80cc15d1c0 msgr2=0x7f80cc0b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80cc15d1c0 0x7f80cc0b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.001+0000 7f80cbe00640 1 -- 172.21.15.16:0/2808737842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1fa220 con 0x7f80cc0ce630 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80cbe00640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 0x7f80cc0af730 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f80b8001ec0 tx=0x7f80b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8017070 con 0x7f80cc0ce630 2024-09-17T05:04:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80b8003600 con 0x7f80cc0ce630 2024-09-17T05:04:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1fa500 con 0x7f80cc0ce630 2024-09-17T05:04:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003a50 con 0x7f80cc0ce630 2024-09-17T05:04:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1faac0 con 0x7f80cc0ce630 2024-09-17T05:04:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc073ec0 con 0x7f80cc0ce630 2024-09-17T05:04:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.005+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f80b801a070 con 0x7f80cc0ce630 2024-09-17T05:04:14.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.009+0000 7f80c8c00640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f80ac073230 0x7f80ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.009+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7f80b80932c0 con 0x7f80cc0ce630 2024-09-17T05:04:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.009+0000 7f80d0c00640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f80ac073230 0x7f80ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.009+0000 7f80d0c00640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f80ac073230 0x7f80ac075740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009b40 tx=0x7f80c00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:14.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.013+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f80cc074810 con 0x7f80cc0ce630 2024-09-17T05:04:14.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.185+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f80cc15dee0 con 0x7f80cc0ce630 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80c8c00640 1 -- 172.21.15.16:0/2808737842 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f80b805de50 con 0x7f80cc0ce630 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:04:14.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:14.191 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f80ac073230 msgr2=0x7f80ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f80ac073230 0x7f80ac075740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009b40 tx=0x7f80c00091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 msgr2=0x7f80cc0af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 0x7f80cc0af730 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f80b8001ec0 tx=0x7f80b800c890 comp rx=0 tx=0).stop 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 shutdown_connections 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f80ac073230 0x7f80ac075740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80cc15d1c0 0x7f80cc0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80cc0ce630 0x7f80cc0af730 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 --2- 172.21.15.16:0/2808737842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f80cc074400 0x7f80cc1fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 >> 172.21.15.16:0/2808737842 conn(0x7f80cc0d10c0 msgr2=0x7f80cc0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 shutdown_connections 2024-09-17T05:04:14.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.189+0000 7f80d3600640 1 -- 172.21.15.16:0/2808737842 wait complete. 2024-09-17T05:04:14.212 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:04:14.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:04:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 -- 172.21.15.16:0/3467998045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c074bf0 msgr2=0x7efd0c15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 --2- 172.21.15.16:0/3467998045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c074bf0 0x7efd0c15f6b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7efd04009c80 tx=0x7efd0401cd70 comp rx=0 tx=0).stop 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 -- 172.21.15.16:0/3467998045 shutdown_connections 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 --2- 172.21.15.16:0/3467998045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c074bf0 0x7efd0c15f6b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 --2- 172.21.15.16:0/3467998045 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd0c0b4620 0x7efd0c06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 -- 172.21.15.16:0/3467998045 >> 172.21.15.16:0/3467998045 conn(0x7efd0c0d0050 msgr2=0x7efd0c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 -- 172.21.15.16:0/3467998045 shutdown_connections 2024-09-17T05:04:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 -- 172.21.15.16:0/3467998045 wait complete. 2024-09-17T05:04:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 Processor -- start 2024-09-17T05:04:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.373+0000 7efd11400640 1 -- start start 2024-09-17T05:04:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd0c06e4e0 0x7efd0c142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd0c074bf0 0x7efd0c1433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 0x7efd0c1f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c168920 con 0x7efd0c0b4620 2024-09-17T05:04:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efd0c15fe00 con 0x7efd0c06e4e0 2024-09-17T05:04:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efd0c168aa0 con 0x7efd0c074bf0 2024-09-17T05:04:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 0x7efd0c1f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd0c06e4e0 0x7efd0c142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 0x7efd0c1f99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38668/0 (socket says 172.21.15.16:38668) 2024-09-17T05:04:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 -- 172.21.15.16:0/1405280599 learned_addr learned my addr 172.21.15.16:0/1405280599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd01e00640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd0c074bf0 0x7efd0c1433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 -- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd0c074bf0 msgr2=0x7efd0c1433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd0c074bf0 0x7efd0c1433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 -- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd0c06e4e0 msgr2=0x7efd0c142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd0c06e4e0 0x7efd0c142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 -- 172.21.15.16:0/1405280599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0c1fa780 con 0x7efd0c0b4620 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0a800640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd0c06e4e0 0x7efd0c142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd0b200640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 0x7efd0c1f99a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efd0401b420 tx=0x7efd04005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd040bb070 con 0x7efd0c0b4620 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd04004b40 con 0x7efd0c0b4620 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0c158dc0 con 0x7efd0c0b4620 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd040b2040 con 0x7efd0c0b4620 2024-09-17T05:04:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.377+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd0c09cdd0 con 0x7efd0c0b4620 2024-09-17T05:04:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.381+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0c1402f0 con 0x7efd0c0b4620 2024-09-17T05:04:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.381+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd040be070 con 0x7efd0c0b4620 2024-09-17T05:04:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.381+0000 7efd03400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efce8073200 0x7efce8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.381+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7efd0412de40 con 0x7efd0c0b4620 2024-09-17T05:04:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.381+0000 7efd0a800640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efce8073200 0x7efce8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.381+0000 7efd0a800640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efce8073200 0x7efce8075710 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009f30 tx=0x7efcfc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.385+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd0c1402f0 con 0x7efd0c0b4620 2024-09-17T05:04:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:14.565+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7efd0c0d07e0 con 0x7efd0c0b4620 2024-09-17T05:04:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.401+0000 7efd03400640 1 -- 172.21.15.16:0/1405280599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7efd04016990 con 0x7efd0c0b4620 2024-09-17T05:04:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:04:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efce8073200 msgr2=0x7efce8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efce8073200 0x7efce8075710 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009f30 tx=0x7efcfc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 msgr2=0x7efd0c1f99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 0x7efd0c1f99a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7efd0401b420 tx=0x7efd04005b00 comp rx=0 tx=0).stop 2024-09-17T05:04:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 shutdown_connections 2024-09-17T05:04:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efce8073200 0x7efce8075710 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd0c0b4620 0x7efd0c1f99a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd0c074bf0 0x7efd0c1433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 --2- 172.21.15.16:0/1405280599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd0c06e4e0 0x7efd0c142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 >> 172.21.15.16:0/1405280599 conn(0x7efd0c0d0050 msgr2=0x7efd0c151a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 shutdown_connections 2024-09-17T05:04:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.405+0000 7efd11400640 1 -- 172.21.15.16:0/1405280599 wait complete. 2024-09-17T05:04:15.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:04:15.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 -- 172.21.15.16:0/1415578232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8068073240 msgr2=0x7f8068073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:15.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 --2- 172.21.15.16:0/1415578232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8068073240 0x7f8068073650 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8060009c80 tx=0x7f806001cd10 comp rx=0 tx=0).stop 2024-09-17T05:04:15.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 -- 172.21.15.16:0/1415578232 shutdown_connections 2024-09-17T05:04:15.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 --2- 172.21.15.16:0/1415578232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8068073240 0x7f8068073650 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 --2- 172.21.15.16:0/1415578232 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f806815c270 0x7f806815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 -- 172.21.15.16:0/1415578232 >> 172.21.15.16:0/1415578232 conn(0x7f806806f900 msgr2=0x7f80680b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:15.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 -- 172.21.15.16:0/1415578232 shutdown_connections 2024-09-17T05:04:15.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.609+0000 7f806f800640 1 -- 172.21.15.16:0/1415578232 wait complete. 2024-09-17T05:04:15.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 Processor -- start 2024-09-17T05:04:15.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 -- start start 2024-09-17T05:04:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8068073240 0x7f8068140410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 0x7f8068140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f806815c270 0x7f8068151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8068168e00 con 0x7f80680b2f50 2024-09-17T05:04:15.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8068168c80 con 0x7f806815c270 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8068168f80 con 0x7f8068073240 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 0x7f8068140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 0x7f8068140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38678/0 (socket says 172.21.15.16:38678) 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 -- 172.21.15.16:0/3326459825 learned_addr learned my addr 172.21.15.16:0/3326459825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806d800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f806815c270 0x7f8068151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806ce00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8068073240 0x7f8068140410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 -- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8068073240 msgr2=0x7f8068140410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8068073240 0x7f8068140410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 -- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f806815c270 msgr2=0x7f8068151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f806815c270 0x7f8068151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 -- 172.21.15.16:0/3326459825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8068150b30 con 0x7f80680b2f50 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806d800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f806815c270 0x7f8068151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806ce00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8068073240 0x7f8068140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f805fe00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 0x7f8068140950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f805000e9f0 tx=0x7f805000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805000ccc0 con 0x7f80680b2f50 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8050004020 con 0x7f80680b2f50 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805000ce60 con 0x7f80680b2f50 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8068154810 con 0x7f80680b2f50 2024-09-17T05:04:15.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.613+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8068154dd0 con 0x7f80680b2f50 2024-09-17T05:04:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.617+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8068073650 con 0x7f80680b2f50 2024-09-17T05:04:15.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.617+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8050003600 con 0x7f80680b2f50 2024-09-17T05:04:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.617+0000 7f8065600640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8044073200 0x7f8044075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.617+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5619+0+0 (secure 0 0 0) 0x7f8050094af0 con 0x7f80680b2f50 2024-09-17T05:04:15.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.617+0000 7f806ce00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8044073200 0x7f8044075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.621+0000 7f806ce00640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8044073200 0x7f8044075710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80580016b0 tx=0x7f8058009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.625+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8068150b30 con 0x7f80680b2f50 2024-09-17T05:04:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:15.797+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8068069500 con 0x7f80680b2f50 2024-09-17T05:04:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.409+0000 7f8065600640 1 -- 172.21.15.16:0/3326459825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f805005fb60 con 0x7f80680b2f50 2024-09-17T05:04:16.413 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8044073200 msgr2=0x7f8044075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8044073200 0x7f8044075710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f80580016b0 tx=0x7f8058009040 comp rx=0 tx=0).stop 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 msgr2=0x7f8068140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 0x7f8068140950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f805000e9f0 tx=0x7f805000eeb0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 shutdown_connections 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8044073200 0x7f8044075710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f806815c270 0x7f8068151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80680b2f50 0x7f8068140950 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 --2- 172.21.15.16:0/3326459825 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8068073240 0x7f8068140410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 >> 172.21.15.16:0/3326459825 conn(0x7f806806f900 msgr2=0x7f80681659b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 shutdown_connections 2024-09-17T05:04:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.413+0000 7f806f800640 1 -- 172.21.15.16:0/3326459825 wait complete. 2024-09-17T05:04:16.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 -- 172.21.15.16:0/3735197065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c06ce90 msgr2=0x7f3e4c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 --2- 172.21.15.16:0/3735197065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c06d2a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c06ea60 tx=0x7f3e4401cac0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 -- 172.21.15.16:0/3735197065 shutdown_connections 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 --2- 172.21.15.16:0/3735197065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c06d2a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 --2- 172.21.15.16:0/3735197065 >> v2:172.21.15.104:3300/0 conn(0x7f3e4c0cb350 0x7f3e4c0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 -- 172.21.15.16:0/3735197065 >> 172.21.15.16:0/3735197065 conn(0x7f3e4c0cd980 msgr2=0x7f3e4c0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 -- 172.21.15.16:0/3735197065 shutdown_connections 2024-09-17T05:04:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.621+0000 7f3e50a00640 1 -- 172.21.15.16:0/3735197065 wait complete. 2024-09-17T05:04:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 Processor -- start 2024-09-17T05:04:16.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 -- start start 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c14b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e4c06ff20 0x7f3e4c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 0x7f3e4c0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c15d110 con 0x7f3e4c0cb350 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c15cf90 con 0x7f3e4c06ce90 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4c15d290 con 0x7f3e4c06ff20 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 0x7f3e4c0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 0x7f3e4c0b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38690/0 (socket says 172.21.15.16:38690) 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 -- 172.21.15.16:0/2540028549 learned_addr learned my addr 172.21.15.16:0/2540028549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e49e00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c14b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e43e00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e4c06ff20 0x7f3e4c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 -- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e4c06ff20 msgr2=0x7f3e4c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e4c06ff20 0x7f3e4c0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 -- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e4c06ce90 msgr2=0x7f3e4c14b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c14b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 -- 172.21.15.16:0/2540028549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c1555a0 con 0x7f3e4c0cb350 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e49e00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c14b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e43e00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e4c06ff20 0x7f3e4c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e4a800640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 0x7f3e4c0b10e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3e4401c530 tx=0x7f3e44005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b2040 con 0x7f3e4c0cb350 2024-09-17T05:04:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e440bb070 con 0x7f3e4c0cb350 2024-09-17T05:04:16.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c155880 con 0x7f3e4c0cb350 2024-09-17T05:04:16.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44004530 con 0x7f3e4c0cb350 2024-09-17T05:04:16.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.625+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c14d8c0 con 0x7f3e4c0cb350 2024-09-17T05:04:16.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.629+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c15d740 con 0x7f3e4c0cb350 2024-09-17T05:04:16.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.629+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3e440046d0 con 0x7f3e4c0cb350 2024-09-17T05:04:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.629+0000 7f3e42000640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e28073200 0x7f3e28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:16.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.629+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6001+0+0 (secure 0 0 0) 0x7f3e4412f850 con 0x7f3e4c0cb350 2024-09-17T05:04:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.629+0000 7f3e49e00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e28073200 0x7f3e28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.629+0000 7f3e49e00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e28073200 0x7f3e28075710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ac0 tx=0x7f3e3c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:16.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.633+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e44016770 con 0x7f3e4c0cb350 2024-09-17T05:04:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:16.841+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3e4c06a180 con 0x7f3e4c0cb350 2024-09-17T05:04:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.413+0000 7f3e42000640 1 -- 172.21.15.16:0/2540028549 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) ==== 174+0+0 (secure 0 0 0) 0x7f3e440fa7b0 con 0x7f3e4c0cb350 2024-09-17T05:04:17.416 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 11 and data pool 12 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e28073200 msgr2=0x7f3e28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e28073200 0x7f3e28075710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ac0 tx=0x7f3e3c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 msgr2=0x7f3e4c0b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 0x7f3e4c0b10e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3e4401c530 tx=0x7f3e44005a60 comp rx=0 tx=0).stop 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 shutdown_connections 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e28073200 0x7f3e28075710 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e4c0cb350 0x7f3e4c0b10e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e4c06ff20 0x7f3e4c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 --2- 172.21.15.16:0/2540028549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e4c06ce90 0x7f3e4c14b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 >> 172.21.15.16:0/2540028549 conn(0x7f3e4c0cd980 msgr2=0x7f3e4c06e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:17.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 shutdown_connections 2024-09-17T05:04:17.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.417+0000 7f3e50a00640 1 -- 172.21.15.16:0/2540028549 wait complete. 2024-09-17T05:04:17.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 -- 172.21.15.16:0/415877875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b4073240 msgr2=0x7f38b4073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 --2- 172.21.15.16:0/415877875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b4073240 0x7f38b4073650 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009c80 tx=0x7f38ac01cd10 comp rx=0 tx=0).stop 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 -- 172.21.15.16:0/415877875 shutdown_connections 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 --2- 172.21.15.16:0/415877875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b4073240 0x7f38b4073650 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 --2- 172.21.15.16:0/415877875 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38b40b2f50 0x7f38b415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 -- 172.21.15.16:0/415877875 >> 172.21.15.16:0/415877875 conn(0x7f38b406f900 msgr2=0x7f38b40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:17.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 -- 172.21.15.16:0/415877875 shutdown_connections 2024-09-17T05:04:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.629+0000 7f38bc000640 1 -- 172.21.15.16:0/415877875 wait complete. 2024-09-17T05:04:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 Processor -- start 2024-09-17T05:04:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 -- start start 2024-09-17T05:04:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38b4073240 0x7f38b41f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 0x7f38b40cee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38b415c0a0 0x7f38b40cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4168d60 con 0x7f38b40b2f50 2024-09-17T05:04:17.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38b4168be0 con 0x7f38b415c0a0 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38bc000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f38b4168ee0 con 0x7f38b4073240 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 0x7f38b40cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b9600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38b4073240 0x7f38b41f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38ba000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38b415c0a0 0x7f38b40cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 0x7f38b40cee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48092/0 (socket says 172.21.15.16:48092) 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 -- 172.21.15.16:0/2099284307 learned_addr learned my addr 172.21.15.16:0/2099284307 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 -- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38b4073240 msgr2=0x7f38b41f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38b4073240 0x7f38b41f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 -- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38b415c0a0 msgr2=0x7f38b40cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38b415c0a0 0x7f38b40cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 -- 172.21.15.16:0/2099284307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b415b6f0 con 0x7f38b40b2f50 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38ba000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38b415c0a0 0x7f38b40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.633+0000 7f38b8c00640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 0x7f38b40cee90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f38a000c3f0 tx=0x7f38a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017070 con 0x7f38b40b2f50 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38a00053c0 con 0x7f38b40b2f50 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0003860 con 0x7f38b40b2f50 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b4146540 con 0x7f38b40b2f50 2024-09-17T05:04:17.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b40cff40 con 0x7f38b40b2f50 2024-09-17T05:04:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b4073650 con 0x7f38b40b2f50 2024-09-17T05:04:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f38a001a070 con 0x7f38b40b2f50 2024-09-17T05:04:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b1600640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3894073200 0x7f3894075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f38a0093bb0 con 0x7f38b40b2f50 2024-09-17T05:04:17.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.637+0000 7f38b9600640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3894073200 0x7f3894075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:17.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.641+0000 7f38b9600640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3894073200 0x7f3894075710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f38a800bb50 tx=0x7f38a800b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:17.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.641+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f38a0021020 con 0x7f38b40b2f50 2024-09-17T05:04:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.817+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38b415c550 con 0x7f38b40b2f50 2024-09-17T05:04:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38b1600640 1 -- 172.21.15.16:0/2099284307 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16946 (secure 0 0 0) 0x7f38b4146540 con 0x7f38b40b2f50 2024-09-17T05:04:17.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:17.822 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:16.846581+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:14.578846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:15.811287+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3894073200 msgr2=0x7f3894075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3894073200 0x7f3894075710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f38a800bb50 tx=0x7f38a800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 msgr2=0x7f38b40cee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 0x7f38b40cee90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f38a000c3f0 tx=0x7f38a000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 shutdown_connections 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3894073200 0x7f3894075710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38b415c0a0 0x7f38b40cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b40b2f50 0x7f38b40cee90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 --2- 172.21.15.16:0/2099284307 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38b4073240 0x7f38b41f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 >> 172.21.15.16:0/2099284307 conn(0x7f38b406f900 msgr2=0x7f38b40b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 shutdown_connections 2024-09-17T05:04:17.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:17.821+0000 7f38bc000640 1 -- 172.21.15.16:0/2099284307 wait complete. 2024-09-17T05:04:17.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 -- 172.21.15.16:0/4095781634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf20165170 msgr2=0x7fdf20167590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/4095781634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf20165170 0x7fdf20167590 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdf20160ed0 tx=0x7fdf1801cba0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 -- 172.21.15.16:0/4095781634 shutdown_connections 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/4095781634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf20165170 0x7fdf20167590 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 -- 172.21.15.16:0/4095781634 >> 172.21.15.16:0/4095781634 conn(0x7fdf20073cf0 msgr2=0x7fdf20074100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 -- 172.21.15.16:0/4095781634 shutdown_connections 2024-09-17T05:04:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.001+0000 7fdf1fe00640 1 -- 172.21.15.16:0/4095781634 wait complete. 2024-09-17T05:04:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 Processor -- start 2024-09-17T05:04:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 -- start start 2024-09-17T05:04:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 0x7fdf2014c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf20160230 0x7fdf201566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf20165170 0x7fdf200b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2016ae70 con 0x7fdf2006efb0 2024-09-17T05:04:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2016acf0 con 0x7fdf20160230 2024-09-17T05:04:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1fe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2016aff0 con 0x7fdf20165170 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 0x7fdf2014c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 0x7fdf2014c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48104/0 (socket says 172.21.15.16:48104) 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 -- 172.21.15.16:0/2854522819 learned_addr learned my addr 172.21.15.16:0/2854522819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1f400640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf20165170 0x7fdf200b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1e000640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf20160230 0x7fdf201566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 -- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf20165170 msgr2=0x7fdf200b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf20165170 0x7fdf200b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 -- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf20160230 msgr2=0x7fdf201566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf20160230 0x7fdf201566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.005+0000 7fdf1ea00640 1 -- 172.21.15.16:0/2854522819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf202006b0 con 0x7fdf2006efb0 2024-09-17T05:04:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf1ea00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 0x7fdf2014c5f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000c3f0 tx=0x7fdf1000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10017070 con 0x7fdf2006efb0 2024-09-17T05:04:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf10003600 con 0x7fdf2006efb0 2024-09-17T05:04:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10003a50 con 0x7fdf2006efb0 2024-09-17T05:04:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf20200990 con 0x7fdf2006efb0 2024-09-17T05:04:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf20200f50 con 0x7fdf2006efb0 2024-09-17T05:04:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf20160640 con 0x7fdf2006efb0 2024-09-17T05:04:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf1001a070 con 0x7fdf2006efb0 2024-09-17T05:04:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.009+0000 7fdf16a00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf00073230 0x7fdf00075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.013+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fdf10093470 con 0x7fdf2006efb0 2024-09-17T05:04:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.013+0000 7fdf1e000640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf00073230 0x7fdf00075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.013+0000 7fdf1e000640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf00073230 0x7fdf00075740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf2015ff40 tx=0x7fdf0c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.017+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf1005ddd0 con 0x7fdf2006efb0 2024-09-17T05:04:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf20074440 con 0x7fdf2006efb0 2024-09-17T05:04:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf16a00640 1 -- 172.21.15.16:0/2854522819 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16946 (secure 0 0 0) 0x7fdf10062800 con 0x7fdf2006efb0 2024-09-17T05:04:18.195 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:18.195 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:16.846581+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:14.578846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:15.811287+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf00073230 msgr2=0x7fdf00075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf00073230 0x7fdf00075740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf2015ff40 tx=0x7fdf0c009040 comp rx=0 tx=0).stop 2024-09-17T05:04:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 msgr2=0x7fdf2014c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 0x7fdf2014c5f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000c3f0 tx=0x7fdf1000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 shutdown_connections 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdf00073230 0x7fdf00075740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf20165170 0x7fdf200b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf20160230 0x7fdf201566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 --2- 172.21.15.16:0/2854522819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2006efb0 0x7fdf2014c5f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 >> 172.21.15.16:0/2854522819 conn(0x7fdf20073cf0 msgr2=0x7fdf2015ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 shutdown_connections 2024-09-17T05:04:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.193+0000 7fdf1fe00640 1 -- 172.21.15.16:0/2854522819 wait complete. 2024-09-17T05:04:18.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:04:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 -- 172.21.15.16:0/27968483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b0159f70 msgr2=0x7fa8b00d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/27968483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b0159f70 0x7fa8b00d1b80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009870 tx=0x7fa8a001c760 comp rx=0 tx=0).stop 2024-09-17T05:04:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 -- 172.21.15.16:0/27968483 shutdown_connections 2024-09-17T05:04:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/27968483 >> v2:172.21.15.104:3300/0 conn(0x7fa8b00cf850 0x7fa8b00b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/27968483 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b0159f70 0x7fa8b00d1b80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 -- 172.21.15.16:0/27968483 >> 172.21.15.16:0/27968483 conn(0x7fa8b00b42d0 msgr2=0x7fa8b00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 -- 172.21.15.16:0/27968483 shutdown_connections 2024-09-17T05:04:18.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.377+0000 7fa8b6e00640 1 -- 172.21.15.16:0/27968483 wait complete. 2024-09-17T05:04:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 Processor -- start 2024-09-17T05:04:18.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 -- start start 2024-09-17T05:04:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 0x7fa8b014e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b0159f70 0x7fa8b01518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b014ea70 0x7fa8b0151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0169e20 con 0x7fa8b00d02d0 2024-09-17T05:04:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b0169ca0 con 0x7fa8b0159f70 2024-09-17T05:04:18.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b6e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b0169fa0 con 0x7fa8b014ea70 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 0x7fa8b014e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 0x7fa8b014e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48106/0 (socket says 172.21.15.16:48106) 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 -- 172.21.15.16:0/1347790980 learned_addr learned my addr 172.21.15.16:0/1347790980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8b4e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b014ea70 0x7fa8b0151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8af400640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b0159f70 0x7fa8b01518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 -- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b014ea70 msgr2=0x7fa8b0151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b014ea70 0x7fa8b0151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 -- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b0159f70 msgr2=0x7fa8b01518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b0159f70 0x7fa8b01518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8afe00640 1 -- 172.21.15.16:0/1347790980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b00719f0 con 0x7fa8b00d02d0 2024-09-17T05:04:18.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.381+0000 7fa8af400640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b0159f70 0x7fa8b01518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8afe00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 0x7fa8b014e2b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa8a001cc40 tx=0x7fa8a0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a00b2040 con 0x7fa8b00d02d0 2024-09-17T05:04:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8a0004040 con 0x7fa8b00d02d0 2024-09-17T05:04:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b0071cd0 con 0x7fa8b00d02d0 2024-09-17T05:04:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004490 con 0x7fa8b00d02d0 2024-09-17T05:04:18.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b0150a10 con 0x7fa8b00d02d0 2024-09-17T05:04:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b0071770 con 0x7fa8b00d02d0 2024-09-17T05:04:18.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.385+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8a00bd030 con 0x7fa8b00d02d0 2024-09-17T05:04:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.389+0000 7fa8a7e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.389+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fa8a012de50 con 0x7fa8b00d02d0 2024-09-17T05:04:18.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.389+0000 7fa8af400640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.389+0000 7fa8af400640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa89c007fd0 tx=0x7fa89c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:18.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.393+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8b0071770 con 0x7fa8b00d02d0 2024-09-17T05:04:18.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:18.605+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa8b00b2100 con 0x7fa8b00d02d0 2024-09-17T05:04:19.396 INFO:tasks.ceph.mds.b.smithi104.stderr:2024-09-17T05:04:19.397+0000 7f8e34e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:04:19.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8a7e00640 1 -- 172.21.15.16:0/1347790980 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7fa8a00c4060 con 0x7fa8b00d02d0 2024-09-17T05:04:19.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 msgr2=0x7fa890075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa89c007fd0 tx=0x7fa89c009040 comp rx=0 tx=0).stop 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 msgr2=0x7fa8b014e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 0x7fa8b014e2b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa8a001cc40 tx=0x7fa8a0005760 comp rx=0 tx=0).stop 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 shutdown_connections 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa890073200 0x7fa890075710 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8b014ea70 0x7fa8b0151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8b0159f70 0x7fa8b01518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 --2- 172.21.15.16:0/1347790980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8b00d02d0 0x7fa8b014e2b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 >> 172.21.15.16:0/1347790980 conn(0x7fa8b00b42d0 msgr2=0x7fa8b00cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 shutdown_connections 2024-09-17T05:04:19.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.441+0000 7fa8b6e00640 1 -- 172.21.15.16:0/1347790980 wait complete. 2024-09-17T05:04:19.471 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:04:19.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 -- 172.21.15.16:0/2642470940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774074a20 msgr2=0x7f977415efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 --2- 172.21.15.16:0/2642470940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774074a20 0x7f977415efe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f97740cca90 tx=0x7f976801c800 comp rx=0 tx=0).stop 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 -- 172.21.15.16:0/2642470940 shutdown_connections 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 --2- 172.21.15.16:0/2642470940 >> v2:172.21.15.104:3300/0 conn(0x7f9774074e30 0x7f977415f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 --2- 172.21.15.16:0/2642470940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774074a20 0x7f977415efe0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 -- 172.21.15.16:0/2642470940 >> 172.21.15.16:0/2642470940 conn(0x7f97740b2250 msgr2=0x7f97740d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 -- 172.21.15.16:0/2642470940 shutdown_connections 2024-09-17T05:04:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 -- 172.21.15.16:0/2642470940 wait complete. 2024-09-17T05:04:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 Processor -- start 2024-09-17T05:04:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.637+0000 7f977ba00640 1 -- start start 2024-09-17T05:04:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9774074a20 0x7f977415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9774074e30 0x7f97740c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 0x7f97740c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9774168e20 con 0x7f977415ba50 2024-09-17T05:04:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9774168ca0 con 0x7f9774074a20 2024-09-17T05:04:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9774168fa0 con 0x7f9774074e30 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 0x7f97740c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 0x7f97740c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48116/0 (socket says 172.21.15.16:48116) 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 -- 172.21.15.16:0/1527078311 learned_addr learned my addr 172.21.15.16:0/1527078311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779000640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9774074a20 0x7f977415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9773e00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9774074e30 0x7f97740c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 -- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9774074e30 msgr2=0x7f97740c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9774074e30 0x7f97740c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 -- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9774074a20 msgr2=0x7f977415a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9774074a20 0x7f977415a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 -- 172.21.15.16:0/1527078311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741f7620 con 0x7f977415ba50 2024-09-17T05:04:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9779a00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 0x7f97740c8db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c9f0 tx=0x7f976c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c016020 con 0x7f977415ba50 2024-09-17T05:04:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976c012920 con 0x7f977415ba50 2024-09-17T05:04:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c012d70 con 0x7f977415ba50 2024-09-17T05:04:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741f9fd0 con 0x7f977415ba50 2024-09-17T05:04:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741fa590 con 0x7f977415ba50 2024-09-17T05:04:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.641+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977415be60 con 0x7f977415ba50 2024-09-17T05:04:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.645+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f976c019020 con 0x7f977415ba50 2024-09-17T05:04:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.645+0000 7f9770c00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9754073200 0x7f9754075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.645+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f976c094410 con 0x7f977415ba50 2024-09-17T05:04:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.645+0000 7f9779000640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9754073200 0x7f9754075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.645+0000 7f9779000640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9754073200 0x7f9754075710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f97680af000 tx=0x7f976801b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.649+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f976c020020 con 0x7f977415ba50 2024-09-17T05:04:19.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:19.861+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f97740cd790 con 0x7f977415ba50 2024-09-17T05:04:20.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.445+0000 7f9770c00640 1 -- 172.21.15.16:0/1527078311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f976c05ec50 con 0x7f977415ba50 2024-09-17T05:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9754073200 msgr2=0x7f9754075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9754073200 0x7f9754075710 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f97680af000 tx=0x7f976801b040 comp rx=0 tx=0).stop 2024-09-17T05:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 msgr2=0x7f97740c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 0x7f97740c8db0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c9f0 tx=0x7f976c00cae0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 shutdown_connections 2024-09-17T05:04:20.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9754073200 0x7f9754075710 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f977415ba50 0x7f97740c8db0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9774074e30 0x7f97740c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 --2- 172.21.15.16:0/1527078311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9774074a20 0x7f977415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 >> 172.21.15.16:0/1527078311 conn(0x7f97740b2250 msgr2=0x7f97740ca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 shutdown_connections 2024-09-17T05:04:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.449+0000 7f977ba00640 1 -- 172.21.15.16:0/1527078311 wait complete. 2024-09-17T05:04:20.476 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:04:20.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 -- 172.21.15.16:0/23951886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0815f080 msgr2=0x7fdf0815f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/23951886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0815f080 0x7fdf0815f490 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdf08072860 tx=0x7fdef401c6e0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 -- 172.21.15.16:0/23951886 shutdown_connections 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/23951886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf0815f080 0x7fdf0815f490 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/23951886 >> v2:172.21.15.104:3300/0 conn(0x7fdf08074b00 0x7fdf080b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 -- 172.21.15.16:0/23951886 >> 172.21.15.16:0/23951886 conn(0x7fdf0806f900 msgr2=0x7fdf080b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 -- 172.21.15.16:0/23951886 shutdown_connections 2024-09-17T05:04:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 -- 172.21.15.16:0/23951886 wait complete. 2024-09-17T05:04:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 Processor -- start 2024-09-17T05:04:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.637+0000 7fdf0cc00640 1 -- start start 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 0x7fdf081fc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0815ba90 0x7fdf080cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf0815f080 0x7fdf08153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08168fc0 con 0x7fdf08074b00 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08168e40 con 0x7fdf0815ba90 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdf08169140 con 0x7fdf0815f080 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 0x7fdf081fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 0x7fdf081fc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48136/0 (socket says 172.21.15.16:48136) 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 -- 172.21.15.16:0/2296726768 learned_addr learned my addr 172.21.15.16:0/2296726768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05400640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0815ba90 0x7fdf080cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf06800640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf0815f080 0x7fdf08153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 -- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf0815f080 msgr2=0x7fdf08153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf0815f080 0x7fdf08153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 -- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0815ba90 msgr2=0x7fdf080cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0815ba90 0x7fdf080cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 -- 172.21.15.16:0/2296726768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf08155a30 con 0x7fdf08074b00 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05400640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0815ba90 0x7fdf080cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf05e00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 0x7fdf081fc990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdef80058d0 tx=0x7fdef800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8017070 con 0x7fdf08074b00 2024-09-17T05:04:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdef8009d20 con 0x7fdf08074b00 2024-09-17T05:04:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef80106c0 con 0x7fdf08074b00 2024-09-17T05:04:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf08155d10 con 0x7fdf08074b00 2024-09-17T05:04:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf081fc470 con 0x7fdf08074b00 2024-09-17T05:04:20.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.641+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf081fbf80 con 0x7fdf08074b00 2024-09-17T05:04:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.645+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdef801a070 con 0x7fdf08074b00 2024-09-17T05:04:20.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.645+0000 7fdefe000640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdee8073200 0x7fdee8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.645+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fdef8094620 con 0x7fdf08074b00 2024-09-17T05:04:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.645+0000 7fdf05400640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdee8073200 0x7fdee8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:20.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.645+0000 7fdf05400640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdee8073200 0x7fdee8075710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdef40af000 tx=0x7fdef401af20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:20.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.649+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdef805f410 con 0x7fdf08074b00 2024-09-17T05:04:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.857+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf080b2f50 con 0x7fdf08074b00 2024-09-17T05:04:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdefe000640 1 -- 172.21.15.16:0/2296726768 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7fdef805ed10 con 0x7fdf08074b00 2024-09-17T05:04:20.862 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:20.862 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdee8073200 msgr2=0x7fdee8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdee8073200 0x7fdee8075710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdef40af000 tx=0x7fdef401af20 comp rx=0 tx=0).stop 2024-09-17T05:04:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 msgr2=0x7fdf081fc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 0x7fdf081fc990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fdef80058d0 tx=0x7fdef800dde0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 shutdown_connections 2024-09-17T05:04:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdee8073200 0x7fdee8075710 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdf0815f080 0x7fdf08153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0815ba90 0x7fdf080cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 --2- 172.21.15.16:0/2296726768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf08074b00 0x7fdf081fc990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 >> 172.21.15.16:0/2296726768 conn(0x7fdf0806f900 msgr2=0x7fdf080b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 shutdown_connections 2024-09-17T05:04:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:20.861+0000 7fdf0cc00640 1 -- 172.21.15.16:0/2296726768 wait complete. 2024-09-17T05:04:20.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 -- 172.21.15.16:0/4026465478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b80b2f50 msgr2=0x7ff9b815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 --2- 172.21.15.16:0/4026465478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b80b2f50 0x7ff9b815bb60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009870 tx=0x7ff9a801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 -- 172.21.15.16:0/4026465478 shutdown_connections 2024-09-17T05:04:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 --2- 172.21.15.16:0/4026465478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b80b2f50 0x7ff9b815bb60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 -- 172.21.15.16:0/4026465478 >> 172.21.15.16:0/4026465478 conn(0x7ff9b806f900 msgr2=0x7ff9b80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:21.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 -- 172.21.15.16:0/4026465478 shutdown_connections 2024-09-17T05:04:21.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 -- 172.21.15.16:0/4026465478 wait complete. 2024-09-17T05:04:21.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 Processor -- start 2024-09-17T05:04:21.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.041+0000 7ff9bea00640 1 -- start start 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b8073240 0x7ff9b81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 0x7ff9b815d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b815c0a0 0x7ff9b815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8169db0 con 0x7ff9b80b2f50 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b8169c30 con 0x7ff9b8073240 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b8169f30 con 0x7ff9b815c0a0 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 0x7ff9b815d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 0x7ff9b815d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48146/0 (socket says 172.21.15.16:48146) 2024-09-17T05:04:21.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 -- 172.21.15.16:0/3704953950 learned_addr learned my addr 172.21.15.16:0/3704953950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bca00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b815c0a0 0x7ff9b815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9b7e00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b8073240 0x7ff9b81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 -- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b815c0a0 msgr2=0x7ff9b815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b815c0a0 0x7ff9b815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 -- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b8073240 msgr2=0x7ff9b81503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b8073240 0x7ff9b81503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 -- 172.21.15.16:0/3704953950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b8160320 con 0x7ff9b80b2f50 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9afe00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 0x7ff9b815d850 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0001ec0 tx=0x7ff9a000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0011790 con 0x7ff9b80b2f50 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b81605c0 con 0x7ff9b80b2f50 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9b80b2f50 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0011d40 con 0x7ff9b80b2f50 2024-09-17T05:04:21.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b8160b80 con 0x7ff9b80b2f50 2024-09-17T05:04:21.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.045+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b81f6ad0 con 0x7ff9b80b2f50 2024-09-17T05:04:21.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.049+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff9a001c460 con 0x7ff9b80b2f50 2024-09-17T05:04:21.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.049+0000 7ff9b4c00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff994073200 0x7ff994075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.049+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7ff9a00940a0 con 0x7ff9b80b2f50 2024-09-17T05:04:21.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.049+0000 7ff9b7e00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff994073200 0x7ff994075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.049+0000 7ff9b7e00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff994073200 0x7ff994075710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80af000 tx=0x7ff9a801b010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:21.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.053+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff9a005e850 con 0x7ff9b80b2f50 2024-09-17T05:04:21.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.229+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9b80cfdb0 con 0x7ff9b80b2f50 2024-09-17T05:04:21.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.229+0000 7ff9b4c00640 1 -- 172.21.15.16:0/3704953950 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16933 (secure 0 0 0) 0x7ff9a0063280 con 0x7ff9b80b2f50 2024-09-17T05:04:21.233 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:21.234 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:18.428662+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:14.578846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:15.811287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:21.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff994073200 msgr2=0x7ff994075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff994073200 0x7ff994075710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80af000 tx=0x7ff9a801b010 comp rx=0 tx=0).stop 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 msgr2=0x7ff9b815d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 0x7ff9b815d850 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0001ec0 tx=0x7ff9a000c820 comp rx=0 tx=0).stop 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 shutdown_connections 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff994073200 0x7ff994075710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9b815c0a0 0x7ff9b815dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9b80b2f50 0x7ff9b815d850 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 --2- 172.21.15.16:0/3704953950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9b8073240 0x7ff9b81503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 >> 172.21.15.16:0/3704953950 conn(0x7ff9b806f900 msgr2=0x7ff9b8154c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 shutdown_connections 2024-09-17T05:04:21.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.233+0000 7ff9bea00640 1 -- 172.21.15.16:0/3704953950 wait complete. 2024-09-17T05:04:21.256 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:04:21.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.409+0000 7f11d9200640 1 -- 172.21.15.16:0/2418835156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d40cf590 msgr2=0x7f11d40cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.409+0000 7f11d9200640 1 --2- 172.21.15.16:0/2418835156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d40cf590 0x7f11d40cf9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009870 tx=0x7f11bc01c780 comp rx=0 tx=0).stop 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 -- 172.21.15.16:0/2418835156 shutdown_connections 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 --2- 172.21.15.16:0/2418835156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d40cf590 0x7f11d40cf9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 -- 172.21.15.16:0/2418835156 >> 172.21.15.16:0/2418835156 conn(0x7f11d40d18e0 msgr2=0x7f11d40d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 -- 172.21.15.16:0/2418835156 shutdown_connections 2024-09-17T05:04:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 -- 172.21.15.16:0/2418835156 wait complete. 2024-09-17T05:04:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 Processor -- start 2024-09-17T05:04:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 -- start start 2024-09-17T05:04:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.413+0000 7f11d9200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11d40cf590 0x7f11d41483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 0x7f11d4148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d4162290 0x7f11d4148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11d40d3f20 con 0x7f11d41601d0 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11d40d3da0 con 0x7f11d40cf590 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f11d40d40a0 con 0x7f11d4162290 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 0x7f11d4148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 0x7f11d4148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48162/0 (socket says 172.21.15.16:48162) 2024-09-17T05:04:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 -- 172.21.15.16:0/2403544222 learned_addr learned my addr 172.21.15.16:0/2403544222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 -- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d4162290 msgr2=0x7f11d4148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d1e00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11d40cf590 0x7f11d41483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d4162290 0x7f11d4148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 -- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11d40cf590 msgr2=0x7f11d41483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11d40cf590 0x7f11d41483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 -- 172.21.15.16:0/2403544222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d40d2a90 con 0x7f11d41601d0 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d1e00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11d40cf590 0x7f11d41483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11cbe00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 0x7f11d4148920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f11bc01a3b0 tx=0x7f11bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc004750 con 0x7f11d41601d0 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11bc0048f0 con 0x7f11d41601d0 2024-09-17T05:04:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d40c6f40 con 0x7f11d41601d0 2024-09-17T05:04:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc004d40 con 0x7f11d41601d0 2024-09-17T05:04:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d40c82d0 con 0x7f11d41601d0 2024-09-17T05:04:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d40cf9a0 con 0x7f11d41601d0 2024-09-17T05:04:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.417+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f11bc0a8520 con 0x7f11d41601d0 2024-09-17T05:04:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.421+0000 7f11ca000640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11b0073200 0x7f11b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.421+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f11bc12eb00 con 0x7f11d41601d0 2024-09-17T05:04:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.421+0000 7f11d1e00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11b0073200 0x7f11b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.421+0000 7f11d1e00640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11b0073200 0x7f11b0075710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11c4007fd0 tx=0x7f11c4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.425+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11bc0c9080 con 0x7f11d41601d0 2024-09-17T05:04:21.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.593+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11d406e740 con 0x7f11b0073200 2024-09-17T05:04:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.601+0000 7f11ca000640 1 -- 172.21.15.16:0/2403544222 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f11d406e740 con 0x7f11b0073200 2024-09-17T05:04:21.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:04:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11b0073200 msgr2=0x7f11b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11b0073200 0x7f11b0075710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11c4007fd0 tx=0x7f11c4009040 comp rx=0 tx=0).stop 2024-09-17T05:04:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 msgr2=0x7f11d4148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 0x7f11d4148920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f11bc01a3b0 tx=0x7f11bc0056d0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 shutdown_connections 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f11b0073200 0x7f11b0075710 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f11d4162290 0x7f11d4148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11d41601d0 0x7f11d4148920 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 --2- 172.21.15.16:0/2403544222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11d40cf590 0x7f11d41483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 >> 172.21.15.16:0/2403544222 conn(0x7f11d40d18e0 msgr2=0x7f11d41626a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 shutdown_connections 2024-09-17T05:04:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.605+0000 7f11d9200640 1 -- 172.21.15.16:0/2403544222 wait complete. 2024-09-17T05:04:21.626 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-17T05:04:21.511708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253532,"kb_used_data":8164,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721188,"statfs":{"total":773094113280,"available":772834496512,"internally_reserved":0,"allocated":8359936,"data_stored":4394446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001420"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.451528+0000","last_change":"2024-09-17T05:04:17.442664+0000","last_active":"2024-09-17T05:04:17.451528+0000","last_peered":"2024-09-17T05:04:17.451528+0000","last_clean":"2024-09-17T05:04:17.451528+0000","last_became_active":"2024-09-17T05:04:17.442351+0000","last_became_peered":"2024-09-17T05:04:17.442351+0000","last_unstale":"2024-09-17T05:04:17.451528+0000","last_undegraded":"2024-09-17T05:04:17.451528+0000","last_fullsized":"2024-09-17T05:04:17.451528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:27.874024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464586+0000","last_change":"2024-09-17T05:04:17.459817+0000","last_active":"2024-09-17T05:04:17.464586+0000","last_peered":"2024-09-17T05:04:17.464586+0000","last_clean":"2024-09-17T05:04:17.464586+0000","last_became_active":"2024-09-17T05:04:17.454983+0000","last_became_peered":"2024-09-17T05:04:17.454983+0000","last_unstale":"2024-09-17T05:04:17.464586+0000","last_undegraded":"2024-09-17T05:04:17.464586+0000","last_fullsized":"2024-09-17T05:04:17.464586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:20.175934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.450538+0000","last_change":"2024-09-17T05:04:17.447649+0000","last_active":"2024-09-17T05:04:17.450538+0000","last_peered":"2024-09-17T05:04:17.450538+0000","last_clean":"2024-09-17T05:04:17.450538+0000","last_became_active":"2024-09-17T05:04:17.447323+0000","last_became_peered":"2024-09-17T05:04:17.447323+0000","last_unstale":"2024-09-17T05:04:17.450538+0000","last_undegraded":"2024-09-17T05:04:17.450538+0000","last_fullsized":"2024-09-17T05:04:17.450538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:43.617145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.446500+0000","last_change":"2024-09-17T05:04:17.444031+0000","last_active":"2024-09-17T05:04:17.446500+0000","last_peered":"2024-09-17T05:04:17.446500+0000","last_clean":"2024-09-17T05:04:17.446500+0000","last_became_active":"2024-09-17T05:04:17.443661+0000","last_became_peered":"2024-09-17T05:04:17.443661+0000","last_unstale":"2024-09-17T05:04:17.446500+0000","last_undegraded":"2024-09-17T05:04:17.446500+0000","last_fullsized":"2024-09-17T05:04:17.446500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:20.143372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469888+0000","last_change":"2024-09-17T05:04:17.466049+0000","last_active":"2024-09-17T05:04:17.469888+0000","last_peered":"2024-09-17T05:04:17.469888+0000","last_clean":"2024-09-17T05:04:17.469888+0000","last_became_active":"2024-09-17T05:04:17.465715+0000","last_became_peered":"2024-09-17T05:04:17.465715+0000","last_unstale":"2024-09-17T05:04:17.469888+0000","last_undegraded":"2024-09-17T05:04:17.469888+0000","last_fullsized":"2024-09-17T05:04:17.469888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:07.872979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807584+0000","last_change":"2024-09-17T05:04:17.444706+0000","last_active":"2024-09-17T05:04:18.807584+0000","last_peered":"2024-09-17T05:04:18.807584+0000","last_clean":"2024-09-17T05:04:18.807584+0000","last_became_active":"2024-09-17T05:04:17.444320+0000","last_became_peered":"2024-09-17T05:04:17.444320+0000","last_unstale":"2024-09-17T05:04:18.807584+0000","last_undegraded":"2024-09-17T05:04:18.807584+0000","last_fullsized":"2024-09-17T05:04:18.807584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:52.449310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457059+0000","last_change":"2024-09-17T05:04:17.449489+0000","last_active":"2024-09-17T05:04:17.457059+0000","last_peered":"2024-09-17T05:04:17.457059+0000","last_clean":"2024-09-17T05:04:17.457059+0000","last_became_active":"2024-09-17T05:04:17.449174+0000","last_became_peered":"2024-09-17T05:04:17.449174+0000","last_unstale":"2024-09-17T05:04:17.457059+0000","last_undegraded":"2024-09-17T05:04:17.457059+0000","last_fullsized":"2024-09-17T05:04:17.457059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:38.933711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457492+0000","last_change":"2024-09-17T05:04:17.443462+0000","last_active":"2024-09-17T05:04:17.457492+0000","last_peered":"2024-09-17T05:04:17.457492+0000","last_clean":"2024-09-17T05:04:17.457492+0000","last_became_active":"2024-09-17T05:04:17.443151+0000","last_became_peered":"2024-09-17T05:04:17.443151+0000","last_unstale":"2024-09-17T05:04:17.457492+0000","last_undegraded":"2024-09-17T05:04:17.457492+0000","last_fullsized":"2024-09-17T05:04:17.457492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:01.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458079+0000","last_change":"2024-09-17T05:04:17.454992+0000","last_active":"2024-09-17T05:04:17.458079+0000","last_peered":"2024-09-17T05:04:17.458079+0000","last_clean":"2024-09-17T05:04:17.458079+0000","last_became_active":"2024-09-17T05:04:17.453327+0000","last_became_peered":"2024-09-17T05:04:17.453327+0000","last_unstale":"2024-09-17T05:04:17.458079+0000","last_undegraded":"2024-09-17T05:04:17.458079+0000","last_fullsized":"2024-09-17T05:04:17.458079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:43.617145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.456032+0000","last_change":"2024-09-17T05:04:17.453458+0000","last_active":"2024-09-17T05:04:17.456032+0000","last_peered":"2024-09-17T05:04:17.456032+0000","last_clean":"2024-09-17T05:04:17.456032+0000","last_became_active":"2024-09-17T05:04:17.453166+0000","last_became_peered":"2024-09-17T05:04:17.453166+0000","last_unstale":"2024-09-17T05:04:17.456032+0000","last_undegraded":"2024-09-17T05:04:17.456032+0000","last_fullsized":"2024-09-17T05:04:17.456032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:52.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.462130+0000","last_change":"2024-09-17T05:04:17.453688+0000","last_active":"2024-09-17T05:04:17.462130+0000","last_peered":"2024-09-17T05:04:17.462130+0000","last_clean":"2024-09-17T05:04:17.462130+0000","last_became_active":"2024-09-17T05:04:17.453006+0000","last_became_peered":"2024-09-17T05:04:17.453006+0000","last_unstale":"2024-09-17T05:04:17.462130+0000","last_undegraded":"2024-09-17T05:04:17.462130+0000","last_fullsized":"2024-09-17T05:04:17.462130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:08.770442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454966+0000","last_change":"2024-09-17T05:04:17.447816+0000","last_active":"2024-09-17T05:04:17.454966+0000","last_peered":"2024-09-17T05:04:17.454966+0000","last_clean":"2024-09-17T05:04:17.454966+0000","last_became_active":"2024-09-17T05:04:17.447508+0000","last_became_peered":"2024-09-17T05:04:17.447508+0000","last_unstale":"2024-09-17T05:04:17.454966+0000","last_undegraded":"2024-09-17T05:04:17.454966+0000","last_fullsized":"2024-09-17T05:04:17.454966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.339567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.466729+0000","last_change":"2024-09-17T05:04:17.461226+0000","last_active":"2024-09-17T05:04:17.466729+0000","last_peered":"2024-09-17T05:04:17.466729+0000","last_clean":"2024-09-17T05:04:17.466729+0000","last_became_active":"2024-09-17T05:04:17.460716+0000","last_became_peered":"2024-09-17T05:04:17.460716+0000","last_unstale":"2024-09-17T05:04:17.466729+0000","last_undegraded":"2024-09-17T05:04:17.466729+0000","last_fullsized":"2024-09-17T05:04:17.466729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:44.049692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459351+0000","last_change":"2024-09-17T05:04:17.447882+0000","last_active":"2024-09-17T05:04:17.459351+0000","last_peered":"2024-09-17T05:04:17.459351+0000","last_clean":"2024-09-17T05:04:17.459351+0000","last_became_active":"2024-09-17T05:04:17.447553+0000","last_became_peered":"2024-09-17T05:04:17.447553+0000","last_unstale":"2024-09-17T05:04:17.459351+0000","last_undegraded":"2024-09-17T05:04:17.459351+0000","last_fullsized":"2024-09-17T05:04:17.459351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:44.358823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468490+0000","last_change":"2024-09-17T05:04:17.461112+0000","last_active":"2024-09-17T05:04:17.468490+0000","last_peered":"2024-09-17T05:04:17.468490+0000","last_clean":"2024-09-17T05:04:17.468490+0000","last_became_active":"2024-09-17T05:04:17.460799+0000","last_became_peered":"2024-09-17T05:04:17.460799+0000","last_unstale":"2024-09-17T05:04:17.468490+0000","last_undegraded":"2024-09-17T05:04:17.468490+0000","last_fullsized":"2024-09-17T05:04:17.468490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:31.247288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454713+0000","last_change":"2024-09-17T05:04:17.448651+0000","last_active":"2024-09-17T05:04:17.454713+0000","last_peered":"2024-09-17T05:04:17.454713+0000","last_clean":"2024-09-17T05:04:17.454713+0000","last_became_active":"2024-09-17T05:04:17.447495+0000","last_became_peered":"2024-09-17T05:04:17.447495+0000","last_unstale":"2024-09-17T05:04:17.454713+0000","last_undegraded":"2024-09-17T05:04:17.454713+0000","last_fullsized":"2024-09-17T05:04:17.454713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:28.651823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458932+0000","last_change":"2024-09-17T05:04:17.447180+0000","last_active":"2024-09-17T05:04:17.458932+0000","last_peered":"2024-09-17T05:04:17.458932+0000","last_clean":"2024-09-17T05:04:17.458932+0000","last_became_active":"2024-09-17T05:04:17.446852+0000","last_became_peered":"2024-09-17T05:04:17.446852+0000","last_unstale":"2024-09-17T05:04:17.458932+0000","last_undegraded":"2024-09-17T05:04:17.458932+0000","last_fullsized":"2024-09-17T05:04:17.458932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:32:51.404799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464616+0000","last_change":"2024-09-17T05:04:17.460158+0000","last_active":"2024-09-17T05:04:17.464616+0000","last_peered":"2024-09-17T05:04:17.464616+0000","last_clean":"2024-09-17T05:04:17.464616+0000","last_became_active":"2024-09-17T05:04:17.455481+0000","last_became_peered":"2024-09-17T05:04:17.455481+0000","last_unstale":"2024-09-17T05:04:17.464616+0000","last_undegraded":"2024-09-17T05:04:17.464616+0000","last_fullsized":"2024-09-17T05:04:17.464616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:11.790270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.472383+0000","last_change":"2024-09-17T05:04:17.466812+0000","last_active":"2024-09-17T05:04:17.472383+0000","last_peered":"2024-09-17T05:04:17.472383+0000","last_clean":"2024-09-17T05:04:17.472383+0000","last_became_active":"2024-09-17T05:04:17.466487+0000","last_became_peered":"2024-09-17T05:04:17.466487+0000","last_unstale":"2024-09-17T05:04:17.472383+0000","last_undegraded":"2024-09-17T05:04:17.472383+0000","last_fullsized":"2024-09-17T05:04:17.472383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:13.329981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807596+0000","last_change":"2024-09-17T05:04:17.455756+0000","last_active":"2024-09-17T05:04:18.807596+0000","last_peered":"2024-09-17T05:04:18.807596+0000","last_clean":"2024-09-17T05:04:18.807596+0000","last_became_active":"2024-09-17T05:04:17.455437+0000","last_became_peered":"2024-09-17T05:04:17.455437+0000","last_unstale":"2024-09-17T05:04:18.807596+0000","last_undegraded":"2024-09-17T05:04:18.807596+0000","last_fullsized":"2024-09-17T05:04:18.807596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:07.476987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.461829+0000","last_change":"2024-09-17T05:04:17.459594+0000","last_active":"2024-09-17T05:04:17.461829+0000","last_peered":"2024-09-17T05:04:17.461829+0000","last_clean":"2024-09-17T05:04:17.461829+0000","last_became_active":"2024-09-17T05:04:17.459085+0000","last_became_peered":"2024-09-17T05:04:17.459085+0000","last_unstale":"2024-09-17T05:04:17.461829+0000","last_undegraded":"2024-09-17T05:04:17.461829+0000","last_fullsized":"2024-09-17T05:04:17.461829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:48.014480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460209+0000","last_change":"2024-09-17T05:04:17.452690+0000","last_active":"2024-09-17T05:04:17.460209+0000","last_peered":"2024-09-17T05:04:17.460209+0000","last_clean":"2024-09-17T05:04:17.460209+0000","last_became_active":"2024-09-17T05:04:17.452356+0000","last_became_peered":"2024-09-17T05:04:17.452356+0000","last_unstale":"2024-09-17T05:04:17.460209+0000","last_undegraded":"2024-09-17T05:04:17.460209+0000","last_fullsized":"2024-09-17T05:04:17.460209+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:47.020135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.440210+0000","last_change":"2024-09-17T05:04:17.431263+0000","last_active":"2024-09-17T05:04:17.440210+0000","last_peered":"2024-09-17T05:04:17.440210+0000","last_clean":"2024-09-17T05:04:17.440210+0000","last_became_active":"2024-09-17T05:04:17.430952+0000","last_became_peered":"2024-09-17T05:04:17.430952+0000","last_unstale":"2024-09-17T05:04:17.440210+0000","last_undegraded":"2024-09-17T05:04:17.440210+0000","last_fullsized":"2024-09-17T05:04:17.440210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:26.010989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459774+0000","last_change":"2024-09-17T05:04:17.450645+0000","last_active":"2024-09-17T05:04:17.459774+0000","last_peered":"2024-09-17T05:04:17.459774+0000","last_clean":"2024-09-17T05:04:17.459774+0000","last_became_active":"2024-09-17T05:04:17.449831+0000","last_became_peered":"2024-09-17T05:04:17.449831+0000","last_unstale":"2024-09-17T05:04:17.459774+0000","last_undegraded":"2024-09-17T05:04:17.459774+0000","last_fullsized":"2024-09-17T05:04:17.459774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:01:43.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807970+0000","last_change":"2024-09-17T05:04:17.450178+0000","last_active":"2024-09-17T05:04:18.807970+0000","last_peered":"2024-09-17T05:04:18.807970+0000","last_clean":"2024-09-17T05:04:18.807970+0000","last_became_active":"2024-09-17T05:04:17.449780+0000","last_became_peered":"2024-09-17T05:04:17.449780+0000","last_unstale":"2024-09-17T05:04:18.807970+0000","last_undegraded":"2024-09-17T05:04:18.807970+0000","last_fullsized":"2024-09-17T05:04:18.807970+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:34.770065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.456627+0000","last_change":"2024-09-17T05:04:17.453491+0000","last_active":"2024-09-17T05:04:17.456627+0000","last_peered":"2024-09-17T05:04:17.456627+0000","last_clean":"2024-09-17T05:04:17.456627+0000","last_became_active":"2024-09-17T05:04:17.453173+0000","last_became_peered":"2024-09-17T05:04:17.453173+0000","last_unstale":"2024-09-17T05:04:17.456627+0000","last_undegraded":"2024-09-17T05:04:17.456627+0000","last_fullsized":"2024-09-17T05:04:17.456627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:44.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.456361+0000","last_change":"2024-09-17T05:04:17.451591+0000","last_active":"2024-09-17T05:04:17.456361+0000","last_peered":"2024-09-17T05:04:17.456361+0000","last_clean":"2024-09-17T05:04:17.456361+0000","last_became_active":"2024-09-17T05:04:17.451279+0000","last_became_peered":"2024-09-17T05:04:17.451279+0000","last_unstale":"2024-09-17T05:04:17.456361+0000","last_undegraded":"2024-09-17T05:04:17.456361+0000","last_fullsized":"2024-09-17T05:04:17.456361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:41.193297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469832+0000","last_change":"2024-09-17T05:04:17.466009+0000","last_active":"2024-09-17T05:04:17.469832+0000","last_peered":"2024-09-17T05:04:17.469832+0000","last_clean":"2024-09-17T05:04:17.469832+0000","last_became_active":"2024-09-17T05:04:17.465666+0000","last_became_peered":"2024-09-17T05:04:17.465666+0000","last_unstale":"2024-09-17T05:04:17.469832+0000","last_undegraded":"2024-09-17T05:04:17.469832+0000","last_fullsized":"2024-09-17T05:04:17.469832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:19.467327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807504+0000","last_change":"2024-09-17T05:04:17.433194+0000","last_active":"2024-09-17T05:04:18.807504+0000","last_peered":"2024-09-17T05:04:18.807504+0000","last_clean":"2024-09-17T05:04:18.807504+0000","last_became_active":"2024-09-17T05:04:17.431679+0000","last_became_peered":"2024-09-17T05:04:17.431679+0000","last_unstale":"2024-09-17T05:04:18.807504+0000","last_undegraded":"2024-09-17T05:04:18.807504+0000","last_fullsized":"2024-09-17T05:04:18.807504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:12.696484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.446973+0000","last_change":"2024-09-17T05:04:17.438821+0000","last_active":"2024-09-17T05:04:17.446973+0000","last_peered":"2024-09-17T05:04:17.446973+0000","last_clean":"2024-09-17T05:04:17.446973+0000","last_became_active":"2024-09-17T05:04:17.436552+0000","last_became_peered":"2024-09-17T05:04:17.436552+0000","last_unstale":"2024-09-17T05:04:17.446973+0000","last_undegraded":"2024-09-17T05:04:17.446973+0000","last_fullsized":"2024-09-17T05:04:17.446973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:31.889570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460422+0000","last_change":"2024-09-17T05:04:17.450932+0000","last_active":"2024-09-17T05:04:17.460422+0000","last_peered":"2024-09-17T05:04:17.460422+0000","last_clean":"2024-09-17T05:04:17.460422+0000","last_became_active":"2024-09-17T05:04:17.449856+0000","last_became_peered":"2024-09-17T05:04:17.449856+0000","last_unstale":"2024-09-17T05:04:17.460422+0000","last_undegraded":"2024-09-17T05:04:17.460422+0000","last_fullsized":"2024-09-17T05:04:17.460422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:10.578051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454651+0000","last_change":"2024-09-17T05:04:17.447868+0000","last_active":"2024-09-17T05:04:17.454651+0000","last_peered":"2024-09-17T05:04:17.454651+0000","last_clean":"2024-09-17T05:04:17.454651+0000","last_became_active":"2024-09-17T05:04:17.445741+0000","last_became_peered":"2024-09-17T05:04:17.445741+0000","last_unstale":"2024-09-17T05:04:17.454651+0000","last_undegraded":"2024-09-17T05:04:17.454651+0000","last_fullsized":"2024-09-17T05:04:17.454651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:37.843774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468218+0000","last_change":"2024-09-17T05:04:17.458984+0000","last_active":"2024-09-17T05:04:17.468218+0000","last_peered":"2024-09-17T05:04:17.468218+0000","last_clean":"2024-09-17T05:04:17.468218+0000","last_became_active":"2024-09-17T05:04:17.458664+0000","last_became_peered":"2024-09-17T05:04:17.458664+0000","last_unstale":"2024-09-17T05:04:17.468218+0000","last_undegraded":"2024-09-17T05:04:17.468218+0000","last_fullsized":"2024-09-17T05:04:17.468218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:34.829225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468423+0000","last_change":"2024-09-17T05:04:16.442302+0000","last_active":"2024-09-17T05:04:17.468423+0000","last_peered":"2024-09-17T05:04:17.468423+0000","last_clean":"2024-09-17T05:04:17.468423+0000","last_became_active":"2024-09-17T05:04:16.441959+0000","last_became_peered":"2024-09-17T05:04:16.441959+0000","last_unstale":"2024-09-17T05:04:17.468423+0000","last_undegraded":"2024-09-17T05:04:17.468423+0000","last_fullsized":"2024-09-17T05:04:17.468423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:00.712657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.465529+0000","last_change":"2024-09-17T05:04:17.449749+0000","last_active":"2024-09-17T05:04:17.465529+0000","last_peered":"2024-09-17T05:04:17.465529+0000","last_clean":"2024-09-17T05:04:17.465529+0000","last_became_active":"2024-09-17T05:04:17.449353+0000","last_became_peered":"2024-09-17T05:04:17.449353+0000","last_unstale":"2024-09-17T05:04:17.465529+0000","last_undegraded":"2024-09-17T05:04:17.465529+0000","last_fullsized":"2024-09-17T05:04:17.465529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:48.683758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464594+0000","last_change":"2024-09-17T05:04:16.452150+0000","last_active":"2024-09-17T05:04:17.464594+0000","last_peered":"2024-09-17T05:04:17.464594+0000","last_clean":"2024-09-17T05:04:17.464594+0000","last_became_active":"2024-09-17T05:04:16.451803+0000","last_became_peered":"2024-09-17T05:04:16.451803+0000","last_unstale":"2024-09-17T05:04:17.464594+0000","last_undegraded":"2024-09-17T05:04:17.464594+0000","last_fullsized":"2024-09-17T05:04:17.464594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:02.922815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.452768+0000","last_change":"2024-09-17T05:04:17.447126+0000","last_active":"2024-09-17T05:04:17.452768+0000","last_peered":"2024-09-17T05:04:17.452768+0000","last_clean":"2024-09-17T05:04:17.452768+0000","last_became_active":"2024-09-17T05:04:17.446704+0000","last_became_peered":"2024-09-17T05:04:17.446704+0000","last_unstale":"2024-09-17T05:04:17.452768+0000","last_undegraded":"2024-09-17T05:04:17.452768+0000","last_fullsized":"2024-09-17T05:04:17.452768+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:44.049692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460262+0000","last_change":"2024-09-17T05:04:16.436756+0000","last_active":"2024-09-17T05:04:17.460262+0000","last_peered":"2024-09-17T05:04:17.460262+0000","last_clean":"2024-09-17T05:04:17.460262+0000","last_became_active":"2024-09-17T05:04:16.436415+0000","last_became_peered":"2024-09-17T05:04:16.436415+0000","last_unstale":"2024-09-17T05:04:17.460262+0000","last_undegraded":"2024-09-17T05:04:17.460262+0000","last_fullsized":"2024-09-17T05:04:17.460262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:01.351567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454684+0000","last_change":"2024-09-17T05:04:17.449523+0000","last_active":"2024-09-17T05:04:17.454684+0000","last_peered":"2024-09-17T05:04:17.454684+0000","last_clean":"2024-09-17T05:04:17.454684+0000","last_became_active":"2024-09-17T05:04:17.449193+0000","last_became_peered":"2024-09-17T05:04:17.449193+0000","last_unstale":"2024-09-17T05:04:17.454684+0000","last_undegraded":"2024-09-17T05:04:17.454684+0000","last_fullsized":"2024-09-17T05:04:17.454684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:54.872877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458834+0000","last_change":"2024-09-17T05:04:16.440778+0000","last_active":"2024-09-17T05:04:17.458834+0000","last_peered":"2024-09-17T05:04:17.458834+0000","last_clean":"2024-09-17T05:04:17.458834+0000","last_became_active":"2024-09-17T05:04:16.440416+0000","last_became_peered":"2024-09-17T05:04:16.440416+0000","last_unstale":"2024-09-17T05:04:17.458834+0000","last_undegraded":"2024-09-17T05:04:17.458834+0000","last_fullsized":"2024-09-17T05:04:17.458834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:07.767221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807579+0000","last_change":"2024-09-17T05:04:17.455425+0000","last_active":"2024-09-17T05:04:18.807579+0000","last_peered":"2024-09-17T05:04:18.807579+0000","last_clean":"2024-09-17T05:04:18.807579+0000","last_became_active":"2024-09-17T05:04:17.455012+0000","last_became_peered":"2024-09-17T05:04:17.455012+0000","last_unstale":"2024-09-17T05:04:18.807579+0000","last_undegraded":"2024-09-17T05:04:18.807579+0000","last_fullsized":"2024-09-17T05:04:18.807579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:03.218826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.455983+0000","last_change":"2024-09-17T05:04:16.443460+0000","last_active":"2024-09-17T05:04:17.455983+0000","last_peered":"2024-09-17T05:04:17.455983+0000","last_clean":"2024-09-17T05:04:17.455983+0000","last_became_active":"2024-09-17T05:04:16.439515+0000","last_became_peered":"2024-09-17T05:04:16.439515+0000","last_unstale":"2024-09-17T05:04:17.455983+0000","last_undegraded":"2024-09-17T05:04:17.455983+0000","last_fullsized":"2024-09-17T05:04:17.455983+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:19.172713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460461+0000","last_change":"2024-09-17T05:04:17.451069+0000","last_active":"2024-09-17T05:04:17.460461+0000","last_peered":"2024-09-17T05:04:17.460461+0000","last_clean":"2024-09-17T05:04:17.460461+0000","last_became_active":"2024-09-17T05:04:17.450281+0000","last_became_peered":"2024-09-17T05:04:17.450281+0000","last_unstale":"2024-09-17T05:04:17.460461+0000","last_undegraded":"2024-09-17T05:04:17.460461+0000","last_fullsized":"2024-09-17T05:04:17.460461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:31.334395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.467646+0000","last_change":"2024-09-17T05:04:16.442691+0000","last_active":"2024-09-17T05:04:17.467646+0000","last_peered":"2024-09-17T05:04:17.467646+0000","last_clean":"2024-09-17T05:04:17.467646+0000","last_became_active":"2024-09-17T05:04:16.442154+0000","last_became_peered":"2024-09-17T05:04:16.442154+0000","last_unstale":"2024-09-17T05:04:17.467646+0000","last_undegraded":"2024-09-17T05:04:17.467646+0000","last_fullsized":"2024-09-17T05:04:17.467646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:00.163306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454942+0000","last_change":"2024-09-17T05:04:17.451396+0000","last_active":"2024-09-17T05:04:17.454942+0000","last_peered":"2024-09-17T05:04:17.454942+0000","last_clean":"2024-09-17T05:04:17.454942+0000","last_became_active":"2024-09-17T05:04:17.450633+0000","last_became_peered":"2024-09-17T05:04:17.450633+0000","last_unstale":"2024-09-17T05:04:17.454942+0000","last_undegraded":"2024-09-17T05:04:17.454942+0000","last_fullsized":"2024-09-17T05:04:17.454942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:58.348078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.431285+0000","last_change":"2024-09-17T05:04:16.437012+0000","last_active":"2024-09-17T05:04:17.431285+0000","last_peered":"2024-09-17T05:04:17.431285+0000","last_clean":"2024-09-17T05:04:17.431285+0000","last_became_active":"2024-09-17T05:04:16.436709+0000","last_became_peered":"2024-09-17T05:04:16.436709+0000","last_unstale":"2024-09-17T05:04:17.431285+0000","last_undegraded":"2024-09-17T05:04:17.431285+0000","last_fullsized":"2024-09-17T05:04:17.431285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:24.735292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464871+0000","last_change":"2024-09-17T05:04:17.461532+0000","last_active":"2024-09-17T05:04:17.464871+0000","last_peered":"2024-09-17T05:04:17.464871+0000","last_clean":"2024-09-17T05:04:17.464871+0000","last_became_active":"2024-09-17T05:04:17.461217+0000","last_became_peered":"2024-09-17T05:04:17.461217+0000","last_unstale":"2024-09-17T05:04:17.464871+0000","last_undegraded":"2024-09-17T05:04:17.464871+0000","last_fullsized":"2024-09-17T05:04:17.464871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:33.022867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.433614+0000","last_change":"2024-09-17T05:04:16.439787+0000","last_active":"2024-09-17T05:04:17.433614+0000","last_peered":"2024-09-17T05:04:17.433614+0000","last_clean":"2024-09-17T05:04:17.433614+0000","last_became_active":"2024-09-17T05:04:16.439130+0000","last_became_peered":"2024-09-17T05:04:16.439130+0000","last_unstale":"2024-09-17T05:04:17.433614+0000","last_undegraded":"2024-09-17T05:04:17.433614+0000","last_fullsized":"2024-09-17T05:04:17.433614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:55.643709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469629+0000","last_change":"2024-09-17T05:04:17.461794+0000","last_active":"2024-09-17T05:04:17.469629+0000","last_peered":"2024-09-17T05:04:17.469629+0000","last_clean":"2024-09-17T05:04:17.469629+0000","last_became_active":"2024-09-17T05:04:17.461457+0000","last_became_peered":"2024-09-17T05:04:17.461457+0000","last_unstale":"2024-09-17T05:04:17.469629+0000","last_undegraded":"2024-09-17T05:04:17.469629+0000","last_fullsized":"2024-09-17T05:04:17.469629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:53.912834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.438921+0000","last_change":"2024-09-17T05:04:16.440619+0000","last_active":"2024-09-17T05:04:17.438921+0000","last_peered":"2024-09-17T05:04:17.438921+0000","last_clean":"2024-09-17T05:04:17.438921+0000","last_became_active":"2024-09-17T05:04:16.440214+0000","last_became_peered":"2024-09-17T05:04:16.440214+0000","last_unstale":"2024-09-17T05:04:17.438921+0000","last_undegraded":"2024-09-17T05:04:17.438921+0000","last_fullsized":"2024-09-17T05:04:17.438921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:12.273507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.452491+0000","last_change":"2024-09-17T05:04:17.442139+0000","last_active":"2024-09-17T05:04:17.452491+0000","last_peered":"2024-09-17T05:04:17.452491+0000","last_clean":"2024-09-17T05:04:17.452491+0000","last_became_active":"2024-09-17T05:04:17.441809+0000","last_became_peered":"2024-09-17T05:04:17.441809+0000","last_unstale":"2024-09-17T05:04:17.452491+0000","last_undegraded":"2024-09-17T05:04:17.452491+0000","last_fullsized":"2024-09-17T05:04:17.452491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:11.790270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.422718+0000","last_change":"2024-09-17T05:04:16.450058+0000","last_active":"2024-09-17T05:04:17.422718+0000","last_peered":"2024-09-17T05:04:17.422718+0000","last_clean":"2024-09-17T05:04:17.422718+0000","last_became_active":"2024-09-17T05:04:16.449701+0000","last_became_peered":"2024-09-17T05:04:16.449701+0000","last_unstale":"2024-09-17T05:04:17.422718+0000","last_undegraded":"2024-09-17T05:04:17.422718+0000","last_fullsized":"2024-09-17T05:04:17.422718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:43.335136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459323+0000","last_change":"2024-09-17T05:04:17.447100+0000","last_active":"2024-09-17T05:04:17.459323+0000","last_peered":"2024-09-17T05:04:17.459323+0000","last_clean":"2024-09-17T05:04:17.459323+0000","last_became_active":"2024-09-17T05:04:17.446396+0000","last_became_peered":"2024-09-17T05:04:17.446396+0000","last_unstale":"2024-09-17T05:04:17.459323+0000","last_undegraded":"2024-09-17T05:04:17.459323+0000","last_fullsized":"2024-09-17T05:04:17.459323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:08.333394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807271+0000","last_change":"2024-09-17T05:04:16.433675+0000","last_active":"2024-09-17T05:04:18.807271+0000","last_peered":"2024-09-17T05:04:18.807271+0000","last_clean":"2024-09-17T05:04:18.807271+0000","last_became_active":"2024-09-17T05:04:16.433330+0000","last_became_peered":"2024-09-17T05:04:16.433330+0000","last_unstale":"2024-09-17T05:04:18.807271+0000","last_undegraded":"2024-09-17T05:04:18.807271+0000","last_fullsized":"2024-09-17T05:04:18.807271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:38.828077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807378+0000","last_change":"2024-09-17T05:04:17.458481+0000","last_active":"2024-09-17T05:04:18.807378+0000","last_peered":"2024-09-17T05:04:18.807378+0000","last_clean":"2024-09-17T05:04:18.807378+0000","last_became_active":"2024-09-17T05:04:17.458188+0000","last_became_peered":"2024-09-17T05:04:17.458188+0000","last_unstale":"2024-09-17T05:04:18.807378+0000","last_undegraded":"2024-09-17T05:04:18.807378+0000","last_fullsized":"2024-09-17T05:04:18.807378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:35.159360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468612+0000","last_change":"2024-09-17T05:04:16.450009+0000","last_active":"2024-09-17T05:04:17.468612+0000","last_peered":"2024-09-17T05:04:17.468612+0000","last_clean":"2024-09-17T05:04:17.468612+0000","last_became_active":"2024-09-17T05:04:16.444154+0000","last_became_peered":"2024-09-17T05:04:16.444154+0000","last_unstale":"2024-09-17T05:04:17.468612+0000","last_undegraded":"2024-09-17T05:04:17.468612+0000","last_fullsized":"2024-09-17T05:04:17.468612+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:04.237584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.453886+0000","last_change":"2024-09-17T05:04:17.446638+0000","last_active":"2024-09-17T05:04:17.453886+0000","last_peered":"2024-09-17T05:04:17.453886+0000","last_clean":"2024-09-17T05:04:17.453886+0000","last_became_active":"2024-09-17T05:04:17.446297+0000","last_became_peered":"2024-09-17T05:04:17.446297+0000","last_unstale":"2024-09-17T05:04:17.453886+0000","last_undegraded":"2024-09-17T05:04:17.453886+0000","last_fullsized":"2024-09-17T05:04:17.453886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:57.864805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457308+0000","last_change":"2024-09-17T05:04:16.439931+0000","last_active":"2024-09-17T05:04:17.457308+0000","last_peered":"2024-09-17T05:04:17.457308+0000","last_clean":"2024-09-17T05:04:17.457308+0000","last_became_active":"2024-09-17T05:04:16.438817+0000","last_became_peered":"2024-09-17T05:04:16.438817+0000","last_unstale":"2024-09-17T05:04:17.457308+0000","last_undegraded":"2024-09-17T05:04:17.457308+0000","last_fullsized":"2024-09-17T05:04:17.457308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:30.244067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464419+0000","last_change":"2024-09-17T05:04:17.461527+0000","last_active":"2024-09-17T05:04:17.464419+0000","last_peered":"2024-09-17T05:04:17.464419+0000","last_clean":"2024-09-17T05:04:17.464419+0000","last_became_active":"2024-09-17T05:04:17.461065+0000","last_became_peered":"2024-09-17T05:04:17.461065+0000","last_unstale":"2024-09-17T05:04:17.464419+0000","last_undegraded":"2024-09-17T05:04:17.464419+0000","last_fullsized":"2024-09-17T05:04:17.464419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:10.230753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.428499+0000","last_change":"2024-09-17T05:04:16.439186+0000","last_active":"2024-09-17T05:04:17.428499+0000","last_peered":"2024-09-17T05:04:17.428499+0000","last_clean":"2024-09-17T05:04:17.428499+0000","last_became_active":"2024-09-17T05:04:16.438799+0000","last_became_peered":"2024-09-17T05:04:16.438799+0000","last_unstale":"2024-09-17T05:04:17.428499+0000","last_undegraded":"2024-09-17T05:04:17.428499+0000","last_fullsized":"2024-09-17T05:04:17.428499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:46.016914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.455794+0000","last_change":"2024-09-17T05:04:17.454512+0000","last_active":"2024-09-17T05:04:17.455794+0000","last_peered":"2024-09-17T05:04:17.455794+0000","last_clean":"2024-09-17T05:04:17.455794+0000","last_became_active":"2024-09-17T05:04:17.451077+0000","last_became_peered":"2024-09-17T05:04:17.451077+0000","last_unstale":"2024-09-17T05:04:17.455794+0000","last_undegraded":"2024-09-17T05:04:17.455794+0000","last_fullsized":"2024-09-17T05:04:17.455794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:47.954107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.437956+0000","last_change":"2024-09-17T05:04:16.442319+0000","last_active":"2024-09-17T05:04:17.437956+0000","last_peered":"2024-09-17T05:04:17.437956+0000","last_clean":"2024-09-17T05:04:17.437956+0000","last_became_active":"2024-09-17T05:04:16.441887+0000","last_became_peered":"2024-09-17T05:04:16.441887+0000","last_unstale":"2024-09-17T05:04:17.437956+0000","last_undegraded":"2024-09-17T05:04:17.437956+0000","last_fullsized":"2024-09-17T05:04:17.437956+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:09.136966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.440049+0000","last_change":"2024-09-17T05:04:17.429711+0000","last_active":"2024-09-17T05:04:17.440049+0000","last_peered":"2024-09-17T05:04:17.440049+0000","last_clean":"2024-09-17T05:04:17.440049+0000","last_became_active":"2024-09-17T05:04:17.429352+0000","last_became_peered":"2024-09-17T05:04:17.429352+0000","last_unstale":"2024-09-17T05:04:17.440049+0000","last_undegraded":"2024-09-17T05:04:17.440049+0000","last_fullsized":"2024-09-17T05:04:17.440049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:09.402644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807108+0000","last_change":"2024-09-17T05:04:16.435462+0000","last_active":"2024-09-17T05:04:18.807108+0000","last_peered":"2024-09-17T05:04:18.807108+0000","last_clean":"2024-09-17T05:04:18.807108+0000","last_became_active":"2024-09-17T05:04:16.435083+0000","last_became_peered":"2024-09-17T05:04:16.435083+0000","last_unstale":"2024-09-17T05:04:18.807108+0000","last_undegraded":"2024-09-17T05:04:18.807108+0000","last_fullsized":"2024-09-17T05:04:18.807108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:05.691319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468530+0000","last_change":"2024-09-17T05:04:17.462582+0000","last_active":"2024-09-17T05:04:17.468530+0000","last_peered":"2024-09-17T05:04:17.468530+0000","last_clean":"2024-09-17T05:04:17.468530+0000","last_became_active":"2024-09-17T05:04:17.462192+0000","last_became_peered":"2024-09-17T05:04:17.462192+0000","last_unstale":"2024-09-17T05:04:17.468530+0000","last_undegraded":"2024-09-17T05:04:17.468530+0000","last_fullsized":"2024-09-17T05:04:17.468530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:02.354788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.449587+0000","last_change":"2024-09-17T05:04:16.445016+0000","last_active":"2024-09-17T05:04:17.449587+0000","last_peered":"2024-09-17T05:04:17.449587+0000","last_clean":"2024-09-17T05:04:17.449587+0000","last_became_active":"2024-09-17T05:04:16.444694+0000","last_became_peered":"2024-09-17T05:04:16.444694+0000","last_unstale":"2024-09-17T05:04:17.449587+0000","last_undegraded":"2024-09-17T05:04:17.449587+0000","last_fullsized":"2024-09-17T05:04:17.449587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:08.399423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.443191+0000","last_change":"2024-09-17T05:04:17.439211+0000","last_active":"2024-09-17T05:04:17.443191+0000","last_peered":"2024-09-17T05:04:17.443191+0000","last_clean":"2024-09-17T05:04:17.443191+0000","last_became_active":"2024-09-17T05:04:17.437945+0000","last_became_peered":"2024-09-17T05:04:17.437945+0000","last_unstale":"2024-09-17T05:04:17.443191+0000","last_undegraded":"2024-09-17T05:04:17.443191+0000","last_fullsized":"2024-09-17T05:04:17.443191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:44.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.446787+0000","last_change":"2024-09-17T05:04:16.441762+0000","last_active":"2024-09-17T05:04:17.446787+0000","last_peered":"2024-09-17T05:04:17.446787+0000","last_clean":"2024-09-17T05:04:17.446787+0000","last_became_active":"2024-09-17T05:04:16.441468+0000","last_became_peered":"2024-09-17T05:04:16.441468+0000","last_unstale":"2024-09-17T05:04:17.446787+0000","last_undegraded":"2024-09-17T05:04:17.446787+0000","last_fullsized":"2024-09-17T05:04:17.446787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:25.007767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.461796+0000","last_change":"2024-09-17T05:04:17.452552+0000","last_active":"2024-09-17T05:04:17.461796+0000","last_peered":"2024-09-17T05:04:17.461796+0000","last_clean":"2024-09-17T05:04:17.461796+0000","last_became_active":"2024-09-17T05:04:17.452206+0000","last_became_peered":"2024-09-17T05:04:17.452206+0000","last_unstale":"2024-09-17T05:04:17.461796+0000","last_undegraded":"2024-09-17T05:04:17.461796+0000","last_fullsized":"2024-09-17T05:04:17.461796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:13.292533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458423+0000","last_change":"2024-09-17T05:04:16.446523+0000","last_active":"2024-09-17T05:04:17.458423+0000","last_peered":"2024-09-17T05:04:17.458423+0000","last_clean":"2024-09-17T05:04:17.458423+0000","last_became_active":"2024-09-17T05:04:16.446194+0000","last_became_peered":"2024-09-17T05:04:16.446194+0000","last_unstale":"2024-09-17T05:04:17.458423+0000","last_undegraded":"2024-09-17T05:04:17.458423+0000","last_fullsized":"2024-09-17T05:04:17.458423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.515722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.472490+0000","last_change":"2024-09-17T05:04:17.467996+0000","last_active":"2024-09-17T05:04:17.472490+0000","last_peered":"2024-09-17T05:04:17.472490+0000","last_clean":"2024-09-17T05:04:17.472490+0000","last_became_active":"2024-09-17T05:04:17.467687+0000","last_became_peered":"2024-09-17T05:04:17.467687+0000","last_unstale":"2024-09-17T05:04:17.472490+0000","last_undegraded":"2024-09-17T05:04:17.472490+0000","last_fullsized":"2024-09-17T05:04:17.472490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:44.338357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.441600+0000","last_change":"2024-09-17T05:04:16.442945+0000","last_active":"2024-09-17T05:04:17.441600+0000","last_peered":"2024-09-17T05:04:17.441600+0000","last_clean":"2024-09-17T05:04:17.441600+0000","last_became_active":"2024-09-17T05:04:16.442697+0000","last_became_peered":"2024-09-17T05:04:16.442697+0000","last_unstale":"2024-09-17T05:04:17.441600+0000","last_undegraded":"2024-09-17T05:04:17.441600+0000","last_fullsized":"2024-09-17T05:04:17.441600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:03.658735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.466279+0000","last_change":"2024-09-17T05:04:17.446766+0000","last_active":"2024-09-17T05:04:17.466279+0000","last_peered":"2024-09-17T05:04:17.466279+0000","last_clean":"2024-09-17T05:04:17.466279+0000","last_became_active":"2024-09-17T05:04:17.446469+0000","last_became_peered":"2024-09-17T05:04:17.446469+0000","last_unstale":"2024-09-17T05:04:17.466279+0000","last_undegraded":"2024-09-17T05:04:17.466279+0000","last_fullsized":"2024-09-17T05:04:17.466279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:46.398936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469930+0000","last_change":"2024-09-17T05:04:16.438284+0000","last_active":"2024-09-17T05:04:17.469930+0000","last_peered":"2024-09-17T05:04:17.469930+0000","last_clean":"2024-09-17T05:04:17.469930+0000","last_became_active":"2024-09-17T05:04:16.437883+0000","last_became_peered":"2024-09-17T05:04:16.437883+0000","last_unstale":"2024-09-17T05:04:17.469930+0000","last_undegraded":"2024-09-17T05:04:17.469930+0000","last_fullsized":"2024-09-17T05:04:17.469930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:27.706213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468246+0000","last_change":"2024-09-17T05:04:17.459422+0000","last_active":"2024-09-17T05:04:17.468246+0000","last_peered":"2024-09-17T05:04:17.468246+0000","last_clean":"2024-09-17T05:04:17.468246+0000","last_became_active":"2024-09-17T05:04:17.458966+0000","last_became_peered":"2024-09-17T05:04:17.458966+0000","last_unstale":"2024-09-17T05:04:17.468246+0000","last_undegraded":"2024-09-17T05:04:17.468246+0000","last_fullsized":"2024-09-17T05:04:17.468246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:26.632431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.431559+0000","last_change":"2024-09-17T05:04:16.436978+0000","last_active":"2024-09-17T05:04:17.431559+0000","last_peered":"2024-09-17T05:04:17.431559+0000","last_clean":"2024-09-17T05:04:17.431559+0000","last_became_active":"2024-09-17T05:04:16.436673+0000","last_became_peered":"2024-09-17T05:04:16.436673+0000","last_unstale":"2024-09-17T05:04:17.431559+0000","last_undegraded":"2024-09-17T05:04:17.431559+0000","last_fullsized":"2024-09-17T05:04:17.431559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:15.335263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.451742+0000","last_change":"2024-09-17T05:04:17.444157+0000","last_active":"2024-09-17T05:04:17.451742+0000","last_peered":"2024-09-17T05:04:17.451742+0000","last_clean":"2024-09-17T05:04:17.451742+0000","last_became_active":"2024-09-17T05:04:17.443839+0000","last_became_peered":"2024-09-17T05:04:17.443839+0000","last_unstale":"2024-09-17T05:04:17.451742+0000","last_undegraded":"2024-09-17T05:04:17.451742+0000","last_fullsized":"2024-09-17T05:04:17.451742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:26.612250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.433621+0000","last_change":"2024-09-17T05:04:16.437758+0000","last_active":"2024-09-17T05:04:17.433621+0000","last_peered":"2024-09-17T05:04:17.433621+0000","last_clean":"2024-09-17T05:04:17.433621+0000","last_became_active":"2024-09-17T05:04:16.434675+0000","last_became_peered":"2024-09-17T05:04:16.434675+0000","last_unstale":"2024-09-17T05:04:17.433621+0000","last_undegraded":"2024-09-17T05:04:17.433621+0000","last_fullsized":"2024-09-17T05:04:17.433621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:37.930490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454683+0000","last_change":"2024-09-17T05:04:17.448240+0000","last_active":"2024-09-17T05:04:17.454683+0000","last_peered":"2024-09-17T05:04:17.454683+0000","last_clean":"2024-09-17T05:04:17.454683+0000","last_became_active":"2024-09-17T05:04:17.447044+0000","last_became_peered":"2024-09-17T05:04:17.447044+0000","last_unstale":"2024-09-17T05:04:17.454683+0000","last_undegraded":"2024-09-17T05:04:17.454683+0000","last_fullsized":"2024-09-17T05:04:17.454683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:09.718696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464362+0000","last_change":"2024-09-17T05:04:16.443585+0000","last_active":"2024-09-17T05:04:17.464362+0000","last_peered":"2024-09-17T05:04:17.464362+0000","last_clean":"2024-09-17T05:04:17.464362+0000","last_became_active":"2024-09-17T05:04:16.442150+0000","last_became_peered":"2024-09-17T05:04:16.442150+0000","last_unstale":"2024-09-17T05:04:17.464362+0000","last_undegraded":"2024-09-17T05:04:17.464362+0000","last_fullsized":"2024-09-17T05:04:17.464362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:06.869758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.463158+0000","last_change":"2024-09-17T05:04:17.461197+0000","last_active":"2024-09-17T05:04:17.463158+0000","last_peered":"2024-09-17T05:04:17.463158+0000","last_clean":"2024-09-17T05:04:17.463158+0000","last_became_active":"2024-09-17T05:04:17.460791+0000","last_became_peered":"2024-09-17T05:04:17.460791+0000","last_unstale":"2024-09-17T05:04:17.463158+0000","last_undegraded":"2024-09-17T05:04:17.463158+0000","last_fullsized":"2024-09-17T05:04:17.463158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:37.443092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807739+0000","last_change":"2024-09-17T05:04:16.441692+0000","last_active":"2024-09-17T05:04:18.807739+0000","last_peered":"2024-09-17T05:04:18.807739+0000","last_clean":"2024-09-17T05:04:18.807739+0000","last_became_active":"2024-09-17T05:04:16.441313+0000","last_became_peered":"2024-09-17T05:04:16.441313+0000","last_unstale":"2024-09-17T05:04:18.807739+0000","last_undegraded":"2024-09-17T05:04:18.807739+0000","last_fullsized":"2024-09-17T05:04:18.807739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:38.276524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.452795+0000","last_change":"2024-09-17T05:04:17.446970+0000","last_active":"2024-09-17T05:04:17.452795+0000","last_peered":"2024-09-17T05:04:17.452795+0000","last_clean":"2024-09-17T05:04:17.452795+0000","last_became_active":"2024-09-17T05:04:17.446645+0000","last_became_peered":"2024-09-17T05:04:17.446645+0000","last_unstale":"2024-09-17T05:04:17.452795+0000","last_undegraded":"2024-09-17T05:04:17.452795+0000","last_fullsized":"2024-09-17T05:04:17.452795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:53.912834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.422229+0000","last_change":"2024-09-17T05:04:16.437820+0000","last_active":"2024-09-17T05:04:17.422229+0000","last_peered":"2024-09-17T05:04:17.422229+0000","last_clean":"2024-09-17T05:04:17.422229+0000","last_became_active":"2024-09-17T05:04:16.437481+0000","last_became_peered":"2024-09-17T05:04:16.437481+0000","last_unstale":"2024-09-17T05:04:17.422229+0000","last_undegraded":"2024-09-17T05:04:17.422229+0000","last_fullsized":"2024-09-17T05:04:17.422229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:18.464106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464807+0000","last_change":"2024-09-17T05:04:17.454180+0000","last_active":"2024-09-17T05:04:17.464807+0000","last_peered":"2024-09-17T05:04:17.464807+0000","last_clean":"2024-09-17T05:04:17.464807+0000","last_became_active":"2024-09-17T05:04:17.453896+0000","last_became_peered":"2024-09-17T05:04:17.453896+0000","last_unstale":"2024-09-17T05:04:17.464807+0000","last_undegraded":"2024-09-17T05:04:17.464807+0000","last_fullsized":"2024-09-17T05:04:17.464807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:17.585986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.437589+0000","last_change":"2024-09-17T05:04:16.447649+0000","last_active":"2024-09-17T05:04:17.437589+0000","last_peered":"2024-09-17T05:04:17.437589+0000","last_clean":"2024-09-17T05:04:17.437589+0000","last_became_active":"2024-09-17T05:04:16.447262+0000","last_became_peered":"2024-09-17T05:04:16.447262+0000","last_unstale":"2024-09-17T05:04:17.437589+0000","last_undegraded":"2024-09-17T05:04:17.437589+0000","last_fullsized":"2024-09-17T05:04:17.437589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:49.371370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454996+0000","last_change":"2024-09-17T05:04:17.449809+0000","last_active":"2024-09-17T05:04:17.454996+0000","last_peered":"2024-09-17T05:04:17.454996+0000","last_clean":"2024-09-17T05:04:17.454996+0000","last_became_active":"2024-09-17T05:04:17.449523+0000","last_became_peered":"2024-09-17T05:04:17.449523+0000","last_unstale":"2024-09-17T05:04:17.454996+0000","last_undegraded":"2024-09-17T05:04:17.454996+0000","last_fullsized":"2024-09-17T05:04:17.454996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:06.239329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.437910+0000","last_change":"2024-09-17T05:04:16.433349+0000","last_active":"2024-09-17T05:04:17.437910+0000","last_peered":"2024-09-17T05:04:17.437910+0000","last_clean":"2024-09-17T05:04:17.437910+0000","last_became_active":"2024-09-17T05:04:16.433023+0000","last_became_peered":"2024-09-17T05:04:16.433023+0000","last_unstale":"2024-09-17T05:04:17.437910+0000","last_undegraded":"2024-09-17T05:04:17.437910+0000","last_fullsized":"2024-09-17T05:04:17.437910+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:50.928143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.445606+0000","last_change":"2024-09-17T05:04:17.443354+0000","last_active":"2024-09-17T05:04:17.445606+0000","last_peered":"2024-09-17T05:04:17.445606+0000","last_clean":"2024-09-17T05:04:17.445606+0000","last_became_active":"2024-09-17T05:04:17.443049+0000","last_became_peered":"2024-09-17T05:04:17.443049+0000","last_unstale":"2024-09-17T05:04:17.445606+0000","last_undegraded":"2024-09-17T05:04:17.445606+0000","last_fullsized":"2024-09-17T05:04:17.445606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:29.121778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807777+0000","last_change":"2024-09-17T05:04:16.437951+0000","last_active":"2024-09-17T05:04:18.807777+0000","last_peered":"2024-09-17T05:04:18.807777+0000","last_clean":"2024-09-17T05:04:18.807777+0000","last_became_active":"2024-09-17T05:04:16.437600+0000","last_became_peered":"2024-09-17T05:04:16.437600+0000","last_unstale":"2024-09-17T05:04:18.807777+0000","last_undegraded":"2024-09-17T05:04:18.807777+0000","last_fullsized":"2024-09-17T05:04:18.807777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:04.125443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459644+0000","last_change":"2024-09-17T05:04:17.448943+0000","last_active":"2024-09-17T05:04:17.459644+0000","last_peered":"2024-09-17T05:04:17.459644+0000","last_clean":"2024-09-17T05:04:17.459644+0000","last_became_active":"2024-09-17T05:04:17.448570+0000","last_became_peered":"2024-09-17T05:04:17.448570+0000","last_unstale":"2024-09-17T05:04:17.459644+0000","last_undegraded":"2024-09-17T05:04:17.459644+0000","last_fullsized":"2024-09-17T05:04:17.459644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:24.862987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.436640+0000","last_change":"2024-09-17T05:04:16.449561+0000","last_active":"2024-09-17T05:04:17.436640+0000","last_peered":"2024-09-17T05:04:17.436640+0000","last_clean":"2024-09-17T05:04:17.436640+0000","last_became_active":"2024-09-17T05:04:16.449173+0000","last_became_peered":"2024-09-17T05:04:16.449173+0000","last_unstale":"2024-09-17T05:04:17.436640+0000","last_undegraded":"2024-09-17T05:04:17.436640+0000","last_fullsized":"2024-09-17T05:04:17.436640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:47.082784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468328+0000","last_change":"2024-09-17T05:04:17.460956+0000","last_active":"2024-09-17T05:04:17.468328+0000","last_peered":"2024-09-17T05:04:17.468328+0000","last_clean":"2024-09-17T05:04:17.468328+0000","last_became_active":"2024-09-17T05:04:17.460628+0000","last_became_peered":"2024-09-17T05:04:17.460628+0000","last_unstale":"2024-09-17T05:04:17.468328+0000","last_undegraded":"2024-09-17T05:04:17.468328+0000","last_fullsized":"2024-09-17T05:04:17.468328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:14.634121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.433874+0000","last_change":"2024-09-17T05:04:16.444709+0000","last_active":"2024-09-17T05:04:17.433874+0000","last_peered":"2024-09-17T05:04:17.433874+0000","last_clean":"2024-09-17T05:04:17.433874+0000","last_became_active":"2024-09-17T05:04:16.444327+0000","last_became_peered":"2024-09-17T05:04:16.444327+0000","last_unstale":"2024-09-17T05:04:17.433874+0000","last_undegraded":"2024-09-17T05:04:17.433874+0000","last_fullsized":"2024-09-17T05:04:17.433874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:57.314622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457947+0000","last_change":"2024-09-17T05:04:17.448248+0000","last_active":"2024-09-17T05:04:17.457947+0000","last_peered":"2024-09-17T05:04:17.457947+0000","last_clean":"2024-09-17T05:04:17.457947+0000","last_became_active":"2024-09-17T05:04:17.447904+0000","last_became_peered":"2024-09-17T05:04:17.447904+0000","last_unstale":"2024-09-17T05:04:17.457947+0000","last_undegraded":"2024-09-17T05:04:17.457947+0000","last_fullsized":"2024-09-17T05:04:17.457947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:20.651318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807584+0000","last_change":"2024-09-17T05:04:16.433895+0000","last_active":"2024-09-17T05:04:18.807584+0000","last_peered":"2024-09-17T05:04:18.807584+0000","last_clean":"2024-09-17T05:04:18.807584+0000","last_became_active":"2024-09-17T05:04:16.433554+0000","last_became_peered":"2024-09-17T05:04:16.433554+0000","last_unstale":"2024-09-17T05:04:18.807584+0000","last_undegraded":"2024-09-17T05:04:18.807584+0000","last_fullsized":"2024-09-17T05:04:18.807584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:47.555968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.420520+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:17.420520+0000","last_peered":"2024-09-17T05:04:17.420520+0000","last_clean":"2024-09-17T05:04:17.420520+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:17.420520+0000","last_undegraded":"2024-09-17T05:04:17.420520+0000","last_fullsized":"2024-09-17T05:04:17.420520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":905216,"data_stored":404829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":996171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":404179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1507328,"data_stored":1001307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:04:21.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:04:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 -- 172.21.15.16:0/2650125959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 msgr2=0x7fab24073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 --2- 172.21.15.16:0/2650125959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab24073650 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab24071ce0 tx=0x7fab1c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 -- 172.21.15.16:0/2650125959 shutdown_connections 2024-09-17T05:04:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 --2- 172.21.15.16:0/2650125959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab24073650 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 --2- 172.21.15.16:0/2650125959 >> v2:172.21.15.104:3300/0 conn(0x7fab240b2f50 0x7fab2415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 -- 172.21.15.16:0/2650125959 >> 172.21.15.16:0/2650125959 conn(0x7fab2406f900 msgr2=0x7fab240b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 -- 172.21.15.16:0/2650125959 shutdown_connections 2024-09-17T05:04:21.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 -- 172.21.15.16:0/2650125959 wait complete. 2024-09-17T05:04:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 Processor -- start 2024-09-17T05:04:21.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.825+0000 7fab2b400640 1 -- start start 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab240cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fab240b2f50 0x7fab2415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2415c0a0 0x7fab240ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab24168d10 con 0x7fab24073240 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fab24168b90 con 0x7fab2415c0a0 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fab24168e90 con 0x7fab240b2f50 2024-09-17T05:04:21.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab240cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab240cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48184/0 (socket says 172.21.15.16:48184) 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 -- 172.21.15.16:0/1280771645 learned_addr learned my addr 172.21.15.16:0/1280771645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab23e00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fab240b2f50 0x7fab2415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab29400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2415c0a0 0x7fab240ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 -- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fab240b2f50 msgr2=0x7fab2415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fab240b2f50 0x7fab2415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 -- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2415c0a0 msgr2=0x7fab240ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2415c0a0 0x7fab240ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 -- 172.21.15.16:0/1280771645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab24142020 con 0x7fab24073240 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab28a00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab240cb920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab1800d720 tx=0x7fab1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18017070 con 0x7fab24073240 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab18009d20 con 0x7fab24073240 2024-09-17T05:04:21.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab24142300 con 0x7fab24073240 2024-09-17T05:04:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18005280 con 0x7fab24073240 2024-09-17T05:04:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab240ce120 con 0x7fab24073240 2024-09-17T05:04:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.829+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab24073650 con 0x7fab24073240 2024-09-17T05:04:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.833+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fab18005420 con 0x7fab24073240 2024-09-17T05:04:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.833+0000 7fab20c00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fab04073200 0x7fab04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.833+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fab18094580 con 0x7fab24073240 2024-09-17T05:04:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.833+0000 7fab23e00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fab04073200 0x7fab04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.833+0000 7fab23e00640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fab04073200 0x7fab04075710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab2415e310 tx=0x7fab10009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:21.837+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fab1805f370 con 0x7fab24073240 2024-09-17T05:04:22.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.009+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab2406a7c0 con 0x7fab04073200 2024-09-17T05:04:22.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.013+0000 7fab20c00640 1 -- 172.21.15.16:0/1280771645 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fab2406a7c0 con 0x7fab04073200 2024-09-17T05:04:22.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:22.020 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fab04073200 msgr2=0x7fab04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fab04073200 0x7fab04075710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fab2415e310 tx=0x7fab10009040 comp rx=0 tx=0).stop 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 msgr2=0x7fab240cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab240cb920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab1800d720 tx=0x7fab1800dbe0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 shutdown_connections 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fab04073200 0x7fab04075710 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2415c0a0 0x7fab240ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fab240b2f50 0x7fab2415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 --2- 172.21.15.16:0/1280771645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab24073240 0x7fab240cb920 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 >> 172.21.15.16:0/1280771645 conn(0x7fab2406f900 msgr2=0x7fab240cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 shutdown_connections 2024-09-17T05:04:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.017+0000 7fab2b400640 1 -- 172.21.15.16:0/1280771645 wait complete. 2024-09-17T05:04:22.041 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-17T05:04:21.511708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253532,"kb_used_data":8164,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721188,"statfs":{"total":773094113280,"available":772834496512,"internally_reserved":0,"allocated":8359936,"data_stored":4394446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001420"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.451528+0000","last_change":"2024-09-17T05:04:17.442664+0000","last_active":"2024-09-17T05:04:17.451528+0000","last_peered":"2024-09-17T05:04:17.451528+0000","last_clean":"2024-09-17T05:04:17.451528+0000","last_became_active":"2024-09-17T05:04:17.442351+0000","last_became_peered":"2024-09-17T05:04:17.442351+0000","last_unstale":"2024-09-17T05:04:17.451528+0000","last_undegraded":"2024-09-17T05:04:17.451528+0000","last_fullsized":"2024-09-17T05:04:17.451528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:27.874024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464586+0000","last_change":"2024-09-17T05:04:17.459817+0000","last_active":"2024-09-17T05:04:17.464586+0000","last_peered":"2024-09-17T05:04:17.464586+0000","last_clean":"2024-09-17T05:04:17.464586+0000","last_became_active":"2024-09-17T05:04:17.454983+0000","last_became_peered":"2024-09-17T05:04:17.454983+0000","last_unstale":"2024-09-17T05:04:17.464586+0000","last_undegraded":"2024-09-17T05:04:17.464586+0000","last_fullsized":"2024-09-17T05:04:17.464586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:20.175934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.450538+0000","last_change":"2024-09-17T05:04:17.447649+0000","last_active":"2024-09-17T05:04:17.450538+0000","last_peered":"2024-09-17T05:04:17.450538+0000","last_clean":"2024-09-17T05:04:17.450538+0000","last_became_active":"2024-09-17T05:04:17.447323+0000","last_became_peered":"2024-09-17T05:04:17.447323+0000","last_unstale":"2024-09-17T05:04:17.450538+0000","last_undegraded":"2024-09-17T05:04:17.450538+0000","last_fullsized":"2024-09-17T05:04:17.450538+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:43.617145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.446500+0000","last_change":"2024-09-17T05:04:17.444031+0000","last_active":"2024-09-17T05:04:17.446500+0000","last_peered":"2024-09-17T05:04:17.446500+0000","last_clean":"2024-09-17T05:04:17.446500+0000","last_became_active":"2024-09-17T05:04:17.443661+0000","last_became_peered":"2024-09-17T05:04:17.443661+0000","last_unstale":"2024-09-17T05:04:17.446500+0000","last_undegraded":"2024-09-17T05:04:17.446500+0000","last_fullsized":"2024-09-17T05:04:17.446500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:20.143372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469888+0000","last_change":"2024-09-17T05:04:17.466049+0000","last_active":"2024-09-17T05:04:17.469888+0000","last_peered":"2024-09-17T05:04:17.469888+0000","last_clean":"2024-09-17T05:04:17.469888+0000","last_became_active":"2024-09-17T05:04:17.465715+0000","last_became_peered":"2024-09-17T05:04:17.465715+0000","last_unstale":"2024-09-17T05:04:17.469888+0000","last_undegraded":"2024-09-17T05:04:17.469888+0000","last_fullsized":"2024-09-17T05:04:17.469888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:07.872979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807584+0000","last_change":"2024-09-17T05:04:17.444706+0000","last_active":"2024-09-17T05:04:18.807584+0000","last_peered":"2024-09-17T05:04:18.807584+0000","last_clean":"2024-09-17T05:04:18.807584+0000","last_became_active":"2024-09-17T05:04:17.444320+0000","last_became_peered":"2024-09-17T05:04:17.444320+0000","last_unstale":"2024-09-17T05:04:18.807584+0000","last_undegraded":"2024-09-17T05:04:18.807584+0000","last_fullsized":"2024-09-17T05:04:18.807584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:52.449310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457059+0000","last_change":"2024-09-17T05:04:17.449489+0000","last_active":"2024-09-17T05:04:17.457059+0000","last_peered":"2024-09-17T05:04:17.457059+0000","last_clean":"2024-09-17T05:04:17.457059+0000","last_became_active":"2024-09-17T05:04:17.449174+0000","last_became_peered":"2024-09-17T05:04:17.449174+0000","last_unstale":"2024-09-17T05:04:17.457059+0000","last_undegraded":"2024-09-17T05:04:17.457059+0000","last_fullsized":"2024-09-17T05:04:17.457059+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:38.933711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457492+0000","last_change":"2024-09-17T05:04:17.443462+0000","last_active":"2024-09-17T05:04:17.457492+0000","last_peered":"2024-09-17T05:04:17.457492+0000","last_clean":"2024-09-17T05:04:17.457492+0000","last_became_active":"2024-09-17T05:04:17.443151+0000","last_became_peered":"2024-09-17T05:04:17.443151+0000","last_unstale":"2024-09-17T05:04:17.457492+0000","last_undegraded":"2024-09-17T05:04:17.457492+0000","last_fullsized":"2024-09-17T05:04:17.457492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:01.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458079+0000","last_change":"2024-09-17T05:04:17.454992+0000","last_active":"2024-09-17T05:04:17.458079+0000","last_peered":"2024-09-17T05:04:17.458079+0000","last_clean":"2024-09-17T05:04:17.458079+0000","last_became_active":"2024-09-17T05:04:17.453327+0000","last_became_peered":"2024-09-17T05:04:17.453327+0000","last_unstale":"2024-09-17T05:04:17.458079+0000","last_undegraded":"2024-09-17T05:04:17.458079+0000","last_fullsized":"2024-09-17T05:04:17.458079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:57:43.617145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.456032+0000","last_change":"2024-09-17T05:04:17.453458+0000","last_active":"2024-09-17T05:04:17.456032+0000","last_peered":"2024-09-17T05:04:17.456032+0000","last_clean":"2024-09-17T05:04:17.456032+0000","last_became_active":"2024-09-17T05:04:17.453166+0000","last_became_peered":"2024-09-17T05:04:17.453166+0000","last_unstale":"2024-09-17T05:04:17.456032+0000","last_undegraded":"2024-09-17T05:04:17.456032+0000","last_fullsized":"2024-09-17T05:04:17.456032+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:52.536071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.462130+0000","last_change":"2024-09-17T05:04:17.453688+0000","last_active":"2024-09-17T05:04:17.462130+0000","last_peered":"2024-09-17T05:04:17.462130+0000","last_clean":"2024-09-17T05:04:17.462130+0000","last_became_active":"2024-09-17T05:04:17.453006+0000","last_became_peered":"2024-09-17T05:04:17.453006+0000","last_unstale":"2024-09-17T05:04:17.462130+0000","last_undegraded":"2024-09-17T05:04:17.462130+0000","last_fullsized":"2024-09-17T05:04:17.462130+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:08.770442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454966+0000","last_change":"2024-09-17T05:04:17.447816+0000","last_active":"2024-09-17T05:04:17.454966+0000","last_peered":"2024-09-17T05:04:17.454966+0000","last_clean":"2024-09-17T05:04:17.454966+0000","last_became_active":"2024-09-17T05:04:17.447508+0000","last_became_peered":"2024-09-17T05:04:17.447508+0000","last_unstale":"2024-09-17T05:04:17.454966+0000","last_undegraded":"2024-09-17T05:04:17.454966+0000","last_fullsized":"2024-09-17T05:04:17.454966+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:45.339567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.466729+0000","last_change":"2024-09-17T05:04:17.461226+0000","last_active":"2024-09-17T05:04:17.466729+0000","last_peered":"2024-09-17T05:04:17.466729+0000","last_clean":"2024-09-17T05:04:17.466729+0000","last_became_active":"2024-09-17T05:04:17.460716+0000","last_became_peered":"2024-09-17T05:04:17.460716+0000","last_unstale":"2024-09-17T05:04:17.466729+0000","last_undegraded":"2024-09-17T05:04:17.466729+0000","last_fullsized":"2024-09-17T05:04:17.466729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:44.049692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459351+0000","last_change":"2024-09-17T05:04:17.447882+0000","last_active":"2024-09-17T05:04:17.459351+0000","last_peered":"2024-09-17T05:04:17.459351+0000","last_clean":"2024-09-17T05:04:17.459351+0000","last_became_active":"2024-09-17T05:04:17.447553+0000","last_became_peered":"2024-09-17T05:04:17.447553+0000","last_unstale":"2024-09-17T05:04:17.459351+0000","last_undegraded":"2024-09-17T05:04:17.459351+0000","last_fullsized":"2024-09-17T05:04:17.459351+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:44.358823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468490+0000","last_change":"2024-09-17T05:04:17.461112+0000","last_active":"2024-09-17T05:04:17.468490+0000","last_peered":"2024-09-17T05:04:17.468490+0000","last_clean":"2024-09-17T05:04:17.468490+0000","last_became_active":"2024-09-17T05:04:17.460799+0000","last_became_peered":"2024-09-17T05:04:17.460799+0000","last_unstale":"2024-09-17T05:04:17.468490+0000","last_undegraded":"2024-09-17T05:04:17.468490+0000","last_fullsized":"2024-09-17T05:04:17.468490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:31.247288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454713+0000","last_change":"2024-09-17T05:04:17.448651+0000","last_active":"2024-09-17T05:04:17.454713+0000","last_peered":"2024-09-17T05:04:17.454713+0000","last_clean":"2024-09-17T05:04:17.454713+0000","last_became_active":"2024-09-17T05:04:17.447495+0000","last_became_peered":"2024-09-17T05:04:17.447495+0000","last_unstale":"2024-09-17T05:04:17.454713+0000","last_undegraded":"2024-09-17T05:04:17.454713+0000","last_fullsized":"2024-09-17T05:04:17.454713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:28.651823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458932+0000","last_change":"2024-09-17T05:04:17.447180+0000","last_active":"2024-09-17T05:04:17.458932+0000","last_peered":"2024-09-17T05:04:17.458932+0000","last_clean":"2024-09-17T05:04:17.458932+0000","last_became_active":"2024-09-17T05:04:17.446852+0000","last_became_peered":"2024-09-17T05:04:17.446852+0000","last_unstale":"2024-09-17T05:04:17.458932+0000","last_undegraded":"2024-09-17T05:04:17.458932+0000","last_fullsized":"2024-09-17T05:04:17.458932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:32:51.404799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464616+0000","last_change":"2024-09-17T05:04:17.460158+0000","last_active":"2024-09-17T05:04:17.464616+0000","last_peered":"2024-09-17T05:04:17.464616+0000","last_clean":"2024-09-17T05:04:17.464616+0000","last_became_active":"2024-09-17T05:04:17.455481+0000","last_became_peered":"2024-09-17T05:04:17.455481+0000","last_unstale":"2024-09-17T05:04:17.464616+0000","last_undegraded":"2024-09-17T05:04:17.464616+0000","last_fullsized":"2024-09-17T05:04:17.464616+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:11.790270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.472383+0000","last_change":"2024-09-17T05:04:17.466812+0000","last_active":"2024-09-17T05:04:17.472383+0000","last_peered":"2024-09-17T05:04:17.472383+0000","last_clean":"2024-09-17T05:04:17.472383+0000","last_became_active":"2024-09-17T05:04:17.466487+0000","last_became_peered":"2024-09-17T05:04:17.466487+0000","last_unstale":"2024-09-17T05:04:17.472383+0000","last_undegraded":"2024-09-17T05:04:17.472383+0000","last_fullsized":"2024-09-17T05:04:17.472383+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:13.329981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807596+0000","last_change":"2024-09-17T05:04:17.455756+0000","last_active":"2024-09-17T05:04:18.807596+0000","last_peered":"2024-09-17T05:04:18.807596+0000","last_clean":"2024-09-17T05:04:18.807596+0000","last_became_active":"2024-09-17T05:04:17.455437+0000","last_became_peered":"2024-09-17T05:04:17.455437+0000","last_unstale":"2024-09-17T05:04:18.807596+0000","last_undegraded":"2024-09-17T05:04:18.807596+0000","last_fullsized":"2024-09-17T05:04:18.807596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:07.476987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.461829+0000","last_change":"2024-09-17T05:04:17.459594+0000","last_active":"2024-09-17T05:04:17.461829+0000","last_peered":"2024-09-17T05:04:17.461829+0000","last_clean":"2024-09-17T05:04:17.461829+0000","last_became_active":"2024-09-17T05:04:17.459085+0000","last_became_peered":"2024-09-17T05:04:17.459085+0000","last_unstale":"2024-09-17T05:04:17.461829+0000","last_undegraded":"2024-09-17T05:04:17.461829+0000","last_fullsized":"2024-09-17T05:04:17.461829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:48.014480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460209+0000","last_change":"2024-09-17T05:04:17.452690+0000","last_active":"2024-09-17T05:04:17.460209+0000","last_peered":"2024-09-17T05:04:17.460209+0000","last_clean":"2024-09-17T05:04:17.460209+0000","last_became_active":"2024-09-17T05:04:17.452356+0000","last_became_peered":"2024-09-17T05:04:17.452356+0000","last_unstale":"2024-09-17T05:04:17.460209+0000","last_undegraded":"2024-09-17T05:04:17.460209+0000","last_fullsized":"2024-09-17T05:04:17.460209+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:47.020135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.440210+0000","last_change":"2024-09-17T05:04:17.431263+0000","last_active":"2024-09-17T05:04:17.440210+0000","last_peered":"2024-09-17T05:04:17.440210+0000","last_clean":"2024-09-17T05:04:17.440210+0000","last_became_active":"2024-09-17T05:04:17.430952+0000","last_became_peered":"2024-09-17T05:04:17.430952+0000","last_unstale":"2024-09-17T05:04:17.440210+0000","last_undegraded":"2024-09-17T05:04:17.440210+0000","last_fullsized":"2024-09-17T05:04:17.440210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:26.010989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459774+0000","last_change":"2024-09-17T05:04:17.450645+0000","last_active":"2024-09-17T05:04:17.459774+0000","last_peered":"2024-09-17T05:04:17.459774+0000","last_clean":"2024-09-17T05:04:17.459774+0000","last_became_active":"2024-09-17T05:04:17.449831+0000","last_became_peered":"2024-09-17T05:04:17.449831+0000","last_unstale":"2024-09-17T05:04:17.459774+0000","last_undegraded":"2024-09-17T05:04:17.459774+0000","last_fullsized":"2024-09-17T05:04:17.459774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:01:43.872510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807970+0000","last_change":"2024-09-17T05:04:17.450178+0000","last_active":"2024-09-17T05:04:18.807970+0000","last_peered":"2024-09-17T05:04:18.807970+0000","last_clean":"2024-09-17T05:04:18.807970+0000","last_became_active":"2024-09-17T05:04:17.449780+0000","last_became_peered":"2024-09-17T05:04:17.449780+0000","last_unstale":"2024-09-17T05:04:18.807970+0000","last_undegraded":"2024-09-17T05:04:18.807970+0000","last_fullsized":"2024-09-17T05:04:18.807970+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:34.770065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.456627+0000","last_change":"2024-09-17T05:04:17.453491+0000","last_active":"2024-09-17T05:04:17.456627+0000","last_peered":"2024-09-17T05:04:17.456627+0000","last_clean":"2024-09-17T05:04:17.456627+0000","last_became_active":"2024-09-17T05:04:17.453173+0000","last_became_peered":"2024-09-17T05:04:17.453173+0000","last_unstale":"2024-09-17T05:04:17.456627+0000","last_undegraded":"2024-09-17T05:04:17.456627+0000","last_fullsized":"2024-09-17T05:04:17.456627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:44.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.456361+0000","last_change":"2024-09-17T05:04:17.451591+0000","last_active":"2024-09-17T05:04:17.456361+0000","last_peered":"2024-09-17T05:04:17.456361+0000","last_clean":"2024-09-17T05:04:17.456361+0000","last_became_active":"2024-09-17T05:04:17.451279+0000","last_became_peered":"2024-09-17T05:04:17.451279+0000","last_unstale":"2024-09-17T05:04:17.456361+0000","last_undegraded":"2024-09-17T05:04:17.456361+0000","last_fullsized":"2024-09-17T05:04:17.456361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:35:41.193297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469832+0000","last_change":"2024-09-17T05:04:17.466009+0000","last_active":"2024-09-17T05:04:17.469832+0000","last_peered":"2024-09-17T05:04:17.469832+0000","last_clean":"2024-09-17T05:04:17.469832+0000","last_became_active":"2024-09-17T05:04:17.465666+0000","last_became_peered":"2024-09-17T05:04:17.465666+0000","last_unstale":"2024-09-17T05:04:17.469832+0000","last_undegraded":"2024-09-17T05:04:17.469832+0000","last_fullsized":"2024-09-17T05:04:17.469832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:19.467327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807504+0000","last_change":"2024-09-17T05:04:17.433194+0000","last_active":"2024-09-17T05:04:18.807504+0000","last_peered":"2024-09-17T05:04:18.807504+0000","last_clean":"2024-09-17T05:04:18.807504+0000","last_became_active":"2024-09-17T05:04:17.431679+0000","last_became_peered":"2024-09-17T05:04:17.431679+0000","last_unstale":"2024-09-17T05:04:18.807504+0000","last_undegraded":"2024-09-17T05:04:18.807504+0000","last_fullsized":"2024-09-17T05:04:18.807504+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:12.696484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.446973+0000","last_change":"2024-09-17T05:04:17.438821+0000","last_active":"2024-09-17T05:04:17.446973+0000","last_peered":"2024-09-17T05:04:17.446973+0000","last_clean":"2024-09-17T05:04:17.446973+0000","last_became_active":"2024-09-17T05:04:17.436552+0000","last_became_peered":"2024-09-17T05:04:17.436552+0000","last_unstale":"2024-09-17T05:04:17.446973+0000","last_undegraded":"2024-09-17T05:04:17.446973+0000","last_fullsized":"2024-09-17T05:04:17.446973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:31.889570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460422+0000","last_change":"2024-09-17T05:04:17.450932+0000","last_active":"2024-09-17T05:04:17.460422+0000","last_peered":"2024-09-17T05:04:17.460422+0000","last_clean":"2024-09-17T05:04:17.460422+0000","last_became_active":"2024-09-17T05:04:17.449856+0000","last_became_peered":"2024-09-17T05:04:17.449856+0000","last_unstale":"2024-09-17T05:04:17.460422+0000","last_undegraded":"2024-09-17T05:04:17.460422+0000","last_fullsized":"2024-09-17T05:04:17.460422+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:10.578051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454651+0000","last_change":"2024-09-17T05:04:17.447868+0000","last_active":"2024-09-17T05:04:17.454651+0000","last_peered":"2024-09-17T05:04:17.454651+0000","last_clean":"2024-09-17T05:04:17.454651+0000","last_became_active":"2024-09-17T05:04:17.445741+0000","last_became_peered":"2024-09-17T05:04:17.445741+0000","last_unstale":"2024-09-17T05:04:17.454651+0000","last_undegraded":"2024-09-17T05:04:17.454651+0000","last_fullsized":"2024-09-17T05:04:17.454651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:37.843774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468218+0000","last_change":"2024-09-17T05:04:17.458984+0000","last_active":"2024-09-17T05:04:17.468218+0000","last_peered":"2024-09-17T05:04:17.468218+0000","last_clean":"2024-09-17T05:04:17.468218+0000","last_became_active":"2024-09-17T05:04:17.458664+0000","last_became_peered":"2024-09-17T05:04:17.458664+0000","last_unstale":"2024-09-17T05:04:17.468218+0000","last_undegraded":"2024-09-17T05:04:17.468218+0000","last_fullsized":"2024-09-17T05:04:17.468218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:34.829225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":17,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468423+0000","last_change":"2024-09-17T05:04:16.442302+0000","last_active":"2024-09-17T05:04:17.468423+0000","last_peered":"2024-09-17T05:04:17.468423+0000","last_clean":"2024-09-17T05:04:17.468423+0000","last_became_active":"2024-09-17T05:04:16.441959+0000","last_became_peered":"2024-09-17T05:04:16.441959+0000","last_unstale":"2024-09-17T05:04:17.468423+0000","last_undegraded":"2024-09-17T05:04:17.468423+0000","last_fullsized":"2024-09-17T05:04:17.468423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:59:00.712657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.465529+0000","last_change":"2024-09-17T05:04:17.449749+0000","last_active":"2024-09-17T05:04:17.465529+0000","last_peered":"2024-09-17T05:04:17.465529+0000","last_clean":"2024-09-17T05:04:17.465529+0000","last_became_active":"2024-09-17T05:04:17.449353+0000","last_became_peered":"2024-09-17T05:04:17.449353+0000","last_unstale":"2024-09-17T05:04:17.465529+0000","last_undegraded":"2024-09-17T05:04:17.465529+0000","last_fullsized":"2024-09-17T05:04:17.465529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:48.683758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464594+0000","last_change":"2024-09-17T05:04:16.452150+0000","last_active":"2024-09-17T05:04:17.464594+0000","last_peered":"2024-09-17T05:04:17.464594+0000","last_clean":"2024-09-17T05:04:17.464594+0000","last_became_active":"2024-09-17T05:04:16.451803+0000","last_became_peered":"2024-09-17T05:04:16.451803+0000","last_unstale":"2024-09-17T05:04:17.464594+0000","last_undegraded":"2024-09-17T05:04:17.464594+0000","last_fullsized":"2024-09-17T05:04:17.464594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:02.922815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.452768+0000","last_change":"2024-09-17T05:04:17.447126+0000","last_active":"2024-09-17T05:04:17.452768+0000","last_peered":"2024-09-17T05:04:17.452768+0000","last_clean":"2024-09-17T05:04:17.452768+0000","last_became_active":"2024-09-17T05:04:17.446704+0000","last_became_peered":"2024-09-17T05:04:17.446704+0000","last_unstale":"2024-09-17T05:04:17.452768+0000","last_undegraded":"2024-09-17T05:04:17.452768+0000","last_fullsized":"2024-09-17T05:04:17.452768+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:44.049692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460262+0000","last_change":"2024-09-17T05:04:16.436756+0000","last_active":"2024-09-17T05:04:17.460262+0000","last_peered":"2024-09-17T05:04:17.460262+0000","last_clean":"2024-09-17T05:04:17.460262+0000","last_became_active":"2024-09-17T05:04:16.436415+0000","last_became_peered":"2024-09-17T05:04:16.436415+0000","last_unstale":"2024-09-17T05:04:17.460262+0000","last_undegraded":"2024-09-17T05:04:17.460262+0000","last_fullsized":"2024-09-17T05:04:17.460262+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:01.351567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454684+0000","last_change":"2024-09-17T05:04:17.449523+0000","last_active":"2024-09-17T05:04:17.454684+0000","last_peered":"2024-09-17T05:04:17.454684+0000","last_clean":"2024-09-17T05:04:17.454684+0000","last_became_active":"2024-09-17T05:04:17.449193+0000","last_became_peered":"2024-09-17T05:04:17.449193+0000","last_unstale":"2024-09-17T05:04:17.454684+0000","last_undegraded":"2024-09-17T05:04:17.454684+0000","last_fullsized":"2024-09-17T05:04:17.454684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:54.872877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458834+0000","last_change":"2024-09-17T05:04:16.440778+0000","last_active":"2024-09-17T05:04:17.458834+0000","last_peered":"2024-09-17T05:04:17.458834+0000","last_clean":"2024-09-17T05:04:17.458834+0000","last_became_active":"2024-09-17T05:04:16.440416+0000","last_became_peered":"2024-09-17T05:04:16.440416+0000","last_unstale":"2024-09-17T05:04:17.458834+0000","last_undegraded":"2024-09-17T05:04:17.458834+0000","last_fullsized":"2024-09-17T05:04:17.458834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:07.767221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807579+0000","last_change":"2024-09-17T05:04:17.455425+0000","last_active":"2024-09-17T05:04:18.807579+0000","last_peered":"2024-09-17T05:04:18.807579+0000","last_clean":"2024-09-17T05:04:18.807579+0000","last_became_active":"2024-09-17T05:04:17.455012+0000","last_became_peered":"2024-09-17T05:04:17.455012+0000","last_unstale":"2024-09-17T05:04:18.807579+0000","last_undegraded":"2024-09-17T05:04:18.807579+0000","last_fullsized":"2024-09-17T05:04:18.807579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:03.218826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.455983+0000","last_change":"2024-09-17T05:04:16.443460+0000","last_active":"2024-09-17T05:04:17.455983+0000","last_peered":"2024-09-17T05:04:17.455983+0000","last_clean":"2024-09-17T05:04:17.455983+0000","last_became_active":"2024-09-17T05:04:16.439515+0000","last_became_peered":"2024-09-17T05:04:16.439515+0000","last_unstale":"2024-09-17T05:04:17.455983+0000","last_undegraded":"2024-09-17T05:04:17.455983+0000","last_fullsized":"2024-09-17T05:04:17.455983+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:19.172713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.460461+0000","last_change":"2024-09-17T05:04:17.451069+0000","last_active":"2024-09-17T05:04:17.460461+0000","last_peered":"2024-09-17T05:04:17.460461+0000","last_clean":"2024-09-17T05:04:17.460461+0000","last_became_active":"2024-09-17T05:04:17.450281+0000","last_became_peered":"2024-09-17T05:04:17.450281+0000","last_unstale":"2024-09-17T05:04:17.460461+0000","last_undegraded":"2024-09-17T05:04:17.460461+0000","last_fullsized":"2024-09-17T05:04:17.460461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:31.334395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.467646+0000","last_change":"2024-09-17T05:04:16.442691+0000","last_active":"2024-09-17T05:04:17.467646+0000","last_peered":"2024-09-17T05:04:17.467646+0000","last_clean":"2024-09-17T05:04:17.467646+0000","last_became_active":"2024-09-17T05:04:16.442154+0000","last_became_peered":"2024-09-17T05:04:16.442154+0000","last_unstale":"2024-09-17T05:04:17.467646+0000","last_undegraded":"2024-09-17T05:04:17.467646+0000","last_fullsized":"2024-09-17T05:04:17.467646+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:00.163306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454942+0000","last_change":"2024-09-17T05:04:17.451396+0000","last_active":"2024-09-17T05:04:17.454942+0000","last_peered":"2024-09-17T05:04:17.454942+0000","last_clean":"2024-09-17T05:04:17.454942+0000","last_became_active":"2024-09-17T05:04:17.450633+0000","last_became_peered":"2024-09-17T05:04:17.450633+0000","last_unstale":"2024-09-17T05:04:17.454942+0000","last_undegraded":"2024-09-17T05:04:17.454942+0000","last_fullsized":"2024-09-17T05:04:17.454942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:58.348078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.431285+0000","last_change":"2024-09-17T05:04:16.437012+0000","last_active":"2024-09-17T05:04:17.431285+0000","last_peered":"2024-09-17T05:04:17.431285+0000","last_clean":"2024-09-17T05:04:17.431285+0000","last_became_active":"2024-09-17T05:04:16.436709+0000","last_became_peered":"2024-09-17T05:04:16.436709+0000","last_unstale":"2024-09-17T05:04:17.431285+0000","last_undegraded":"2024-09-17T05:04:17.431285+0000","last_fullsized":"2024-09-17T05:04:17.431285+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:24.735292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464871+0000","last_change":"2024-09-17T05:04:17.461532+0000","last_active":"2024-09-17T05:04:17.464871+0000","last_peered":"2024-09-17T05:04:17.464871+0000","last_clean":"2024-09-17T05:04:17.464871+0000","last_became_active":"2024-09-17T05:04:17.461217+0000","last_became_peered":"2024-09-17T05:04:17.461217+0000","last_unstale":"2024-09-17T05:04:17.464871+0000","last_undegraded":"2024-09-17T05:04:17.464871+0000","last_fullsized":"2024-09-17T05:04:17.464871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:33.022867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.433614+0000","last_change":"2024-09-17T05:04:16.439787+0000","last_active":"2024-09-17T05:04:17.433614+0000","last_peered":"2024-09-17T05:04:17.433614+0000","last_clean":"2024-09-17T05:04:17.433614+0000","last_became_active":"2024-09-17T05:04:16.439130+0000","last_became_peered":"2024-09-17T05:04:16.439130+0000","last_unstale":"2024-09-17T05:04:17.433614+0000","last_undegraded":"2024-09-17T05:04:17.433614+0000","last_fullsized":"2024-09-17T05:04:17.433614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:55.643709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469629+0000","last_change":"2024-09-17T05:04:17.461794+0000","last_active":"2024-09-17T05:04:17.469629+0000","last_peered":"2024-09-17T05:04:17.469629+0000","last_clean":"2024-09-17T05:04:17.469629+0000","last_became_active":"2024-09-17T05:04:17.461457+0000","last_became_peered":"2024-09-17T05:04:17.461457+0000","last_unstale":"2024-09-17T05:04:17.469629+0000","last_undegraded":"2024-09-17T05:04:17.469629+0000","last_fullsized":"2024-09-17T05:04:17.469629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:53.912834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.438921+0000","last_change":"2024-09-17T05:04:16.440619+0000","last_active":"2024-09-17T05:04:17.438921+0000","last_peered":"2024-09-17T05:04:17.438921+0000","last_clean":"2024-09-17T05:04:17.438921+0000","last_became_active":"2024-09-17T05:04:16.440214+0000","last_became_peered":"2024-09-17T05:04:16.440214+0000","last_unstale":"2024-09-17T05:04:17.438921+0000","last_undegraded":"2024-09-17T05:04:17.438921+0000","last_fullsized":"2024-09-17T05:04:17.438921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:12.273507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.452491+0000","last_change":"2024-09-17T05:04:17.442139+0000","last_active":"2024-09-17T05:04:17.452491+0000","last_peered":"2024-09-17T05:04:17.452491+0000","last_clean":"2024-09-17T05:04:17.452491+0000","last_became_active":"2024-09-17T05:04:17.441809+0000","last_became_peered":"2024-09-17T05:04:17.441809+0000","last_unstale":"2024-09-17T05:04:17.452491+0000","last_undegraded":"2024-09-17T05:04:17.452491+0000","last_fullsized":"2024-09-17T05:04:17.452491+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:11.790270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.422718+0000","last_change":"2024-09-17T05:04:16.450058+0000","last_active":"2024-09-17T05:04:17.422718+0000","last_peered":"2024-09-17T05:04:17.422718+0000","last_clean":"2024-09-17T05:04:17.422718+0000","last_became_active":"2024-09-17T05:04:16.449701+0000","last_became_peered":"2024-09-17T05:04:16.449701+0000","last_unstale":"2024-09-17T05:04:17.422718+0000","last_undegraded":"2024-09-17T05:04:17.422718+0000","last_fullsized":"2024-09-17T05:04:17.422718+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:43.335136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459323+0000","last_change":"2024-09-17T05:04:17.447100+0000","last_active":"2024-09-17T05:04:17.459323+0000","last_peered":"2024-09-17T05:04:17.459323+0000","last_clean":"2024-09-17T05:04:17.459323+0000","last_became_active":"2024-09-17T05:04:17.446396+0000","last_became_peered":"2024-09-17T05:04:17.446396+0000","last_unstale":"2024-09-17T05:04:17.459323+0000","last_undegraded":"2024-09-17T05:04:17.459323+0000","last_fullsized":"2024-09-17T05:04:17.459323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:08.333394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807271+0000","last_change":"2024-09-17T05:04:16.433675+0000","last_active":"2024-09-17T05:04:18.807271+0000","last_peered":"2024-09-17T05:04:18.807271+0000","last_clean":"2024-09-17T05:04:18.807271+0000","last_became_active":"2024-09-17T05:04:16.433330+0000","last_became_peered":"2024-09-17T05:04:16.433330+0000","last_unstale":"2024-09-17T05:04:18.807271+0000","last_undegraded":"2024-09-17T05:04:18.807271+0000","last_fullsized":"2024-09-17T05:04:18.807271+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:54:38.828077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807378+0000","last_change":"2024-09-17T05:04:17.458481+0000","last_active":"2024-09-17T05:04:18.807378+0000","last_peered":"2024-09-17T05:04:18.807378+0000","last_clean":"2024-09-17T05:04:18.807378+0000","last_became_active":"2024-09-17T05:04:17.458188+0000","last_became_peered":"2024-09-17T05:04:17.458188+0000","last_unstale":"2024-09-17T05:04:18.807378+0000","last_undegraded":"2024-09-17T05:04:18.807378+0000","last_fullsized":"2024-09-17T05:04:18.807378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:17:35.159360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":19,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468612+0000","last_change":"2024-09-17T05:04:16.450009+0000","last_active":"2024-09-17T05:04:17.468612+0000","last_peered":"2024-09-17T05:04:17.468612+0000","last_clean":"2024-09-17T05:04:17.468612+0000","last_became_active":"2024-09-17T05:04:16.444154+0000","last_became_peered":"2024-09-17T05:04:16.444154+0000","last_unstale":"2024-09-17T05:04:17.468612+0000","last_undegraded":"2024-09-17T05:04:17.468612+0000","last_fullsized":"2024-09-17T05:04:17.468612+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:04.237584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.453886+0000","last_change":"2024-09-17T05:04:17.446638+0000","last_active":"2024-09-17T05:04:17.453886+0000","last_peered":"2024-09-17T05:04:17.453886+0000","last_clean":"2024-09-17T05:04:17.453886+0000","last_became_active":"2024-09-17T05:04:17.446297+0000","last_became_peered":"2024-09-17T05:04:17.446297+0000","last_unstale":"2024-09-17T05:04:17.453886+0000","last_undegraded":"2024-09-17T05:04:17.453886+0000","last_fullsized":"2024-09-17T05:04:17.453886+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:57.864805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457308+0000","last_change":"2024-09-17T05:04:16.439931+0000","last_active":"2024-09-17T05:04:17.457308+0000","last_peered":"2024-09-17T05:04:17.457308+0000","last_clean":"2024-09-17T05:04:17.457308+0000","last_became_active":"2024-09-17T05:04:16.438817+0000","last_became_peered":"2024-09-17T05:04:16.438817+0000","last_unstale":"2024-09-17T05:04:17.457308+0000","last_undegraded":"2024-09-17T05:04:17.457308+0000","last_fullsized":"2024-09-17T05:04:17.457308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:30.244067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464419+0000","last_change":"2024-09-17T05:04:17.461527+0000","last_active":"2024-09-17T05:04:17.464419+0000","last_peered":"2024-09-17T05:04:17.464419+0000","last_clean":"2024-09-17T05:04:17.464419+0000","last_became_active":"2024-09-17T05:04:17.461065+0000","last_became_peered":"2024-09-17T05:04:17.461065+0000","last_unstale":"2024-09-17T05:04:17.464419+0000","last_undegraded":"2024-09-17T05:04:17.464419+0000","last_fullsized":"2024-09-17T05:04:17.464419+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:10.230753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.428499+0000","last_change":"2024-09-17T05:04:16.439186+0000","last_active":"2024-09-17T05:04:17.428499+0000","last_peered":"2024-09-17T05:04:17.428499+0000","last_clean":"2024-09-17T05:04:17.428499+0000","last_became_active":"2024-09-17T05:04:16.438799+0000","last_became_peered":"2024-09-17T05:04:16.438799+0000","last_unstale":"2024-09-17T05:04:17.428499+0000","last_undegraded":"2024-09-17T05:04:17.428499+0000","last_fullsized":"2024-09-17T05:04:17.428499+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:46.016914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.455794+0000","last_change":"2024-09-17T05:04:17.454512+0000","last_active":"2024-09-17T05:04:17.455794+0000","last_peered":"2024-09-17T05:04:17.455794+0000","last_clean":"2024-09-17T05:04:17.455794+0000","last_became_active":"2024-09-17T05:04:17.451077+0000","last_became_peered":"2024-09-17T05:04:17.451077+0000","last_unstale":"2024-09-17T05:04:17.455794+0000","last_undegraded":"2024-09-17T05:04:17.455794+0000","last_fullsized":"2024-09-17T05:04:17.455794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:47.954107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.437956+0000","last_change":"2024-09-17T05:04:16.442319+0000","last_active":"2024-09-17T05:04:17.437956+0000","last_peered":"2024-09-17T05:04:17.437956+0000","last_clean":"2024-09-17T05:04:17.437956+0000","last_became_active":"2024-09-17T05:04:16.441887+0000","last_became_peered":"2024-09-17T05:04:16.441887+0000","last_unstale":"2024-09-17T05:04:17.437956+0000","last_undegraded":"2024-09-17T05:04:17.437956+0000","last_fullsized":"2024-09-17T05:04:17.437956+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:09.136966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.440049+0000","last_change":"2024-09-17T05:04:17.429711+0000","last_active":"2024-09-17T05:04:17.440049+0000","last_peered":"2024-09-17T05:04:17.440049+0000","last_clean":"2024-09-17T05:04:17.440049+0000","last_became_active":"2024-09-17T05:04:17.429352+0000","last_became_peered":"2024-09-17T05:04:17.429352+0000","last_unstale":"2024-09-17T05:04:17.440049+0000","last_undegraded":"2024-09-17T05:04:17.440049+0000","last_fullsized":"2024-09-17T05:04:17.440049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:09.402644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807108+0000","last_change":"2024-09-17T05:04:16.435462+0000","last_active":"2024-09-17T05:04:18.807108+0000","last_peered":"2024-09-17T05:04:18.807108+0000","last_clean":"2024-09-17T05:04:18.807108+0000","last_became_active":"2024-09-17T05:04:16.435083+0000","last_became_peered":"2024-09-17T05:04:16.435083+0000","last_unstale":"2024-09-17T05:04:18.807108+0000","last_undegraded":"2024-09-17T05:04:18.807108+0000","last_fullsized":"2024-09-17T05:04:18.807108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:05.691319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468530+0000","last_change":"2024-09-17T05:04:17.462582+0000","last_active":"2024-09-17T05:04:17.468530+0000","last_peered":"2024-09-17T05:04:17.468530+0000","last_clean":"2024-09-17T05:04:17.468530+0000","last_became_active":"2024-09-17T05:04:17.462192+0000","last_became_peered":"2024-09-17T05:04:17.462192+0000","last_unstale":"2024-09-17T05:04:17.468530+0000","last_undegraded":"2024-09-17T05:04:17.468530+0000","last_fullsized":"2024-09-17T05:04:17.468530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:02.354788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.449587+0000","last_change":"2024-09-17T05:04:16.445016+0000","last_active":"2024-09-17T05:04:17.449587+0000","last_peered":"2024-09-17T05:04:17.449587+0000","last_clean":"2024-09-17T05:04:17.449587+0000","last_became_active":"2024-09-17T05:04:16.444694+0000","last_became_peered":"2024-09-17T05:04:16.444694+0000","last_unstale":"2024-09-17T05:04:17.449587+0000","last_undegraded":"2024-09-17T05:04:17.449587+0000","last_fullsized":"2024-09-17T05:04:17.449587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:08.399423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.443191+0000","last_change":"2024-09-17T05:04:17.439211+0000","last_active":"2024-09-17T05:04:17.443191+0000","last_peered":"2024-09-17T05:04:17.443191+0000","last_clean":"2024-09-17T05:04:17.443191+0000","last_became_active":"2024-09-17T05:04:17.437945+0000","last_became_peered":"2024-09-17T05:04:17.437945+0000","last_unstale":"2024-09-17T05:04:17.443191+0000","last_undegraded":"2024-09-17T05:04:17.443191+0000","last_fullsized":"2024-09-17T05:04:17.443191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:44.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.446787+0000","last_change":"2024-09-17T05:04:16.441762+0000","last_active":"2024-09-17T05:04:17.446787+0000","last_peered":"2024-09-17T05:04:17.446787+0000","last_clean":"2024-09-17T05:04:17.446787+0000","last_became_active":"2024-09-17T05:04:16.441468+0000","last_became_peered":"2024-09-17T05:04:16.441468+0000","last_unstale":"2024-09-17T05:04:17.446787+0000","last_undegraded":"2024-09-17T05:04:17.446787+0000","last_fullsized":"2024-09-17T05:04:17.446787+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:24:25.007767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.461796+0000","last_change":"2024-09-17T05:04:17.452552+0000","last_active":"2024-09-17T05:04:17.461796+0000","last_peered":"2024-09-17T05:04:17.461796+0000","last_clean":"2024-09-17T05:04:17.461796+0000","last_became_active":"2024-09-17T05:04:17.452206+0000","last_became_peered":"2024-09-17T05:04:17.452206+0000","last_unstale":"2024-09-17T05:04:17.461796+0000","last_undegraded":"2024-09-17T05:04:17.461796+0000","last_fullsized":"2024-09-17T05:04:17.461796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:13.292533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.458423+0000","last_change":"2024-09-17T05:04:16.446523+0000","last_active":"2024-09-17T05:04:17.458423+0000","last_peered":"2024-09-17T05:04:17.458423+0000","last_clean":"2024-09-17T05:04:17.458423+0000","last_became_active":"2024-09-17T05:04:16.446194+0000","last_became_peered":"2024-09-17T05:04:16.446194+0000","last_unstale":"2024-09-17T05:04:17.458423+0000","last_undegraded":"2024-09-17T05:04:17.458423+0000","last_fullsized":"2024-09-17T05:04:17.458423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:18.515722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.472490+0000","last_change":"2024-09-17T05:04:17.467996+0000","last_active":"2024-09-17T05:04:17.472490+0000","last_peered":"2024-09-17T05:04:17.472490+0000","last_clean":"2024-09-17T05:04:17.472490+0000","last_became_active":"2024-09-17T05:04:17.467687+0000","last_became_peered":"2024-09-17T05:04:17.467687+0000","last_unstale":"2024-09-17T05:04:17.472490+0000","last_undegraded":"2024-09-17T05:04:17.472490+0000","last_fullsized":"2024-09-17T05:04:17.472490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:44.338357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.441600+0000","last_change":"2024-09-17T05:04:16.442945+0000","last_active":"2024-09-17T05:04:17.441600+0000","last_peered":"2024-09-17T05:04:17.441600+0000","last_clean":"2024-09-17T05:04:17.441600+0000","last_became_active":"2024-09-17T05:04:16.442697+0000","last_became_peered":"2024-09-17T05:04:16.442697+0000","last_unstale":"2024-09-17T05:04:17.441600+0000","last_undegraded":"2024-09-17T05:04:17.441600+0000","last_fullsized":"2024-09-17T05:04:17.441600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:03.658735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.466279+0000","last_change":"2024-09-17T05:04:17.446766+0000","last_active":"2024-09-17T05:04:17.466279+0000","last_peered":"2024-09-17T05:04:17.466279+0000","last_clean":"2024-09-17T05:04:17.466279+0000","last_became_active":"2024-09-17T05:04:17.446469+0000","last_became_peered":"2024-09-17T05:04:17.446469+0000","last_unstale":"2024-09-17T05:04:17.466279+0000","last_undegraded":"2024-09-17T05:04:17.466279+0000","last_fullsized":"2024-09-17T05:04:17.466279+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:46.398936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.469930+0000","last_change":"2024-09-17T05:04:16.438284+0000","last_active":"2024-09-17T05:04:17.469930+0000","last_peered":"2024-09-17T05:04:17.469930+0000","last_clean":"2024-09-17T05:04:17.469930+0000","last_became_active":"2024-09-17T05:04:16.437883+0000","last_became_peered":"2024-09-17T05:04:16.437883+0000","last_unstale":"2024-09-17T05:04:17.469930+0000","last_undegraded":"2024-09-17T05:04:17.469930+0000","last_fullsized":"2024-09-17T05:04:17.469930+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:27.706213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468246+0000","last_change":"2024-09-17T05:04:17.459422+0000","last_active":"2024-09-17T05:04:17.468246+0000","last_peered":"2024-09-17T05:04:17.468246+0000","last_clean":"2024-09-17T05:04:17.468246+0000","last_became_active":"2024-09-17T05:04:17.458966+0000","last_became_peered":"2024-09-17T05:04:17.458966+0000","last_unstale":"2024-09-17T05:04:17.468246+0000","last_undegraded":"2024-09-17T05:04:17.468246+0000","last_fullsized":"2024-09-17T05:04:17.468246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:26.632431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.431559+0000","last_change":"2024-09-17T05:04:16.436978+0000","last_active":"2024-09-17T05:04:17.431559+0000","last_peered":"2024-09-17T05:04:17.431559+0000","last_clean":"2024-09-17T05:04:17.431559+0000","last_became_active":"2024-09-17T05:04:16.436673+0000","last_became_peered":"2024-09-17T05:04:16.436673+0000","last_unstale":"2024-09-17T05:04:17.431559+0000","last_undegraded":"2024-09-17T05:04:17.431559+0000","last_fullsized":"2024-09-17T05:04:17.431559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:49:15.335263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.451742+0000","last_change":"2024-09-17T05:04:17.444157+0000","last_active":"2024-09-17T05:04:17.451742+0000","last_peered":"2024-09-17T05:04:17.451742+0000","last_clean":"2024-09-17T05:04:17.451742+0000","last_became_active":"2024-09-17T05:04:17.443839+0000","last_became_peered":"2024-09-17T05:04:17.443839+0000","last_unstale":"2024-09-17T05:04:17.451742+0000","last_undegraded":"2024-09-17T05:04:17.451742+0000","last_fullsized":"2024-09-17T05:04:17.451742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:31:26.612250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.433621+0000","last_change":"2024-09-17T05:04:16.437758+0000","last_active":"2024-09-17T05:04:17.433621+0000","last_peered":"2024-09-17T05:04:17.433621+0000","last_clean":"2024-09-17T05:04:17.433621+0000","last_became_active":"2024-09-17T05:04:16.434675+0000","last_became_peered":"2024-09-17T05:04:16.434675+0000","last_unstale":"2024-09-17T05:04:17.433621+0000","last_undegraded":"2024-09-17T05:04:17.433621+0000","last_fullsized":"2024-09-17T05:04:17.433621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:37.930490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454683+0000","last_change":"2024-09-17T05:04:17.448240+0000","last_active":"2024-09-17T05:04:17.454683+0000","last_peered":"2024-09-17T05:04:17.454683+0000","last_clean":"2024-09-17T05:04:17.454683+0000","last_became_active":"2024-09-17T05:04:17.447044+0000","last_became_peered":"2024-09-17T05:04:17.447044+0000","last_unstale":"2024-09-17T05:04:17.454683+0000","last_undegraded":"2024-09-17T05:04:17.454683+0000","last_fullsized":"2024-09-17T05:04:17.454683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:09.718696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464362+0000","last_change":"2024-09-17T05:04:16.443585+0000","last_active":"2024-09-17T05:04:17.464362+0000","last_peered":"2024-09-17T05:04:17.464362+0000","last_clean":"2024-09-17T05:04:17.464362+0000","last_became_active":"2024-09-17T05:04:16.442150+0000","last_became_peered":"2024-09-17T05:04:16.442150+0000","last_unstale":"2024-09-17T05:04:17.464362+0000","last_undegraded":"2024-09-17T05:04:17.464362+0000","last_fullsized":"2024-09-17T05:04:17.464362+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:44:06.869758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.463158+0000","last_change":"2024-09-17T05:04:17.461197+0000","last_active":"2024-09-17T05:04:17.463158+0000","last_peered":"2024-09-17T05:04:17.463158+0000","last_clean":"2024-09-17T05:04:17.463158+0000","last_became_active":"2024-09-17T05:04:17.460791+0000","last_became_peered":"2024-09-17T05:04:17.460791+0000","last_unstale":"2024-09-17T05:04:17.463158+0000","last_undegraded":"2024-09-17T05:04:17.463158+0000","last_fullsized":"2024-09-17T05:04:17.463158+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:37.443092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807739+0000","last_change":"2024-09-17T05:04:16.441692+0000","last_active":"2024-09-17T05:04:18.807739+0000","last_peered":"2024-09-17T05:04:18.807739+0000","last_clean":"2024-09-17T05:04:18.807739+0000","last_became_active":"2024-09-17T05:04:16.441313+0000","last_became_peered":"2024-09-17T05:04:16.441313+0000","last_unstale":"2024-09-17T05:04:18.807739+0000","last_undegraded":"2024-09-17T05:04:18.807739+0000","last_fullsized":"2024-09-17T05:04:18.807739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:38.276524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.452795+0000","last_change":"2024-09-17T05:04:17.446970+0000","last_active":"2024-09-17T05:04:17.452795+0000","last_peered":"2024-09-17T05:04:17.452795+0000","last_clean":"2024-09-17T05:04:17.452795+0000","last_became_active":"2024-09-17T05:04:17.446645+0000","last_became_peered":"2024-09-17T05:04:17.446645+0000","last_unstale":"2024-09-17T05:04:17.452795+0000","last_undegraded":"2024-09-17T05:04:17.452795+0000","last_fullsized":"2024-09-17T05:04:17.452795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:26:53.912834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.422229+0000","last_change":"2024-09-17T05:04:16.437820+0000","last_active":"2024-09-17T05:04:17.422229+0000","last_peered":"2024-09-17T05:04:17.422229+0000","last_clean":"2024-09-17T05:04:17.422229+0000","last_became_active":"2024-09-17T05:04:16.437481+0000","last_became_peered":"2024-09-17T05:04:16.437481+0000","last_unstale":"2024-09-17T05:04:17.422229+0000","last_undegraded":"2024-09-17T05:04:17.422229+0000","last_fullsized":"2024-09-17T05:04:17.422229+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:18.464106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.464807+0000","last_change":"2024-09-17T05:04:17.454180+0000","last_active":"2024-09-17T05:04:17.464807+0000","last_peered":"2024-09-17T05:04:17.464807+0000","last_clean":"2024-09-17T05:04:17.464807+0000","last_became_active":"2024-09-17T05:04:17.453896+0000","last_became_peered":"2024-09-17T05:04:17.453896+0000","last_unstale":"2024-09-17T05:04:17.464807+0000","last_undegraded":"2024-09-17T05:04:17.464807+0000","last_fullsized":"2024-09-17T05:04:17.464807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:17.585986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.437589+0000","last_change":"2024-09-17T05:04:16.447649+0000","last_active":"2024-09-17T05:04:17.437589+0000","last_peered":"2024-09-17T05:04:17.437589+0000","last_clean":"2024-09-17T05:04:17.437589+0000","last_became_active":"2024-09-17T05:04:16.447262+0000","last_became_peered":"2024-09-17T05:04:16.447262+0000","last_unstale":"2024-09-17T05:04:17.437589+0000","last_undegraded":"2024-09-17T05:04:17.437589+0000","last_fullsized":"2024-09-17T05:04:17.437589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:49.371370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.454996+0000","last_change":"2024-09-17T05:04:17.449809+0000","last_active":"2024-09-17T05:04:17.454996+0000","last_peered":"2024-09-17T05:04:17.454996+0000","last_clean":"2024-09-17T05:04:17.454996+0000","last_became_active":"2024-09-17T05:04:17.449523+0000","last_became_peered":"2024-09-17T05:04:17.449523+0000","last_unstale":"2024-09-17T05:04:17.454996+0000","last_undegraded":"2024-09-17T05:04:17.454996+0000","last_fullsized":"2024-09-17T05:04:17.454996+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:06.239329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.437910+0000","last_change":"2024-09-17T05:04:16.433349+0000","last_active":"2024-09-17T05:04:17.437910+0000","last_peered":"2024-09-17T05:04:17.437910+0000","last_clean":"2024-09-17T05:04:17.437910+0000","last_became_active":"2024-09-17T05:04:16.433023+0000","last_became_peered":"2024-09-17T05:04:16.433023+0000","last_unstale":"2024-09-17T05:04:17.437910+0000","last_undegraded":"2024-09-17T05:04:17.437910+0000","last_fullsized":"2024-09-17T05:04:17.437910+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:50.928143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.445606+0000","last_change":"2024-09-17T05:04:17.443354+0000","last_active":"2024-09-17T05:04:17.445606+0000","last_peered":"2024-09-17T05:04:17.445606+0000","last_clean":"2024-09-17T05:04:17.445606+0000","last_became_active":"2024-09-17T05:04:17.443049+0000","last_became_peered":"2024-09-17T05:04:17.443049+0000","last_unstale":"2024-09-17T05:04:17.445606+0000","last_undegraded":"2024-09-17T05:04:17.445606+0000","last_fullsized":"2024-09-17T05:04:17.445606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:29.121778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807777+0000","last_change":"2024-09-17T05:04:16.437951+0000","last_active":"2024-09-17T05:04:18.807777+0000","last_peered":"2024-09-17T05:04:18.807777+0000","last_clean":"2024-09-17T05:04:18.807777+0000","last_became_active":"2024-09-17T05:04:16.437600+0000","last_became_peered":"2024-09-17T05:04:16.437600+0000","last_unstale":"2024-09-17T05:04:18.807777+0000","last_undegraded":"2024-09-17T05:04:18.807777+0000","last_fullsized":"2024-09-17T05:04:18.807777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:02:04.125443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.459644+0000","last_change":"2024-09-17T05:04:17.448943+0000","last_active":"2024-09-17T05:04:17.459644+0000","last_peered":"2024-09-17T05:04:17.459644+0000","last_clean":"2024-09-17T05:04:17.459644+0000","last_became_active":"2024-09-17T05:04:17.448570+0000","last_became_peered":"2024-09-17T05:04:17.448570+0000","last_unstale":"2024-09-17T05:04:17.459644+0000","last_undegraded":"2024-09-17T05:04:17.459644+0000","last_fullsized":"2024-09-17T05:04:17.459644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:24.862987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.436640+0000","last_change":"2024-09-17T05:04:16.449561+0000","last_active":"2024-09-17T05:04:17.436640+0000","last_peered":"2024-09-17T05:04:17.436640+0000","last_clean":"2024-09-17T05:04:17.436640+0000","last_became_active":"2024-09-17T05:04:16.449173+0000","last_became_peered":"2024-09-17T05:04:16.449173+0000","last_unstale":"2024-09-17T05:04:17.436640+0000","last_undegraded":"2024-09-17T05:04:17.436640+0000","last_fullsized":"2024-09-17T05:04:17.436640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:47.082784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.468328+0000","last_change":"2024-09-17T05:04:17.460956+0000","last_active":"2024-09-17T05:04:17.468328+0000","last_peered":"2024-09-17T05:04:17.468328+0000","last_clean":"2024-09-17T05:04:17.468328+0000","last_became_active":"2024-09-17T05:04:17.460628+0000","last_became_peered":"2024-09-17T05:04:17.460628+0000","last_unstale":"2024-09-17T05:04:17.468328+0000","last_undegraded":"2024-09-17T05:04:17.468328+0000","last_fullsized":"2024-09-17T05:04:17.468328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:14.634121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.433874+0000","last_change":"2024-09-17T05:04:16.444709+0000","last_active":"2024-09-17T05:04:17.433874+0000","last_peered":"2024-09-17T05:04:17.433874+0000","last_clean":"2024-09-17T05:04:17.433874+0000","last_became_active":"2024-09-17T05:04:16.444327+0000","last_became_peered":"2024-09-17T05:04:16.444327+0000","last_unstale":"2024-09-17T05:04:17.433874+0000","last_undegraded":"2024-09-17T05:04:17.433874+0000","last_fullsized":"2024-09-17T05:04:17.433874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:05:57.314622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.457947+0000","last_change":"2024-09-17T05:04:17.448248+0000","last_active":"2024-09-17T05:04:17.457947+0000","last_peered":"2024-09-17T05:04:17.457947+0000","last_clean":"2024-09-17T05:04:17.457947+0000","last_became_active":"2024-09-17T05:04:17.447904+0000","last_became_peered":"2024-09-17T05:04:17.447904+0000","last_unstale":"2024-09-17T05:04:17.457947+0000","last_undegraded":"2024-09-17T05:04:17.457947+0000","last_fullsized":"2024-09-17T05:04:17.457947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:16.408155+0000","last_clean_scrub_stamp":"2024-09-17T05:04:16.408155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:20.651318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-17T05:04:18.807584+0000","last_change":"2024-09-17T05:04:16.433895+0000","last_active":"2024-09-17T05:04:18.807584+0000","last_peered":"2024-09-17T05:04:18.807584+0000","last_clean":"2024-09-17T05:04:18.807584+0000","last_became_active":"2024-09-17T05:04:16.433554+0000","last_became_peered":"2024-09-17T05:04:16.433554+0000","last_unstale":"2024-09-17T05:04:18.807584+0000","last_undegraded":"2024-09-17T05:04:18.807584+0000","last_fullsized":"2024-09-17T05:04:18.807584+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:15.404933+0000","last_clean_scrub_stamp":"2024-09-17T05:04:15.404933+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:12:47.555968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":211,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-17T05:04:17.420520+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:17.420520+0000","last_peered":"2024-09-17T05:04:17.420520+0000","last_clean":"2024-09-17T05:04:17.420520+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:17.420520+0000","last_undegraded":"2024-09-17T05:04:17.420520+0000","last_fullsized":"2024-09-17T05:04:17.420520+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":905216,"data_stored":404829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":396990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":996171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509484,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32068,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339772,"statfs":{"total":96636764160,"available":96603926528,"internally_reserved":0,"allocated":901120,"data_stored":404179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1507328,"data_stored":1001307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:04:22.044 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:04:22.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 -- 172.21.15.16:0/2889316489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb6406e6b0 msgr2=0x7ffb641661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 --2- 172.21.15.16:0/2889316489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb6406e6b0 0x7ffb641661a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009bc0 tx=0x7ffb5c01ca80 comp rx=0 tx=0).stop 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 -- 172.21.15.16:0/2889316489 shutdown_connections 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 --2- 172.21.15.16:0/2889316489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb6406e6b0 0x7ffb641661a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 -- 172.21.15.16:0/2889316489 >> 172.21.15.16:0/2889316489 conn(0x7ffb6406f9c0 msgr2=0x7ffb640b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 -- 172.21.15.16:0/2889316489 shutdown_connections 2024-09-17T05:04:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.249+0000 7ffb69800640 1 -- 172.21.15.16:0/2889316489 wait complete. 2024-09-17T05:04:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 Processor -- start 2024-09-17T05:04:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 -- start start 2024-09-17T05:04:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb64152300 0x7ffb641557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb64152710 0x7ffb64147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 0x7ffb6414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64169940 con 0x7ffb64150bf0 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641697c0 con 0x7ffb64152300 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ffb64169ac0 con 0x7ffb64152710 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 0x7ffb6414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 0x7ffb6414a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48198/0 (socket says 172.21.15.16:48198) 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 -- 172.21.15.16:0/130213197 learned_addr learned my addr 172.21.15.16:0/130213197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb63e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb64152300 0x7ffb641557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb5b400640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb64152710 0x7ffb64147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 -- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb64152710 msgr2=0x7ffb64147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb64152710 0x7ffb64147480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 -- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb64152300 msgr2=0x7ffb641557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb64152300 0x7ffb641557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 -- 172.21.15.16:0/130213197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb641479c0 con 0x7ffb64150bf0 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb5b400640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb64152710 0x7ffb64147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb63e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb64152300 0x7ffb641557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:22.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb68e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 0x7ffb6414a490 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0b0040 tx=0x7ffb5c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0b2040 con 0x7ffb64150bf0 2024-09-17T05:04:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb5c0bb070 con 0x7ffb64150bf0 2024-09-17T05:04:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c004570 con 0x7ffb64150bf0 2024-09-17T05:04:22.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb640afd20 con 0x7ffb64150bf0 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.253+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb64151620 con 0x7ffb64150bf0 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.257+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb640b2110 con 0x7ffb64150bf0 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.257+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffb5c004710 con 0x7ffb64150bf0 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.257+0000 7ffb60c00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffb40073200 0x7ffb40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.257+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7ffb5c12ed80 con 0x7ffb64150bf0 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.257+0000 7ffb63e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffb40073200 0x7ffb40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.257+0000 7ffb63e00640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffb40073200 0x7ffb40075710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb540046c0 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:22.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.261+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffb640b2110 con 0x7ffb64150bf0 2024-09-17T05:04:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.501+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ffb640b4b10 con 0x7ffb64150bf0 2024-09-17T05:04:22.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.501+0000 7ffb60c00640 1 -- 172.21.15.16:0/130213197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) ==== 190+0+147 (secure 0 0 0) 0x7ffb5c016790 con 0x7ffb64150bf0 2024-09-17T05:04:22.505 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:04:22.505 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:04:22.505 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:04:22.505 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:04:22.505 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:04:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:04:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffb40073200 msgr2=0x7ffb40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffb40073200 0x7ffb40075710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffb540046c0 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2024-09-17T05:04:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 msgr2=0x7ffb6414a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 0x7ffb6414a490 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c0b0040 tx=0x7ffb5c005ae0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 shutdown_connections 2024-09-17T05:04:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffb40073200 0x7ffb40075710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb64150bf0 0x7ffb6414a490 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffb64152710 0x7ffb64147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 --2- 172.21.15.16:0/130213197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb64152300 0x7ffb641557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 >> 172.21.15.16:0/130213197 conn(0x7ffb6406f9c0 msgr2=0x7ffb640d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 shutdown_connections 2024-09-17T05:04:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.505+0000 7ffb69800640 1 -- 172.21.15.16:0/130213197 wait complete. 2024-09-17T05:04:22.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:22.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 -- 172.21.15.16:0/3845025331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cc8c0 msgr2=0x7f886815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 --2- 172.21.15.16:0/3845025331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cc8c0 0x7f886815cab0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f88680ce800 tx=0x7f885801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 -- 172.21.15.16:0/3845025331 shutdown_connections 2024-09-17T05:04:22.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 --2- 172.21.15.16:0/3845025331 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f88680d0010 0x7f88680d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 --2- 172.21.15.16:0/3845025331 >> v2:172.21.15.104:3300/0 conn(0x7f88680d1580 0x7f88680d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 --2- 172.21.15.16:0/3845025331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cc8c0 0x7f886815cab0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 -- 172.21.15.16:0/3845025331 >> 172.21.15.16:0/3845025331 conn(0x7f88680d2270 msgr2=0x7f88680d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:22.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 -- 172.21.15.16:0/3845025331 shutdown_connections 2024-09-17T05:04:22.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 -- 172.21.15.16:0/3845025331 wait complete. 2024-09-17T05:04:22.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 Processor -- start 2024-09-17T05:04:22.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 -- start start 2024-09-17T05:04:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.685+0000 7f886d800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f88680cc8c0 0x7f88680715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88680d0010 0x7f88680b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 0x7f886815ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88680ca0c0 con 0x7f88680d1580 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f88680c9f40 con 0x7f88680d0010 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f88680ca240 con 0x7f88680cc8c0 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 0x7f886815ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 0x7f886815ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48216/0 (socket says 172.21.15.16:48216) 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 -- 172.21.15.16:0/3890173751 learned_addr learned my addr 172.21.15.16:0/3890173751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8866a00640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f88680cc8c0 0x7f88680715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f885fe00640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88680d0010 0x7f88680b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 -- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f88680cc8c0 msgr2=0x7f88680715f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f88680cc8c0 0x7f88680715f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 -- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88680d0010 msgr2=0x7f88680b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88680d0010 0x7f88680b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 -- 172.21.15.16:0/3890173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88681fc820 con 0x7f88680d1580 2024-09-17T05:04:22.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f885fe00640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88680d0010 0x7f88680b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f8867400640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 0x7f886815ef20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f886000cb00 tx=0x7f886000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886000ae30 con 0x7f88680d1580 2024-09-17T05:04:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8860003980 con 0x7f88680d1580 2024-09-17T05:04:22.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860003df0 con 0x7f88680d1580 2024-09-17T05:04:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88681fcb00 con 0x7f88680d1580 2024-09-17T05:04:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88681fd0c0 con 0x7f88680d1580 2024-09-17T05:04:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88680d0420 con 0x7f88680d1580 2024-09-17T05:04:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.689+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8860004030 con 0x7f88680d1580 2024-09-17T05:04:22.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.693+0000 7f885ea00640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8844073230 0x7f8844075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:22.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.693+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f8860094c60 con 0x7f88680d1580 2024-09-17T05:04:22.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.693+0000 7f8866a00640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8844073230 0x7f8844075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:22.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.693+0000 7f8866a00640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8844073230 0x7f8844075740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88580af000 tx=0x7f885801b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:22.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.697+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8860014070 con 0x7f88680d1580 2024-09-17T05:04:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.905+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f886815d130 con 0x7f88680d1580 2024-09-17T05:04:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.905+0000 7f885ea00640 1 -- 172.21.15.16:0/3890173751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7f8860068020 con 0x7f88680d1580 2024-09-17T05:04:22.909 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:22.909 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8844073230 msgr2=0x7f8844075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8844073230 0x7f8844075740 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88580af000 tx=0x7f885801b100 comp rx=0 tx=0).stop 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 msgr2=0x7f886815ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 0x7f886815ef20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f886000cb00 tx=0x7f886000a250 comp rx=0 tx=0).stop 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 shutdown_connections 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8844073230 0x7f8844075740 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680d1580 0x7f886815ef20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88680d0010 0x7f88680b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 --2- 172.21.15.16:0/3890173751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f88680cc8c0 0x7f88680715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 >> 172.21.15.16:0/3890173751 conn(0x7f88680d2270 msgr2=0x7f8868156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 shutdown_connections 2024-09-17T05:04:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:22.909+0000 7f886d800640 1 -- 172.21.15.16:0/3890173751 wait complete. 2024-09-17T05:04:23.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:04:16.846538+0000', 'modified': '2024-09-17T05:04:19.864243+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5926}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5926': {'gid': 5926, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6837/556849924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 556849924}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 556849924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5926, 'qdb_cluster': [5926]} 2024-09-17T05:04:23.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:04:23.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T05:04:23.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 -- 172.21.15.16:0/98413966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0cadc0 msgr2=0x7fd64c1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 --2- 172.21.15.16:0/98413966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0cadc0 0x7fd64c1590a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd634009840 tx=0x7fd63401c820 comp rx=0 tx=0).stop 2024-09-17T05:04:23.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 -- 172.21.15.16:0/98413966 shutdown_connections 2024-09-17T05:04:23.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 --2- 172.21.15.16:0/98413966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c0cadc0 0x7fd64c1590a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 -- 172.21.15.16:0/98413966 >> 172.21.15.16:0/98413966 conn(0x7fd64c0b2250 msgr2=0x7fd64c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 -- 172.21.15.16:0/98413966 shutdown_connections 2024-09-17T05:04:23.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.497+0000 7fd651a00640 1 -- 172.21.15.16:0/98413966 wait complete. 2024-09-17T05:04:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 Processor -- start 2024-09-17T05:04:23.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 -- start start 2024-09-17T05:04:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c073f30 0x7fd64c14f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd64c0d0940 0x7fd64c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 0x7fd64c152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c16a040 con 0x7fd64c150050 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c169ec0 con 0x7fd64c073f30 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd651a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd64c16a1c0 con 0x7fd64c0d0940 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd64b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 0x7fd64c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd64b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 0x7fd64c152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48236/0 (socket says 172.21.15.16:48236) 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd64b400640 1 -- 172.21.15.16:0/426670197 learned_addr learned my addr 172.21.15.16:0/426670197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd64aa00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c073f30 0x7fd64c14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.501+0000 7fd643e00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd64c0d0940 0x7fd64c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd64b400640 1 -- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd64c0d0940 msgr2=0x7fd64c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd64b400640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd64c0d0940 0x7fd64c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd64b400640 1 -- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c073f30 msgr2=0x7fd64c14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd64b400640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c073f30 0x7fd64c14f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd64b400640 1 -- 172.21.15.16:0/426670197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd64c15b350 con 0x7fd64c150050 2024-09-17T05:04:23.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd64b400640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 0x7fd64c152300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd64400cb10 tx=0x7fd64400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd642a00640 1 -- 172.21.15.16:0/426670197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd644016020 con 0x7fd64c150050 2024-09-17T05:04:23.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd642a00640 1 -- 172.21.15.16:0/426670197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd64400ce50 con 0x7fd64c150050 2024-09-17T05:04:23.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd642a00640 1 -- 172.21.15.16:0/426670197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64400ade0 con 0x7fd64c150050 2024-09-17T05:04:23.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd64c1f89c0 con 0x7fd64c150050 2024-09-17T05:04:23.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd64c1f8f80 con 0x7fd64c150050 2024-09-17T05:04:23.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd642a00640 1 -- 172.21.15.16:0/426670197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd644019020 con 0x7fd64c150050 2024-09-17T05:04:23.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.505+0000 7fd642a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd628073200 0x7fd628075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd642a00640 1 -- 172.21.15.16:0/426670197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fd644094200 con 0x7fd64c150050 2024-09-17T05:04:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd628073200 0x7fd628075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 Processor -- start 2024-09-17T05:04:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 -- start start 2024-09-17T05:04:23.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd628073200 0x7fd628075710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd63c007fd0 tx=0x7fd63c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 0x562d80b606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80b60be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562d80a35410 con 0x562d8061f460 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562d80972b40 con 0x562d80a0f950 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x562d80961f70 con 0x562d809f8ca0 2024-09-17T05:04:23.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 0x562d80b606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 0x562d80b606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48240/0 (socket says 172.21.15.16:48240) 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 -- 172.21.15.16:0/1266003368 learned_addr learned my addr 172.21.15.16:0/1266003368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd643e00640 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80b60be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64b400640 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 -- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 msgr2=0x562d80b61120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b61120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 -- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 msgr2=0x562d80b60be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80b60be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 -- 172.21.15.16:0/1266003368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d809295b0 con 0x562d8061f460 2024-09-17T05:04:23.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd64aa00640 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 0x562d80b606a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00f990 tx=0x7fd63c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9b26b813ad1f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd63b400640 1 -- 172.21.15.16:0/1266003368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c025070 con 0x562d8061f460 2024-09-17T05:04:23.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd63b400640 1 -- 172.21.15.16:0/1266003368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd63c01b050 con 0x562d8061f460 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.509+0000 7fd63b400640 1 -- 172.21.15.16:0/1266003368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c014e40 con 0x562d8061f460 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- 172.21.15.16:0/1266003368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 msgr2=0x562d80b606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 0x562d80b606a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00f990 tx=0x7fd63c012000 comp rx=0 tx=0).stop 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- 172.21.15.16:0/1266003368 shutdown_connections 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b61120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80b60be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- 172.21.15.16:0/1266003368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d8061f460 0x562d80b606a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- 172.21.15.16:0/1266003368 >> 172.21.15.16:0/1266003368 conn(0x562d80ae5d90 msgr2=0x562d80b15690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- 172.21.15.16:0/1266003368 shutdown_connections 2024-09-17T05:04:23.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- 172.21.15.16:0/1266003368 wait complete. 2024-09-17T05:04:23.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- start start 2024-09-17T05:04:23.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 Processor -- start 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d8061f460 0x562d80b67920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b67e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80b683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562d808fc5f0 con 0x562d80a0f950 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562d80965d80 con 0x562d8061f460 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x562d80908520 con 0x562d809f8ca0 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd6591e2000 10 client.0 _ll_register_callbacks cb 0x562d8090c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80b683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80b683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48254/0 (socket says 172.21.15.16:48254) 2024-09-17T05:04:23.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64b400640 1 -- 172.21.15.16:0/417666621 learned_addr learned my addr 172.21.15.16:0/417666621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64aa00640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d8061f460 0x562d80b67920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd643e00640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b67e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64b400640 1 -- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 msgr2=0x562d80b67e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64b400640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b67e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.513+0000 7fd64b400640 1 -- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d8061f460 msgr2=0x562d80b67920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd64b400640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d8061f460 0x562d80b67920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd64b400640 1 -- 172.21.15.16:0/417666621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d80900190 con 0x562d80a0f950 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd64b400640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80b683a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd644002240 tx=0x7fd64405aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23dd09acf9d8fb18 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 1 -- 172.21.15.16:0/417666621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64405de90 con 0x562d80a0f950 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 1 -- 172.21.15.16:0/417666621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fd64405a380 con 0x562d80a0f950 2024-09-17T05:04:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d808fa110 con 0x562d80a0f950 2024-09-17T05:04:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 1 -- 172.21.15.16:0/417666621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd64407f050 con 0x562d80a0f950 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 10 client.6237 fetch_fsmap learned FSMap version 44 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x562d809e6f10 con 0x562d80a0f950 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 1 -- 172.21.15.16:0/417666621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562d808fa110 con 0x562d80a0f950 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 1 -- 172.21.15.16:0/417666621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) ==== 3502+0+0 (secure 0 0 0) 0x7fd64405a580 con 0x562d80a0f950 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 10 client.6237 fetch_fsmap finished waiting for FSMap version 44 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 10 client.6237 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 20 client.6237 populate_metadata read hostname 'smithi016' 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d806047d0 0x562d80b71d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 4 client.6237 mds_command: new command op to 5926 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 --> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d80974a90 con 0x562d806047d0 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd64aa00640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d806047d0 0x562d80b71d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 10 client.6237 ms_handle_connect on v2:172.21.15.104:6836/556849924 2024-09-17T05:04:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd64aa00640 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d806047d0 0x562d80b71d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db8afa56944a53d6 server_cookie=6b7e01b88d4f9f56 in_seq=0 out_seq=0 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 1 -- 172.21.15.16:0/417666621 <== mds.0 v2:172.21.15.104:6836/556849924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d80974a90 con 0x562d806047d0 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd62fe00640 10 client.6237 handle_command_reply: tid=0 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.517+0000 7fd6591e2000 1 client.6237 shutdown 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 msgr2=0x562d80b683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80b683a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd644002240 tx=0x7fd64405aa00 comp rx=0 tx=0).stop 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 shutdown_connections 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d806047d0 0x562d80b71d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80b683a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d809f8ca0 0x562d80b67e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd62fe00640 0 client.6237 ms_handle_reset on v2:172.21.15.104:6836/556849924 2024-09-17T05:04:23.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 --2- 172.21.15.16:0/417666621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d8061f460 0x562d80b67920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 >> 172.21.15.16:0/417666621 conn(0x562d80ae5d90 msgr2=0x562d80b1c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 shutdown_connections 2024-09-17T05:04:23.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 1 -- 172.21.15.16:0/417666621 wait complete. 2024-09-17T05:04:23.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.521+0000 7fd6591e2000 20 client.6237 trim_cache size 0 max 16384 2024-09-17T05:04:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.537+0000 7fd6591e2000 1 Processor -- start 2024-09-17T05:04:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- start start 2024-09-17T05:04:23.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 0x562d80bd2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80bd30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c205b40 con 0x562d809f8ca0 2024-09-17T05:04:23.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd64c204bc0 con 0x562d80a0f950 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd64c203ff0 con 0x562d8061f460 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 0x562d80bd2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 0x562d80bd2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48266/0 (socket says 172.21.15.16:48266) 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 -- 172.21.15.16:0/471598553 learned_addr learned my addr 172.21.15.16:0/471598553 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd64aa00640 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd64b400640 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80bd30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 -- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 msgr2=0x562d80bd2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 -- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 msgr2=0x562d80bd30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80bd30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 -- 172.21.15.16:0/471598553 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d809bddb0 con 0x562d809f8ca0 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd64b400640 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80bd30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd643e00640 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 0x562d80bd2b60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd63401a450 tx=0x7fd6340195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07e196c0b79660a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd63aa00640 1 -- 172.21.15.16:0/471598553 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634004020 con 0x562d809f8ca0 2024-09-17T05:04:23.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd63aa00640 1 -- 172.21.15.16:0/471598553 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6340042b0 con 0x562d809f8ca0 2024-09-17T05:04:23.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd63aa00640 1 -- 172.21.15.16:0/471598553 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6340a6980 con 0x562d809f8ca0 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- 172.21.15.16:0/471598553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 msgr2=0x562d80bd2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 0x562d80bd2b60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd63401a450 tx=0x7fd6340195c0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- 172.21.15.16:0/471598553 shutdown_connections 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d80a0f950 0x562d80bd30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d809f8ca0 0x562d80bd2b60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 --2- 172.21.15.16:0/471598553 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- 172.21.15.16:0/471598553 >> 172.21.15.16:0/471598553 conn(0x562d80ae5d90 msgr2=0x562d80b5e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- 172.21.15.16:0/471598553 shutdown_connections 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.541+0000 7fd6591e2000 1 -- 172.21.15.16:0/471598553 wait complete. 2024-09-17T05:04:23.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 -- start start 2024-09-17T05:04:23.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 Processor -- start 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d809f8ca0 0x562d80bd68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80bd6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562d80965d80 con 0x562d80a0f950 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562d8096a3b0 con 0x562d809f8ca0 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x562d8092dee0 con 0x562d8061f460 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 10 client.0 _ll_register_callbacks cb 0x7fd64400ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:04:23.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80bd6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80bd6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48272/0 (socket says 172.21.15.16:48272) 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 -- 172.21.15.16:0/2260536901 learned_addr learned my addr 172.21.15.16:0/2260536901 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd643e00640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d809f8ca0 0x562d80bd68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64aa00640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 -- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 msgr2=0x562d80bd6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 -- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d809f8ca0 msgr2=0x562d80bd68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d809f8ca0 0x562d80bd68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 -- 172.21.15.16:0/2260536901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d80928ab0 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd64b400640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80bd6e10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd644001ce0 tx=0x7fd64405a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dc01b2a4792d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd62fe00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd62fe00640 1 -- 172.21.15.16:0/2260536901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64405e800 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd62fe00640 1 -- 172.21.15.16:0/2260536901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fd644069da0 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6340042b0 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd62fe00640 1 -- 172.21.15.16:0/2260536901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd64405e1a0 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 10 client.6243 fetch_fsmap learned FSMap version 44 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.545+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x562d80907070 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd62fe00640 1 -- 172.21.15.16:0/2260536901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6340042b0 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd62fe00640 1 -- 172.21.15.16:0/2260536901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 44) ==== 3502+0+0 (secure 0 0 0) 0x7fd644074aa0 con 0x562d80a0f950 2024-09-17T05:04:23.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 10 client.6243 fetch_fsmap finished waiting for FSMap version 44 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 10 client.6243 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 20 client.6243 populate_metadata read hostname 'smithi016' 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d805f2830 0x562d80b926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 4 client.6243 mds_command: new command op to 5926 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 --> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] -- command(tid 0: {"prefix": "status"}) -- 0x562d80970540 con 0x562d805f2830 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd64aa00640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d805f2830 0x562d80b926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd64aa00640 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d805f2830 0x562d80b926a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc00186264844201 server_cookie=f4dedc16530eef51 in_seq=0 out_seq=0 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd62fe00640 10 client.6243 ms_handle_connect on v2:172.21.15.104:6836/556849924 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd62fe00640 1 -- 172.21.15.16:0/2260536901 <== mds.0 v2:172.21.15.104:6836/556849924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562d80970540 con 0x562d805f2830 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd62fe00640 10 client.6243 handle_command_reply: tid=0 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 client.6243 shutdown 2024-09-17T05:04:23.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 msgr2=0x562d80bd6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80bd6e10 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd644001ce0 tx=0x7fd64405a5b0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 shutdown_connections 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:6836/556849924,v1:172.21.15.104:6837/556849924] conn(0x562d805f2830 0x562d80b926a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562d80a0f950 0x562d80bd6e10 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562d809f8ca0 0x562d80bd68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd62fe00640 0 client.6243 ms_handle_reset on v2:172.21.15.104:6836/556849924 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 --2- 172.21.15.16:0/2260536901 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562d8061f460 0x562d80bd6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 >> 172.21.15.16:0/2260536901 conn(0x562d80ae5d90 msgr2=0x562d80b5efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 shutdown_connections 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 1 -- 172.21.15.16:0/2260536901 wait complete. 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd6591e2000 20 client.6243 trim_cache size 0 max 16384 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:04:23.557 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5926, 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1343361720000003, 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 44, 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 75, 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 75, 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.536644838000001 2024-09-17T05:04:23.558 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:04:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd628073200 msgr2=0x7fd628075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd628073200 0x7fd628075710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd63c007fd0 tx=0x7fd63c009040 comp rx=0 tx=0).stop 2024-09-17T05:04:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 msgr2=0x7fd64c152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 0x7fd64c152300 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd64400cb10 tx=0x7fd64400cc00 comp rx=0 tx=0).stop 2024-09-17T05:04:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 shutdown_connections 2024-09-17T05:04:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd628073200 0x7fd628075710 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd64c150050 0x7fd64c152300 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd64c0d0940 0x7fd64c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 --2- 172.21.15.16:0/426670197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd64c073f30 0x7fd64c14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 >> 172.21.15.16:0/426670197 conn(0x7fd64c0b2250 msgr2=0x7fd64c153320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 shutdown_connections 2024-09-17T05:04:23.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.549+0000 7fd651a00640 1 -- 172.21.15.16:0/426670197 wait complete. 2024-09-17T05:04:23.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:04:23.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 -- 172.21.15.16:0/4148542445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 msgr2=0x7f8b28004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 --2- 172.21.15.16:0/4148542445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b28004c20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0016f0 tx=0x7f8b2c01c750 comp rx=0 tx=0).stop 2024-09-17T05:04:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 -- 172.21.15.16:0/4148542445 shutdown_connections 2024-09-17T05:04:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 --2- 172.21.15.16:0/4148542445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b28004c20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 -- 172.21.15.16:0/4148542445 >> 172.21.15.16:0/4148542445 conn(0x7f8b4006f900 msgr2=0x7f8b400b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 -- 172.21.15.16:0/4148542445 shutdown_connections 2024-09-17T05:04:23.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 -- 172.21.15.16:0/4148542445 wait complete. 2024-09-17T05:04:23.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.729+0000 7f8b48400640 1 Processor -- start 2024-09-17T05:04:23.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- start start 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b400ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b400af180 0x7f8b401f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b40154230 0x7f8b40152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4015ff30 con 0x7f8b28004810 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4015fdb0 con 0x7f8b400af180 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8b40073240 con 0x7f8b40154230 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b400ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b400ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48300/0 (socket says 172.21.15.16:48300) 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 -- 172.21.15.16:0/2105404384 learned_addr learned my addr 172.21.15.16:0/2105404384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45000640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b400af180 0x7f8b401f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b46400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b40154230 0x7f8b40152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 -- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b40154230 msgr2=0x7f8b40152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b40154230 0x7f8b40152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 -- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b400af180 msgr2=0x7f8b401f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b400af180 0x7f8b401f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 -- 172.21.15.16:0/2105404384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b400ad740 con 0x7f8b28004810 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b46400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b40154230 0x7f8b40152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:23.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b45a00640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b400ae500 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8b34006600 tx=0x7f8b340039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34015070 con 0x7f8b28004810 2024-09-17T05:04:23.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b34011920 con 0x7f8b28004810 2024-09-17T05:04:23.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b34011d70 con 0x7f8b28004810 2024-09-17T05:04:23.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b400ada20 con 0x7f8b28004810 2024-09-17T05:04:23.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b400adfe0 con 0x7f8b28004810 2024-09-17T05:04:23.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.733+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b400737a0 con 0x7f8b28004810 2024-09-17T05:04:23.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.737+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b34018070 con 0x7f8b28004810 2024-09-17T05:04:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.737+0000 7f8b3d600640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b20073200 0x7f8b20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:23.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.737+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f8b34095170 con 0x7f8b28004810 2024-09-17T05:04:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.737+0000 7f8b45000640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b20073200 0x7f8b20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:23.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.737+0000 7f8b45000640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b20073200 0x7f8b20075710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0016c0 tx=0x7f8b2c019540 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:23.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.741+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b3405ff60 con 0x7f8b28004810 2024-09-17T05:04:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.953+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b40069130 con 0x7f8b28004810 2024-09-17T05:04:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.953+0000 7f8b3d600640 1 -- 172.21.15.16:0/2105404384 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f8b3405f860 con 0x7f8b28004810 2024-09-17T05:04:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b20073200 msgr2=0x7f8b20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b20073200 0x7f8b20075710 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c0016c0 tx=0x7f8b2c019540 comp rx=0 tx=0).stop 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 msgr2=0x7f8b400ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b400ae500 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8b34006600 tx=0x7f8b340039d0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 shutdown_connections 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b20073200 0x7f8b20075710 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b40154230 0x7f8b40152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b400af180 0x7f8b401f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 --2- 172.21.15.16:0/2105404384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b28004810 0x7f8b400ae500 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 >> 172.21.15.16:0/2105404384 conn(0x7f8b4006f900 msgr2=0x7f8b400b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:23.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 shutdown_connections 2024-09-17T05:04:23.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:23.957+0000 7f8b48400640 1 -- 172.21.15.16:0/2105404384 wait complete. 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:04:23.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:04:23.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:23.980 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:23.989 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42229sec preferred_lft 42229sec 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:23.990 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:04:23.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.992 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:04:23.992 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:04:23.992 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:04:23.992 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:23.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:23.993 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:04:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:04:24.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:24.053 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:04:24.107 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:04:24.107 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:04:24.107 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:04:24.107 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:04:24.107 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:04:24.107 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:04:24.108 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:04:24.108 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:04:24.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:04:24.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:04:24.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:04:24.108 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:04:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:04:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:24.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:24.246 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:04:24.255 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:04:24.255 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:04:24.255 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:04:24.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:24.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:04:24.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:04:24.317 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:04:24.370 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:04:24.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:04:24.370 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:04:24.420 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:04:24.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:24.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:04:24.523 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:04:24.523 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:04:24.523 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:04:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:04:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:04:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:04:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:04:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:04:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:04:24.556 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:04:24.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:24.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:04:24.576 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:04:24.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:04:24.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:24.688 INFO:teuthology.orchestra.run.smithi016.stdout:client5558 2024-09-17T05:04:24.689 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:24.689 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:24.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:04:24.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 -- 172.21.15.16:0/1665255400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1ce0 msgr2=0x7f51ec0b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 --2- 172.21.15.16:0/1665255400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1ce0 0x7f51ec0b22c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009bc0 tx=0x7f51e401cb10 comp rx=0 tx=0).stop 2024-09-17T05:04:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 -- 172.21.15.16:0/1665255400 shutdown_connections 2024-09-17T05:04:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 --2- 172.21.15.16:0/1665255400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec0d1ce0 0x7f51ec0b22c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 -- 172.21.15.16:0/1665255400 >> 172.21.15.16:0/1665255400 conn(0x7f51ec0cecc0 msgr2=0x7f51ec0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 -- 172.21.15.16:0/1665255400 shutdown_connections 2024-09-17T05:04:24.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.901+0000 7f51f1800640 1 -- 172.21.15.16:0/1665255400 wait complete. 2024-09-17T05:04:24.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 Processor -- start 2024-09-17T05:04:24.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 -- start start 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 0x7f51ec0b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51ec1f5b00 0x7f51ec14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51ec1f98f0 0x7f51ec15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec16b150 con 0x7f51ec15a300 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec16afd0 con 0x7f51ec1f5b00 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec16b2d0 con 0x7f51ec1f98f0 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 0x7f51ec0b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 0x7f51ec0b7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48330/0 (socket says 172.21.15.16:48330) 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 -- 172.21.15.16:0/3403666021 learned_addr learned my addr 172.21.15.16:0/3403666021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:24.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51e3e00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51ec1f5b00 0x7f51ec14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eb400640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51ec1f98f0 0x7f51ec15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 -- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51ec1f98f0 msgr2=0x7f51ec15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51ec1f98f0 0x7f51ec15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 -- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51ec1f5b00 msgr2=0x7f51ec14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51ec1f5b00 0x7f51ec14eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 -- 172.21.15.16:0/3403666021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec155b70 con 0x7f51ec15a300 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51e3e00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51ec1f5b00 0x7f51ec14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51eaa00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 0x7f51ec0b7ce0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c400 tx=0x7f51dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51ec15a300 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51dc003600 con 0x7f51ec15a300 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc003a50 con 0x7f51ec15a300 2024-09-17T05:04:24.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec155e50 con 0x7f51ec15a300 2024-09-17T05:04:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.905+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1f90f0 con 0x7f51ec15a300 2024-09-17T05:04:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.909+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec0d1ce0 con 0x7f51ec15a300 2024-09-17T05:04:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.909+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f51dc01a070 con 0x7f51ec15a300 2024-09-17T05:04:24.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.909+0000 7f51e2a00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f51c8073230 0x7f51c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.909+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f51dc093a30 con 0x7f51ec15a300 2024-09-17T05:04:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.909+0000 7f51e3e00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f51c8073230 0x7f51c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:24.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.909+0000 7f51e3e00640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f51c8073230 0x7f51c8075740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0b5d80 tx=0x7f51d4008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:24.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:24.913+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f51dc021020 con 0x7f51ec15a300 2024-09-17T05:04:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.121+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f51ec14f530 con 0x7f51ec15a300 2024-09-17T05:04:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51e2a00640 1 -- 172.21.15.16:0/3403666021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7f51dc05e2e0 con 0x7f51ec15a300 2024-09-17T05:04:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f51c8073230 msgr2=0x7f51c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f51c8073230 0x7f51c8075740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0b5d80 tx=0x7f51d4008040 comp rx=0 tx=0).stop 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 msgr2=0x7f51ec0b7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 0x7f51ec0b7ce0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c400 tx=0x7f51dc00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 shutdown_connections 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f51c8073230 0x7f51c8075740 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51ec1f98f0 0x7f51ec15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51ec1f5b00 0x7f51ec14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 --2- 172.21.15.16:0/3403666021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51ec15a300 0x7f51ec0b7ce0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 >> 172.21.15.16:0/3403666021 conn(0x7f51ec0cecc0 msgr2=0x7f51ec167400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 shutdown_connections 2024-09-17T05:04:25.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:25.125+0000 7f51f1800640 1 -- 172.21.15.16:0/3403666021 wait complete. 2024-09-17T05:04:25.147 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:04:25.148 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:04:25.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.149 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:25.159 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42223sec preferred_lft 42223sec 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:04:25.160 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:04:25.161 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:04:25.162 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.162 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:04:25.162 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:04:25.162 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:04:25.162 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:04:25.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.163 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:04:25.224 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:04:25.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.226 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:04:25.280 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:04:25.280 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:04:25.281 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:04:25.282 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:04:25.282 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:04:25.282 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:04:25.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:04:25.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:04:25.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:04:25.282 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:04:25.375 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:04:25.418 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:25.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.425 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:04:25.434 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:04:25.434 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:04:25.435 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:04:25.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.435 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:04:25.496 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:04:25.497 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:25.552 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:04:25.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:04:25.552 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:25.607 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:04:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.607 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:25.707 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:25.708 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:04:25.708 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:04:25.725 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:25.726 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:04:25.726 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:04:25.726 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:04:25.726 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:04:25.735 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:04:25.736 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:04:25.739 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:04:25.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:25.740 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:25.758 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:04:25.854 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:04:25.857 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:25.864 INFO:teuthology.orchestra.run.smithi104.stdout:client5564 2024-09-17T05:04:25.864 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:25.864 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:25.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 -- 172.21.15.16:0/586915604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8070630 msgr2=0x7efdf80cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 --2- 172.21.15.16:0/586915604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8070630 0x7efdf80cb760 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7efde4009900 tx=0x7efde401c820 comp rx=0 tx=0).stop 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 -- 172.21.15.16:0/586915604 shutdown_connections 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 --2- 172.21.15.16:0/586915604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efdf806ce90 0x7efdf806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 --2- 172.21.15.16:0/586915604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf8070630 0x7efdf80cb760 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 -- 172.21.15.16:0/586915604 >> 172.21.15.16:0/586915604 conn(0x7efdf80cd980 msgr2=0x7efdf80cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 -- 172.21.15.16:0/586915604 shutdown_connections 2024-09-17T05:04:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 -- 172.21.15.16:0/586915604 wait complete. 2024-09-17T05:04:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 Processor -- start 2024-09-17T05:04:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.037+0000 7efe00600640 1 -- start start 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 0x7efdf814b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf80cb350 0x7efdf80c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efdf81fc4a0 0x7efdf80c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efdf815d250 con 0x7efdf806ce90 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efdf815d0d0 con 0x7efdf80cb350 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efdf815d3d0 con 0x7efdf81fc4a0 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 0x7efdf814b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 0x7efdf814b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48350/0 (socket says 172.21.15.16:48350) 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 -- 172.21.15.16:0/561061535 learned_addr learned my addr 172.21.15.16:0/561061535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfd200640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf80cb350 0x7efdf80c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfe600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efdf81fc4a0 0x7efdf80c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 -- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efdf81fc4a0 msgr2=0x7efdf80c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efdf81fc4a0 0x7efdf80c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 -- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf80cb350 msgr2=0x7efdf80c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf80cb350 0x7efdf80c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 -- 172.21.15.16:0/561061535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf814b9a0 con 0x7efdf806ce90 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfd200640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf80cb350 0x7efdf80c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdfdc00640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 0x7efdf814b460 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c3c0 tx=0x7efdec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec017070 con 0x7efdf806ce90 2024-09-17T05:04:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdec0053c0 con 0x7efdf806ce90 2024-09-17T05:04:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf80cf2f0 con 0x7efdf806ce90 2024-09-17T05:04:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec003860 con 0x7efdf806ce90 2024-09-17T05:04:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf80cf8b0 con 0x7efdf806ce90 2024-09-17T05:04:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.041+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf806d2a0 con 0x7efdf806ce90 2024-09-17T05:04:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.045+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efdec01a070 con 0x7efdf806ce90 2024-09-17T05:04:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.045+0000 7efdf5600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efdd8073230 0x7efdd8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.045+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7efdec093600 con 0x7efdf806ce90 2024-09-17T05:04:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.045+0000 7efdfd200640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efdd8073230 0x7efdd8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.045+0000 7efdfd200640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efdd8073230 0x7efdd8075740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efde40af000 tx=0x7efde401b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.049+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efdec05dec0 con 0x7efdf806ce90 2024-09-17T05:04:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.225+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7efdf806a180 con 0x7efdf806ce90 2024-09-17T05:04:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.225+0000 7efdf5600640 1 -- 172.21.15.16:0/561061535 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v40) ==== 106+0+4 (secure 0 0 0) 0x7efdec0628f0 con 0x7efdf806ce90 2024-09-17T05:04:26.228 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-17T05:04:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efdd8073230 msgr2=0x7efdd8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efdd8073230 0x7efdd8075740 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efde40af000 tx=0x7efde401b060 comp rx=0 tx=0).stop 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 msgr2=0x7efdf814b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 0x7efdf814b460 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c3c0 tx=0x7efdec00c880 comp rx=0 tx=0).stop 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 shutdown_connections 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efdd8073230 0x7efdd8075740 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efdf81fc4a0 0x7efdf80c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efdf80cb350 0x7efdf80c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 --2- 172.21.15.16:0/561061535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdf806ce90 0x7efdf814b460 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 >> 172.21.15.16:0/561061535 conn(0x7efdf80cd980 msgr2=0x7efdf806f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 shutdown_connections 2024-09-17T05:04:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.229+0000 7efe00600640 1 -- 172.21.15.16:0/561061535 wait complete. 2024-09-17T05:04:26.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2024-09-17T05:04:26.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 -- 172.21.15.16:0/3337978337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 msgr2=0x7f01a80ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 --2- 172.21.15.16:0/3337978337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 0x7f01a80ceb10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009bc0 tx=0x7f01a001cba0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 -- 172.21.15.16:0/3337978337 shutdown_connections 2024-09-17T05:04:26.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 --2- 172.21.15.16:0/3337978337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 0x7f01a80ceb10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 --2- 172.21.15.16:0/3337978337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f01a8074e30 0x7f01a80ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 -- 172.21.15.16:0/3337978337 >> 172.21.15.16:0/3337978337 conn(0x7f01a80b2250 msgr2=0x7f01a80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:26.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 -- 172.21.15.16:0/3337978337 shutdown_connections 2024-09-17T05:04:26.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.409+0000 7f01ad200640 1 -- 172.21.15.16:0/3337978337 wait complete. 2024-09-17T05:04:26.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 Processor -- start 2024-09-17T05:04:26.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 -- start start 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f01a8074a20 0x7f01a815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 0x7f01a815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 0x7f01a80b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8168e80 con 0x7f01a8074e30 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8168d00 con 0x7f01a815f6f0 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01ad200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f01a8169000 con 0x7f01a8074a20 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 0x7f01a815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 0x7f01a815a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48356/0 (socket says 172.21.15.16:48356) 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 -- 172.21.15.16:0/93887512 learned_addr learned my addr 172.21.15.16:0/93887512 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01a6800640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 0x7f01a80b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f01a5e00640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f01a8074a20 0x7f01a815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 -- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f01a8074a20 msgr2=0x7f01a815a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f01a8074a20 0x7f01a815a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 -- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 msgr2=0x7f01a80b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 0x7f01a80b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.413+0000 7f019fe00640 1 -- 172.21.15.16:0/93887512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a81f9d00 con 0x7f01a8074e30 2024-09-17T05:04:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f019fe00640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 0x7f01a815a9a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f019000e7a0 tx=0x7f019000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019000ccc0 con 0x7f01a8074e30 2024-09-17T05:04:26.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0190010430 con 0x7f01a8074e30 2024-09-17T05:04:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a81f8f40 con 0x7f01a8074e30 2024-09-17T05:04:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190010860 con 0x7f01a8074e30 2024-09-17T05:04:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a81f9500 con 0x7f01a8074e30 2024-09-17T05:04:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a81fd060 con 0x7f01a8074e30 2024-09-17T05:04:26.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.417+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0190010a00 con 0x7f01a8074e30 2024-09-17T05:04:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.421+0000 7f019e000640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0184073200 0x7f0184075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.421+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f0190094d20 con 0x7f01a8074e30 2024-09-17T05:04:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.421+0000 7f01a5e00640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0184073200 0x7f0184075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:26.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.421+0000 7f01a5e00640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0184073200 0x7f0184075710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f01980046c0 tx=0x7f0198007ec0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:26.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.425+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f019005f5e0 con 0x7f01a8074e30 2024-09-17T05:04:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.601+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f01a815efe0 con 0x7f01a8074e30 2024-09-17T05:04:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f019e000640 1 -- 172.21.15.16:0/93887512 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v41)=0 v41) ==== 123+0+0 (secure 0 0 0) 0x7f0190064010 con 0x7f01a8074e30 2024-09-17T05:04:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0184073200 msgr2=0x7f0184075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0184073200 0x7f0184075710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f01980046c0 tx=0x7f0198007ec0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 msgr2=0x7f01a815a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 0x7f01a815a9a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f019000e7a0 tx=0x7f019000ec60 comp rx=0 tx=0).stop 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 shutdown_connections 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0184073200 0x7f0184075710 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01a815f6f0 0x7f01a80b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01a8074e30 0x7f01a815a9a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 --2- 172.21.15.16:0/93887512 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f01a8074a20 0x7f01a815a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.605+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 >> 172.21.15.16:0/93887512 conn(0x7f01a80b2250 msgr2=0x7f01a80b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.609+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 shutdown_connections 2024-09-17T05:04:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:26.609+0000 7f01ad200640 1 -- 172.21.15.16:0/93887512 wait complete. 2024-09-17T05:04:26.630 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:26.630 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:26.639 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> import uuid 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> n = 600 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = False 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> if 0 > 0: 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> create_hard_links = True 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> dpath = os.path.dirname(path) 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> fnameprefix = os.path.basename(path) 2024-09-17T05:04:26.640 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(dpath, exist_ok=True) 2024-09-17T05:04:26.641 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:04:26.641 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:04:26.641 DEBUG:teuthology.orchestra.run.smithi016:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-09-17T05:04:26.641 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:04:26.641 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(n): 2024-09-17T05:04:26.642 DEBUG:teuthology.orchestra.run.smithi016:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-09-17T05:04:26.642 DEBUG:teuthology.orchestra.run.smithi016:> with open(fpath, '"'"'w'"'"') as f: 2024-09-17T05:04:26.642 DEBUG:teuthology.orchestra.run.smithi016:> f.write(f"{i}") 2024-09-17T05:04:26.642 DEBUG:teuthology.orchestra.run.smithi016:> if True: 2024-09-17T05:04:26.642 DEBUG:teuthology.orchestra.run.smithi016:> f.flush() 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(f.fileno()) 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> os.unlink(fpath) 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> if create_hard_links: 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> for j in range(0): 2024-09-17T05:04:26.643 DEBUG:teuthology.orchestra.run.smithi016:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-09-17T05:04:26.644 DEBUG:teuthology.orchestra.run.smithi016:> if False: 2024-09-17T05:04:26.644 DEBUG:teuthology.orchestra.run.smithi016:> os.fsync(dirfd) 2024-09-17T05:04:26.644 DEBUG:teuthology.orchestra.run.smithi016:> finally: 2024-09-17T05:04:26.644 DEBUG:teuthology.orchestra.run.smithi016:> os.close(dirfd) 2024-09-17T05:04:26.644 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:04:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:29.895 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:29.895 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:29.946 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:04:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:04:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:04:30.048 INFO:teuthology.orchestra.run.smithi016.stdout:client5558 2024-09-17T05:04:30.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 -- 172.21.15.16:0/796990055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6840b2f50 msgr2=0x7fc68415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 --2- 172.21.15.16:0/796990055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6840b2f50 0x7fc68415bb60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc674009870 tx=0x7fc67401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 -- 172.21.15.16:0/796990055 shutdown_connections 2024-09-17T05:04:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 --2- 172.21.15.16:0/796990055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6840b2f50 0x7fc68415bb60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 -- 172.21.15.16:0/796990055 >> 172.21.15.16:0/796990055 conn(0x7fc68406f900 msgr2=0x7fc6840b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 -- 172.21.15.16:0/796990055 shutdown_connections 2024-09-17T05:04:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 -- 172.21.15.16:0/796990055 wait complete. 2024-09-17T05:04:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 Processor -- start 2024-09-17T05:04:30.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.204+0000 7fc68a400640 1 -- start start 2024-09-17T05:04:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc684073240 0x7fc684165660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6840b2f50 0x7fc6841f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 0x7fc6841f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc68406e210 con 0x7fc68415c0a0 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc68406e090 con 0x7fc6840b2f50 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc68406e390 con 0x7fc684073240 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 0x7fc6841f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 0x7fc6841f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47066/0 (socket says 172.21.15.16:47066) 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 -- 172.21.15.16:0/1115940816 learned_addr learned my addr 172.21.15.16:0/1115940816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc684073240 0x7fc684165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc67aa00640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6840b2f50 0x7fc6841f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 -- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc684073240 msgr2=0x7fc684165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc684073240 0x7fc684165660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 -- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6840b2f50 msgr2=0x7fc6841f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6840b2f50 0x7fc6841f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 -- 172.21.15.16:0/1115940816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68414fe90 con 0x7fc68415c0a0 2024-09-17T05:04:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc684073240 0x7fc684165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc683e00640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 0x7fc6841f8260 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c3c0 tx=0x7fc67c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c011790 con 0x7fc68415c0a0 2024-09-17T05:04:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc67c017070 con 0x7fc68415c0a0 2024-09-17T05:04:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c011ce0 con 0x7fc68415c0a0 2024-09-17T05:04:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc684150170 con 0x7fc68415c0a0 2024-09-17T05:04:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc684150730 con 0x7fc68415c0a0 2024-09-17T05:04:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.208+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68415e750 con 0x7fc68415c0a0 2024-09-17T05:04:30.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.212+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc67c01c460 con 0x7fc68415c0a0 2024-09-17T05:04:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.212+0000 7fc67be00640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073200 0x7fc660075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.212+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fc67c0940a0 con 0x7fc68415c0a0 2024-09-17T05:04:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.212+0000 7fc683400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073200 0x7fc660075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.212+0000 7fc683400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073200 0x7fc660075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc6740af000 tx=0x7fc67401b010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.216+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc67c05e7c0 con 0x7fc68415c0a0 2024-09-17T05:04:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.424+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc68415e470 con 0x7fc68415c0a0 2024-09-17T05:04:30.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.424+0000 7fc67be00640 1 -- 172.21.15.16:0/1115940816 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7fc67c0631f0 con 0x7fc68415c0a0 2024-09-17T05:04:30.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:30.429 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:30.429 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073200 msgr2=0x7fc660075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073200 0x7fc660075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc6740af000 tx=0x7fc67401b010 comp rx=0 tx=0).stop 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 msgr2=0x7fc6841f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 0x7fc6841f8260 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00c3c0 tx=0x7fc67c00c880 comp rx=0 tx=0).stop 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 shutdown_connections 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc660073200 0x7fc660075710 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc68415c0a0 0x7fc6841f8260 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6840b2f50 0x7fc6841f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 --2- 172.21.15.16:0/1115940816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc684073240 0x7fc684165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 >> 172.21.15.16:0/1115940816 conn(0x7fc68406f900 msgr2=0x7fc684166600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 shutdown_connections 2024-09-17T05:04:30.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:30.428+0000 7fc68a400640 1 -- 172.21.15.16:0/1115940816 wait complete. 2024-09-17T05:04:30.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-17T05:04:30.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:04:30.541 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:04:30.623 INFO:teuthology.orchestra.run.smithi104.stdout:[{"id":5564,"entity":{"name":{"type":"client","num":5564},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1912993728}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":4.8759525730000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9711039654936768,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5564 v1:192.168.0.1:0/1912993728","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":5558,"entity":{"name":{"type":"client","num":5558},"addr":{"type":"v1","addr":"172.21.15.16:0","nonce":228855557}},"state":"open","num_leases":1,"num_caps":602,"request_load_avg":612,"uptime":6.0599410489999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":1127.844606479712,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":785.38939316917003,"halflife":1.5},"recall_caps_throttle2o":{"value":485.525954631753,"halflife":0.5},"session_cache_liveness":{"value":1198.6340382404267,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5558 v1:172.21.15.16:0/228855557","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:04:30.624 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1912993728, "type": "v1" }, "name": { "num": 5564, "type": "client" } }, "id": 5564, "inst": "client.5564 v1:192.168.0.1:0/1912993728", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.971103965493677 }, "state": "open", "uptime": 4.875952573 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 228855557, "type": "v1" }, "name": { "num": 5558, "type": "client" } }, "id": 5558, "inst": "client.5558 v1:172.21.15.16:0/228855557", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 602, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 1127.844606479712 }, "recall_caps_throttle": { "halflife": 1.5, "value": 785.38939316917 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 485.525954631753 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 612, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1198.6340382404267 }, "state": "open", "uptime": 6.059941049 } ] 2024-09-17T05:04:30.624 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-17T05:04:35.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 -- 172.21.15.16:0/189243531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c00cf390 msgr2=0x7f74c0160170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 --2- 172.21.15.16:0/189243531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c00cf390 0x7f74c0160170 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f74c0072860 tx=0x7f74ac01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 -- 172.21.15.16:0/189243531 shutdown_connections 2024-09-17T05:04:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 --2- 172.21.15.16:0/189243531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c00cf390 0x7f74c0160170 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 --2- 172.21.15.16:0/189243531 >> v2:172.21.15.104:3300/0 conn(0x7f74c015d600 0x7f74c00cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 -- 172.21.15.16:0/189243531 >> 172.21.15.16:0/189243531 conn(0x7f74c006f900 msgr2=0x7f74c00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 -- 172.21.15.16:0/189243531 shutdown_connections 2024-09-17T05:04:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.792+0000 7f74c7e00640 1 -- 172.21.15.16:0/189243531 wait complete. 2024-09-17T05:04:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 Processor -- start 2024-09-17T05:04:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 -- start start 2024-09-17T05:04:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74c00cf390 0x7f74c015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 0x7f74c00a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01606b0 0x7f74c013f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00b48b0 con 0x7f74c015d600 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74c00b5280 con 0x7f74c01606b0 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f74c00b4a30 con 0x7f74c00cf390 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 0x7f74c00a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 0x7f74c00a9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47086/0 (socket says 172.21.15.16:47086) 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 -- 172.21.15.16:0/4134643692 learned_addr learned my addr 172.21.15.16:0/4134643692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c5400640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74c00cf390 0x7f74c015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c5e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01606b0 0x7f74c013f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 -- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74c00cf390 msgr2=0x7f74c015a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74c00cf390 0x7f74c015a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 -- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01606b0 msgr2=0x7f74c013f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01606b0 0x7f74c013f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 -- 172.21.15.16:0/4134643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c00b0f00 con 0x7f74c015d600 2024-09-17T05:04:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c4a00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 0x7f74c00a9450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0194d0 tx=0x7f74ac0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0bc070 con 0x7f74c015d600 2024-09-17T05:04:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74ac004430 con 0x7f74c015d600 2024-09-17T05:04:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0b2040 con 0x7f74c015d600 2024-09-17T05:04:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.796+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c00b11e0 con 0x7f74c015d600 2024-09-17T05:04:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.800+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c0157e40 con 0x7f74c015d600 2024-09-17T05:04:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.800+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c0160170 con 0x7f74c015d600 2024-09-17T05:04:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.800+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f74ac005290 con 0x7f74c015d600 2024-09-17T05:04:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.800+0000 7f74bd600640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f74a0073230 0x7f74a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.800+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f74ac12dba0 con 0x7f74c015d600 2024-09-17T05:04:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.804+0000 7f74c5400640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f74a0073230 0x7f74a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.804+0000 7f74c5400640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f74a0073230 0x7f74a0075740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74c00afb90 tx=0x7f74b400b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:35.808+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74ac0164b0 con 0x7f74c015d600 2024-09-17T05:04:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.016+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74c00b3530 con 0x7f74c015d600 2024-09-17T05:04:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74bd600640 1 -- 172.21.15.16:0/4134643692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7f74ac0f8340 con 0x7f74c015d600 2024-09-17T05:04:36.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:36.022 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f74a0073230 msgr2=0x7f74a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f74a0073230 0x7f74a0075740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f74c00afb90 tx=0x7f74b400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 msgr2=0x7f74c00a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 0x7f74c00a9450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0194d0 tx=0x7f74ac0a57d0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 shutdown_connections 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f74a0073230 0x7f74a0075740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01606b0 0x7f74c013f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74c015d600 0x7f74c00a9450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 --2- 172.21.15.16:0/4134643692 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74c00cf390 0x7f74c015a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 >> 172.21.15.16:0/4134643692 conn(0x7f74c006f900 msgr2=0x7f74c00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 shutdown_connections 2024-09-17T05:04:36.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.020+0000 7f74c7e00640 1 -- 172.21.15.16:0/4134643692 wait complete. 2024-09-17T05:04:36.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-17T05:04:36.227 INFO:teuthology.orchestra.run.smithi104.stdout:[{"id":5564,"entity":{"name":{"type":"client","num":5564},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":1912993728}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":10.479898145,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9328824938495601,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5564 v1:192.168.0.1:0/1912993728","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":5558,"entity":{"name":{"type":"client","num":5558},"addr":{"type":"v1","addr":"172.21.15.16:0","nonce":228855557}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":574,"uptime":11.663886621,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":2059.4332317374042,"halflife":60},"release_caps":{"value":487.03310345717387,"halflife":60},"recall_caps_throttle":{"value":383.40184523745734,"halflife":1.5},"recall_caps_throttle2o":{"value":26.761686816869819,"halflife":0.5},"session_cache_liveness":{"value":1183.2143297965767,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5558 v1:172.21.15.16:0/228855557","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:04:36.228 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1912993728, "type": "v1" }, "name": { "num": 5564, "type": "client" } }, "id": 5564, "inst": "client.5564 v1:192.168.0.1:0/1912993728", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.93288249384956 }, "state": "open", "uptime": 10.479898145 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 228855557, "type": "v1" }, "name": { "num": 5558, "type": "client" } }, "id": 5558, "inst": "client.5558 v1:172.21.15.16:0/228855557", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 2059.433231737404 }, "recall_caps_throttle": { "halflife": 1.5, "value": 383.40184523745734 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 26.76168681686982 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 487.03310345717387 }, "request_load_avg": 574, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1183.2143297965767 }, "state": "open", "uptime": 11.663886621 } ] 2024-09-17T05:04:36.228 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:04:36.229 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:04:36.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:36.312 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:36.395 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:04:36.395 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:36.395 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:36.395 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:36.404 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:36.404 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:36.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.652+0000 7f3951e00640 1 -- 172.21.15.16:0/1424677006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 msgr2=0x7f394c071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.652+0000 7f3951e00640 1 --2- 172.21.15.16:0/1424677006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c071970 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f394c072f60 tx=0x7f393401c740 comp rx=0 tx=0).stop 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- 172.21.15.16:0/1424677006 shutdown_connections 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 --2- 172.21.15.16:0/1424677006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c071970 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 --2- 172.21.15.16:0/1424677006 >> v2:172.21.15.104:3300/0 conn(0x7f394c0cd1d0 0x7f394c0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- 172.21.15.16:0/1424677006 >> 172.21.15.16:0/1424677006 conn(0x7f394c0d1170 msgr2=0x7f394c0734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- 172.21.15.16:0/1424677006 shutdown_connections 2024-09-17T05:04:36.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- 172.21.15.16:0/1424677006 wait complete. 2024-09-17T05:04:36.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 Processor -- start 2024-09-17T05:04:36.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- start start 2024-09-17T05:04:36.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c1f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:36.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c0cd1d0 0x7f394c143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f394c1529e0 0x7f394c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f394c15e340 con 0x7f394c071560 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f394c15e1c0 con 0x7f394c0cd1d0 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.656+0000 7f3951e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f394c15e4c0 con 0x7f394c1529e0 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c1f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c1f93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47102/0 (socket says 172.21.15.16:47102) 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 -- 172.21.15.16:0/2833853996 learned_addr learned my addr 172.21.15.16:0/2833853996 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 -- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f394c1529e0 msgr2=0x7f394c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f394c1529e0 0x7f394c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 -- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c0cd1d0 msgr2=0x7f394c143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c0cd1d0 0x7f394c143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 -- 172.21.15.16:0/2833853996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c144450 con 0x7f394c071560 2024-09-17T05:04:36.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3950a00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c1f93a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e7a0 tx=0x7f393c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c00ccc0 con 0x7f394c071560 2024-09-17T05:04:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f393c010430 con 0x7f394c071560 2024-09-17T05:04:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c150c40 con 0x7f394c071560 2024-09-17T05:04:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c010860 con 0x7f394c071560 2024-09-17T05:04:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c151200 con 0x7f394c071560 2024-09-17T05:04:36.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c071970 con 0x7f394c071560 2024-09-17T05:04:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.660+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f393c010a00 con 0x7f394c071560 2024-09-17T05:04:36.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.664+0000 7f3949600640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3928073200 0x7f3928075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.664+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f393c094df0 con 0x7f394c071560 2024-09-17T05:04:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.664+0000 7f3943e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3928073200 0x7f3928075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:36.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.664+0000 7f3943e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3928073200 0x7f3928075710 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f39340af000 tx=0x7f39340194e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:36.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.668+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f393c05fc70 con 0x7f394c071560 2024-09-17T05:04:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.852+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f394c150080 con 0x7f394c071560 2024-09-17T05:04:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.852+0000 7f3949600640 1 -- 172.21.15.16:0/2833853996 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7f393c05f570 con 0x7f394c071560 2024-09-17T05:04:36.855 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.852+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3928073200 msgr2=0x7f3928075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.852+0000 7f3951e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3928073200 0x7f3928075710 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f39340af000 tx=0x7f39340194e0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.852+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 msgr2=0x7f394c1f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.852+0000 7f3951e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c1f93a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e7a0 tx=0x7f393c00ec60 comp rx=0 tx=0).stop 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 shutdown_connections 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3928073200 0x7f3928075710 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f394c1529e0 0x7f394c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c0cd1d0 0x7f394c143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 --2- 172.21.15.16:0/2833853996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c071560 0x7f394c1f93a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 >> 172.21.15.16:0/2833853996 conn(0x7f394c0d1170 msgr2=0x7f394c1491b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:36.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 shutdown_connections 2024-09-17T05:04:36.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:36.856+0000 7f3951e00640 1 -- 172.21.15.16:0/2833853996 wait complete. 2024-09-17T05:04:36.878 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:04:36.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:36.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:04:37.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:04:37.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:37.026 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:04:37.035 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:37.036 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:37.036 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:37.045 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:37.045 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:37.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:04:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21ac800640 1 -- 172.21.15.16:0/1495107821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c003a10 msgr2=0x7f218c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21ac800640 1 --2- 172.21.15.16:0/1495107821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f218c003a10 0x7f218c003e80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f21a40722a0 tx=0x7f219801d2d0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21aa800640 1 -- 172.21.15.16:0/1495107821 reap_dead start 2024-09-17T05:04:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21ac800640 1 -- 172.21.15.16:0/1495107821 shutdown_connections 2024-09-17T05:04:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21ac800640 1 -- 172.21.15.16:0/1495107821 >> 172.21.15.16:0/1495107821 conn(0x7f21a40b2250 msgr2=0x7f21a40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21ac800640 1 -- 172.21.15.16:0/1495107821 shutdown_connections 2024-09-17T05:04:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.256+0000 7f21ac800640 1 -- 172.21.15.16:0/1495107821 wait complete. 2024-09-17T05:04:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 Processor -- start 2024-09-17T05:04:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 -- start start 2024-09-17T05:04:37.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f21a41f7e50 0x7f21a4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a4158590 0x7f21a41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 0x7f21a4163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21a4166d60 con 0x7f21a41638d0 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21a4166be0 con 0x7f21a4158590 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21ac800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f21a4166ee0 con 0x7f21a41f7e50 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21aa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 0x7f21a4163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21aa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 0x7f21a4163ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39328/0 (socket says 172.21.15.16:39328) 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21aa800640 1 -- 172.21.15.16:0/53670888 learned_addr learned my addr 172.21.15.16:0/53670888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21a9e00640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f21a41f7e50 0x7f21a4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.260+0000 7f21a9400640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a4158590 0x7f21a41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21aa800640 1 -- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f21a41f7e50 msgr2=0x7f21a4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21aa800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f21a41f7e50 0x7f21a4153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21aa800640 1 -- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a4158590 msgr2=0x7f21a41589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21aa800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a4158590 0x7f21a41589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21aa800640 1 -- 172.21.15.16:0/53670888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a41538d0 con 0x7f21a41638d0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21aa800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 0x7f21a4163ce0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f21a40d0c90 tx=0x7f21a0006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a000fbc0 con 0x7f21a41638d0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21a000fd80 con 0x7f21a41638d0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a4158ee0 con 0x7f21a41638d0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0011690 con 0x7f21a41638d0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a4156200 con 0x7f21a41638d0 2024-09-17T05:04:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a415f500 con 0x7f21a41638d0 2024-09-17T05:04:37.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.264+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21a0011830 con 0x7f21a41638d0 2024-09-17T05:04:37.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.268+0000 7f219e000640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2188073200 0x7f2188075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.268+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f21a0094d40 con 0x7f21a41638d0 2024-09-17T05:04:37.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.268+0000 7f21a9e00640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2188073200 0x7f2188075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.268+0000 7f21a9e00640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2188073200 0x7f2188075710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f21a40722a0 tx=0x7f219801a070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.272+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21a4158ee0 con 0x7f21a41638d0 2024-09-17T05:04:37.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.456+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21a415bf00 con 0x7f21a41638d0 2024-09-17T05:04:37.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.456+0000 7f219e000640 1 -- 172.21.15.16:0/53670888 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7f21a005f580 con 0x7f21a41638d0 2024-09-17T05:04:37.461 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:04:37.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2188073200 msgr2=0x7f2188075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2188073200 0x7f2188075710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f21a40722a0 tx=0x7f219801a070 comp rx=0 tx=0).stop 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 msgr2=0x7f21a4163ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 0x7f21a4163ce0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f21a40d0c90 tx=0x7f21a0006720 comp rx=0 tx=0).stop 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 shutdown_connections 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2188073200 0x7f2188075710 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21a41638d0 0x7f21a4163ce0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f21a4158590 0x7f21a41589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 --2- 172.21.15.16:0/53670888 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f21a41f7e50 0x7f21a4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 >> 172.21.15.16:0/53670888 conn(0x7f21a40b2250 msgr2=0x7f21a4155460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 shutdown_connections 2024-09-17T05:04:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.460+0000 7f21ac800640 1 -- 172.21.15.16:0/53670888 wait complete. 2024-09-17T05:04:37.483 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:04:37.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:37.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:04:37.509 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:04:37.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:04:37.509 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:04:37.562 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:37.562 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:37.563 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:37.571 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:04:37.572 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:04:37.572 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:37.572 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:37.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:04:37.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 -- 172.21.15.16:0/528649251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840cf390 msgr2=0x7ff084160170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 --2- 172.21.15.16:0/528649251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840cf390 0x7ff084160170 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff084072860 tx=0x7ff06c01c800 comp rx=0 tx=0).stop 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 -- 172.21.15.16:0/528649251 shutdown_connections 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 --2- 172.21.15.16:0/528649251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0840cf390 0x7ff084160170 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 --2- 172.21.15.16:0/528649251 >> v2:172.21.15.104:3300/0 conn(0x7ff08415d600 0x7ff0840cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 -- 172.21.15.16:0/528649251 >> 172.21.15.16:0/528649251 conn(0x7ff08406f900 msgr2=0x7ff0840b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 -- 172.21.15.16:0/528649251 shutdown_connections 2024-09-17T05:04:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.772+0000 7ff089200640 1 -- 172.21.15.16:0/528649251 wait complete. 2024-09-17T05:04:37.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 Processor -- start 2024-09-17T05:04:37.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 -- start start 2024-09-17T05:04:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0840cf390 0x7ff0841393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08415d600 0x7ff08413a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 0x7ff08413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0840b48b0 con 0x7ff0841606b0 2024-09-17T05:04:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0840b5280 con 0x7ff08415d600 2024-09-17T05:04:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff0840b4a30 con 0x7ff0840cf390 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 0x7ff08413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 0x7ff08413ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39348/0 (socket says 172.21.15.16:39348) 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 -- 172.21.15.16:0/255136848 learned_addr learned my addr 172.21.15.16:0/255136848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff079400640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08415d600 0x7ff08413a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff081e00640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0840cf390 0x7ff0841393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 -- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0840cf390 msgr2=0x7ff0841393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0840cf390 0x7ff0841393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 -- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08415d600 msgr2=0x7ff08413a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08415d600 0x7ff08413a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 -- 172.21.15.16:0/255136848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff084139c10 con 0x7ff0841606b0 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff082800640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 0x7ff08413ad60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff07c010e60 tx=0x7ff07c010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c01d070 con 0x7ff0841606b0 2024-09-17T05:04:37.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff07c00aa30 con 0x7ff0841606b0 2024-09-17T05:04:37.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c018410 con 0x7ff0841606b0 2024-09-17T05:04:37.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08413d4f0 con 0x7ff0841606b0 2024-09-17T05:04:37.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.776+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841f75d0 con 0x7ff0841606b0 2024-09-17T05:04:37.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.780+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff084160170 con 0x7ff0841606b0 2024-09-17T05:04:37.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.780+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff07c020070 con 0x7ff0841606b0 2024-09-17T05:04:37.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.780+0000 7ff07aa00640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff060073200 0x7ff060075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.780+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7ff07c097be0 con 0x7ff0841606b0 2024-09-17T05:04:37.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.780+0000 7ff081e00640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff060073200 0x7ff060075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:37.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.780+0000 7ff081e00640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff060073200 0x7ff060075710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff0840afb90 tx=0x7ff0740091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:37.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.784+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff07c062420 con 0x7ff0841606b0 2024-09-17T05:04:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.992+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff084139e10 con 0x7ff0841606b0 2024-09-17T05:04:37.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff07aa00640 1 -- 172.21.15.16:0/255136848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7ff07c066e50 con 0x7ff0841606b0 2024-09-17T05:04:37.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:37.998 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff060073200 msgr2=0x7ff060075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff060073200 0x7ff060075710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff0840afb90 tx=0x7ff0740091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 msgr2=0x7ff08413ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 0x7ff08413ad60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff07c010e60 tx=0x7ff07c010f50 comp rx=0 tx=0).stop 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 shutdown_connections 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff060073200 0x7ff060075710 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0841606b0 0x7ff08413ad60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff08415d600 0x7ff08413a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 --2- 172.21.15.16:0/255136848 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0840cf390 0x7ff0841393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 >> 172.21.15.16:0/255136848 conn(0x7ff08406f900 msgr2=0x7ff08415e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 shutdown_connections 2024-09-17T05:04:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:37.996+0000 7ff089200640 1 -- 172.21.15.16:0/255136848 wait complete. 2024-09-17T05:04:38.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 -- 172.21.15.16:0/889332344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c0b2f50 msgr2=0x7f122c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 --2- 172.21.15.16:0/889332344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c0b2f50 0x7f122c15bb60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1220009900 tx=0x7f122001c870 comp rx=0 tx=0).stop 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 -- 172.21.15.16:0/889332344 shutdown_connections 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 --2- 172.21.15.16:0/889332344 >> v2:172.21.15.16:3300/0 conn(0x7f122c073240 0x7f122c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 --2- 172.21.15.16:0/889332344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c0b2f50 0x7f122c15bb60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 -- 172.21.15.16:0/889332344 >> 172.21.15.16:0/889332344 conn(0x7f122c06f900 msgr2=0x7f122c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:38.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 -- 172.21.15.16:0/889332344 shutdown_connections 2024-09-17T05:04:38.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.176+0000 7f1233600640 1 -- 172.21.15.16:0/889332344 wait complete. 2024-09-17T05:04:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 Processor -- start 2024-09-17T05:04:38.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 -- start start 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f122c073240 0x7f122c1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 0x7f122c0d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c15c0a0 0x7f122c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f122c168d60 con 0x7f122c0b2f50 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f122c168be0 con 0x7f122c073240 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f122c168ee0 con 0x7f122c15c0a0 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 0x7f122c0d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1230c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f122c073240 0x7f122c1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 0x7f122c0d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39366/0 (socket says 172.21.15.16:39366) 2024-09-17T05:04:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 -- 172.21.15.16:0/1178018304 learned_addr learned my addr 172.21.15.16:0/1178018304 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1231600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c15c0a0 0x7f122c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 -- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c15c0a0 msgr2=0x7f122c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c15c0a0 0x7f122c0d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 -- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f122c073240 msgr2=0x7f122c1508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f122c073240 0x7f122c1508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 -- 172.21.15.16:0/1178018304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122c162640 con 0x7f122c0b2f50 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f122be00640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 0x7f122c0d0100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f121800c340 tx=0x7f121800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218017070 con 0x7f122c0b2f50 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1218003600 con 0x7f122c0b2f50 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218003a90 con 0x7f122c0b2f50 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122c161ac0 con 0x7f122c0b2f50 2024-09-17T05:04:38.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.180+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122c162080 con 0x7f122c0b2f50 2024-09-17T05:04:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.184+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122c073650 con 0x7f122c0b2f50 2024-09-17T05:04:38.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.184+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f121801a070 con 0x7f122c0b2f50 2024-09-17T05:04:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.184+0000 7f1228c00640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f120c073200 0x7f120c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.184+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f1218093540 con 0x7f122c0b2f50 2024-09-17T05:04:38.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.184+0000 7f1230c00640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f120c073200 0x7f120c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.184+0000 7f1230c00640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f120c073200 0x7f120c075710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f12200af000 tx=0x7f122001b0b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:38.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.188+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f121805de00 con 0x7f122c0b2f50 2024-09-17T05:04:38.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.396+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f122c06e150 con 0x7f122c0b2f50 2024-09-17T05:04:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.396+0000 7f1228c00640 1 -- 172.21.15.16:0/1178018304 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7f1218062830 con 0x7f122c0b2f50 2024-09-17T05:04:38.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:38.400 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:38.400 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f120c073200 msgr2=0x7f120c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f120c073200 0x7f120c075710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f12200af000 tx=0x7f122001b0b0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 msgr2=0x7f122c0d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 0x7f122c0d0100 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f121800c340 tx=0x7f121800c800 comp rx=0 tx=0).stop 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 shutdown_connections 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f120c073200 0x7f120c075710 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f122c15c0a0 0x7f122c0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f122c0b2f50 0x7f122c0d0100 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 --2- 172.21.15.16:0/1178018304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f122c073240 0x7f122c1508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 >> 172.21.15.16:0/1178018304 conn(0x7f122c06f900 msgr2=0x7f122c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 shutdown_connections 2024-09-17T05:04:38.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.400+0000 7f1233600640 1 -- 172.21.15.16:0/1178018304 wait complete. 2024-09-17T05:04:38.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 -- 172.21.15.16:0/328621124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980d1c90 msgr2=0x7f679815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 --2- 172.21.15.16:0/328621124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980d1c90 0x7f679815cab0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6780009900 tx=0x7f678001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 -- 172.21.15.16:0/328621124 shutdown_connections 2024-09-17T05:04:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 --2- 172.21.15.16:0/328621124 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 0x7f67980c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 --2- 172.21.15.16:0/328621124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980d1c90 0x7f679815cab0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 -- 172.21.15.16:0/328621124 >> 172.21.15.16:0/328621124 conn(0x7f67980d2270 msgr2=0x7f67980d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 -- 172.21.15.16:0/328621124 shutdown_connections 2024-09-17T05:04:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.576+0000 7f679d600640 1 -- 172.21.15.16:0/328621124 wait complete. 2024-09-17T05:04:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 Processor -- start 2024-09-17T05:04:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 -- start start 2024-09-17T05:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 0x7f67981fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 0x7f6798153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67980d1120 0x7f67980d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6798169fd0 con 0x7f67980cc8c0 2024-09-17T05:04:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6798169e50 con 0x7f67980d1120 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f679d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f679816a150 con 0x7f67980c9440 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f678fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 0x7f6798153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f678fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 0x7f6798153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39388/0 (socket says 172.21.15.16:39388) 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f678fe00640 1 -- 172.21.15.16:0/1603057960 learned_addr learned my addr 172.21.15.16:0/1603057960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f6797400640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67980d1120 0x7f67980d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.580+0000 7f6796a00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 0x7f67981fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678fe00640 1 -- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 msgr2=0x7f67981fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678fe00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 0x7f67981fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678fe00640 1 -- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67980d1120 msgr2=0x7f67980d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678fe00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67980d1120 0x7f67980d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678fe00640 1 -- 172.21.15.16:0/1603057960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67981fd120 con 0x7f67980cc8c0 2024-09-17T05:04:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f6796a00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 0x7f67981fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678fe00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 0x7f6798153210 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6780009850 tx=0x7f67800a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67800b2040 con 0x7f67980cc8c0 2024-09-17T05:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67800043b0 con 0x7f67980cc8c0 2024-09-17T05:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780005290 con 0x7f67980cc8c0 2024-09-17T05:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67980d1a70 con 0x7f67980cc8c0 2024-09-17T05:04:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679806e8d0 con 0x7f67980cc8c0 2024-09-17T05:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f679806e360 con 0x7f67980cc8c0 2024-09-17T05:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.584+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6780005430 con 0x7f67980cc8c0 2024-09-17T05:04:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.588+0000 7f678ea00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6774073230 0x7f6774075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.588+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f678012d960 con 0x7f67980cc8c0 2024-09-17T05:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.588+0000 7f6796a00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6774073230 0x7f6774075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.588+0000 7f6796a00640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6774073230 0x7f6774075740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f67980b1bc0 tx=0x7f6788009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:38.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.592+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f67980d1a70 con 0x7f67980cc8c0 2024-09-17T05:04:38.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.764+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67980c9850 con 0x7f67980cc8c0 2024-09-17T05:04:38.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f678ea00640 1 -- 172.21.15.16:0/1603057960 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16933 (secure 0 0 0) 0x7f67800bc060 con 0x7f67980cc8c0 2024-09-17T05:04:38.770 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:38.770 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:18.428662+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:14.578846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:15.811287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6774073230 msgr2=0x7f6774075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6774073230 0x7f6774075740 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f67980b1bc0 tx=0x7f6788009040 comp rx=0 tx=0).stop 2024-09-17T05:04:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 msgr2=0x7f6798153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 0x7f6798153210 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6780009850 tx=0x7f67800a5730 comp rx=0 tx=0).stop 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 shutdown_connections 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6774073230 0x7f6774075740 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67980d1120 0x7f67980d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67980cc8c0 0x7f6798153210 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 --2- 172.21.15.16:0/1603057960 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f67980c9440 0x7f67981fcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 >> 172.21.15.16:0/1603057960 conn(0x7f67980d2270 msgr2=0x7f67980cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 shutdown_connections 2024-09-17T05:04:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.768+0000 7f679d600640 1 -- 172.21.15.16:0/1603057960 wait complete. 2024-09-17T05:04:38.792 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:04:38.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 -- 172.21.15.16:0/2034633190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc15c0a0 msgr2=0x7f13bc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 --2- 172.21.15.16:0/2034633190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc15c0a0 0x7f13bc15c4b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009840 tx=0x7f13a401c790 comp rx=0 tx=0).stop 2024-09-17T05:04:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 -- 172.21.15.16:0/2034633190 shutdown_connections 2024-09-17T05:04:38.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 --2- 172.21.15.16:0/2034633190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc15c0a0 0x7f13bc15c4b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 --2- 172.21.15.16:0/2034633190 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13bc0b2f50 0x7f13bc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 -- 172.21.15.16:0/2034633190 >> 172.21.15.16:0/2034633190 conn(0x7f13bc06f900 msgr2=0x7f13bc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 -- 172.21.15.16:0/2034633190 shutdown_connections 2024-09-17T05:04:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.948+0000 7f13c3200640 1 -- 172.21.15.16:0/2034633190 wait complete. 2024-09-17T05:04:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 Processor -- start 2024-09-17T05:04:38.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 -- start start 2024-09-17T05:04:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc0b2f50 0x7f13bc1650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 0x7f13bc165600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13bc15f150 0x7f13bc1632f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc169150 con 0x7f13bc15c0a0 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc168fd0 con 0x7f13bc0b2f50 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f13bc1692d0 con 0x7f13bc15f150 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 0x7f13bc165600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 0x7f13bc165600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39402/0 (socket says 172.21.15.16:39402) 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 -- 172.21.15.16:0/3746453648 learned_addr learned my addr 172.21.15.16:0/3746453648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13bbe00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc0b2f50 0x7f13bc1650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c1200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13bc15f150 0x7f13bc1632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 -- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13bc15f150 msgr2=0x7f13bc1632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13bc15f150 0x7f13bc1632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 -- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc0b2f50 msgr2=0x7f13bc1650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc0b2f50 0x7f13bc1650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 -- 172.21.15.16:0/3746453648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc165e60 con 0x7f13bc15c0a0 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b3e00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 0x7f13bc165600 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009790 tx=0x7f13a40a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40bb070 con 0x7f13bc15c0a0 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13a4005030 con 0x7f13bc15c0a0 2024-09-17T05:04:38.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a40b2040 con 0x7f13bc15c0a0 2024-09-17T05:04:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc13fcf0 con 0x7f13bc15c0a0 2024-09-17T05:04:38.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.952+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc13d6d0 con 0x7f13bc15c0a0 2024-09-17T05:04:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.956+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc165bd0 con 0x7f13bc15c0a0 2024-09-17T05:04:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.956+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13a4005460 con 0x7f13bc15c0a0 2024-09-17T05:04:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.956+0000 7f13b8c00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1398073200 0x7f1398075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:38.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.956+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f13a412dd20 con 0x7f13bc15c0a0 2024-09-17T05:04:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.956+0000 7f13bbe00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1398073200 0x7f1398075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:38.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.960+0000 7f13bbe00640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1398073200 0x7f1398075710 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ad0 tx=0x7f13ac009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:38.960+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13a40f1db0 con 0x7f13bc15c0a0 2024-09-17T05:04:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.176+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13bc15bb60 con 0x7f13bc15c0a0 2024-09-17T05:04:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.176+0000 7f13b8c00640 1 -- 172.21.15.16:0/3746453648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7f13a4016410 con 0x7f13bc15c0a0 2024-09-17T05:04:39.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:39.181 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:39.181 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:39.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1398073200 msgr2=0x7f1398075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1398073200 0x7f1398075710 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ad0 tx=0x7f13ac009260 comp rx=0 tx=0).stop 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 msgr2=0x7f13bc165600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 0x7f13bc165600 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009790 tx=0x7f13a40a5780 comp rx=0 tx=0).stop 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 shutdown_connections 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1398073200 0x7f1398075710 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13bc15f150 0x7f13bc1632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13bc15c0a0 0x7f13bc165600 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 --2- 172.21.15.16:0/3746453648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc0b2f50 0x7f13bc1650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 >> 172.21.15.16:0/3746453648 conn(0x7f13bc06f900 msgr2=0x7f13bc1518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 shutdown_connections 2024-09-17T05:04:39.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.180+0000 7f13c3200640 1 -- 172.21.15.16:0/3746453648 wait complete. 2024-09-17T05:04:39.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 -- 172.21.15.16:0/2457557309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6000714a0 msgr2=0x7fc60015caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 --2- 172.21.15.16:0/2457557309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6000714a0 0x7fc60015caf0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc6000d0240 tx=0x7fc5f801ccc0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 -- 172.21.15.16:0/2457557309 shutdown_connections 2024-09-17T05:04:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 --2- 172.21.15.16:0/2457557309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6000714a0 0x7fc60015caf0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 --2- 172.21.15.16:0/2457557309 >> v2:172.21.15.104:3300/0 conn(0x7fc6000d0a00 0x7fc60015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 -- 172.21.15.16:0/2457557309 >> 172.21.15.16:0/2457557309 conn(0x7fc6000b4160 msgr2=0x7fc6000b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 -- 172.21.15.16:0/2457557309 shutdown_connections 2024-09-17T05:04:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 -- 172.21.15.16:0/2457557309 wait complete. 2024-09-17T05:04:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 Processor -- start 2024-09-17T05:04:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.360+0000 7fc5ff200640 1 -- start start 2024-09-17T05:04:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6000714a0 0x7fc600152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6000d1aa0 0x7fc600157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 0x7fc6001582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc600169a10 con 0x7fc6001546b0 2024-09-17T05:04:39.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc600169890 con 0x7fc6000714a0 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc600169b90 con 0x7fc6000d1aa0 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 0x7fc6001582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 0x7fc6001582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39418/0 (socket says 172.21.15.16:39418) 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 -- 172.21.15.16:0/4274494436 learned_addr learned my addr 172.21.15.16:0/4274494436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fde00640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6000714a0 0x7fc600152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fd400640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6000d1aa0 0x7fc600157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 -- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6000d1aa0 msgr2=0x7fc600157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6000d1aa0 0x7fc600157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 -- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6000714a0 msgr2=0x7fc600152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6000714a0 0x7fc600152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 -- 172.21.15.16:0/4274494436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc600157190 con 0x7fc6001546b0 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fde00640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6000714a0 0x7fc600152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5fe800640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 0x7fc6001582e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009bd0 tx=0x7fc5f8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:39.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80bb070 con 0x7fc6001546b0 2024-09-17T05:04:39.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5f8004d40 con 0x7fc6001546b0 2024-09-17T05:04:39.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6001541d0 con 0x7fc6001546b0 2024-09-17T05:04:39.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b2040 con 0x7fc6001546b0 2024-09-17T05:04:39.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6000b52c0 con 0x7fc6001546b0 2024-09-17T05:04:39.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60015c5b0 con 0x7fc6001546b0 2024-09-17T05:04:39.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.364+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5f80be070 con 0x7fc6001546b0 2024-09-17T05:04:39.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.368+0000 7fc5f6000640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc5e0073230 0x7fc5e0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.368+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fc5f812e1e0 con 0x7fc6001546b0 2024-09-17T05:04:39.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.368+0000 7fc5fde00640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc5e0073230 0x7fc5e0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.368+0000 7fc5fde00640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc5e0073230 0x7fc5e0075740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00016b0 tx=0x7fc5f0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:39.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.372+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5f8016970 con 0x7fc6001546b0 2024-09-17T05:04:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.580+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6000ccdb0 con 0x7fc6001546b0 2024-09-17T05:04:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.580+0000 7fc5f6000640 1 -- 172.21.15.16:0/4274494436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5225 (secure 0 0 0) 0x7fc5f80f8a10 con 0x7fc6001546b0 2024-09-17T05:04:39.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:39.584 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":44,"btime":"2024-09-17T05:04:20:444289+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:16.846538+0000","modified":"2024-09-17T05:04:19.864243+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5926":{"gid":5926,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/556849924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":556849924},{"type":"v1","addr":"172.21.15.104:6837","nonce":556849924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5926,"qdb_cluster":[5926]},"id":5}]} 2024-09-17T05:04:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 44 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc5e0073230 msgr2=0x7fc5e0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc5e0073230 0x7fc5e0075740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00016b0 tx=0x7fc5f0009040 comp rx=0 tx=0).stop 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 msgr2=0x7fc6001582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 0x7fc6001582e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009bd0 tx=0x7fc5f8005b00 comp rx=0 tx=0).stop 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 shutdown_connections 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc5e0073230 0x7fc5e0075740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6001546b0 0x7fc6001582e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6000d1aa0 0x7fc600157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 --2- 172.21.15.16:0/4274494436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6000714a0 0x7fc600152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 >> 172.21.15.16:0/4274494436 conn(0x7fc6000b4160 msgr2=0x7fc6000b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 shutdown_connections 2024-09-17T05:04:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.584+0000 7fc5ff200640 1 -- 172.21.15.16:0/4274494436 wait complete. 2024-09-17T05:04:39.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 -- 172.21.15.16:0/1168153035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 msgr2=0x7ff07006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 --2- 172.21.15.16:0/1168153035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07006dfa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff07006eb30 tx=0x7ff05c01c830 comp rx=0 tx=0).stop 2024-09-17T05:04:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 -- 172.21.15.16:0/1168153035 shutdown_connections 2024-09-17T05:04:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 --2- 172.21.15.16:0/1168153035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07006dfa0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 --2- 172.21.15.16:0/1168153035 >> v2:172.21.15.104:3300/0 conn(0x7ff070152300 0x7ff0700b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 -- 172.21.15.16:0/1168153035 >> 172.21.15.16:0/1168153035 conn(0x7ff07006f9c0 msgr2=0x7ff0700b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 -- 172.21.15.16:0/1168153035 shutdown_connections 2024-09-17T05:04:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 -- 172.21.15.16:0/1168153035 wait complete. 2024-09-17T05:04:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.764+0000 7ff076a00640 1 Processor -- start 2024-09-17T05:04:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 -- start start 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0700b5050 0x7ff070145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff070152300 0x7ff07014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff070168d20 con 0x7ff070152710 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff070168ba0 con 0x7ff070152300 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff070168ea0 con 0x7ff0700b5050 2024-09-17T05:04:39.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff075600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0700b5050 0x7ff070145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07014b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39430/0 (socket says 172.21.15.16:39430) 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 -- 172.21.15.16:0/1350070581 learned_addr learned my addr 172.21.15.16:0/1350070581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff074c00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff070152300 0x7ff07014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 -- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0700b5050 msgr2=0x7ff070145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0700b5050 0x7ff070145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 -- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff070152300 msgr2=0x7ff07014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff070152300 0x7ff07014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 -- 172.21.15.16:0/1350070581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07015d340 con 0x7ff070152710 2024-09-17T05:04:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076000640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07014b800 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff06800ba10 tx=0x7ff06800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068019070 con 0x7ff070152710 2024-09-17T05:04:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff06800cc30 con 0x7ff070152710 2024-09-17T05:04:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06800ce60 con 0x7ff070152710 2024-09-17T05:04:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07015d620 con 0x7ff070152710 2024-09-17T05:04:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.768+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff070153930 con 0x7ff070152710 2024-09-17T05:04:39.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.772+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0700b4b10 con 0x7ff070152710 2024-09-17T05:04:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.772+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff06801c070 con 0x7ff070152710 2024-09-17T05:04:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.772+0000 7ff06d600640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff050073230 0x7ff050075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.772+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7ff068093e90 con 0x7ff070152710 2024-09-17T05:04:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.772+0000 7ff075600640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff050073230 0x7ff050075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:39.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.772+0000 7ff075600640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff050073230 0x7ff050075740 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff06400bb50 tx=0x7ff06400b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.776+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff06805e750 con 0x7ff070152710 2024-09-17T05:04:39.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.952+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff070153b60 con 0x7ff070152710 2024-09-17T05:04:39.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff06d600640 1 -- 172.21.15.16:0/1350070581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16933 (secure 0 0 0) 0x7ff068063180 con 0x7ff070152710 2024-09-17T05:04:39.958 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:39.958 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:18.428662+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:14.578846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:15.811287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff050073230 msgr2=0x7ff050075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff050073230 0x7ff050075740 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff06400bb50 tx=0x7ff06400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 msgr2=0x7ff07014b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07014b800 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff06800ba10 tx=0x7ff06800bed0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 shutdown_connections 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff050073230 0x7ff050075740 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff070152710 0x7ff07014b800 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff070152300 0x7ff07014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 --2- 172.21.15.16:0/1350070581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff0700b5050 0x7ff070145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 >> 172.21.15.16:0/1350070581 conn(0x7ff07006f9c0 msgr2=0x7ff0700cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 shutdown_connections 2024-09-17T05:04:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:39.956+0000 7ff076a00640 1 -- 172.21.15.16:0/1350070581 wait complete. 2024-09-17T05:04:39.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:04:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 -- 172.21.15.16:0/3555234821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f762015f6f0 msgr2=0x7f76200ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 --2- 172.21.15.16:0/3555234821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f762015f6f0 0x7f76200ceb10 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7618009c80 tx=0x7f761801cda0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 -- 172.21.15.16:0/3555234821 shutdown_connections 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 --2- 172.21.15.16:0/3555234821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f762015f6f0 0x7f76200ceb10 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 --2- 172.21.15.16:0/3555234821 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 0x7f76200ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 -- 172.21.15.16:0/3555234821 >> 172.21.15.16:0/3555234821 conn(0x7f76200b2250 msgr2=0x7f76200d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 -- 172.21.15.16:0/3555234821 shutdown_connections 2024-09-17T05:04:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.136+0000 7f7625400640 1 -- 172.21.15.16:0/3555234821 wait complete. 2024-09-17T05:04:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 Processor -- start 2024-09-17T05:04:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 -- start start 2024-09-17T05:04:40.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 0x7f76200c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 0x7f762014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762015f6f0 0x7f7620152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7620168f30 con 0x7f7620074a20 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7620168db0 con 0x7f762015f6f0 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f7625400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f76201690b0 con 0x7f7620074e30 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.140+0000 7f761e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 0x7f76200c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 0x7f76200c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39448/0 (socket says 172.21.15.16:39448) 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 -- 172.21.15.16:0/2027674219 learned_addr learned my addr 172.21.15.16:0/2027674219 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7617e00640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 0x7f762014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761f200640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762015f6f0 0x7f7620152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 -- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 msgr2=0x7f762014d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 0x7f762014d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 -- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762015f6f0 msgr2=0x7f7620152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762015f6f0 0x7f7620152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 -- 172.21.15.16:0/2027674219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762014f8d0 con 0x7f7620074a20 2024-09-17T05:04:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761f200640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762015f6f0 0x7f7620152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7617e00640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 0x7f762014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f761e800640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 0x7f76200c7110 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f761000c3c0 tx=0x7f761000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100052d0 con 0x7f7620074a20 2024-09-17T05:04:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7610007020 con 0x7f7620074a20 2024-09-17T05:04:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76201fa530 con 0x7f7620074a20 2024-09-17T05:04:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610007470 con 0x7f7620074a20 2024-09-17T05:04:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76201faaf0 con 0x7f7620074a20 2024-09-17T05:04:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762015efe0 con 0x7f7620074a20 2024-09-17T05:04:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.144+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76100067e0 con 0x7f7620074a20 2024-09-17T05:04:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.148+0000 7f7616a00640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f75fc073200 0x7f75fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.148+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f7610093870 con 0x7f7620074a20 2024-09-17T05:04:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.148+0000 7f7617e00640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f75fc073200 0x7f75fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.148+0000 7f7617e00640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f75fc073200 0x7f75fc075710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f762014dc30 tx=0x7f76080091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:40.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.152+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f761005e700 con 0x7f7620074a20 2024-09-17T05:04:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.360+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7620150800 con 0x7f7620074a20 2024-09-17T05:04:40.365 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:04:40.364+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:04:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.364+0000 7f7616a00640 1 -- 172.21.15.16:0/2027674219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f7610001090 con 0x7f7620074a20 2024-09-17T05:04:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:04:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f75fc073200 msgr2=0x7f75fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f75fc073200 0x7f75fc075710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f762014dc30 tx=0x7f76080091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 msgr2=0x7f76200c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 0x7f76200c7110 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f761000c3c0 tx=0x7f761000c880 comp rx=0 tx=0).stop 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 shutdown_connections 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f75fc073200 0x7f75fc075710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762015f6f0 0x7f7620152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7620074e30 0x7f762014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 --2- 172.21.15.16:0/2027674219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7620074a20 0x7f76200c7110 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 >> 172.21.15.16:0/2027674219 conn(0x7f76200b2250 msgr2=0x7f76201537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 shutdown_connections 2024-09-17T05:04:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.368+0000 7f7625400640 1 -- 172.21.15.16:0/2027674219 wait complete. 2024-09-17T05:04:40.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:04:40.514 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:04:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 -- 172.21.15.16:0/941098871 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 msgr2=0x7fa1f815c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/941098871 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f815c490 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009840 tx=0x7fa1ec01c820 comp rx=0 tx=0).stop 2024-09-17T05:04:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 -- 172.21.15.16:0/941098871 shutdown_connections 2024-09-17T05:04:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/941098871 >> v2:172.21.15.104:3300/0 conn(0x7fa1f80744d0 0x7fa1f806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/941098871 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f815c490 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 -- 172.21.15.16:0/941098871 >> 172.21.15.16:0/941098871 conn(0x7fa1f80b2dd0 msgr2=0x7fa1f80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 -- 172.21.15.16:0/941098871 shutdown_connections 2024-09-17T05:04:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.552+0000 7fa1ffe00640 1 -- 172.21.15.16:0/941098871 wait complete. 2024-09-17T05:04:40.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 Processor -- start 2024-09-17T05:04:40.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 -- start start 2024-09-17T05:04:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f81480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80c6040 0x7fa1f81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1f8150910 0x7fa1f80c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f816a440 con 0x7fa1f80c6040 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f816a2c0 con 0x7fa1f8150910 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1ffe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f816a5c0 con 0x7fa1f806f5f0 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1fd400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f81480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1fca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80c6040 0x7fa1f81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1fde00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1f8150910 0x7fa1f80c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1fd400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f81480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:55718/0 (socket says 172.21.15.16:55718) 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.556+0000 7fa1fd400640 1 -- 172.21.15.16:0/4231713669 learned_addr learned my addr 172.21.15.16:0/4231713669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1fde00640 1 -- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 msgr2=0x7fa1f81480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1fde00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f81480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1fde00640 1 -- 172.21.15.16:0/4231713669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80c6040 msgr2=0x7fa1f81503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1fde00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80c6040 0x7fa1f81503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1fde00640 1 -- 172.21.15.16:0/4231713669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f81f6e00 con 0x7fa1f8150910 2024-09-17T05:04:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1fde00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1f8150910 0x7fa1f80c98c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e9c0 tx=0x7fa1f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000cde0 con 0x7fa1f8150910 2024-09-17T05:04:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1f0004c20 con 0x7fa1f8150910 2024-09-17T05:04:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0005020 con 0x7fa1f8150910 2024-09-17T05:04:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f81f70e0 con 0x7fa1f8150910 2024-09-17T05:04:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f81584c0 con 0x7fa1f8150910 2024-09-17T05:04:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f815c490 con 0x7fa1f8150910 2024-09-17T05:04:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.560+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa1f00051c0 con 0x7fa1f8150910 2024-09-17T05:04:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.564+0000 7fa1f5600640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1d8073200 0x7fa1d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.564+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fa1f00956c0 con 0x7fa1f8150910 2024-09-17T05:04:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.564+0000 7fa1fd400640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1d8073200 0x7fa1d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.564+0000 7fa1fd400640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1d8073200 0x7fa1d8075710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0b0040 tx=0x7fa1ec01b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.568+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa1f005fe90 con 0x7fa1f8150910 2024-09-17T05:04:40.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:40.776+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa1f8158760 con 0x7fa1f8150910 2024-09-17T05:04:41.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.368+0000 7fa1f5600640 1 -- 172.21.15.16:0/4231713669 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) ==== 104+0+0 (secure 0 0 0) 0x7fa1f00648c0 con 0x7fa1f8150910 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1d8073200 msgr2=0x7fa1d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1d8073200 0x7fa1d8075710 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0b0040 tx=0x7fa1ec01b060 comp rx=0 tx=0).stop 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1f8150910 msgr2=0x7fa1f80c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1f8150910 0x7fa1f80c98c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e9c0 tx=0x7fa1f000ee80 comp rx=0 tx=0).stop 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 shutdown_connections 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa1d8073200 0x7fa1d8075710 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1f8150910 0x7fa1f80c98c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1f80c6040 0x7fa1f81503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 --2- 172.21.15.16:0/4231713669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa1f806f5f0 0x7fa1f81480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 >> 172.21.15.16:0/4231713669 conn(0x7fa1f80b2dd0 msgr2=0x7fa1f8155910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 shutdown_connections 2024-09-17T05:04:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.372+0000 7fa1ffe00640 1 -- 172.21.15.16:0/4231713669 wait complete. 2024-09-17T05:04:41.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 -- 172.21.15.16:0/1975054521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d80b4620 msgr2=0x7ff6d806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/1975054521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d80b4620 0x7ff6d806dfa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8072ce0 tx=0x7ff6c801c890 comp rx=0 tx=0).stop 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 -- 172.21.15.16:0/1975054521 shutdown_connections 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/1975054521 >> v2:172.21.15.104:3300/0 conn(0x7ff6d8074bf0 0x7ff6d815f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/1975054521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d80b4620 0x7ff6d806dfa0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 -- 172.21.15.16:0/1975054521 >> 172.21.15.16:0/1975054521 conn(0x7ff6d80d0050 msgr2=0x7ff6d80d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 -- 172.21.15.16:0/1975054521 shutdown_connections 2024-09-17T05:04:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 -- 172.21.15.16:0/1975054521 wait complete. 2024-09-17T05:04:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 Processor -- start 2024-09-17T05:04:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.564+0000 7ff6ddc00640 1 -- start start 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 0x7ff6d8156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d8074bf0 0x7ff6d81f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6d80b4620 0x7ff6d813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d8152840 con 0x7ff6d806e4e0 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d815fe00 con 0x7ff6d8074bf0 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d8168920 con 0x7ff6d80b4620 2024-09-17T05:04:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 0x7ff6d8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 0x7ff6d8156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39476/0 (socket says 172.21.15.16:39476) 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 -- 172.21.15.16:0/2918177283 learned_addr learned my addr 172.21.15.16:0/2918177283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6cfe00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d8074bf0 0x7ff6d81f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d7400640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6d80b4620 0x7ff6d813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 -- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6d80b4620 msgr2=0x7ff6d813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6d80b4620 0x7ff6d813db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 -- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d8074bf0 msgr2=0x7ff6d81f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d8074bf0 0x7ff6d81f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 -- 172.21.15.16:0/2918177283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d81569d0 con 0x7ff6d806e4e0 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d7400640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6d80b4620 0x7ff6d813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6d6a00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 0x7ff6d8156210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80194a0 tx=0x7ff6c8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80bc070 con 0x7ff6d806e4e0 2024-09-17T05:04:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6c8005320 con 0x7ff6d806e4e0 2024-09-17T05:04:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d81f7870 con 0x7ff6d806e4e0 2024-09-17T05:04:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80b2040 con 0x7ff6d806e4e0 2024-09-17T05:04:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d81f7e30 con 0x7ff6d806e4e0 2024-09-17T05:04:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.568+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d806dfa0 con 0x7ff6d806e4e0 2024-09-17T05:04:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.572+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6c80046d0 con 0x7ff6d806e4e0 2024-09-17T05:04:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.572+0000 7ff6cea00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6b4073200 0x7ff6b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.572+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7ff6c812e1b0 con 0x7ff6d806e4e0 2024-09-17T05:04:41.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.572+0000 7ff6cfe00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6b4073200 0x7ff6b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:41.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.572+0000 7ff6cfe00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6b4073200 0x7ff6b4075710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff6d806f8d0 tx=0x7ff6c0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:41.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.576+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6c80164b0 con 0x7ff6d806e4e0 2024-09-17T05:04:41.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:41.752+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6d8070990 con 0x7ff6d806e4e0 2024-09-17T05:04:42.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.648+0000 7ff6cea00640 1 -- 172.21.15.16:0/2918177283 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) ==== 181+0+0 (secure 0 0 0) 0x7ff6c80f8f50 con 0x7ff6d806e4e0 2024-09-17T05:04:42.654 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:04:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6b4073200 msgr2=0x7ff6b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6b4073200 0x7ff6b4075710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff6d806f8d0 tx=0x7ff6c0009040 comp rx=0 tx=0).stop 2024-09-17T05:04:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 msgr2=0x7ff6d8156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 0x7ff6d8156210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80194a0 tx=0x7ff6c8004600 comp rx=0 tx=0).stop 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 shutdown_connections 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6b4073200 0x7ff6b4075710 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6d80b4620 0x7ff6d813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6d8074bf0 0x7ff6d81f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 --2- 172.21.15.16:0/2918177283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6d806e4e0 0x7ff6d8156210 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 >> 172.21.15.16:0/2918177283 conn(0x7ff6d80d0050 msgr2=0x7ff6d80b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 shutdown_connections 2024-09-17T05:04:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.656+0000 7ff6ddc00640 1 -- 172.21.15.16:0/2918177283 wait complete. 2024-09-17T05:04:42.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:04:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 -- 172.21.15.16:0/616720968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b640742f0 msgr2=0x7f6b64074700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 --2- 172.21.15.16:0/616720968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b640742f0 0x7f6b64074700 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009bc0 tx=0x7f6b5c01cb70 comp rx=0 tx=0).stop 2024-09-17T05:04:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 -- 172.21.15.16:0/616720968 shutdown_connections 2024-09-17T05:04:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 --2- 172.21.15.16:0/616720968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b640742f0 0x7f6b64074700 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 -- 172.21.15.16:0/616720968 >> 172.21.15.16:0/616720968 conn(0x7f6b640d0050 msgr2=0x7f6b640d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 -- 172.21.15.16:0/616720968 shutdown_connections 2024-09-17T05:04:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 -- 172.21.15.16:0/616720968 wait complete. 2024-09-17T05:04:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 Processor -- start 2024-09-17T05:04:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.848+0000 7f6b6c600640 1 -- start start 2024-09-17T05:04:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b640742f0 0x7f6b64147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b640b4620 0x7f6b64148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 0x7f6b64149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64169d70 con 0x7f6b641f9740 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6b64169bf0 con 0x7f6b640b4620 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6b64169ef0 con 0x7f6b640742f0 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 0x7f6b64149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 0x7f6b64149490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39492/0 (socket says 172.21.15.16:39492) 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 -- 172.21.15.16:0/850388372 learned_addr learned my addr 172.21.15.16:0/850388372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b69200640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b640b4620 0x7f6b64148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b69c00640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b640742f0 0x7f6b64147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 -- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b640742f0 msgr2=0x7f6b64147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b640742f0 0x7f6b64147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 -- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b640b4620 msgr2=0x7f6b64148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b640b4620 0x7f6b64148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 -- 172.21.15.16:0/850388372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6414fd50 con 0x7f6b641f9740 2024-09-17T05:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b69c00640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b640742f0 0x7f6b64147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6a600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 0x7f6b64149490 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c019780 tx=0x7f6b5c0a5ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0bb070 con 0x7f6b641f9740 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b5c004d40 con 0x7f6b641f9740 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c0b2040 con 0x7f6b641f9740 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b641f9e90 con 0x7f6b641f9740 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b641fa450 con 0x7f6b641f9740 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.852+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b641fa690 con 0x7f6b641f9740 2024-09-17T05:04:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.856+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b5c0be070 con 0x7f6b641f9740 2024-09-17T05:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.856+0000 7f6b61600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b44073200 0x7f6b44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.856+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5749+0+0 (secure 0 0 0) 0x7f6b5c12dfd0 con 0x7f6b641f9740 2024-09-17T05:04:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.856+0000 7f6b69c00640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b44073200 0x7f6b44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.856+0000 7f6b69c00640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b44073200 0x7f6b44075710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6b58000f70 tx=0x7f6b58009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:42.860+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b641fa690 con 0x7f6b641f9740 2024-09-17T05:04:43.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.032+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6b64074700 con 0x7f6b641f9740 2024-09-17T05:04:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.656+0000 7f6b61600640 1 -- 172.21.15.16:0/850388372 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) ==== 169+0+0 (secure 0 0 0) 0x7f6b5c016790 con 0x7f6b641f9740 2024-09-17T05:04:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:04:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.660+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b44073200 msgr2=0x7f6b44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.660+0000 7f6b6c600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b44073200 0x7f6b44075710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6b58000f70 tx=0x7f6b58009040 comp rx=0 tx=0).stop 2024-09-17T05:04:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.660+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 msgr2=0x7f6b64149490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.660+0000 7f6b6c600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 0x7f6b64149490 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c019780 tx=0x7f6b5c0a5ba0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 shutdown_connections 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b44073200 0x7f6b44075710 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b641f9740 0x7f6b64149490 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b640b4620 0x7f6b64148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 --2- 172.21.15.16:0/850388372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b640742f0 0x7f6b64147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 >> 172.21.15.16:0/850388372 conn(0x7f6b640d0050 msgr2=0x7f6b6406fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 shutdown_connections 2024-09-17T05:04:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.664+0000 7f6b6c600640 1 -- 172.21.15.16:0/850388372 wait complete. 2024-09-17T05:04:43.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:04:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 -- 172.21.15.16:0/2885140628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa0074a20 msgr2=0x7f7fa015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/2885140628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa0074a20 0x7f7fa015efe0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00cca90 tx=0x7f7f9001c800 comp rx=0 tx=0).stop 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 -- 172.21.15.16:0/2885140628 shutdown_connections 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/2885140628 >> v2:172.21.15.104:3300/0 conn(0x7f7fa0074e30 0x7f7fa015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/2885140628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa0074a20 0x7f7fa015efe0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 -- 172.21.15.16:0/2885140628 >> 172.21.15.16:0/2885140628 conn(0x7f7fa00b2250 msgr2=0x7f7fa00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.856+0000 7f7fa6c00640 1 -- 172.21.15.16:0/2885140628 shutdown_connections 2024-09-17T05:04:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 -- 172.21.15.16:0/2885140628 wait complete. 2024-09-17T05:04:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 Processor -- start 2024-09-17T05:04:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 -- start start 2024-09-17T05:04:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fa0074a20 0x7f7fa0150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fa0074e30 0x7f7fa00c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 0x7f7fa00c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:43.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0168e20 con 0x7f7fa015ba50 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0168ca0 con 0x7f7fa0074e30 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa6c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa0168fa0 con 0x7f7fa0074a20 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.860+0000 7f7fa4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 0x7f7fa00c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 0x7f7fa00c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39514/0 (socket says 172.21.15.16:39514) 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 -- 172.21.15.16:0/1520923834 learned_addr learned my addr 172.21.15.16:0/1520923834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fa0074a20 0x7f7fa0150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7f97e00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fa0074e30 0x7f7fa00c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 -- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fa0074a20 msgr2=0x7f7fa0150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fa0074a20 0x7f7fa0150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 -- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fa0074e30 msgr2=0x7f7fa00c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fa0074e30 0x7f7fa00c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 -- 172.21.15.16:0/1520923834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa00b4120 con 0x7f7fa015ba50 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa4c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 0x7f7fa00c91a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800ca80 tx=0x7f7f9800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98016020 con 0x7f7fa015ba50 2024-09-17T05:04:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f9800a950 con 0x7f7fa015ba50 2024-09-17T05:04:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa00b4400 con 0x7f7fa015ba50 2024-09-17T05:04:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800ade0 con 0x7f7fa015ba50 2024-09-17T05:04:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa00b49c0 con 0x7f7fa015ba50 2024-09-17T05:04:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa00b3e90 con 0x7f7fa015ba50 2024-09-17T05:04:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.864+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7f98019020 con 0x7f7fa015ba50 2024-09-17T05:04:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.868+0000 7f7f9cc00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f7c073230 0x7f7f7c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.868+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f7f98094220 con 0x7f7fa015ba50 2024-09-17T05:04:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.868+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f7c073230 0x7f7f7c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.868+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f7c073230 0x7f7f7c075740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f900af000 tx=0x7f7f9001b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:43.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:43.872+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f98020020 con 0x7f7fa015ba50 2024-09-17T05:04:44.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.044+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f7fa015be60 con 0x7f7fa015ba50 2024-09-17T05:04:44.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.048+0000 7f7f9cc00640 1 -- 172.21.15.16:0/1520923834 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v42) ==== 71+0+0 (secure 0 0 0) 0x7f7f9805ecc0 con 0x7f7fa015ba50 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f7c073230 msgr2=0x7f7f7c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f7c073230 0x7f7f7c075740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f900af000 tx=0x7f7f9001b040 comp rx=0 tx=0).stop 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 msgr2=0x7f7fa00c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 0x7f7fa00c91a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800ca80 tx=0x7f7f9800cb70 comp rx=0 tx=0).stop 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 shutdown_connections 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f7c073230 0x7f7f7c075740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa015ba50 0x7f7fa00c91a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fa0074e30 0x7f7fa00c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 --2- 172.21.15.16:0/1520923834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7fa0074a20 0x7f7fa0150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 >> 172.21.15.16:0/1520923834 conn(0x7f7fa00b2250 msgr2=0x7f7fa00cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 shutdown_connections 2024-09-17T05:04:44.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.052+0000 7f7fa6c00640 1 -- 172.21.15.16:0/1520923834 wait complete. 2024-09-17T05:04:44.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 -- 172.21.15.16:0/4160550183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b7815ce40 msgr2=0x7f0b7815d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 --2- 172.21.15.16:0/4160550183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b7815ce40 0x7f0b7815d250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009bc0 tx=0x7f0b7001cb10 comp rx=0 tx=0).stop 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 -- 172.21.15.16:0/4160550183 shutdown_connections 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 --2- 172.21.15.16:0/4160550183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b7815ce40 0x7f0b7815d250 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 --2- 172.21.15.16:0/4160550183 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b781590d0 0x7f0b780af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 -- 172.21.15.16:0/4160550183 >> 172.21.15.16:0/4160550183 conn(0x7f0b780d0430 msgr2=0x7f0b780d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 -- 172.21.15.16:0/4160550183 shutdown_connections 2024-09-17T05:04:44.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 -- 172.21.15.16:0/4160550183 wait complete. 2024-09-17T05:04:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 Processor -- start 2024-09-17T05:04:44.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.232+0000 7f0b7d000640 1 -- start start 2024-09-17T05:04:44.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 0x7f0b78165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b7815ce40 0x7f0b781f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b7815ef40 0x7f0b781f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b78169090 con 0x7f0b781590d0 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0b78168f10 con 0x7f0b7815ef40 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0b78169210 con 0x7f0b7815ce40 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 0x7f0b78165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 0x7f0b78165990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39544/0 (socket says 172.21.15.16:39544) 2024-09-17T05:04:44.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 -- 172.21.15.16:0/257905361 learned_addr learned my addr 172.21.15.16:0/257905361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b76800640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b7815ef40 0x7f0b781f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 -- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b7815ce40 msgr2=0x7f0b781f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b6fe00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b7815ce40 0x7f0b781f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b7815ce40 0x7f0b781f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 -- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b7815ef40 msgr2=0x7f0b781f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b7815ef40 0x7f0b781f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 -- 172.21.15.16:0/257905361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b78165ed0 con 0x7f0b781590d0 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b76800640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b7815ef40 0x7f0b781f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b6fe00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b7815ce40 0x7f0b781f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b75e00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 0x7f0b78165990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800e7c0 tx=0x7f0b6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800ccc0 con 0x7f0b781590d0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b6800ce80 con 0x7f0b781590d0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68005280 con 0x7f0b781590d0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7815a890 con 0x7f0b781590d0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7815ae50 con 0x7f0b781590d0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.236+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b780af8e0 con 0x7f0b781590d0 2024-09-17T05:04:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.240+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0b68005420 con 0x7f0b781590d0 2024-09-17T05:04:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.240+0000 7f0b6e000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0b54073200 0x7f0b54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.240+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f0b68094b60 con 0x7f0b781590d0 2024-09-17T05:04:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.240+0000 7f0b6fe00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0b54073200 0x7f0b54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.240+0000 7f0b6fe00640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0b54073200 0x7f0b54075710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0b7806e4e0 tx=0x7f0b60009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:44.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.244+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0b6805fc50 con 0x7f0b781590d0 2024-09-17T05:04:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.504+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7f0b780710b0 con 0x7f0b781590d0 2024-09-17T05:04:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.660+0000 7f0b6e000640 1 -- 172.21.15.16:0/257905361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v205) ==== 148+0+0 (secure 0 0 0) 0x7f0b6805f550 con 0x7f0b781590d0 2024-09-17T05:04:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0b54073200 msgr2=0x7f0b54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0b54073200 0x7f0b54075710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0b7806e4e0 tx=0x7f0b60009040 comp rx=0 tx=0).stop 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 msgr2=0x7f0b78165990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 0x7f0b78165990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800e7c0 tx=0x7f0b6800ec80 comp rx=0 tx=0).stop 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 shutdown_connections 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0b54073200 0x7f0b54075710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b7815ef40 0x7f0b781f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0b7815ce40 0x7f0b781f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 --2- 172.21.15.16:0/257905361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b781590d0 0x7f0b78165990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 >> 172.21.15.16:0/257905361 conn(0x7f0b780d0430 msgr2=0x7f0b7806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 shutdown_connections 2024-09-17T05:04:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.664+0000 7f0b7d000640 1 -- 172.21.15.16:0/257905361 wait complete. 2024-09-17T05:04:44.687 INFO:tasks.cephfs_test_runner:That the MDS will not let a client sit above mds_max_caps_per_client caps. ... ok 2024-09-17T05:04:44.688 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) When a client has inodes pinned in its cache (open files), that the MDS 2024-09-17T05:04:44.688 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:04:44.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 -- 172.21.15.16:0/4093929761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d00b2f50 msgr2=0x7f50d015ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 --2- 172.21.15.16:0/4093929761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d00b2f50 0x7f50d015ef80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009840 tx=0x7f50c401c820 comp rx=0 tx=0).stop 2024-09-17T05:04:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 -- 172.21.15.16:0/4093929761 shutdown_connections 2024-09-17T05:04:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 --2- 172.21.15.16:0/4093929761 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d015f4c0 0x7f50d015f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 --2- 172.21.15.16:0/4093929761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d00b2f50 0x7f50d015ef80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 -- 172.21.15.16:0/4093929761 >> 172.21.15.16:0/4093929761 conn(0x7f50d006f900 msgr2=0x7f50d00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 -- 172.21.15.16:0/4093929761 shutdown_connections 2024-09-17T05:04:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.844+0000 7f50d7a00640 1 -- 172.21.15.16:0/4093929761 wait complete. 2024-09-17T05:04:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 Processor -- start 2024-09-17T05:04:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 -- start start 2024-09-17T05:04:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d00b2f50 0x7f50d014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d015b9a0 0x7f50d0154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d015f4c0 0x7f50d0150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0169000 con 0x7f50d015b9a0 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50d0168e80 con 0x7f50d015f4c0 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d7a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f50d0169180 con 0x7f50d00b2f50 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d015b9a0 0x7f50d0154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d015b9a0 0x7f50d0154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39556/0 (socket says 172.21.15.16:39556) 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d00b2f50 0x7f50d014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50cfe00640 1 -- 172.21.15.16:0/1702056523 learned_addr learned my addr 172.21.15.16:0/1702056523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d015f4c0 0x7f50d0150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5000640 1 -- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d015f4c0 msgr2=0x7f50d0150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5000640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d015f4c0 0x7f50d0150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5000640 1 -- 172.21.15.16:0/1702056523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d015b9a0 msgr2=0x7f50d0154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5000640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d015b9a0 0x7f50d0154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.848+0000 7f50d5000640 1 -- 172.21.15.16:0/1702056523 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d01fc8e0 con 0x7f50d00b2f50 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50d5000640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d00b2f50 0x7f50d014fa90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f50c40ac000 tx=0x7f50c4005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c40b2040 con 0x7f50d00b2f50 2024-09-17T05:04:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50c4004330 con 0x7f50d00b2f50 2024-09-17T05:04:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d01fcb80 con 0x7f50d00b2f50 2024-09-17T05:04:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4005280 con 0x7f50d00b2f50 2024-09-17T05:04:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d01fd140 con 0x7f50d00b2f50 2024-09-17T05:04:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d015ef80 con 0x7f50d00b2f50 2024-09-17T05:04:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.852+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50c4005420 con 0x7f50d00b2f50 2024-09-17T05:04:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.856+0000 7f50ccc00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50b0073230 0x7f50b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.856+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f50c412db60 con 0x7f50d00b2f50 2024-09-17T05:04:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.856+0000 7f50cfe00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50b0073230 0x7f50b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:44.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.856+0000 7f50cfe00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50b0073230 0x7f50b0075740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f50d015ecd0 tx=0x7f50bc009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:44.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:44.860+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f50c4016410 con 0x7f50d00b2f50 2024-09-17T05:04:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.036+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50d00b4be0 con 0x7f50d00b2f50 2024-09-17T05:04:45.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.036+0000 7f50ccc00640 1 -- 172.21.15.16:0/1702056523 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+13294 (secure 0 0 0) 0x7f50c40f8600 con 0x7f50d00b2f50 2024-09-17T05:04:45.040 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:45.040 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:43.655867+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6837/556849924":"2024-09-18T05:04:40.367431+0000","172.21.15.104:6836/556849924":"2024-09-18T05:04:40.367431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50b0073230 msgr2=0x7f50b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50b0073230 0x7f50b0075740 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f50d015ecd0 tx=0x7f50bc009260 comp rx=0 tx=0).stop 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d00b2f50 msgr2=0x7f50d014fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d00b2f50 0x7f50d014fa90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f50c40ac000 tx=0x7f50c4005dd0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 shutdown_connections 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50b0073230 0x7f50b0075740 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50d015f4c0 0x7f50d0150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d015b9a0 0x7f50d0154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 --2- 172.21.15.16:0/1702056523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f50d00b2f50 0x7f50d014fa90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 >> 172.21.15.16:0/1702056523 conn(0x7f50d006f900 msgr2=0x7f50d015ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 shutdown_connections 2024-09-17T05:04:45.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.040+0000 7f50d7a00640 1 -- 172.21.15.16:0/1702056523 wait complete. 2024-09-17T05:04:45.062 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:04:45.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:04:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 -- 172.21.15.16:0/1390226699 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4054073170 msgr2=0x7f4054073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 --2- 172.21.15.16:0/1390226699 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4054073170 0x7f4054073580 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4048009900 tx=0x7f404801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 -- 172.21.15.16:0/1390226699 shutdown_connections 2024-09-17T05:04:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 --2- 172.21.15.16:0/1390226699 >> v2:172.21.15.104:3300/0 conn(0x7f40540cbe70 0x7f40540b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 --2- 172.21.15.16:0/1390226699 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4054073170 0x7f4054073580 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 -- 172.21.15.16:0/1390226699 >> 172.21.15.16:0/1390226699 conn(0x7f40540b2250 msgr2=0x7f40540d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 -- 172.21.15.16:0/1390226699 shutdown_connections 2024-09-17T05:04:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.220+0000 7f405c000640 1 -- 172.21.15.16:0/1390226699 wait complete. 2024-09-17T05:04:45.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 Processor -- start 2024-09-17T05:04:45.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 -- start start 2024-09-17T05:04:45.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4054073170 0x7f405414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40540b3230 0x7f405415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 0x7f40540b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4054169da0 con 0x7f40540cbe70 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4054169c20 con 0x7f4054073170 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405c000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4054169f20 con 0x7f40540b3230 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 0x7f40540b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 0x7f40540b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39562/0 (socket says 172.21.15.16:39562) 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f405a000640 1 -- 172.21.15.16:0/523036764 learned_addr learned my addr 172.21.15.16:0/523036764 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f4059600640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4054073170 0x7f405414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.224+0000 7f4058c00640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40540b3230 0x7f405415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405a000640 1 -- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40540b3230 msgr2=0x7f405415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405a000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40540b3230 0x7f405415f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405a000640 1 -- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4054073170 msgr2=0x7f405414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405a000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4054073170 0x7f405414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405a000640 1 -- 172.21.15.16:0/523036764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40541f8ad0 con 0x7f40540cbe70 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405a000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 0x7f40540b5280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f404c00ea80 tx=0x7f404c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c00cea0 con 0x7f40540cbe70 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f404c004b30 con 0x7f40540cbe70 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40540ccb40 con 0x7f40540cbe70 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c005020 con 0x7f40540cbe70 2024-09-17T05:04:45.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40540cd100 con 0x7f40540cbe70 2024-09-17T05:04:45.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4054073580 con 0x7f40540cbe70 2024-09-17T05:04:45.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f404c0051c0 con 0x7f40540cbe70 2024-09-17T05:04:45.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.228+0000 7f4051600640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4034073230 0x7f4034075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.232+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f404c094c50 con 0x7f40540cbe70 2024-09-17T05:04:45.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.232+0000 7f4059600640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4034073230 0x7f4034075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.232+0000 7f4059600640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4034073230 0x7f4034075740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f40480af000 tx=0x7f404801b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.236+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f404c05f700 con 0x7f40540cbe70 2024-09-17T05:04:45.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.408+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f40540b4a80 con 0x7f40540cbe70 2024-09-17T05:04:45.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.408+0000 7f4051600640 1 -- 172.21.15.16:0/523036764 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f404c064130 con 0x7f40540cbe70 2024-09-17T05:04:45.412 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:04:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4034073230 msgr2=0x7f4034075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4034073230 0x7f4034075740 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f40480af000 tx=0x7f404801b0f0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 msgr2=0x7f40540b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 0x7f40540b5280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f404c00ea80 tx=0x7f404c00ef40 comp rx=0 tx=0).stop 2024-09-17T05:04:45.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 shutdown_connections 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4034073230 0x7f4034075740 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40540cbe70 0x7f40540b5280 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40540b3230 0x7f405415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 --2- 172.21.15.16:0/523036764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4054073170 0x7f405414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 >> 172.21.15.16:0/523036764 conn(0x7f40540b2250 msgr2=0x7f40540b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 shutdown_connections 2024-09-17T05:04:45.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.412+0000 7f405c000640 1 -- 172.21.15.16:0/523036764 wait complete. 2024-09-17T05:04:45.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-09-17T05:04:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 -- 172.21.15.16:0/3418446421 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c00b2f50 msgr2=0x7f32c015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 --2- 172.21.15.16:0/3418446421 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c00b2f50 0x7f32c015bb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009840 tx=0x7f32b001c790 comp rx=0 tx=0).stop 2024-09-17T05:04:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 -- 172.21.15.16:0/3418446421 shutdown_connections 2024-09-17T05:04:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 --2- 172.21.15.16:0/3418446421 >> v2:172.21.15.104:3300/0 conn(0x7f32c015c0a0 0x7f32c015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 --2- 172.21.15.16:0/3418446421 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c00b2f50 0x7f32c015bb60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 -- 172.21.15.16:0/3418446421 >> 172.21.15.16:0/3418446421 conn(0x7f32c006f900 msgr2=0x7f32c00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 -- 172.21.15.16:0/3418446421 shutdown_connections 2024-09-17T05:04:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 -- 172.21.15.16:0/3418446421 wait complete. 2024-09-17T05:04:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 Processor -- start 2024-09-17T05:04:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.592+0000 7f32c6800640 1 -- start start 2024-09-17T05:04:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c00b2f50 0x7f32c0144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 0x7f32c01fc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c015f150 0x7f32c0151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0169120 con 0x7f32c015c0a0 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f32c0168fa0 con 0x7f32c00b2f50 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f32c01692a0 con 0x7f32c015f150 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 0x7f32c01fc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 0x7f32c01fc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39566/0 (socket says 172.21.15.16:39566) 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 -- 172.21.15.16:0/1071478057 learned_addr learned my addr 172.21.15.16:0/1071478057 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32bf400640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c00b2f50 0x7f32c0144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32bfe00640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c015f150 0x7f32c0151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 -- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c015f150 msgr2=0x7f32c0151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c015f150 0x7f32c0151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 -- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c00b2f50 msgr2=0x7f32c0144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c00b2f50 0x7f32c0144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 -- 172.21.15.16:0/1071478057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c0144e90 con 0x7f32c015c0a0 2024-09-17T05:04:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7e00640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 0x7f32c01fc710 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f32a800e740 tx=0x7f32a800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:45.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800cc60 con 0x7f32c015c0a0 2024-09-17T05:04:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a800ce00 con 0x7f32c015c0a0 2024-09-17T05:04:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c01fd480 con 0x7f32c015c0a0 2024-09-17T05:04:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8005280 con 0x7f32c015c0a0 2024-09-17T05:04:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c0161140 con 0x7f32c015c0a0 2024-09-17T05:04:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.596+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c01fd1f0 con 0x7f32c015c0a0 2024-09-17T05:04:45.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.600+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f32a8005420 con 0x7f32c015c0a0 2024-09-17T05:04:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.600+0000 7f32b7400640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f329c073230 0x7f329c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:45.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.600+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f32a80948d0 con 0x7f32c015c0a0 2024-09-17T05:04:45.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.600+0000 7f32bf400640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f329c073230 0x7f329c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:45.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.600+0000 7f32bf400640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f329c073230 0x7f329c075740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32b00af000 tx=0x7f32b001afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:45.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.604+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32a805fa60 con 0x7f32c015c0a0 2024-09-17T05:04:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:45.848+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f32c015bb60 con 0x7f32c015c0a0 2024-09-17T05:04:46.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.668+0000 7f32b7400640 1 -- 172.21.15.16:0/1071478057 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v207) ==== 163+0+0 (secure 0 0 0) 0x7f32a805f360 con 0x7f32c015c0a0 2024-09-17T05:04:46.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f329c073230 msgr2=0x7f329c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f329c073230 0x7f329c075740 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32b00af000 tx=0x7f32b001afd0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 msgr2=0x7f32c01fc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 0x7f32c01fc710 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f32a800e740 tx=0x7f32a800ec00 comp rx=0 tx=0).stop 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 shutdown_connections 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f329c073230 0x7f329c075740 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f32c015f150 0x7f32c0151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32c015c0a0 0x7f32c01fc710 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 --2- 172.21.15.16:0/1071478057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32c00b2f50 0x7f32c0144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 >> 172.21.15.16:0/1071478057 conn(0x7f32c006f900 msgr2=0x7f32c015d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 shutdown_connections 2024-09-17T05:04:46.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.672+0000 7f32c6800640 1 -- 172.21.15.16:0/1071478057 wait complete. 2024-09-17T05:04:46.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:04:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 -- 172.21.15.16:0/441190377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680cf560 msgr2=0x7fda680b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 --2- 172.21.15.16:0/441190377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680cf560 0x7fda680b3990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda50009840 tx=0x7fda5001c820 comp rx=0 tx=0).stop 2024-09-17T05:04:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 -- 172.21.15.16:0/441190377 shutdown_connections 2024-09-17T05:04:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 --2- 172.21.15.16:0/441190377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680cf560 0x7fda680b3990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 -- 172.21.15.16:0/441190377 >> 172.21.15.16:0/441190377 conn(0x7fda6806f900 msgr2=0x7fda680b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:46.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 -- 172.21.15.16:0/441190377 shutdown_connections 2024-09-17T05:04:46.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 -- 172.21.15.16:0/441190377 wait complete. 2024-09-17T05:04:46.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 Processor -- start 2024-09-17T05:04:46.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.860+0000 7fda6d400640 1 -- start start 2024-09-17T05:04:46.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 0x7fda6813bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:46.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda680cf560 0x7fda68142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:46.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda6815d600 0x7fda68142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda6816b0c0 con 0x7fda680b3ed0 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda6816af40 con 0x7fda680cf560 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fda6816b240 con 0x7fda6815d600 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 0x7fda6813bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 0x7fda6813bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39584/0 (socket says 172.21.15.16:39584) 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 -- 172.21.15.16:0/872694314 learned_addr learned my addr 172.21.15.16:0/872694314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda5fe00640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda680cf560 0x7fda68142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda67200640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda6815d600 0x7fda68142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 -- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda6815d600 msgr2=0x7fda68142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda6815d600 0x7fda68142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 -- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda680cf560 msgr2=0x7fda68142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda680cf560 0x7fda68142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:46.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 -- 172.21.15.16:0/872694314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6813f140 con 0x7fda680b3ed0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda5fe00640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda680cf560 0x7fda68142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda66800640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 0x7fda6813bd00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c3c0 tx=0x7fda5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58011790 con 0x7fda680b3ed0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda58017070 con 0x7fda680b3ed0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6813f420 con 0x7fda680b3ed0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58011ce0 con 0x7fda680b3ed0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda68141b20 con 0x7fda680b3ed0 2024-09-17T05:04:46.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.864+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda680b3990 con 0x7fda680b3ed0 2024-09-17T05:04:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.868+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda5801c460 con 0x7fda680b3ed0 2024-09-17T05:04:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.868+0000 7fda5ea00640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda44073230 0x7fda44075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.868+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7fda58093f00 con 0x7fda680b3ed0 2024-09-17T05:04:46.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.868+0000 7fda5fe00640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda44073230 0x7fda44075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:46.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.868+0000 7fda5fe00640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda44073230 0x7fda44075740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda500af000 tx=0x7fda5001b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:46.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:46.872+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda5805eff0 con 0x7fda680b3ed0 2024-09-17T05:04:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.044+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fda680b3530 con 0x7fda680b3ed0 2024-09-17T05:04:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.044+0000 7fda5ea00640 1 -- 172.21.15.16:0/872694314 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v42) ==== 59+0+2294 (secure 0 0 0) 0x7fda58001090 con 0x7fda680b3ed0 2024-09-17T05:04:47.049 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:04:47.049 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:04:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:04:47.051 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:04:47.052 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:04:47.052 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:04:47.052 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda44073230 msgr2=0x7fda44075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda44073230 0x7fda44075740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda500af000 tx=0x7fda5001b060 comp rx=0 tx=0).stop 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 msgr2=0x7fda6813bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 0x7fda6813bd00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c3c0 tx=0x7fda5800c880 comp rx=0 tx=0).stop 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 shutdown_connections 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda44073230 0x7fda44075740 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda6815d600 0x7fda68142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda680cf560 0x7fda68142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 --2- 172.21.15.16:0/872694314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda680b3ed0 0x7fda6813bd00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 >> 172.21.15.16:0/872694314 conn(0x7fda6806f900 msgr2=0x7fda680b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 shutdown_connections 2024-09-17T05:04:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.048+0000 7fda6d400640 1 -- 172.21.15.16:0/872694314 wait complete. 2024-09-17T05:04:47.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 -- 172.21.15.16:0/2039403808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 msgr2=0x7f7570160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 --2- 172.21.15.16:0/2039403808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f7570160130 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f75700b61f0 tx=0x7f755c01c870 comp rx=0 tx=0).stop 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 -- 172.21.15.16:0/2039403808 shutdown_connections 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 --2- 172.21.15.16:0/2039403808 >> v2:172.21.15.104:3300/0 conn(0x7f7570160670 0x7f7570160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 --2- 172.21.15.16:0/2039403808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f7570160130 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 --2- 172.21.15.16:0/2039403808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75700733e0 0x7f757015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 -- 172.21.15.16:0/2039403808 >> 172.21.15.16:0/2039403808 conn(0x7f75700b2250 msgr2=0x7f75700d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 -- 172.21.15.16:0/2039403808 shutdown_connections 2024-09-17T05:04:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 -- 172.21.15.16:0/2039403808 wait complete. 2024-09-17T05:04:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 Processor -- start 2024-09-17T05:04:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.232+0000 7f7576600640 1 -- start start 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75700733e0 0x7f75701550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f75701fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7570160670 0x7f75701fb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7570073970 con 0x7f757015d680 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75700737f0 con 0x7f7570160670 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f757006d6a0 con 0x7f75700733e0 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f75701fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f75701fae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60960/0 (socket says 172.21.15.16:60960) 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 -- 172.21.15.16:0/2497097680 learned_addr learned my addr 172.21.15.16:0/2497097680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756f400640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75700733e0 0x7f75701550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756fe00640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7570160670 0x7f75701fb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 -- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75700733e0 msgr2=0x7f75701550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75700733e0 0x7f75701550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 -- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7570160670 msgr2=0x7f75701fb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7570160670 0x7f75701fb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 -- 172.21.15.16:0/2497097680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757015f640 con 0x7f757015d680 2024-09-17T05:04:47.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f756ea00640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f75701fae60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f755c009850 tx=0x7f755c0a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:47.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0b2040 con 0x7f757015d680 2024-09-17T05:04:47.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f755c0043b0 con 0x7f757015d680 2024-09-17T05:04:47.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7570164e00 con 0x7f757015d680 2024-09-17T05:04:47.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c005280 con 0x7f757015d680 2024-09-17T05:04:47.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701faba0 con 0x7f757015d680 2024-09-17T05:04:47.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.236+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757015f3b0 con 0x7f757015d680 2024-09-17T05:04:47.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.240+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f755c005420 con 0x7f757015d680 2024-09-17T05:04:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.240+0000 7f7567400640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7550073230 0x7f7550075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.240+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f755c12e120 con 0x7f757015d680 2024-09-17T05:04:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.240+0000 7f756f400640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7550073230 0x7f7550075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.240+0000 7f756f400640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7550073230 0x7f7550075740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7560009a50 tx=0x7f75600091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:47.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.244+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f755c016490 con 0x7f757015d680 2024-09-17T05:04:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.420+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f757006db60 con 0x7f757015d680 2024-09-17T05:04:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.424+0000 7f7567400640 1 -- 172.21.15.16:0/2497097680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v43)=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7f755c0f92b0 con 0x7f757015d680 2024-09-17T05:04:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7550073230 msgr2=0x7f7550075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7550073230 0x7f7550075740 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7560009a50 tx=0x7f75600091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 msgr2=0x7f75701fae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f75701fae60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f755c009850 tx=0x7f755c0a5770 comp rx=0 tx=0).stop 2024-09-17T05:04:47.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 shutdown_connections 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7550073230 0x7f7550075740 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7570160670 0x7f75701fb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f757015d680 0x7f75701fae60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 --2- 172.21.15.16:0/2497097680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f75700733e0 0x7f75701550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 >> 172.21.15.16:0/2497097680 conn(0x7f75700b2250 msgr2=0x7f757015bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 shutdown_connections 2024-09-17T05:04:47.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.428+0000 7f7576600640 1 -- 172.21.15.16:0/2497097680 wait complete. 2024-09-17T05:04:47.450 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:04:47.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:47.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:04:47.647 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:04:47.647 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:47.648 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:47.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:04:47.658 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:47.658 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:47.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:04:47.668 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:04:47.668 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:47.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:04:47.714 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:04:47.715 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:04:47.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:04:47.724 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:04:47.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 -- 172.21.15.16:0/2889341770 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 msgr2=0x7f7d0415c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 --2- 172.21.15.16:0/2889341770 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 0x7f7d0415c4b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009840 tx=0x7f7cec01c790 comp rx=0 tx=0).stop 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 -- 172.21.15.16:0/2889341770 shutdown_connections 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 --2- 172.21.15.16:0/2889341770 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 0x7f7d0415c4b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 --2- 172.21.15.16:0/2889341770 >> v2:172.21.15.104:3300/0 conn(0x7f7d040b2f50 0x7f7d0415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 -- 172.21.15.16:0/2889341770 >> 172.21.15.16:0/2889341770 conn(0x7f7d0406f900 msgr2=0x7f7d040b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 -- 172.21.15.16:0/2889341770 shutdown_connections 2024-09-17T05:04:47.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.916+0000 7f7d0a400640 1 -- 172.21.15.16:0/2889341770 wait complete. 2024-09-17T05:04:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 Processor -- start 2024-09-17T05:04:47.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 -- start start 2024-09-17T05:04:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d040b2f50 0x7f7d0413e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 0x7f7d041f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 0x7f7d04154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04169120 con 0x7f7d0415f150 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04168fa0 con 0x7f7d040b2f50 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7d041692a0 con 0x7f7d0415c0a0 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 0x7f7d04154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 0x7f7d04154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60966/0 (socket says 172.21.15.16:60966) 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 -- 172.21.15.16:0/3309716715 learned_addr learned my addr 172.21.15.16:0/3309716715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d040b2f50 0x7f7d0413e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7cfbe00640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 0x7f7d041f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 -- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 msgr2=0x7f7d041f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 0x7f7d041f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 -- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d040b2f50 msgr2=0x7f7d0413e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d040b2f50 0x7f7d0413e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 -- 172.21.15.16:0/3309716715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d04141be0 con 0x7f7d0415f150 2024-09-17T05:04:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d03e00640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 0x7f7d04154a50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00cb10 tx=0x7f7cfc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc016020 con 0x7f7d0415f150 2024-09-17T05:04:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cfc00ce50 con 0x7f7d0415f150 2024-09-17T05:04:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d04140e20 con 0x7f7d0415f150 2024-09-17T05:04:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00ade0 con 0x7f7d0415f150 2024-09-17T05:04:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.920+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041413e0 con 0x7f7d0415f150 2024-09-17T05:04:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.924+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d041f9a50 con 0x7f7d0415f150 2024-09-17T05:04:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.924+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7cfc019020 con 0x7f7d0415f150 2024-09-17T05:04:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.924+0000 7f7cfb400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7ce0073230 0x7f7ce0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.924+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7f7cfc093e30 con 0x7f7d0415f150 2024-09-17T05:04:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.924+0000 7f7d03400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7ce0073230 0x7f7ce0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.924+0000 7f7d03400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7ce0073230 0x7f7ce0075740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009600 tx=0x7f7cf40091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:47.928+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7cfc020030 con 0x7f7d0415f150 2024-09-17T05:04:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.140+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d0415bb60 con 0x7f7d0415f150 2024-09-17T05:04:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.140+0000 7f7cfb400640 1 -- 172.21.15.16:0/3309716715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+3914 (secure 0 0 0) 0x7f7cfc05e960 con 0x7f7d0415f150 2024-09-17T05:04:48.143 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:48.144 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"btime":"2024-09-17T05:04:41:369802+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2360191379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2360191379},{"type":"v1","addr":"172.21.15.16:6834","nonce":2360191379}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5477,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/149880143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":149880143},{"type":"v1","addr":"172.21.15.16:6835","nonce":149880143}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1673625337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1673625337},{"type":"v1","addr":"172.21.15.104:6837","nonce":1673625337}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":5917,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1245862064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1245862064},{"type":"v1","addr":"172.21.15.104:6835","nonce":1245862064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-09-17T05:04:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 46 2024-09-17T05:04:48.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.140+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7ce0073230 msgr2=0x7f7ce0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.140+0000 7f7d0a400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7ce0073230 0x7f7ce0075740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009600 tx=0x7f7cf40091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.140+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 msgr2=0x7f7d04154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.140+0000 7f7d0a400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 0x7f7d04154a50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00cb10 tx=0x7f7cfc00cc00 comp rx=0 tx=0).stop 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 shutdown_connections 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7ce0073230 0x7f7ce0075740 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d0415f150 0x7f7d04154a50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d0415c0a0 0x7f7d041f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 --2- 172.21.15.16:0/3309716715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d040b2f50 0x7f7d0413e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 >> 172.21.15.16:0/3309716715 conn(0x7f7d0406f900 msgr2=0x7f7d040d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 shutdown_connections 2024-09-17T05:04:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.144+0000 7f7d0a400640 1 -- 172.21.15.16:0/3309716715 wait complete. 2024-09-17T05:04:48.166 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:04:48.166 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:04:48.166 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:04:48.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:48.167 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:04:48.167 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:04:48.167 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:04:48.167 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:48.168 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:04:48.168 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:04:48.168 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:04:48.168 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:48.168 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:04:48.168 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:04:48.168 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:04:48.168 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:04:48.181 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:04:48.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:04:48.183 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:04:48.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:04:48.184 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:04:48.184 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:04:48.192 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:04:48.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:04:48.227 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:04:48.227 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:48.230 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:04:48.230 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:48.234 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:04:48.235 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:48.237 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:04:48.237 DEBUG:teuthology.parallel:result is None 2024-09-17T05:04:48.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:04:48.329 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:04:48.330 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:04:48.335 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:04:48.338 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:04:48.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T05:04:48.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T05:04:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 -- 172.21.15.16:0/2936438875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94074e30 msgr2=0x7efd940ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 --2- 172.21.15.16:0/2936438875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94074e30 0x7efd940ce5d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efd94156030 tx=0x7efd7c01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 -- 172.21.15.16:0/2936438875 shutdown_connections 2024-09-17T05:04:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 --2- 172.21.15.16:0/2936438875 >> v2:172.21.15.104:3300/0 conn(0x7efd9415f6f0 0x7efd940ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 --2- 172.21.15.16:0/2936438875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94074e30 0x7efd940ce5d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 -- 172.21.15.16:0/2936438875 >> 172.21.15.16:0/2936438875 conn(0x7efd940b2250 msgr2=0x7efd940d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 -- 172.21.15.16:0/2936438875 shutdown_connections 2024-09-17T05:04:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.400+0000 7efd9b200640 1 -- 172.21.15.16:0/2936438875 wait complete. 2024-09-17T05:04:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 Processor -- start 2024-09-17T05:04:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 -- start start 2024-09-17T05:04:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd94074a20 0x7efd941fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 0x7efd941542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 0x7efd940b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd94168e20 con 0x7efd9415f6f0 2024-09-17T05:04:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efd94168ca0 con 0x7efd94074a20 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd9b200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efd94168fa0 con 0x7efd94074e30 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd99200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 0x7efd940b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 0x7efd941542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd93e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd94074a20 0x7efd941fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd99200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 0x7efd940b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32792/0 (socket says 172.21.15.16:32792) 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd99200640 1 -- 172.21.15.16:0/2540756710 learned_addr learned my addr 172.21.15.16:0/2540756710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 0x7efd941542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:36534/0 (socket says 172.21.15.16:36534) 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 -- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd94074a20 msgr2=0x7efd941fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd94074a20 0x7efd941fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 -- 172.21.15.16:0/2540756710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 msgr2=0x7efd940b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 0x7efd940b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd8be00640 1 -- 172.21.15.16:0/2540756710 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd94154820 con 0x7efd94074e30 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.404+0000 7efd99200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 0x7efd940b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd8be00640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 0x7efd941542e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0ac000 tx=0x7efd7c005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0b2040 con 0x7efd94074e30 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd7c004350 con 0x7efd94074e30 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd940b4a70 con 0x7efd94074e30 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c003860 con 0x7efd94074e30 2024-09-17T05:04:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd940ada10 con 0x7efd94074e30 2024-09-17T05:04:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9415efe0 con 0x7efd94074e30 2024-09-17T05:04:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd7c003a00 con 0x7efd94074e30 2024-09-17T05:04:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd90c00640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd70073230 0x7efd70075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.408+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5335+0+0 (secure 0 0 0) 0x7efd7c12df50 con 0x7efd94074e30 2024-09-17T05:04:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.412+0000 7efd93e00640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd70073230 0x7efd70075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:48.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.412+0000 7efd93e00640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd70073230 0x7efd70075740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efd84007fd0 tx=0x7efd84009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.416+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd7c0164b0 con 0x7efd94074e30 2024-09-17T05:04:48.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:48.604+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7efd94069770 con 0x7efd94074e30 2024-09-17T05:04:49.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.440+0000 7efd90c00640 1 -- 172.21.15.16:0/2540756710 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) ==== 97+0+0 (secure 0 0 0) 0x7efd7c0f8a80 con 0x7efd94074e30 2024-09-17T05:04:49.445 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:04:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd70073230 msgr2=0x7efd70075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd70073230 0x7efd70075740 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efd84007fd0 tx=0x7efd84009040 comp rx=0 tx=0).stop 2024-09-17T05:04:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 msgr2=0x7efd941542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 0x7efd941542e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0ac000 tx=0x7efd7c005e70 comp rx=0 tx=0).stop 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 shutdown_connections 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd70073230 0x7efd70075740 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd9415f6f0 0x7efd940b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd94074e30 0x7efd941542e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 --2- 172.21.15.16:0/2540756710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd94074a20 0x7efd941fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 >> 172.21.15.16:0/2540756710 conn(0x7efd940b2250 msgr2=0x7efd940ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 shutdown_connections 2024-09-17T05:04:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.444+0000 7efd9b200640 1 -- 172.21.15.16:0/2540756710 wait complete. 2024-09-17T05:04:49.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 -- 172.21.15.16:0/3346124955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 msgr2=0x7f595c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 --2- 172.21.15.16:0/3346124955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c073650 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5954009c80 tx=0x7f595401cd10 comp rx=0 tx=0).stop 2024-09-17T05:04:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 -- 172.21.15.16:0/3346124955 shutdown_connections 2024-09-17T05:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 --2- 172.21.15.16:0/3346124955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c073650 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 --2- 172.21.15.16:0/3346124955 >> v2:172.21.15.104:3300/0 conn(0x7f595c0b2f50 0x7f595c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 -- 172.21.15.16:0/3346124955 >> 172.21.15.16:0/3346124955 conn(0x7f595c06f900 msgr2=0x7f595c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 -- 172.21.15.16:0/3346124955 shutdown_connections 2024-09-17T05:04:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.632+0000 7f5963c00640 1 -- 172.21.15.16:0/3346124955 wait complete. 2024-09-17T05:04:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 Processor -- start 2024-09-17T05:04:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 -- start start 2024-09-17T05:04:49.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:49.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 0x7f595c13c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:49.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c15c0a0 0x7f595c0d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f595c168d30 con 0x7f595c0b2f50 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f595c168bb0 con 0x7f595c15c0a0 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5963c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f595c168eb0 con 0x7f595c073240 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 0x7f595c13c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 0x7f595c13c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32802/0 (socket says 172.21.15.16:32802) 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 -- 172.21.15.16:0/1466244892 learned_addr learned my addr 172.21.15.16:0/1466244892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5961200640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5961c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c15c0a0 0x7f595c0d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 -- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 msgr2=0x7f595c0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 -- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c15c0a0 msgr2=0x7f595c0d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.636+0000 7f5953e00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c15c0a0 0x7f595c0d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5953e00640 1 -- 172.21.15.16:0/1466244892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c13f830 con 0x7f595c0b2f50 2024-09-17T05:04:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5961200640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5961c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c15c0a0 0x7f595c0d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5953e00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 0x7f595c13c4a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f594400c3c0 tx=0x7f594400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:49.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944017070 con 0x7f595c0b2f50 2024-09-17T05:04:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59440053c0 con 0x7f595c0b2f50 2024-09-17T05:04:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5944003860 con 0x7f595c0b2f50 2024-09-17T05:04:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c0cf770 con 0x7f595c0b2f50 2024-09-17T05:04:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c0cfd30 con 0x7f595c0b2f50 2024-09-17T05:04:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c073650 con 0x7f595c0b2f50 2024-09-17T05:04:49.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f594401a070 con 0x7f595c0b2f50 2024-09-17T05:04:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5959600640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5938073200 0x7f5938075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.640+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f5944093310 con 0x7f595c0b2f50 2024-09-17T05:04:49.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.644+0000 7f5961200640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5938073200 0x7f5938075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:49.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.644+0000 7f5961200640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5938073200 0x7f5938075710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f594c009f30 tx=0x7f594c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:49.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.644+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f594405dea0 con 0x7f595c0b2f50 2024-09-17T05:04:49.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.880+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f595c0b1d50 con 0x7f595c0b2f50 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.880+0000 7f5959600640 1 -- 172.21.15.16:0/1466244892 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7f59440930d0 con 0x7f595c0b2f50 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:04:49.884 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:49.885 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.886 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:04:49.887 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.888 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:04:49.889 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.890 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:04:49.891 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.892 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:04:49.893 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:04:49.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:04:49.896 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:04:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:49.898 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:04:49.898 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.880+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5938073200 msgr2=0x7f5938075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.880+0000 7f5963c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5938073200 0x7f5938075710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f594c009f30 tx=0x7f594c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 msgr2=0x7f595c13c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 0x7f595c13c4a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f594400c3c0 tx=0x7f594400c880 comp rx=0 tx=0).stop 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 shutdown_connections 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5938073200 0x7f5938075710 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595c15c0a0 0x7f595c0d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f595c0b2f50 0x7f595c13c4a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 --2- 172.21.15.16:0/1466244892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f595c073240 0x7f595c0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 >> 172.21.15.16:0/1466244892 conn(0x7f595c06f900 msgr2=0x7f595c160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:49.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 shutdown_connections 2024-09-17T05:04:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:49.884+0000 7f5963c00640 1 -- 172.21.15.16:0/1466244892 wait complete. 2024-09-17T05:04:49.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.060+0000 7f2167000640 1 -- 172.21.15.16:0/3553524421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216006f6c0 msgr2=0x7f2160071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.060+0000 7f2167000640 1 --2- 172.21.15.16:0/3553524421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216006f6c0 0x7f2160071790 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f214c009840 tx=0x7f214c01c820 comp rx=0 tx=0).stop 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 -- 172.21.15.16:0/3553524421 shutdown_connections 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 --2- 172.21.15.16:0/3553524421 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f21600cf1e0 0x7f21600cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 --2- 172.21.15.16:0/3553524421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216006f6c0 0x7f2160071790 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 -- 172.21.15.16:0/3553524421 >> 172.21.15.16:0/3553524421 conn(0x7f2160073ca0 msgr2=0x7f21600740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 -- 172.21.15.16:0/3553524421 shutdown_connections 2024-09-17T05:04:50.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 -- 172.21.15.16:0/3553524421 wait complete. 2024-09-17T05:04:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 Processor -- start 2024-09-17T05:04:50.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 -- start start 2024-09-17T05:04:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2160071380 0x7f216015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.064+0000 7f2167000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 0x7f216015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f216014a0c0 0x7f216014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2160168920 con 0x7f21600cf1e0 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f21601687a0 con 0x7f2160071380 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2160168aa0 con 0x7f216014a0c0 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 0x7f216015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 0x7f216015dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32816/0 (socket says 172.21.15.16:32816) 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 -- 172.21.15.16:0/3697492719 learned_addr learned my addr 172.21.15.16:0/3697492719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2166600640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f216014a0c0 0x7f216014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165c00640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2160071380 0x7f216015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 -- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f216014a0c0 msgr2=0x7f216014e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f216014a0c0 0x7f216014e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 -- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2160071380 msgr2=0x7f216015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2160071380 0x7f216015d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 -- 172.21.15.16:0/3697492719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2160153a50 con 0x7f21600cf1e0 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2166600640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f216014a0c0 0x7f216014e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2165200640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 0x7f216015dd40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f214c009790 tx=0x7f214c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b2040 con 0x7f21600cf1e0 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f214c004330 con 0x7f21600cf1e0 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216015e460 con 0x7f21600cf1e0 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c003800 con 0x7f21600cf1e0 2024-09-17T05:04:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2160152ec0 con 0x7f21600cf1e0 2024-09-17T05:04:50.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2160071790 con 0x7f21600cf1e0 2024-09-17T05:04:50.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.068+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f214c0bd060 con 0x7f21600cf1e0 2024-09-17T05:04:50.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.072+0000 7f215d600640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2140073200 0x7f2140075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.072+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7f214c12d870 con 0x7f21600cf1e0 2024-09-17T05:04:50.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.072+0000 7f2165c00640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2140073200 0x7f2140075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.072+0000 7f2165c00640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2140073200 0x7f2140075710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2154000f70 tx=0x7f2154009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:50.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.076+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f214c016410 con 0x7f21600cf1e0 2024-09-17T05:04:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2160153c50 con 0x7f21600cf1e0 2024-09-17T05:04:50.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f215d600640 1 -- 172.21.15.16:0/3697492719 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7f214c0f8a40 con 0x7f21600cf1e0 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:04:50.255 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:04:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2140073200 msgr2=0x7f2140075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2140073200 0x7f2140075710 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2154000f70 tx=0x7f2154009040 comp rx=0 tx=0).stop 2024-09-17T05:04:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 msgr2=0x7f216015dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 0x7f216015dd40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f214c009790 tx=0x7f214c005db0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 shutdown_connections 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2140073200 0x7f2140075710 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f216014a0c0 0x7f216014e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21600cf1e0 0x7f216015dd40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 --2- 172.21.15.16:0/3697492719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2160071380 0x7f216015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 >> 172.21.15.16:0/3697492719 conn(0x7f2160073ca0 msgr2=0x7f21600b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 shutdown_connections 2024-09-17T05:04:50.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.252+0000 7f2167000640 1 -- 172.21.15.16:0/3697492719 wait complete. 2024-09-17T05:04:50.276 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:04:50.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.436+0000 7fbaa1600640 1 -- 172.21.15.16:0/3853889910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c0cfa10 msgr2=0x7fba9c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.436+0000 7fbaa1600640 1 --2- 172.21.15.16:0/3853889910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c0cfa10 0x7fba9c06eeb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009840 tx=0x7fba8c01c7c0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.436+0000 7fbaa1600640 1 -- 172.21.15.16:0/3853889910 shutdown_connections 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 --2- 172.21.15.16:0/3853889910 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba9c156a90 0x7fba9c1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 --2- 172.21.15.16:0/3853889910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c0cfa10 0x7fba9c06eeb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- 172.21.15.16:0/3853889910 >> 172.21.15.16:0/3853889910 conn(0x7fba9c0d0050 msgr2=0x7fba9c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- 172.21.15.16:0/3853889910 shutdown_connections 2024-09-17T05:04:50.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- 172.21.15.16:0/3853889910 wait complete. 2024-09-17T05:04:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 Processor -- start 2024-09-17T05:04:50.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- start start 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba9c0cfa10 0x7fba9c148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba9c156a90 0x7fba9c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 0x7fba9c1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c0cf270 con 0x7fba9c1606e0 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c1570b0 con 0x7fba9c0cfa10 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.440+0000 7fbaa1600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c0cf3f0 con 0x7fba9c156a90 2024-09-17T05:04:50.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 0x7fba9c1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 0x7fba9c1514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32838/0 (socket says 172.21.15.16:32838) 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 -- 172.21.15.16:0/2420893359 learned_addr learned my addr 172.21.15.16:0/2420893359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9aa00640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba9c0cfa10 0x7fba9c148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba93e00640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba9c156a90 0x7fba9c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 -- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba9c156a90 msgr2=0x7fba9c150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba9c156a90 0x7fba9c150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 -- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba9c0cfa10 msgr2=0x7fba9c148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba9c0cfa10 0x7fba9c148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 -- 172.21.15.16:0/2420893359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c149120 con 0x7fba9c1606e0 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9aa00640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba9c0cfa10 0x7fba9c148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:04:50.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba9b400640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 0x7fba9c1514f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c760 tx=0x7fba9400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94003db0 con 0x7fba9c1606e0 2024-09-17T05:04:50.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba9400ce10 con 0x7fba9c1606e0 2024-09-17T05:04:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba940052d0 con 0x7fba9c1606e0 2024-09-17T05:04:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c15c240 con 0x7fba9c1606e0 2024-09-17T05:04:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c14dc00 con 0x7fba9c1606e0 2024-09-17T05:04:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9c06eeb0 con 0x7fba9c1606e0 2024-09-17T05:04:50.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.444+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba94005470 con 0x7fba9c1606e0 2024-09-17T05:04:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.448+0000 7fba92a00640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fba78073230 0x7fba78075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.448+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5249+0+0 (secure 0 0 0) 0x7fba94093cf0 con 0x7fba9c1606e0 2024-09-17T05:04:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.448+0000 7fba9aa00640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fba78073230 0x7fba78075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.448+0000 7fba9aa00640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fba78073230 0x7fba78075740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0af000 tx=0x7fba8c01b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.452+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba9c074f50 con 0x7fba9c1606e0 2024-09-17T05:04:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:50.628+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fba9c0adfc0 con 0x7fba9c1606e0 2024-09-17T05:04:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.440+0000 7fba92a00640 1 -- 172.21.15.16:0/2420893359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7fba9405e7e0 con 0x7fba9c1606e0 2024-09-17T05:04:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:04:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fba78073230 msgr2=0x7fba78075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fba78073230 0x7fba78075740 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0af000 tx=0x7fba8c01b060 comp rx=0 tx=0).stop 2024-09-17T05:04:51.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 msgr2=0x7fba9c1514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 0x7fba9c1514f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fba9400c760 tx=0x7fba9400cc20 comp rx=0 tx=0).stop 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 shutdown_connections 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fba78073230 0x7fba78075740 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba9c1606e0 0x7fba9c1514f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fba9c156a90 0x7fba9c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 --2- 172.21.15.16:0/2420893359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba9c0cfa10 0x7fba9c148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 >> 172.21.15.16:0/2420893359 conn(0x7fba9c0d0050 msgr2=0x7fba9c15f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 shutdown_connections 2024-09-17T05:04:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.444+0000 7fbaa1600640 1 -- 172.21.15.16:0/2420893359 wait complete. 2024-09-17T05:04:51.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 -- 172.21.15.16:0/2465203059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a4073240 msgr2=0x7f97a4073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 --2- 172.21.15.16:0/2465203059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a4073240 0x7f97a4073650 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f979c009c80 tx=0x7f979c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 -- 172.21.15.16:0/2465203059 shutdown_connections 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 --2- 172.21.15.16:0/2465203059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a4073240 0x7f97a4073650 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 --2- 172.21.15.16:0/2465203059 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97a415c270 0x7f97a415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 -- 172.21.15.16:0/2465203059 >> 172.21.15.16:0/2465203059 conn(0x7f97a406f900 msgr2=0x7f97a40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 -- 172.21.15.16:0/2465203059 shutdown_connections 2024-09-17T05:04:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 -- 172.21.15.16:0/2465203059 wait complete. 2024-09-17T05:04:51.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 Processor -- start 2024-09-17T05:04:51.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.648+0000 7f97a9800640 1 -- start start 2024-09-17T05:04:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97a4073240 0x7f97a414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97a40b2f50 0x7f97a4155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 0x7f97a41559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:51.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4168e00 con 0x7f97a415c270 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4168c80 con 0x7f97a4073240 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f97a4168f80 con 0x7f97a40b2f50 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 0x7f97a41559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 0x7f97a41559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32852/0 (socket says 172.21.15.16:32852) 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 -- 172.21.15.16:0/725412845 learned_addr learned my addr 172.21.15.16:0/725412845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f979be00640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97a40b2f50 0x7f97a4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a2a00640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97a4073240 0x7f97a414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 -- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97a40b2f50 msgr2=0x7f97a4155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97a40b2f50 0x7f97a4155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 -- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97a4073240 msgr2=0x7f97a414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97a4073240 0x7f97a414c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 -- 172.21.15.16:0/725412845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a4163930 con 0x7f97a415c270 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a2a00640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97a4073240 0x7f97a414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a3400640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 0x7f97a41559a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f979c01b3c0 tx=0x7f979c009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0bb070 con 0x7f97a415c270 2024-09-17T05:04:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f979c004d40 con 0x7f97a415c270 2024-09-17T05:04:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a41fc6c0 con 0x7f97a415c270 2024-09-17T05:04:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0b2040 con 0x7f97a415c270 2024-09-17T05:04:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a41fcc80 con 0x7f97a415c270 2024-09-17T05:04:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.652+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a41fc410 con 0x7f97a415c270 2024-09-17T05:04:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.656+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f979c0be070 con 0x7f97a415c270 2024-09-17T05:04:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.656+0000 7f979aa00640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9780073200 0x7f9780075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.656+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5619+0+0 (secure 0 0 0) 0x7f979c12dfe0 con 0x7f97a415c270 2024-09-17T05:04:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.656+0000 7f97a2a00640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9780073200 0x7f9780075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.656+0000 7f97a2a00640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9780073200 0x7f9780075710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9794001630 tx=0x7f9794009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.660+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f979c016990 con 0x7f97a415c270 2024-09-17T05:04:51.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:51.836+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f97a415be40 con 0x7f97a415c270 2024-09-17T05:04:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.448+0000 7f979aa00640 1 -- 172.21.15.16:0/725412845 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f979c0f9310 con 0x7f97a415c270 2024-09-17T05:04:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:04:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9780073200 msgr2=0x7f9780075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9780073200 0x7f9780075710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9794001630 tx=0x7f9794009040 comp rx=0 tx=0).stop 2024-09-17T05:04:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 msgr2=0x7f97a41559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 0x7f97a41559a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f979c01b3c0 tx=0x7f979c009350 comp rx=0 tx=0).stop 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 shutdown_connections 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9780073200 0x7f9780075710 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97a415c270 0x7f97a41559a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97a40b2f50 0x7f97a4155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 --2- 172.21.15.16:0/725412845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97a4073240 0x7f97a414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 >> 172.21.15.16:0/725412845 conn(0x7f97a406f900 msgr2=0x7f97a40b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 shutdown_connections 2024-09-17T05:04:52.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.452+0000 7f97a9800640 1 -- 172.21.15.16:0/725412845 wait complete. 2024-09-17T05:04:52.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:04:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 -- 172.21.15.16:0/1899654034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80d09c0 msgr2=0x7feff80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 --2- 172.21.15.16:0/1899654034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80d09c0 0x7feff80cf6e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7feff80b3a70 tx=0x7fefe001c740 comp rx=0 tx=0).stop 2024-09-17T05:04:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 -- 172.21.15.16:0/1899654034 shutdown_connections 2024-09-17T05:04:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 --2- 172.21.15.16:0/1899654034 >> v2:172.21.15.104:3300/0 conn(0x7feff80ccb40 0x7feff80ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 --2- 172.21.15.16:0/1899654034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80d09c0 0x7feff80cf6e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 -- 172.21.15.16:0/1899654034 >> 172.21.15.16:0/1899654034 conn(0x7feff80b42d0 msgr2=0x7feff80b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 -- 172.21.15.16:0/1899654034 shutdown_connections 2024-09-17T05:04:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 -- 172.21.15.16:0/1899654034 wait complete. 2024-09-17T05:04:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 Processor -- start 2024-09-17T05:04:52.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.664+0000 7feffe200640 1 -- start start 2024-09-17T05:04:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 0x7feff8152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff80cf2d0 0x7feff80b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feff81598a0 0x7feff80712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feff816be10 con 0x7feff80ccb40 2024-09-17T05:04:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feff816bc90 con 0x7feff80cf2d0 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7feff816bf90 con 0x7feff81598a0 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 0x7feff8152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 0x7feff8152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32858/0 (socket says 172.21.15.16:32858) 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 -- 172.21.15.16:0/4098373686 learned_addr learned my addr 172.21.15.16:0/4098373686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7fefefe00640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff80cf2d0 0x7feff80b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7e00640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feff81598a0 0x7feff80712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 -- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feff81598a0 msgr2=0x7feff80712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feff81598a0 0x7feff80712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 -- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff80cf2d0 msgr2=0x7feff80b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff80cf2d0 0x7feff80b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 -- 172.21.15.16:0/4098373686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff8158800 con 0x7feff80ccb40 2024-09-17T05:04:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feff7400640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 0x7feff8152790 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c430 tx=0x7fefe800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011790 con 0x7feff80ccb40 2024-09-17T05:04:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefe8017070 con 0x7feff80ccb40 2024-09-17T05:04:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff8158ae0 con 0x7feff80ccb40 2024-09-17T05:04:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8011ce0 con 0x7feff80ccb40 2024-09-17T05:04:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff81590a0 con 0x7feff80ccb40 2024-09-17T05:04:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.668+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feff80ccf50 con 0x7feff80ccb40 2024-09-17T05:04:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.672+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefe801c460 con 0x7feff80ccb40 2024-09-17T05:04:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.672+0000 7fefef400640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefd4073200 0x7fefd4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.672+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6001+0+0 (secure 0 0 0) 0x7fefe8094c90 con 0x7feff80ccb40 2024-09-17T05:04:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.672+0000 7fefefe00640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefd4073200 0x7fefd4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.672+0000 7fefefe00640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefd4073200 0x7fefd4075710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fefe00af000 tx=0x7fefe001af80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.676+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fefe805fc20 con 0x7feff80ccb40 2024-09-17T05:04:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:52.880+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feff815a790 con 0x7feff80ccb40 2024-09-17T05:04:53.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.452+0000 7fefef400640 1 -- 172.21.15.16:0/4098373686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v48) ==== 174+0+0 (secure 0 0 0) 0x7fefe805f520 con 0x7feff80ccb40 2024-09-17T05:04:53.455 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 13 and data pool 14 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefd4073200 msgr2=0x7fefd4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefd4073200 0x7fefd4075710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fefe00af000 tx=0x7fefe001af80 comp rx=0 tx=0).stop 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 msgr2=0x7feff8152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 0x7feff8152790 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c430 tx=0x7fefe800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 shutdown_connections 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefd4073200 0x7fefd4075710 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feff81598a0 0x7feff80712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feff80cf2d0 0x7feff80b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 --2- 172.21.15.16:0/4098373686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff80ccb40 0x7feff8152790 secure :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c430 tx=0x7fefe800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.456+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 >> 172.21.15.16:0/4098373686 conn(0x7feff80b42d0 msgr2=0x7feff80d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.460+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 shutdown_connections 2024-09-17T05:04:53.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.460+0000 7feffe200640 1 -- 172.21.15.16:0/4098373686 wait complete. 2024-09-17T05:04:53.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 -- 172.21.15.16:0/393871922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c06d8a0 msgr2=0x7ff20c0cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 --2- 172.21.15.16:0/393871922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c06d8a0 0x7ff20c0cd900 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff20c074a30 tx=0x7ff1fc01c830 comp rx=0 tx=0).stop 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 -- 172.21.15.16:0/393871922 shutdown_connections 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 --2- 172.21.15.16:0/393871922 >> v2:172.21.15.104:3300/0 conn(0x7ff20c15cd70 0x7ff20c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 --2- 172.21.15.16:0/393871922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c06d8a0 0x7ff20c0cd900 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 -- 172.21.15.16:0/393871922 >> 172.21.15.16:0/393871922 conn(0x7ff20c073ca0 msgr2=0x7ff20c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 -- 172.21.15.16:0/393871922 shutdown_connections 2024-09-17T05:04:53.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.672+0000 7ff211a00640 1 -- 172.21.15.16:0/393871922 wait complete. 2024-09-17T05:04:53.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 Processor -- start 2024-09-17T05:04:53.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 -- start start 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c06d8a0 0x7ff20c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff20c06efb0 0x7ff20c138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 0x7ff20c139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c169b50 con 0x7ff20c15cd70 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c15d8d0 con 0x7ff20c06efb0 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff20c169cd0 con 0x7ff20c06d8a0 2024-09-17T05:04:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 0x7ff20c139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 0x7ff20c139210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32878/0 (socket says 172.21.15.16:32878) 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 -- 172.21.15.16:0/3658276641 learned_addr learned my addr 172.21.15.16:0/3658276641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff20be00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c06d8a0 0x7ff20c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff203e00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff20c06efb0 0x7ff20c138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 -- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c06d8a0 msgr2=0x7ff20c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c06d8a0 0x7ff20c157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 -- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff20c06efb0 msgr2=0x7ff20c138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff20c06efb0 0x7ff20c138cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff211000640 1 -- 172.21.15.16:0/3658276641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c1400c0 con 0x7ff20c15cd70 2024-09-17T05:04:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.676+0000 7ff203e00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff20c06efb0 0x7ff20c138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:53.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff211000640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 0x7ff20c139210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e940 tx=0x7ff20400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:53.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204004d30 con 0x7ff20c15cd70 2024-09-17T05:04:53.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff20400cac0 con 0x7ff20c15cd70 2024-09-17T05:04:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204005020 con 0x7ff20c15cd70 2024-09-17T05:04:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c158170 con 0x7ff20c15cd70 2024-09-17T05:04:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c13d510 con 0x7ff20c15cd70 2024-09-17T05:04:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff20c0cd900 con 0x7ff20c15cd70 2024-09-17T05:04:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff2040051c0 con 0x7ff20c15cd70 2024-09-17T05:04:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.680+0000 7ff208c00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1e8073200 0x7ff1e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.684+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7ff204094c90 con 0x7ff20c15cd70 2024-09-17T05:04:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.684+0000 7ff20be00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1e8073200 0x7ff1e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:53.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.684+0000 7ff20be00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1e8073200 0x7ff1e8075710 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0af000 tx=0x7ff1fc0a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:53.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.688+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff204016030 con 0x7ff20c15cd70 2024-09-17T05:04:53.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.864+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff20c15fd10 con 0x7ff20c15cd70 2024-09-17T05:04:53.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.864+0000 7ff208c00640 1 -- 172.21.15.16:0/3658276641 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16994 (secure 0 0 0) 0x7ff20405f4c0 con 0x7ff20c15cd70 2024-09-17T05:04:53.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:53.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:52.887999+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:50.645738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:51.850045+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:53.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1e8073200 msgr2=0x7ff1e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1e8073200 0x7ff1e8075710 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0af000 tx=0x7ff1fc0a7040 comp rx=0 tx=0).stop 2024-09-17T05:04:53.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 msgr2=0x7ff20c139210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:53.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 0x7ff20c139210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e940 tx=0x7ff20400ee00 comp rx=0 tx=0).stop 2024-09-17T05:04:53.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 shutdown_connections 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1e8073200 0x7ff1e8075710 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c15cd70 0x7ff20c139210 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff20c06efb0 0x7ff20c138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 --2- 172.21.15.16:0/3658276641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c06d8a0 0x7ff20c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 >> 172.21.15.16:0/3658276641 conn(0x7ff20c073ca0 msgr2=0x7ff20c15f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 shutdown_connections 2024-09-17T05:04:53.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:53.868+0000 7ff211a00640 1 -- 172.21.15.16:0/3658276641 wait complete. 2024-09-17T05:04:53.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 -- 172.21.15.16:0/1788938963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc15c0a0 msgr2=0x7f12dc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 --2- 172.21.15.16:0/1788938963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc15c0a0 0x7f12dc15c4b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f12dc071ce0 tx=0x7f12c401c770 comp rx=0 tx=0).stop 2024-09-17T05:04:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 -- 172.21.15.16:0/1788938963 shutdown_connections 2024-09-17T05:04:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 --2- 172.21.15.16:0/1788938963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc15c0a0 0x7f12dc15c4b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 --2- 172.21.15.16:0/1788938963 >> v2:172.21.15.104:3300/0 conn(0x7f12dc0b2f50 0x7f12dc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 -- 172.21.15.16:0/1788938963 >> 172.21.15.16:0/1788938963 conn(0x7f12dc06f900 msgr2=0x7f12dc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 -- 172.21.15.16:0/1788938963 shutdown_connections 2024-09-17T05:04:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.044+0000 7f12e0e00640 1 -- 172.21.15.16:0/1788938963 wait complete. 2024-09-17T05:04:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 Processor -- start 2024-09-17T05:04:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 -- start start 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 0x7f12dc1449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12dc15c0a0 0x7f12dc153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12dc15f150 0x7f12dc165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc169120 con 0x7f12dc0b2f50 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc168fa0 con 0x7f12dc15c0a0 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f12dc1692a0 con 0x7f12dc15f150 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 0x7f12dc1449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 0x7f12dc1449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32890/0 (socket says 172.21.15.16:32890) 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 -- 172.21.15.16:0/409705076 learned_addr learned my addr 172.21.15.16:0/409705076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d3e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12dc15c0a0 0x7f12dc153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12da800640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12dc15f150 0x7f12dc165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 -- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12dc15f150 msgr2=0x7f12dc165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12dc15f150 0x7f12dc165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 -- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12dc15c0a0 msgr2=0x7f12dc153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12dc15c0a0 0x7f12dc153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 -- 172.21.15.16:0/409705076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc1649e0 con 0x7f12dc0b2f50 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d9e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 0x7f12dc1449c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e7f0 tx=0x7f12cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc00ccc0 con 0x7f12dc0b2f50 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12cc00ce80 con 0x7f12dc0b2f50 2024-09-17T05:04:54.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc144f00 con 0x7f12dc0b2f50 2024-09-17T05:04:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc005280 con 0x7f12dc0b2f50 2024-09-17T05:04:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.048+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc164fa0 con 0x7f12dc0b2f50 2024-09-17T05:04:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.052+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc15bb60 con 0x7f12dc0b2f50 2024-09-17T05:04:54.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.052+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f12cc005420 con 0x7f12dc0b2f50 2024-09-17T05:04:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.052+0000 7f12d2000640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f12b8073230 0x7f12b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.052+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f12cc094d10 con 0x7f12dc0b2f50 2024-09-17T05:04:54.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.052+0000 7f12d3e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f12b8073230 0x7f12b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.056+0000 7f12d3e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f12b8073230 0x7f12b8075740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12c40af000 tx=0x7f12c401afb0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.056+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f12cc09a020 con 0x7f12dc0b2f50 2024-09-17T05:04:54.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.232+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12dc15f970 con 0x7f12dc0b2f50 2024-09-17T05:04:54.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.232+0000 7f12d2000640 1 -- 172.21.15.16:0/409705076 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16994 (secure 0 0 0) 0x7f12cc05f4b0 con 0x7f12dc0b2f50 2024-09-17T05:04:54.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:54.237 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:52.887999+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:50.645738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:51.850045+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f12b8073230 msgr2=0x7f12b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f12b8073230 0x7f12b8075740 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12c40af000 tx=0x7f12c401afb0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 msgr2=0x7f12dc1449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 0x7f12dc1449c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f12cc00e7f0 tx=0x7f12cc00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 shutdown_connections 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f12b8073230 0x7f12b8075740 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f12dc15f150 0x7f12dc165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12dc15c0a0 0x7f12dc153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 --2- 172.21.15.16:0/409705076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12dc0b2f50 0x7f12dc1449c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 >> 172.21.15.16:0/409705076 conn(0x7f12dc06f900 msgr2=0x7f12dc15c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 shutdown_connections 2024-09-17T05:04:54.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.236+0000 7f12e0e00640 1 -- 172.21.15.16:0/409705076 wait complete. 2024-09-17T05:04:54.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:04:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 -- 172.21.15.16:0/4102274713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080afbd0 msgr2=0x7f9b0815f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 --2- 172.21.15.16:0/4102274713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080afbd0 0x7f9b0815f310 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009bf0 tx=0x7f9b0001cba0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 -- 172.21.15.16:0/4102274713 shutdown_connections 2024-09-17T05:04:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 --2- 172.21.15.16:0/4102274713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080afbd0 0x7f9b0815f310 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 -- 172.21.15.16:0/4102274713 >> 172.21.15.16:0/4102274713 conn(0x7f9b080cd980 msgr2=0x7f9b080cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 -- 172.21.15.16:0/4102274713 shutdown_connections 2024-09-17T05:04:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.416+0000 7f9b0d400640 1 -- 172.21.15.16:0/4102274713 wait complete. 2024-09-17T05:04:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 Processor -- start 2024-09-17T05:04:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 -- start start 2024-09-17T05:04:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b080afbd0 0x7f9b08140b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 0x7f9b08151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b08155c10 0x7f9b08151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08071ce0 con 0x7f9b080ce4d0 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08071b60 con 0x7f9b08155c10 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b0d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9b08071e60 con 0x7f9b080afbd0 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 0x7f9b08151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 0x7f9b08151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32910/0 (socket says 172.21.15.16:32910) 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 -- 172.21.15.16:0/194067541 learned_addr learned my addr 172.21.15.16:0/194067541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 -- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b080afbd0 msgr2=0x7f9b08140b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b07200640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b08155c10 0x7f9b08151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b06800640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b080afbd0 0x7f9b08140b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b080afbd0 0x7f9b08140b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 -- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b08155c10 msgr2=0x7f9b08151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b08155c10 0x7f9b08151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9afde00640 1 -- 172.21.15.16:0/194067541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b081523d0 con 0x7f9b080ce4d0 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b07200640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b08155c10 0x7f9b08151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.420+0000 7f9b06800640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b080afbd0 0x7f9b08140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9afde00640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 0x7f9b08151950 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b3e0 tx=0x7f9af000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0015020 con 0x7f9b080ce4d0 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9af0011920 con 0x7f9b080ce4d0 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011db0 con 0x7f9b080ce4d0 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0815adb0 con 0x7f9b080ce4d0 2024-09-17T05:04:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0815b370 con 0x7f9b080ce4d0 2024-09-17T05:04:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0815ab20 con 0x7f9b080ce4d0 2024-09-17T05:04:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9af0018020 con 0x7f9b080ce4d0 2024-09-17T05:04:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9aff400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ae4073200 0x7f9ae4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f9af00940d0 con 0x7f9b080ce4d0 2024-09-17T05:04:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.424+0000 7f9b06800640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ae4073200 0x7f9ae4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.428+0000 7f9b06800640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ae4073200 0x7f9ae4075710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9af8000f70 tx=0x7f9af8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.428+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9af001f030 con 0x7f9b080ce4d0 2024-09-17T05:04:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:54.640+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9b0806a180 con 0x7f9b080ce4d0 2024-09-17T05:04:55.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T05:04:55.107 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T05:04:55.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9aff400640 1 -- 172.21.15.16:0/194067541 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) ==== 118+0+0 (secure 0 0 0) 0x7f9af005eec0 con 0x7f9b080ce4d0 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ae4073200 msgr2=0x7f9ae4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ae4073200 0x7f9ae4075710 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9af8000f70 tx=0x7f9af8009040 comp rx=0 tx=0).stop 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 msgr2=0x7f9b08151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 0x7f9b08151950 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b3e0 tx=0x7f9af000b8a0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 shutdown_connections 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9ae4073200 0x7f9ae4075710 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b08155c10 0x7f9b08151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b080ce4d0 0x7f9b08151950 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 --2- 172.21.15.16:0/194067541 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b080afbd0 0x7f9b08140b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 >> 172.21.15.16:0/194067541 conn(0x7f9b080cd980 msgr2=0x7f9b08154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 shutdown_connections 2024-09-17T05:04:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.484+0000 7f9b0d400640 1 -- 172.21.15.16:0/194067541 wait complete. 2024-09-17T05:04:55.513 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:04:55.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:04:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 -- 172.21.15.16:0/4134209765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca0073dc0 msgr2=0x7efca00741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 --2- 172.21.15.16:0/4134209765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca0073dc0 0x7efca00741d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7efca00cec50 tx=0x7efc8801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 -- 172.21.15.16:0/4134209765 shutdown_connections 2024-09-17T05:04:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 --2- 172.21.15.16:0/4134209765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca0073dc0 0x7efca00741d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 --2- 172.21.15.16:0/4134209765 >> v2:172.21.15.104:3300/0 conn(0x7efca00cce00 0x7efca00cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 -- 172.21.15.16:0/4134209765 >> 172.21.15.16:0/4134209765 conn(0x7efca00b2dd0 msgr2=0x7efca00b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 -- 172.21.15.16:0/4134209765 shutdown_connections 2024-09-17T05:04:55.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 -- 172.21.15.16:0/4134209765 wait complete. 2024-09-17T05:04:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 Processor -- start 2024-09-17T05:04:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.680+0000 7efca7800640 1 -- start start 2024-09-17T05:04:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efca0073dc0 0x7efca00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 0x7efca0150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efca015f040 0x7efca006d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efca0168d50 con 0x7efca00cce00 2024-09-17T05:04:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efca0168bd0 con 0x7efca0073dc0 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efca0168ed0 con 0x7efca015f040 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 0x7efca0150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 0x7efca0150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32928/0 (socket says 172.21.15.16:32928) 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 -- 172.21.15.16:0/3038673151 learned_addr learned my addr 172.21.15.16:0/3038673151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca4e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efca0073dc0 0x7efca00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca5800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efca015f040 0x7efca006d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 -- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efca015f040 msgr2=0x7efca006d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efca015f040 0x7efca006d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 -- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efca0073dc0 msgr2=0x7efca00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efca0073dc0 0x7efca00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 -- 172.21.15.16:0/3038673151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca00c9320 con 0x7efca00cce00 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca5800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efca015f040 0x7efca006d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:55.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca4e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efca0073dc0 0x7efca00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc97e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 0x7efca0150a60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7efc880194d0 tx=0x7efc880a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880bc070 con 0x7efca00cce00 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc880043b0 con 0x7efca00cce00 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca00c9600 con 0x7efca00cce00 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880b2040 con 0x7efca00cce00 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca00c9bc0 con 0x7efca00cce00 2024-09-17T05:04:55.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.684+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca00741d0 con 0x7efca00cce00 2024-09-17T05:04:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.688+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc88005290 con 0x7efca00cce00 2024-09-17T05:04:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.688+0000 7efc9d600640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efc7c073200 0x7efc7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.688+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7efc8812dc50 con 0x7efca00cce00 2024-09-17T05:04:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.688+0000 7efca4e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efc7c073200 0x7efc7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.688+0000 7efca4e00640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efc7c073200 0x7efc7c075710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efc90009ac0 tx=0x7efc900091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:55.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.692+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efc880164b0 con 0x7efca00cce00 2024-09-17T05:04:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:55.900+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7efca00cb2b0 con 0x7efca00cce00 2024-09-17T05:04:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.488+0000 7efc9d600640 1 -- 172.21.15.16:0/3038673151 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7efc880f8a50 con 0x7efca00cce00 2024-09-17T05:04:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efc7c073200 msgr2=0x7efc7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efc7c073200 0x7efc7c075710 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efc90009ac0 tx=0x7efc900091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 msgr2=0x7efca0150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 0x7efca0150a60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7efc880194d0 tx=0x7efc880a56f0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 shutdown_connections 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efc7c073200 0x7efc7c075710 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efca015f040 0x7efca006d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca00cce00 0x7efca0150a60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 --2- 172.21.15.16:0/3038673151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efca0073dc0 0x7efca00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 >> 172.21.15.16:0/3038673151 conn(0x7efca00b2dd0 msgr2=0x7efca00b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 shutdown_connections 2024-09-17T05:04:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.492+0000 7efca7800640 1 -- 172.21.15.16:0/3038673151 wait complete. 2024-09-17T05:04:56.517 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:04:56.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:04:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 -- 172.21.15.16:0/462173659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd918003a10 msgr2=0x7fd918003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 --2- 172.21.15.16:0/462173659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd918003a10 0x7fd918003e40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fd924001770 tx=0x7fd92401c860 comp rx=0 tx=0).stop 2024-09-17T05:04:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 -- 172.21.15.16:0/462173659 shutdown_connections 2024-09-17T05:04:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 --2- 172.21.15.16:0/462173659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd918003a10 0x7fd918003e40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 -- 172.21.15.16:0/462173659 >> 172.21.15.16:0/462173659 conn(0x7fd93006f900 msgr2=0x7fd9300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 -- 172.21.15.16:0/462173659 shutdown_connections 2024-09-17T05:04:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 -- 172.21.15.16:0/462173659 wait complete. 2024-09-17T05:04:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 Processor -- start 2024-09-17T05:04:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.676+0000 7fd938200640 1 -- start start 2024-09-17T05:04:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd918003a10 0x7fd930145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 0x7fd93014c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd93014bb20 0x7fd9300b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd9300743e0 con 0x7fd9301fc930 2024-09-17T05:04:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd930074260 con 0x7fd918003a10 2024-09-17T05:04:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd93015f150 con 0x7fd93014bb20 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 0x7fd93014c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 0x7fd93014c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32956/0 (socket says 172.21.15.16:32956) 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 -- 172.21.15.16:0/16195850 learned_addr learned my addr 172.21.15.16:0/16195850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd935800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd918003a10 0x7fd930145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd936200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd93014bb20 0x7fd9300b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 -- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd93014bb20 msgr2=0x7fd9300b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd93014bb20 0x7fd9300b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 -- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd918003a10 msgr2=0x7fd930145a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd918003a10 0x7fd930145a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 -- 172.21.15.16:0/16195850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd930145fc0 con 0x7fd9301fc930 2024-09-17T05:04:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd934e00640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 0x7fd93014c450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011ae0 tx=0x7fd91c011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c01e070 con 0x7fd9301fc930 2024-09-17T05:04:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd91c019430 con 0x7fd9301fc930 2024-09-17T05:04:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c019840 con 0x7fd9301fc930 2024-09-17T05:04:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93014c990 con 0x7fd9301fc930 2024-09-17T05:04:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9300affd0 con 0x7fd9301fc930 2024-09-17T05:04:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.680+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd93015f6b0 con 0x7fd9301fc930 2024-09-17T05:04:56.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.684+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd91c021070 con 0x7fd9301fc930 2024-09-17T05:04:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.684+0000 7fd92d600640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd910073200 0x7fd910075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.684+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fd91c0956d0 con 0x7fd9301fc930 2024-09-17T05:04:56.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.684+0000 7fd935800640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd910073200 0x7fd910075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:56.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.688+0000 7fd935800640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd910073200 0x7fd910075710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd9240a8000 tx=0x7fd924006040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:56.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.688+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd91c05fef0 con 0x7fd9301fc930 2024-09-17T05:04:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.896+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9300d0bd0 con 0x7fd9301fc930 2024-09-17T05:04:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd92d600640 1 -- 172.21.15.16:0/16195850 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5228 (secure 0 0 0) 0x7fd91c064920 con 0x7fd9301fc930 2024-09-17T05:04:56.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:56.901 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"btime":"2024-09-17T05:04:56:488386+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:04:55.907544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:04:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 52 2024-09-17T05:04:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd910073200 msgr2=0x7fd910075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd910073200 0x7fd910075710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd9240a8000 tx=0x7fd924006040 comp rx=0 tx=0).stop 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 msgr2=0x7fd93014c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 0x7fd93014c450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011ae0 tx=0x7fd91c011ee0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 shutdown_connections 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd910073200 0x7fd910075710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd93014bb20 0x7fd9300b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9301fc930 0x7fd93014c450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 --2- 172.21.15.16:0/16195850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd918003a10 0x7fd930145a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 >> 172.21.15.16:0/16195850 conn(0x7fd93006f900 msgr2=0x7fd93006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 shutdown_connections 2024-09-17T05:04:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:56.900+0000 7fd938200640 1 -- 172.21.15.16:0/16195850 wait complete. 2024-09-17T05:04:56.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:04:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 -- 172.21.15.16:0/2054175658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf815d680 msgr2=0x7f5cf80cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 --2- 172.21.15.16:0/2054175658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf815d680 0x7f5cf80cc8f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8146260 tx=0x7f5cec01c8a0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 -- 172.21.15.16:0/2054175658 shutdown_connections 2024-09-17T05:04:57.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 --2- 172.21.15.16:0/2054175658 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5cf81600f0 0x7f5cf8160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 --2- 172.21.15.16:0/2054175658 >> v2:172.21.15.104:3300/0 conn(0x7f5cf80cedc0 0x7f5cf80cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 --2- 172.21.15.16:0/2054175658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf815d680 0x7f5cf80cc8f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 -- 172.21.15.16:0/2054175658 >> 172.21.15.16:0/2054175658 conn(0x7f5cf806f900 msgr2=0x7f5cf80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 -- 172.21.15.16:0/2054175658 shutdown_connections 2024-09-17T05:04:57.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.080+0000 7f5cffa00640 1 -- 172.21.15.16:0/2054175658 wait complete. 2024-09-17T05:04:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 Processor -- start 2024-09-17T05:04:57.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 -- start start 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 0x7f5cf80b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5cf815d680 0x7f5cf814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cf81600f0 0x7f5cf814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf806e3e0 con 0x7f5cf80cedc0 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf806e260 con 0x7f5cf81600f0 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf806e560 con 0x7f5cf815d680 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 0x7f5cf80b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 0x7f5cf80b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41212/0 (socket says 172.21.15.16:41212) 2024-09-17T05:04:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 -- 172.21.15.16:0/3000193024 learned_addr learned my addr 172.21.15.16:0/3000193024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cf7e00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5cf815d680 0x7f5cf814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfda00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cf81600f0 0x7f5cf814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 -- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5cf815d680 msgr2=0x7f5cf814eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5cf815d680 0x7f5cf814eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 -- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cf81600f0 msgr2=0x7f5cf814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cf81600f0 0x7f5cf814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 -- 172.21.15.16:0/3000193024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf8156530 con 0x7f5cf80cedc0 2024-09-17T05:04:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfda00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cf81600f0 0x7f5cf814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cfd000640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 0x7f5cf80b1520 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009850 tx=0x7f5cec0a5520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0b2040 con 0x7f5cf80cedc0 2024-09-17T05:04:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cec0043b0 con 0x7f5cf80cedc0 2024-09-17T05:04:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf8156810 con 0x7f5cf80cedc0 2024-09-17T05:04:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec005270 con 0x7f5cf80cedc0 2024-09-17T05:04:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.084+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf813c680 con 0x7f5cf80cedc0 2024-09-17T05:04:57.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.088+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf80cc8f0 con 0x7f5cf80cedc0 2024-09-17T05:04:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.088+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5cec0bd060 con 0x7f5cf80cedc0 2024-09-17T05:04:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.088+0000 7f5cf4c00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5cd8073200 0x7f5cd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.088+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f5cec12db10 con 0x7f5cf80cedc0 2024-09-17T05:04:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.088+0000 7f5cf7e00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5cd8073200 0x7f5cd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.092+0000 7f5cf7e00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5cd8073200 0x7f5cd8075710 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80b2300 tx=0x7f5ce400b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:57.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.092+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5cec016490 con 0x7f5cf80cedc0 2024-09-17T05:04:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cf815d1e0 con 0x7f5cf80cedc0 2024-09-17T05:04:57.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cf4c00640 1 -- 172.21.15.16:0/3000193024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16981 (secure 0 0 0) 0x7f5cec0f8a20 con 0x7f5cf80cedc0 2024-09-17T05:04:57.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:57.283 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":87,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:04:54.474097+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:50.645738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:51.850045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:04:57.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5cd8073200 msgr2=0x7f5cd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5cd8073200 0x7f5cd8075710 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80b2300 tx=0x7f5ce400b260 comp rx=0 tx=0).stop 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 msgr2=0x7f5cf80b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 0x7f5cf80b1520 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009850 tx=0x7f5cec0a5520 comp rx=0 tx=0).stop 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 shutdown_connections 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5cd8073200 0x7f5cd8075710 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5cf81600f0 0x7f5cf814f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5cf815d680 0x7f5cf814eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 --2- 172.21.15.16:0/3000193024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cf80cedc0 0x7f5cf80b1520 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.280+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 >> 172.21.15.16:0/3000193024 conn(0x7f5cf806f900 msgr2=0x7f5cf814fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:57.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.284+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 shutdown_connections 2024-09-17T05:04:57.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.284+0000 7f5cffa00640 1 -- 172.21.15.16:0/3000193024 wait complete. 2024-09-17T05:04:57.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:04:57.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:04:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 -- 172.21.15.16:0/1994667287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cc820 msgr2=0x7f4bd40ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1994667287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cc820 0x7f4bd40ccc30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009bc0 tx=0x7f4bcc01ca90 comp rx=0 tx=0).stop 2024-09-17T05:04:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 -- 172.21.15.16:0/1994667287 shutdown_connections 2024-09-17T05:04:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1994667287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cc820 0x7f4bd40ccc30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1994667287 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bd40cf0b0 0x7f4bd40cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 -- 172.21.15.16:0/1994667287 >> 172.21.15.16:0/1994667287 conn(0x7f4bd40b4000 msgr2=0x7f4bd40b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 -- 172.21.15.16:0/1994667287 shutdown_connections 2024-09-17T05:04:57.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 -- 172.21.15.16:0/1994667287 wait complete. 2024-09-17T05:04:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 Processor -- start 2024-09-17T05:04:57.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.464+0000 7f4bdb000640 1 -- start start 2024-09-17T05:04:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 0x7f4bd4151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cf0b0 0x7f4bd4152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bd40d1bf0 0x7f4bd414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4153390 con 0x7f4bd40cc820 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4153210 con 0x7f4bd40cf0b0 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd4153510 con 0x7f4bd40d1bf0 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 0x7f4bd4151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 0x7f4bd4151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41220/0 (socket says 172.21.15.16:41220) 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 -- 172.21.15.16:0/1835697064 learned_addr learned my addr 172.21.15.16:0/1835697064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 -- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bd40d1bf0 msgr2=0x7f4bd414afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bd40d1bf0 0x7f4bd414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9200640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cf0b0 0x7f4bd4152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 -- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cf0b0 msgr2=0x7f4bd4152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cf0b0 0x7f4bd4152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 -- 172.21.15.16:0/1835697064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd414a7e0 con 0x7f4bd40cc820 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9200640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cf0b0 0x7f4bd4152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd9c00640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 0x7f4bd4151d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c530 tx=0x7f4bc800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8016020 con 0x7f4bd40cc820 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bc8012940 con 0x7f4bd40cc820 2024-09-17T05:04:57.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd4147500 con 0x7f4bd40cc820 2024-09-17T05:04:57.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8012d90 con 0x7f4bd40cc820 2024-09-17T05:04:57.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd4147ac0 con 0x7f4bd40cc820 2024-09-17T05:04:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.468+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd40ccc30 con 0x7f4bd40cc820 2024-09-17T05:04:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.472+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4bc8019020 con 0x7f4bd40cc820 2024-09-17T05:04:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.472+0000 7f4bd1600640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bb4073230 0x7f4bb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.472+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f4bc80949b0 con 0x7f4bd40cc820 2024-09-17T05:04:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.472+0000 7f4bd9200640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bb4073230 0x7f4bb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.472+0000 7f4bd9200640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bb4073230 0x7f4bb4075740 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4148090 tx=0x7f4bc00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:57.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.476+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4bc8020020 con 0x7f4bd40cc820 2024-09-17T05:04:57.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.652+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bd40cf6f0 con 0x7f4bb4073230 2024-09-17T05:04:57.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.656+0000 7f4bd1600640 1 -- 172.21.15.16:0/1835697064 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247353 (secure 0 0 0) 0x7f4bd40cf6f0 con 0x7f4bb4073230 2024-09-17T05:04:57.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:57.665 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:04:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bb4073230 msgr2=0x7f4bb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bb4073230 0x7f4bb4075740 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4148090 tx=0x7f4bc00091e0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 msgr2=0x7f4bd4151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 0x7f4bd4151d30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c530 tx=0x7f4bc800c9f0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 shutdown_connections 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bb4073230 0x7f4bb4075740 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bd40d1bf0 0x7f4bd414afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bd40cf0b0 0x7f4bd4152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 --2- 172.21.15.16:0/1835697064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bd40cc820 0x7f4bd4151d30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 >> 172.21.15.16:0/1835697064 conn(0x7f4bd40b4000 msgr2=0x7f4bd40b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 shutdown_connections 2024-09-17T05:04:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.660+0000 7f4bdb000640 1 -- 172.21.15.16:0/1835697064 wait complete. 2024-09-17T05:04:57.684 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-17T05:04:57.520432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267088,"kb_used_data":9240,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707632,"statfs":{"total":773094113280,"available":772820615168,"internally_reserved":0,"allocated":9461760,"data_stored":4916240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467906+0000","last_change":"2024-09-17T05:04:52.467906+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467906+0000","last_undegraded":"2024-09-17T05:04:52.467906+0000","last_fullsized":"2024-09-17T05:04:52.467906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464969+0000","last_change":"2024-09-17T05:04:52.464969+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464969+0000","last_undegraded":"2024-09-17T05:04:52.464969+0000","last_fullsized":"2024-09-17T05:04:52.464969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468043+0000","last_change":"2024-09-17T05:04:52.468043+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468043+0000","last_undegraded":"2024-09-17T05:04:52.468043+0000","last_fullsized":"2024-09-17T05:04:52.468043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.472514+0000","last_change":"2024-09-17T05:04:52.472514+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.472514+0000","last_undegraded":"2024-09-17T05:04:52.472514+0000","last_fullsized":"2024-09-17T05:04:52.472514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465690+0000","last_change":"2024-09-17T05:04:52.465690+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465690+0000","last_undegraded":"2024-09-17T05:04:52.465690+0000","last_fullsized":"2024-09-17T05:04:52.465690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460417+0000","last_change":"2024-09-17T05:04:52.460417+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460417+0000","last_undegraded":"2024-09-17T05:04:52.460417+0000","last_fullsized":"2024-09-17T05:04:52.460417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468944+0000","last_change":"2024-09-17T05:04:52.468944+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468944+0000","last_undegraded":"2024-09-17T05:04:52.468944+0000","last_fullsized":"2024-09-17T05:04:52.468944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462576+0000","last_change":"2024-09-17T05:04:52.462576+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462576+0000","last_undegraded":"2024-09-17T05:04:52.462576+0000","last_fullsized":"2024-09-17T05:04:52.462576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.475893+0000","last_change":"2024-09-17T05:04:52.475893+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.475893+0000","last_undegraded":"2024-09-17T05:04:52.475893+0000","last_fullsized":"2024-09-17T05:04:52.475893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469663+0000","last_change":"2024-09-17T05:04:52.469663+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469663+0000","last_undegraded":"2024-09-17T05:04:52.469663+0000","last_fullsized":"2024-09-17T05:04:52.469663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478888+0000","last_change":"2024-09-17T05:04:52.478888+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478888+0000","last_undegraded":"2024-09-17T05:04:52.478888+0000","last_fullsized":"2024-09-17T05:04:52.478888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469566+0000","last_change":"2024-09-17T05:04:52.469566+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469566+0000","last_undegraded":"2024-09-17T05:04:52.469566+0000","last_fullsized":"2024-09-17T05:04:52.469566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.491308+0000","last_change":"2024-09-17T05:04:53.488884+0000","last_active":"2024-09-17T05:04:53.491308+0000","last_peered":"2024-09-17T05:04:53.491308+0000","last_clean":"2024-09-17T05:04:53.491308+0000","last_became_active":"2024-09-17T05:04:53.488567+0000","last_became_peered":"2024-09-17T05:04:53.488567+0000","last_unstale":"2024-09-17T05:04:53.491308+0000","last_undegraded":"2024-09-17T05:04:53.491308+0000","last_fullsized":"2024-09-17T05:04:53.491308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:16.452112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469017+0000","last_change":"2024-09-17T05:04:52.469017+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469017+0000","last_undegraded":"2024-09-17T05:04:52.469017+0000","last_fullsized":"2024-09-17T05:04:52.469017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467045+0000","last_change":"2024-09-17T05:04:52.467045+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467045+0000","last_undegraded":"2024-09-17T05:04:52.467045+0000","last_fullsized":"2024-09-17T05:04:52.467045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462060+0000","last_change":"2024-09-17T05:04:52.462060+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462060+0000","last_undegraded":"2024-09-17T05:04:52.462060+0000","last_fullsized":"2024-09-17T05:04:52.462060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459791+0000","last_change":"2024-09-17T05:04:52.459791+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459791+0000","last_undegraded":"2024-09-17T05:04:52.459791+0000","last_fullsized":"2024-09-17T05:04:52.459791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469183+0000","last_change":"2024-09-17T05:04:52.469183+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469183+0000","last_undegraded":"2024-09-17T05:04:52.469183+0000","last_fullsized":"2024-09-17T05:04:52.469183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465817+0000","last_change":"2024-09-17T05:04:52.465817+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465817+0000","last_undegraded":"2024-09-17T05:04:52.465817+0000","last_fullsized":"2024-09-17T05:04:52.465817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459213+0000","last_change":"2024-09-17T05:04:52.459213+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459213+0000","last_undegraded":"2024-09-17T05:04:52.459213+0000","last_fullsized":"2024-09-17T05:04:52.459213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464345+0000","last_change":"2024-09-17T05:04:52.464345+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464345+0000","last_undegraded":"2024-09-17T05:04:52.464345+0000","last_fullsized":"2024-09-17T05:04:52.464345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464433+0000","last_change":"2024-09-17T05:04:52.464433+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464433+0000","last_undegraded":"2024-09-17T05:04:52.464433+0000","last_fullsized":"2024-09-17T05:04:52.464433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464344+0000","last_change":"2024-09-17T05:04:52.464344+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464344+0000","last_undegraded":"2024-09-17T05:04:52.464344+0000","last_fullsized":"2024-09-17T05:04:52.464344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466713+0000","last_change":"2024-09-17T05:04:52.466713+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466713+0000","last_undegraded":"2024-09-17T05:04:52.466713+0000","last_fullsized":"2024-09-17T05:04:52.466713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464434+0000","last_change":"2024-09-17T05:04:52.464434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464434+0000","last_undegraded":"2024-09-17T05:04:52.464434+0000","last_fullsized":"2024-09-17T05:04:52.464434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463434+0000","last_change":"2024-09-17T05:04:52.463434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463434+0000","last_undegraded":"2024-09-17T05:04:52.463434+0000","last_fullsized":"2024-09-17T05:04:52.463434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467567+0000","last_change":"2024-09-17T05:04:52.467567+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467567+0000","last_undegraded":"2024-09-17T05:04:52.467567+0000","last_fullsized":"2024-09-17T05:04:52.467567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.494978+0000","last_change":"2024-09-17T05:04:53.491786+0000","last_active":"2024-09-17T05:04:53.494978+0000","last_peered":"2024-09-17T05:04:53.494978+0000","last_clean":"2024-09-17T05:04:53.494978+0000","last_became_active":"2024-09-17T05:04:53.491402+0000","last_became_peered":"2024-09-17T05:04:53.491402+0000","last_unstale":"2024-09-17T05:04:53.494978+0000","last_undegraded":"2024-09-17T05:04:53.494978+0000","last_fullsized":"2024-09-17T05:04:53.494978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:40.782888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463766+0000","last_change":"2024-09-17T05:04:52.463766+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463766+0000","last_undegraded":"2024-09-17T05:04:52.463766+0000","last_fullsized":"2024-09-17T05:04:52.463766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466787+0000","last_change":"2024-09-17T05:04:52.466787+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466787+0000","last_undegraded":"2024-09-17T05:04:52.466787+0000","last_fullsized":"2024-09-17T05:04:52.466787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464914+0000","last_change":"2024-09-17T05:04:52.464914+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464914+0000","last_undegraded":"2024-09-17T05:04:52.464914+0000","last_fullsized":"2024-09-17T05:04:52.464914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461563+0000","last_change":"2024-09-17T05:04:52.461563+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461563+0000","last_undegraded":"2024-09-17T05:04:52.461563+0000","last_fullsized":"2024-09-17T05:04:52.461563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467712+0000","last_change":"2024-09-17T05:04:52.467712+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467712+0000","last_undegraded":"2024-09-17T05:04:52.467712+0000","last_fullsized":"2024-09-17T05:04:52.467712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476554+0000","last_change":"2024-09-17T05:04:52.474787+0000","last_active":"2024-09-17T05:04:52.476554+0000","last_peered":"2024-09-17T05:04:52.476554+0000","last_clean":"2024-09-17T05:04:52.476554+0000","last_became_active":"2024-09-17T05:04:52.474407+0000","last_became_peered":"2024-09-17T05:04:52.474407+0000","last_unstale":"2024-09-17T05:04:52.476554+0000","last_undegraded":"2024-09-17T05:04:52.476554+0000","last_fullsized":"2024-09-17T05:04:52.476554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:40.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460299+0000","last_change":"2024-09-17T05:04:52.460299+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460299+0000","last_undegraded":"2024-09-17T05:04:52.460299+0000","last_fullsized":"2024-09-17T05:04:52.460299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489502+0000","last_change":"2024-09-17T05:04:52.488780+0000","last_active":"2024-09-17T05:04:52.489502+0000","last_peered":"2024-09-17T05:04:52.489502+0000","last_clean":"2024-09-17T05:04:52.489502+0000","last_became_active":"2024-09-17T05:04:52.488461+0000","last_became_peered":"2024-09-17T05:04:52.488461+0000","last_unstale":"2024-09-17T05:04:52.489502+0000","last_undegraded":"2024-09-17T05:04:52.489502+0000","last_fullsized":"2024-09-17T05:04:52.489502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:13.981678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477383+0000","last_change":"2024-09-17T05:04:52.477383+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477383+0000","last_undegraded":"2024-09-17T05:04:52.477383+0000","last_fullsized":"2024-09-17T05:04:52.477383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.481366+0000","last_change":"2024-09-17T05:04:52.479681+0000","last_active":"2024-09-17T05:04:52.481366+0000","last_peered":"2024-09-17T05:04:52.481366+0000","last_clean":"2024-09-17T05:04:52.481366+0000","last_became_active":"2024-09-17T05:04:52.477648+0000","last_became_peered":"2024-09-17T05:04:52.477648+0000","last_unstale":"2024-09-17T05:04:52.481366+0000","last_undegraded":"2024-09-17T05:04:52.481366+0000","last_fullsized":"2024-09-17T05:04:52.481366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:58.840632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466570+0000","last_change":"2024-09-17T05:04:52.466570+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466570+0000","last_undegraded":"2024-09-17T05:04:52.466570+0000","last_fullsized":"2024-09-17T05:04:52.466570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490600+0000","last_change":"2024-09-17T05:04:52.487079+0000","last_active":"2024-09-17T05:04:52.490600+0000","last_peered":"2024-09-17T05:04:52.490600+0000","last_clean":"2024-09-17T05:04:52.490600+0000","last_became_active":"2024-09-17T05:04:52.486771+0000","last_became_peered":"2024-09-17T05:04:52.486771+0000","last_unstale":"2024-09-17T05:04:52.490600+0000","last_undegraded":"2024-09-17T05:04:52.490600+0000","last_fullsized":"2024-09-17T05:04:52.490600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:51.755302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.498399+0000","last_change":"2024-09-17T05:04:53.496286+0000","last_active":"2024-09-17T05:04:53.498399+0000","last_peered":"2024-09-17T05:04:53.498399+0000","last_clean":"2024-09-17T05:04:53.498399+0000","last_became_active":"2024-09-17T05:04:53.495971+0000","last_became_peered":"2024-09-17T05:04:53.495971+0000","last_unstale":"2024-09-17T05:04:53.498399+0000","last_undegraded":"2024-09-17T05:04:53.498399+0000","last_fullsized":"2024-09-17T05:04:53.498399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:38.614768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484328+0000","last_change":"2024-09-17T05:04:52.482615+0000","last_active":"2024-09-17T05:04:52.484328+0000","last_peered":"2024-09-17T05:04:52.484328+0000","last_clean":"2024-09-17T05:04:52.484328+0000","last_became_active":"2024-09-17T05:04:52.482267+0000","last_became_peered":"2024-09-17T05:04:52.482267+0000","last_unstale":"2024-09-17T05:04:52.484328+0000","last_undegraded":"2024-09-17T05:04:52.484328+0000","last_fullsized":"2024-09-17T05:04:52.484328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468323+0000","last_change":"2024-09-17T05:04:52.468323+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468323+0000","last_undegraded":"2024-09-17T05:04:52.468323+0000","last_fullsized":"2024-09-17T05:04:52.468323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.495357+0000","last_change":"2024-09-17T05:04:52.476398+0000","last_active":"2024-09-17T05:04:53.495357+0000","last_peered":"2024-09-17T05:04:53.495357+0000","last_clean":"2024-09-17T05:04:53.495357+0000","last_became_active":"2024-09-17T05:04:52.476002+0000","last_became_peered":"2024-09-17T05:04:52.476002+0000","last_unstale":"2024-09-17T05:04:53.495357+0000","last_undegraded":"2024-09-17T05:04:53.495357+0000","last_fullsized":"2024-09-17T05:04:53.495357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:23.106184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459964+0000","last_change":"2024-09-17T05:04:52.459964+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459964+0000","last_undegraded":"2024-09-17T05:04:52.459964+0000","last_fullsized":"2024-09-17T05:04:52.459964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485583+0000","last_change":"2024-09-17T05:04:52.482041+0000","last_active":"2024-09-17T05:04:52.485583+0000","last_peered":"2024-09-17T05:04:52.485583+0000","last_clean":"2024-09-17T05:04:52.485583+0000","last_became_active":"2024-09-17T05:04:52.481707+0000","last_became_peered":"2024-09-17T05:04:52.481707+0000","last_unstale":"2024-09-17T05:04:52.485583+0000","last_undegraded":"2024-09-17T05:04:52.485583+0000","last_fullsized":"2024-09-17T05:04:52.485583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:32.340896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467432+0000","last_change":"2024-09-17T05:04:52.467432+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467432+0000","last_undegraded":"2024-09-17T05:04:52.467432+0000","last_fullsized":"2024-09-17T05:04:52.467432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484343+0000","last_change":"2024-09-17T05:04:52.480980+0000","last_active":"2024-09-17T05:04:52.484343+0000","last_peered":"2024-09-17T05:04:52.484343+0000","last_clean":"2024-09-17T05:04:52.484343+0000","last_became_active":"2024-09-17T05:04:52.477680+0000","last_became_peered":"2024-09-17T05:04:52.477680+0000","last_unstale":"2024-09-17T05:04:52.484343+0000","last_undegraded":"2024-09-17T05:04:52.484343+0000","last_fullsized":"2024-09-17T05:04:52.484343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:06.060774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463355+0000","last_change":"2024-09-17T05:04:52.463355+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463355+0000","last_undegraded":"2024-09-17T05:04:52.463355+0000","last_fullsized":"2024-09-17T05:04:52.463355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479996+0000","last_change":"2024-09-17T05:04:52.478749+0000","last_active":"2024-09-17T05:04:52.479996+0000","last_peered":"2024-09-17T05:04:52.479996+0000","last_clean":"2024-09-17T05:04:52.479996+0000","last_became_active":"2024-09-17T05:04:52.478315+0000","last_became_peered":"2024-09-17T05:04:52.478315+0000","last_unstale":"2024-09-17T05:04:52.479996+0000","last_undegraded":"2024-09-17T05:04:52.479996+0000","last_fullsized":"2024-09-17T05:04:52.479996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:51.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478423+0000","last_change":"2024-09-17T05:04:52.478423+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478423+0000","last_undegraded":"2024-09-17T05:04:52.478423+0000","last_fullsized":"2024-09-17T05:04:52.478423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485541+0000","last_change":"2024-09-17T05:04:52.480538+0000","last_active":"2024-09-17T05:04:52.485541+0000","last_peered":"2024-09-17T05:04:52.485541+0000","last_clean":"2024-09-17T05:04:52.485541+0000","last_became_active":"2024-09-17T05:04:52.480130+0000","last_became_peered":"2024-09-17T05:04:52.480130+0000","last_unstale":"2024-09-17T05:04:52.485541+0000","last_undegraded":"2024-09-17T05:04:52.485541+0000","last_fullsized":"2024-09-17T05:04:52.485541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:06.575023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.470865+0000","last_change":"2024-09-17T05:04:52.470865+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.470865+0000","last_undegraded":"2024-09-17T05:04:52.470865+0000","last_fullsized":"2024-09-17T05:04:52.470865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.486751+0000","last_change":"2024-09-17T05:04:52.484495+0000","last_active":"2024-09-17T05:04:52.486751+0000","last_peered":"2024-09-17T05:04:52.486751+0000","last_clean":"2024-09-17T05:04:52.486751+0000","last_became_active":"2024-09-17T05:04:52.483963+0000","last_became_peered":"2024-09-17T05:04:52.483963+0000","last_unstale":"2024-09-17T05:04:52.486751+0000","last_undegraded":"2024-09-17T05:04:52.486751+0000","last_fullsized":"2024-09-17T05:04:52.486751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:04.344757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459224+0000","last_change":"2024-09-17T05:04:52.459224+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459224+0000","last_undegraded":"2024-09-17T05:04:52.459224+0000","last_fullsized":"2024-09-17T05:04:52.459224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490846+0000","last_change":"2024-09-17T05:04:52.486319+0000","last_active":"2024-09-17T05:04:52.490846+0000","last_peered":"2024-09-17T05:04:52.490846+0000","last_clean":"2024-09-17T05:04:52.490846+0000","last_became_active":"2024-09-17T05:04:52.486037+0000","last_became_peered":"2024-09-17T05:04:52.486037+0000","last_unstale":"2024-09-17T05:04:52.490846+0000","last_undegraded":"2024-09-17T05:04:52.490846+0000","last_fullsized":"2024-09-17T05:04:52.490846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:36.202849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466161+0000","last_change":"2024-09-17T05:04:52.466161+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466161+0000","last_undegraded":"2024-09-17T05:04:52.466161+0000","last_fullsized":"2024-09-17T05:04:52.466161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479955+0000","last_change":"2024-09-17T05:04:52.478046+0000","last_active":"2024-09-17T05:04:52.479955+0000","last_peered":"2024-09-17T05:04:52.479955+0000","last_clean":"2024-09-17T05:04:52.479955+0000","last_became_active":"2024-09-17T05:04:52.477554+0000","last_became_peered":"2024-09-17T05:04:52.477554+0000","last_unstale":"2024-09-17T05:04:52.479955+0000","last_undegraded":"2024-09-17T05:04:52.479955+0000","last_fullsized":"2024-09-17T05:04:52.479955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:31.329683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463789+0000","last_change":"2024-09-17T05:04:52.463789+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463789+0000","last_undegraded":"2024-09-17T05:04:52.463789+0000","last_fullsized":"2024-09-17T05:04:52.463789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485667+0000","last_change":"2024-09-17T05:04:52.481898+0000","last_active":"2024-09-17T05:04:52.485667+0000","last_peered":"2024-09-17T05:04:52.485667+0000","last_clean":"2024-09-17T05:04:52.485667+0000","last_became_active":"2024-09-17T05:04:52.481562+0000","last_became_peered":"2024-09-17T05:04:52.481562+0000","last_unstale":"2024-09-17T05:04:52.485667+0000","last_undegraded":"2024-09-17T05:04:52.485667+0000","last_fullsized":"2024-09-17T05:04:52.485667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466746+0000","last_change":"2024-09-17T05:04:52.466746+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466746+0000","last_undegraded":"2024-09-17T05:04:52.466746+0000","last_fullsized":"2024-09-17T05:04:52.466746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483399+0000","last_change":"2024-09-17T05:04:52.479084+0000","last_active":"2024-09-17T05:04:52.483399+0000","last_peered":"2024-09-17T05:04:52.483399+0000","last_clean":"2024-09-17T05:04:52.483399+0000","last_became_active":"2024-09-17T05:04:52.478401+0000","last_became_peered":"2024-09-17T05:04:52.478401+0000","last_unstale":"2024-09-17T05:04:52.483399+0000","last_undegraded":"2024-09-17T05:04:52.483399+0000","last_fullsized":"2024-09-17T05:04:52.483399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:08.731176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485456+0000","last_change":"2024-09-17T05:04:53.480121+0000","last_active":"2024-09-17T05:04:53.485456+0000","last_peered":"2024-09-17T05:04:53.485456+0000","last_clean":"2024-09-17T05:04:53.485456+0000","last_became_active":"2024-09-17T05:04:53.479707+0000","last_became_peered":"2024-09-17T05:04:53.479707+0000","last_unstale":"2024-09-17T05:04:53.485456+0000","last_undegraded":"2024-09-17T05:04:53.485456+0000","last_fullsized":"2024-09-17T05:04:53.485456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:54.547283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483105+0000","last_change":"2024-09-17T05:04:52.479663+0000","last_active":"2024-09-17T05:04:52.483105+0000","last_peered":"2024-09-17T05:04:52.483105+0000","last_clean":"2024-09-17T05:04:52.483105+0000","last_became_active":"2024-09-17T05:04:52.479313+0000","last_became_peered":"2024-09-17T05:04:52.479313+0000","last_unstale":"2024-09-17T05:04:52.483105+0000","last_undegraded":"2024-09-17T05:04:52.483105+0000","last_fullsized":"2024-09-17T05:04:52.483105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:56.767884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.489610+0000","last_change":"2024-09-17T05:04:53.483761+0000","last_active":"2024-09-17T05:04:53.489610+0000","last_peered":"2024-09-17T05:04:53.489610+0000","last_clean":"2024-09-17T05:04:53.489610+0000","last_became_active":"2024-09-17T05:04:53.483036+0000","last_became_peered":"2024-09-17T05:04:53.483036+0000","last_unstale":"2024-09-17T05:04:53.489610+0000","last_undegraded":"2024-09-17T05:04:53.489610+0000","last_fullsized":"2024-09-17T05:04:53.489610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:05.593222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490673+0000","last_change":"2024-09-17T05:04:52.489855+0000","last_active":"2024-09-17T05:04:52.490673+0000","last_peered":"2024-09-17T05:04:52.490673+0000","last_clean":"2024-09-17T05:04:52.490673+0000","last_became_active":"2024-09-17T05:04:52.487437+0000","last_became_peered":"2024-09-17T05:04:52.487437+0000","last_unstale":"2024-09-17T05:04:52.490673+0000","last_undegraded":"2024-09-17T05:04:52.490673+0000","last_fullsized":"2024-09-17T05:04:52.490673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:39.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477891+0000","last_change":"2024-09-17T05:04:52.477891+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477891+0000","last_undegraded":"2024-09-17T05:04:52.477891+0000","last_fullsized":"2024-09-17T05:04:52.477891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490880+0000","last_change":"2024-09-17T05:04:52.489186+0000","last_active":"2024-09-17T05:04:52.490880+0000","last_peered":"2024-09-17T05:04:52.490880+0000","last_clean":"2024-09-17T05:04:52.490880+0000","last_became_active":"2024-09-17T05:04:52.484432+0000","last_became_peered":"2024-09-17T05:04:52.484432+0000","last_unstale":"2024-09-17T05:04:52.490880+0000","last_undegraded":"2024-09-17T05:04:52.490880+0000","last_fullsized":"2024-09-17T05:04:52.490880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.803390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462678+0000","last_change":"2024-09-17T05:04:52.462678+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462678+0000","last_undegraded":"2024-09-17T05:04:52.462678+0000","last_fullsized":"2024-09-17T05:04:52.462678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488159+0000","last_change":"2024-09-17T05:04:52.486351+0000","last_active":"2024-09-17T05:04:52.488159+0000","last_peered":"2024-09-17T05:04:52.488159+0000","last_clean":"2024-09-17T05:04:52.488159+0000","last_became_active":"2024-09-17T05:04:52.486015+0000","last_became_peered":"2024-09-17T05:04:52.486015+0000","last_unstale":"2024-09-17T05:04:52.488159+0000","last_undegraded":"2024-09-17T05:04:52.488159+0000","last_fullsized":"2024-09-17T05:04:52.488159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.544703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461102+0000","last_change":"2024-09-17T05:04:52.461102+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461102+0000","last_undegraded":"2024-09-17T05:04:52.461102+0000","last_fullsized":"2024-09-17T05:04:52.461102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488049+0000","last_change":"2024-09-17T05:04:52.484413+0000","last_active":"2024-09-17T05:04:52.488049+0000","last_peered":"2024-09-17T05:04:52.488049+0000","last_clean":"2024-09-17T05:04:52.488049+0000","last_became_active":"2024-09-17T05:04:52.483851+0000","last_became_peered":"2024-09-17T05:04:52.483851+0000","last_unstale":"2024-09-17T05:04:52.488049+0000","last_undegraded":"2024-09-17T05:04:52.488049+0000","last_fullsized":"2024-09-17T05:04:52.488049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:23.830173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469110+0000","last_change":"2024-09-17T05:04:52.469110+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469110+0000","last_undegraded":"2024-09-17T05:04:52.469110+0000","last_fullsized":"2024-09-17T05:04:52.469110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478334+0000","last_change":"2024-09-17T05:04:52.475068+0000","last_active":"2024-09-17T05:04:52.478334+0000","last_peered":"2024-09-17T05:04:52.478334+0000","last_clean":"2024-09-17T05:04:52.478334+0000","last_became_active":"2024-09-17T05:04:52.474737+0000","last_became_peered":"2024-09-17T05:04:52.474737+0000","last_unstale":"2024-09-17T05:04:52.478334+0000","last_undegraded":"2024-09-17T05:04:52.478334+0000","last_fullsized":"2024-09-17T05:04:52.478334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:29.910066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463931+0000","last_change":"2024-09-17T05:04:52.463931+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463931+0000","last_undegraded":"2024-09-17T05:04:52.463931+0000","last_fullsized":"2024-09-17T05:04:52.463931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485373+0000","last_change":"2024-09-17T05:04:52.479150+0000","last_active":"2024-09-17T05:04:52.485373+0000","last_peered":"2024-09-17T05:04:52.485373+0000","last_clean":"2024-09-17T05:04:52.485373+0000","last_became_active":"2024-09-17T05:04:52.476295+0000","last_became_peered":"2024-09-17T05:04:52.476295+0000","last_unstale":"2024-09-17T05:04:52.485373+0000","last_undegraded":"2024-09-17T05:04:52.485373+0000","last_fullsized":"2024-09-17T05:04:52.485373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:28.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466916+0000","last_change":"2024-09-17T05:04:52.466916+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466916+0000","last_undegraded":"2024-09-17T05:04:52.466916+0000","last_fullsized":"2024-09-17T05:04:52.466916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.483299+0000","last_change":"2024-09-17T05:04:52.482044+0000","last_active":"2024-09-17T05:04:53.483299+0000","last_peered":"2024-09-17T05:04:53.483299+0000","last_clean":"2024-09-17T05:04:53.483299+0000","last_became_active":"2024-09-17T05:04:52.481677+0000","last_became_peered":"2024-09-17T05:04:52.481677+0000","last_unstale":"2024-09-17T05:04:53.483299+0000","last_undegraded":"2024-09-17T05:04:53.483299+0000","last_fullsized":"2024-09-17T05:04:53.483299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:07.449960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463327+0000","last_change":"2024-09-17T05:04:52.463327+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463327+0000","last_undegraded":"2024-09-17T05:04:52.463327+0000","last_fullsized":"2024-09-17T05:04:52.463327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.492052+0000","last_change":"2024-09-17T05:04:52.491484+0000","last_active":"2024-09-17T05:04:52.492052+0000","last_peered":"2024-09-17T05:04:52.492052+0000","last_clean":"2024-09-17T05:04:52.492052+0000","last_became_active":"2024-09-17T05:04:52.490491+0000","last_became_peered":"2024-09-17T05:04:52.490491+0000","last_unstale":"2024-09-17T05:04:52.492052+0000","last_undegraded":"2024-09-17T05:04:52.492052+0000","last_fullsized":"2024-09-17T05:04:52.492052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:39.450191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485201+0000","last_change":"2024-09-17T05:04:53.480166+0000","last_active":"2024-09-17T05:04:53.485201+0000","last_peered":"2024-09-17T05:04:53.485201+0000","last_clean":"2024-09-17T05:04:53.485201+0000","last_became_active":"2024-09-17T05:04:53.479853+0000","last_became_peered":"2024-09-17T05:04:53.479853+0000","last_unstale":"2024-09-17T05:04:53.485201+0000","last_undegraded":"2024-09-17T05:04:53.485201+0000","last_fullsized":"2024-09-17T05:04:53.485201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:22.807028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.491140+0000","last_change":"2024-09-17T05:04:52.489527+0000","last_active":"2024-09-17T05:04:52.491140+0000","last_peered":"2024-09-17T05:04:52.491140+0000","last_clean":"2024-09-17T05:04:52.491140+0000","last_became_active":"2024-09-17T05:04:52.487078+0000","last_became_peered":"2024-09-17T05:04:52.487078+0000","last_unstale":"2024-09-17T05:04:52.491140+0000","last_undegraded":"2024-09-17T05:04:52.491140+0000","last_fullsized":"2024-09-17T05:04:52.491140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:25.546253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.502905+0000","last_change":"2024-09-17T05:04:53.499221+0000","last_active":"2024-09-17T05:04:53.502905+0000","last_peered":"2024-09-17T05:04:53.502905+0000","last_clean":"2024-09-17T05:04:53.502905+0000","last_became_active":"2024-09-17T05:04:53.498898+0000","last_became_peered":"2024-09-17T05:04:53.498898+0000","last_unstale":"2024-09-17T05:04:53.502905+0000","last_undegraded":"2024-09-17T05:04:53.502905+0000","last_fullsized":"2024-09-17T05:04:53.502905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:25.140293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483295+0000","last_change":"2024-09-17T05:04:52.480615+0000","last_active":"2024-09-17T05:04:52.483295+0000","last_peered":"2024-09-17T05:04:52.483295+0000","last_clean":"2024-09-17T05:04:52.483295+0000","last_became_active":"2024-09-17T05:04:52.479433+0000","last_became_peered":"2024-09-17T05:04:52.479433+0000","last_unstale":"2024-09-17T05:04:52.483295+0000","last_undegraded":"2024-09-17T05:04:52.483295+0000","last_fullsized":"2024-09-17T05:04:52.483295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462471+0000","last_change":"2024-09-17T05:04:52.462471+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462471+0000","last_undegraded":"2024-09-17T05:04:52.462471+0000","last_fullsized":"2024-09-17T05:04:52.462471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489406+0000","last_change":"2024-09-17T05:04:52.488025+0000","last_active":"2024-09-17T05:04:52.489406+0000","last_peered":"2024-09-17T05:04:52.489406+0000","last_clean":"2024-09-17T05:04:52.489406+0000","last_became_active":"2024-09-17T05:04:52.484280+0000","last_became_peered":"2024-09-17T05:04:52.484280+0000","last_unstale":"2024-09-17T05:04:52.489406+0000","last_undegraded":"2024-09-17T05:04:52.489406+0000","last_fullsized":"2024-09-17T05:04:52.489406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:10.565964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.458957+0000","last_change":"2024-09-17T05:04:52.458957+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.458957+0000","last_undegraded":"2024-09-17T05:04:52.458957+0000","last_fullsized":"2024-09-17T05:04:52.458957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478710+0000","last_change":"2024-09-17T05:04:52.474880+0000","last_active":"2024-09-17T05:04:52.478710+0000","last_peered":"2024-09-17T05:04:52.478710+0000","last_clean":"2024-09-17T05:04:52.478710+0000","last_became_active":"2024-09-17T05:04:52.474563+0000","last_became_peered":"2024-09-17T05:04:52.474563+0000","last_unstale":"2024-09-17T05:04:52.478710+0000","last_undegraded":"2024-09-17T05:04:52.478710+0000","last_fullsized":"2024-09-17T05:04:52.478710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:25.901198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466397+0000","last_change":"2024-09-17T05:04:52.466397+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466397+0000","last_undegraded":"2024-09-17T05:04:52.466397+0000","last_fullsized":"2024-09-17T05:04:52.466397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476614+0000","last_change":"2024-09-17T05:04:52.474440+0000","last_active":"2024-09-17T05:04:52.476614+0000","last_peered":"2024-09-17T05:04:52.476614+0000","last_clean":"2024-09-17T05:04:52.476614+0000","last_became_active":"2024-09-17T05:04:52.474097+0000","last_became_peered":"2024-09-17T05:04:52.474097+0000","last_unstale":"2024-09-17T05:04:52.476614+0000","last_undegraded":"2024-09-17T05:04:52.476614+0000","last_fullsized":"2024-09-17T05:04:52.476614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:16.647876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463352+0000","last_change":"2024-09-17T05:04:52.463352+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463352+0000","last_undegraded":"2024-09-17T05:04:52.463352+0000","last_fullsized":"2024-09-17T05:04:52.463352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.468511+0000","last_change":"2024-09-17T05:04:52.466652+0000","last_active":"2024-09-17T05:04:52.468511+0000","last_peered":"2024-09-17T05:04:52.468511+0000","last_clean":"2024-09-17T05:04:52.468511+0000","last_became_active":"2024-09-17T05:04:52.466240+0000","last_became_peered":"2024-09-17T05:04:52.466240+0000","last_unstale":"2024-09-17T05:04:52.468511+0000","last_undegraded":"2024-09-17T05:04:52.468511+0000","last_fullsized":"2024-09-17T05:04:52.468511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:25.791675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459124+0000","last_change":"2024-09-17T05:04:52.459124+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459124+0000","last_undegraded":"2024-09-17T05:04:52.459124+0000","last_fullsized":"2024-09-17T05:04:52.459124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476563+0000","last_change":"2024-09-17T05:04:52.472431+0000","last_active":"2024-09-17T05:04:52.476563+0000","last_peered":"2024-09-17T05:04:52.476563+0000","last_clean":"2024-09-17T05:04:52.476563+0000","last_became_active":"2024-09-17T05:04:52.472066+0000","last_became_peered":"2024-09-17T05:04:52.472066+0000","last_unstale":"2024-09-17T05:04:52.476563+0000","last_undegraded":"2024-09-17T05:04:52.476563+0000","last_fullsized":"2024-09-17T05:04:52.476563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:25.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463436+0000","last_change":"2024-09-17T05:04:52.463436+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463436+0000","last_undegraded":"2024-09-17T05:04:52.463436+0000","last_fullsized":"2024-09-17T05:04:52.463436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490829+0000","last_change":"2024-09-17T05:04:52.486302+0000","last_active":"2024-09-17T05:04:52.490829+0000","last_peered":"2024-09-17T05:04:52.490829+0000","last_clean":"2024-09-17T05:04:52.490829+0000","last_became_active":"2024-09-17T05:04:52.486002+0000","last_became_peered":"2024-09-17T05:04:52.486002+0000","last_unstale":"2024-09-17T05:04:52.490829+0000","last_undegraded":"2024-09-17T05:04:52.490829+0000","last_fullsized":"2024-09-17T05:04:52.490829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.458028+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:52.458028+0000","last_peered":"2024-09-17T05:04:52.458028+0000","last_clean":"2024-09-17T05:04:52.458028+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:52.458028+0000","last_undegraded":"2024-09-17T05:04:52.458028+0000","last_fullsized":"2024-09-17T05:04:52.458028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1040384,"data_stored":470209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36948,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334892,"statfs":{"total":96636764160,"available":96598929408,"internally_reserved":0,"allocated":1634304,"data_stored":1060577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":1067744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:04:57.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 -- 172.21.15.16:0/2299095140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0006ce90 msgr2=0x7f3f0006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 --2- 172.21.15.16:0/2299095140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0006ce90 0x7f3f0006d2a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3f00157010 tx=0x7f3ef801ca60 comp rx=0 tx=0).stop 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 -- 172.21.15.16:0/2299095140 shutdown_connections 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 --2- 172.21.15.16:0/2299095140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0006ce90 0x7f3f0006d2a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 --2- 172.21.15.16:0/2299095140 >> v2:172.21.15.104:3300/0 conn(0x7f3f0015ca90 0x7f3f0015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 -- 172.21.15.16:0/2299095140 >> 172.21.15.16:0/2299095140 conn(0x7f3f000d10c0 msgr2=0x7f3f000d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 -- 172.21.15.16:0/2299095140 shutdown_connections 2024-09-17T05:04:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 -- 172.21.15.16:0/2299095140 wait complete. 2024-09-17T05:04:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 Processor -- start 2024-09-17T05:04:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.880+0000 7f3f07600640 1 -- start start 2024-09-17T05:04:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006ce90 0x7f3f00136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f00073ec0 0x7f3f001454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 0x7f3f00146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00169e20 con 0x7f3f0015ca90 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00169ca0 con 0x7f3f0006ce90 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3f00169fa0 con 0x7f3f00073ec0 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 0x7f3f00146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 0x7f3f00146a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41238/0 (socket says 172.21.15.16:41238) 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 -- 172.21.15.16:0/1610438923 learned_addr learned my addr 172.21.15.16:0/1610438923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f04c00640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006ce90 0x7f3f00136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3effe00640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f00073ec0 0x7f3f001454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 -- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f00073ec0 msgr2=0x7f3f001454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f00073ec0 0x7f3f001454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 -- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006ce90 msgr2=0x7f3f00136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006ce90 0x7f3f00136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 -- 172.21.15.16:0/1610438923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0014d920 con 0x7f3f0015ca90 2024-09-17T05:04:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f05600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 0x7f3f00146a40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3ef801c4d0 tx=0x7f3ef8009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:57.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80b2040 con 0x7f3f0015ca90 2024-09-17T05:04:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ef80bb070 con 0x7f3f0015ca90 2024-09-17T05:04:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f001fc190 con 0x7f3f0015ca90 2024-09-17T05:04:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8004530 con 0x7f3f0015ca90 2024-09-17T05:04:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001fc710 con 0x7f3f0015ca90 2024-09-17T05:04:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.884+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0006d2a0 con 0x7f3f0015ca90 2024-09-17T05:04:57.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.888+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ef80046d0 con 0x7f3f0015ca90 2024-09-17T05:04:57.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.888+0000 7f3efcc00640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ee0073230 0x7f3ee0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:57.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.888+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f3ef812ecb0 con 0x7f3f0015ca90 2024-09-17T05:04:57.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.888+0000 7f3f04c00640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ee0073230 0x7f3ee0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:57.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.888+0000 7f3f04c00640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ee0073230 0x7f3ee0075740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007fd0 tx=0x7f3ef4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:57.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:57.892+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ef8016770 con 0x7f3f0015ca90 2024-09-17T05:04:58.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.064+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f00153090 con 0x7f3ee0073230 2024-09-17T05:04:58.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.068+0000 7f3efcc00640 1 -- 172.21.15.16:0/1610438923 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247353 (secure 0 0 0) 0x7f3f00153090 con 0x7f3ee0073230 2024-09-17T05:04:58.073 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:58.075 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:04:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ee0073230 msgr2=0x7f3ee0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ee0073230 0x7f3ee0075740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007fd0 tx=0x7f3ef4009040 comp rx=0 tx=0).stop 2024-09-17T05:04:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 msgr2=0x7f3f00146a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 0x7f3f00146a40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3ef801c4d0 tx=0x7f3ef8009200 comp rx=0 tx=0).stop 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 shutdown_connections 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ee0073230 0x7f3ee0075740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f0015ca90 0x7f3f00146a40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f00073ec0 0x7f3f001454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 --2- 172.21.15.16:0/1610438923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f0006ce90 0x7f3f00136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 >> 172.21.15.16:0/1610438923 conn(0x7f3f000d10c0 msgr2=0x7f3f0015db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 shutdown_connections 2024-09-17T05:04:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.072+0000 7f3f07600640 1 -- 172.21.15.16:0/1610438923 wait complete. 2024-09-17T05:04:58.096 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-17T05:04:57.520432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267088,"kb_used_data":9240,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707632,"statfs":{"total":773094113280,"available":772820615168,"internally_reserved":0,"allocated":9461760,"data_stored":4916240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467906+0000","last_change":"2024-09-17T05:04:52.467906+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467906+0000","last_undegraded":"2024-09-17T05:04:52.467906+0000","last_fullsized":"2024-09-17T05:04:52.467906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464969+0000","last_change":"2024-09-17T05:04:52.464969+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464969+0000","last_undegraded":"2024-09-17T05:04:52.464969+0000","last_fullsized":"2024-09-17T05:04:52.464969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468043+0000","last_change":"2024-09-17T05:04:52.468043+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468043+0000","last_undegraded":"2024-09-17T05:04:52.468043+0000","last_fullsized":"2024-09-17T05:04:52.468043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.472514+0000","last_change":"2024-09-17T05:04:52.472514+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.472514+0000","last_undegraded":"2024-09-17T05:04:52.472514+0000","last_fullsized":"2024-09-17T05:04:52.472514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465690+0000","last_change":"2024-09-17T05:04:52.465690+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465690+0000","last_undegraded":"2024-09-17T05:04:52.465690+0000","last_fullsized":"2024-09-17T05:04:52.465690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460417+0000","last_change":"2024-09-17T05:04:52.460417+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460417+0000","last_undegraded":"2024-09-17T05:04:52.460417+0000","last_fullsized":"2024-09-17T05:04:52.460417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468944+0000","last_change":"2024-09-17T05:04:52.468944+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468944+0000","last_undegraded":"2024-09-17T05:04:52.468944+0000","last_fullsized":"2024-09-17T05:04:52.468944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462576+0000","last_change":"2024-09-17T05:04:52.462576+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462576+0000","last_undegraded":"2024-09-17T05:04:52.462576+0000","last_fullsized":"2024-09-17T05:04:52.462576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.475893+0000","last_change":"2024-09-17T05:04:52.475893+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.475893+0000","last_undegraded":"2024-09-17T05:04:52.475893+0000","last_fullsized":"2024-09-17T05:04:52.475893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469663+0000","last_change":"2024-09-17T05:04:52.469663+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469663+0000","last_undegraded":"2024-09-17T05:04:52.469663+0000","last_fullsized":"2024-09-17T05:04:52.469663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478888+0000","last_change":"2024-09-17T05:04:52.478888+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478888+0000","last_undegraded":"2024-09-17T05:04:52.478888+0000","last_fullsized":"2024-09-17T05:04:52.478888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469566+0000","last_change":"2024-09-17T05:04:52.469566+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469566+0000","last_undegraded":"2024-09-17T05:04:52.469566+0000","last_fullsized":"2024-09-17T05:04:52.469566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.491308+0000","last_change":"2024-09-17T05:04:53.488884+0000","last_active":"2024-09-17T05:04:53.491308+0000","last_peered":"2024-09-17T05:04:53.491308+0000","last_clean":"2024-09-17T05:04:53.491308+0000","last_became_active":"2024-09-17T05:04:53.488567+0000","last_became_peered":"2024-09-17T05:04:53.488567+0000","last_unstale":"2024-09-17T05:04:53.491308+0000","last_undegraded":"2024-09-17T05:04:53.491308+0000","last_fullsized":"2024-09-17T05:04:53.491308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:16.452112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469017+0000","last_change":"2024-09-17T05:04:52.469017+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469017+0000","last_undegraded":"2024-09-17T05:04:52.469017+0000","last_fullsized":"2024-09-17T05:04:52.469017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467045+0000","last_change":"2024-09-17T05:04:52.467045+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467045+0000","last_undegraded":"2024-09-17T05:04:52.467045+0000","last_fullsized":"2024-09-17T05:04:52.467045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462060+0000","last_change":"2024-09-17T05:04:52.462060+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462060+0000","last_undegraded":"2024-09-17T05:04:52.462060+0000","last_fullsized":"2024-09-17T05:04:52.462060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459791+0000","last_change":"2024-09-17T05:04:52.459791+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459791+0000","last_undegraded":"2024-09-17T05:04:52.459791+0000","last_fullsized":"2024-09-17T05:04:52.459791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469183+0000","last_change":"2024-09-17T05:04:52.469183+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469183+0000","last_undegraded":"2024-09-17T05:04:52.469183+0000","last_fullsized":"2024-09-17T05:04:52.469183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465817+0000","last_change":"2024-09-17T05:04:52.465817+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465817+0000","last_undegraded":"2024-09-17T05:04:52.465817+0000","last_fullsized":"2024-09-17T05:04:52.465817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459213+0000","last_change":"2024-09-17T05:04:52.459213+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459213+0000","last_undegraded":"2024-09-17T05:04:52.459213+0000","last_fullsized":"2024-09-17T05:04:52.459213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464345+0000","last_change":"2024-09-17T05:04:52.464345+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464345+0000","last_undegraded":"2024-09-17T05:04:52.464345+0000","last_fullsized":"2024-09-17T05:04:52.464345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464433+0000","last_change":"2024-09-17T05:04:52.464433+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464433+0000","last_undegraded":"2024-09-17T05:04:52.464433+0000","last_fullsized":"2024-09-17T05:04:52.464433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464344+0000","last_change":"2024-09-17T05:04:52.464344+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464344+0000","last_undegraded":"2024-09-17T05:04:52.464344+0000","last_fullsized":"2024-09-17T05:04:52.464344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466713+0000","last_change":"2024-09-17T05:04:52.466713+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466713+0000","last_undegraded":"2024-09-17T05:04:52.466713+0000","last_fullsized":"2024-09-17T05:04:52.466713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464434+0000","last_change":"2024-09-17T05:04:52.464434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464434+0000","last_undegraded":"2024-09-17T05:04:52.464434+0000","last_fullsized":"2024-09-17T05:04:52.464434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463434+0000","last_change":"2024-09-17T05:04:52.463434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463434+0000","last_undegraded":"2024-09-17T05:04:52.463434+0000","last_fullsized":"2024-09-17T05:04:52.463434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467567+0000","last_change":"2024-09-17T05:04:52.467567+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467567+0000","last_undegraded":"2024-09-17T05:04:52.467567+0000","last_fullsized":"2024-09-17T05:04:52.467567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.494978+0000","last_change":"2024-09-17T05:04:53.491786+0000","last_active":"2024-09-17T05:04:53.494978+0000","last_peered":"2024-09-17T05:04:53.494978+0000","last_clean":"2024-09-17T05:04:53.494978+0000","last_became_active":"2024-09-17T05:04:53.491402+0000","last_became_peered":"2024-09-17T05:04:53.491402+0000","last_unstale":"2024-09-17T05:04:53.494978+0000","last_undegraded":"2024-09-17T05:04:53.494978+0000","last_fullsized":"2024-09-17T05:04:53.494978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:40.782888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463766+0000","last_change":"2024-09-17T05:04:52.463766+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463766+0000","last_undegraded":"2024-09-17T05:04:52.463766+0000","last_fullsized":"2024-09-17T05:04:52.463766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466787+0000","last_change":"2024-09-17T05:04:52.466787+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466787+0000","last_undegraded":"2024-09-17T05:04:52.466787+0000","last_fullsized":"2024-09-17T05:04:52.466787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464914+0000","last_change":"2024-09-17T05:04:52.464914+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464914+0000","last_undegraded":"2024-09-17T05:04:52.464914+0000","last_fullsized":"2024-09-17T05:04:52.464914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461563+0000","last_change":"2024-09-17T05:04:52.461563+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461563+0000","last_undegraded":"2024-09-17T05:04:52.461563+0000","last_fullsized":"2024-09-17T05:04:52.461563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467712+0000","last_change":"2024-09-17T05:04:52.467712+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467712+0000","last_undegraded":"2024-09-17T05:04:52.467712+0000","last_fullsized":"2024-09-17T05:04:52.467712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476554+0000","last_change":"2024-09-17T05:04:52.474787+0000","last_active":"2024-09-17T05:04:52.476554+0000","last_peered":"2024-09-17T05:04:52.476554+0000","last_clean":"2024-09-17T05:04:52.476554+0000","last_became_active":"2024-09-17T05:04:52.474407+0000","last_became_peered":"2024-09-17T05:04:52.474407+0000","last_unstale":"2024-09-17T05:04:52.476554+0000","last_undegraded":"2024-09-17T05:04:52.476554+0000","last_fullsized":"2024-09-17T05:04:52.476554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:40.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460299+0000","last_change":"2024-09-17T05:04:52.460299+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460299+0000","last_undegraded":"2024-09-17T05:04:52.460299+0000","last_fullsized":"2024-09-17T05:04:52.460299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489502+0000","last_change":"2024-09-17T05:04:52.488780+0000","last_active":"2024-09-17T05:04:52.489502+0000","last_peered":"2024-09-17T05:04:52.489502+0000","last_clean":"2024-09-17T05:04:52.489502+0000","last_became_active":"2024-09-17T05:04:52.488461+0000","last_became_peered":"2024-09-17T05:04:52.488461+0000","last_unstale":"2024-09-17T05:04:52.489502+0000","last_undegraded":"2024-09-17T05:04:52.489502+0000","last_fullsized":"2024-09-17T05:04:52.489502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:13.981678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477383+0000","last_change":"2024-09-17T05:04:52.477383+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477383+0000","last_undegraded":"2024-09-17T05:04:52.477383+0000","last_fullsized":"2024-09-17T05:04:52.477383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.481366+0000","last_change":"2024-09-17T05:04:52.479681+0000","last_active":"2024-09-17T05:04:52.481366+0000","last_peered":"2024-09-17T05:04:52.481366+0000","last_clean":"2024-09-17T05:04:52.481366+0000","last_became_active":"2024-09-17T05:04:52.477648+0000","last_became_peered":"2024-09-17T05:04:52.477648+0000","last_unstale":"2024-09-17T05:04:52.481366+0000","last_undegraded":"2024-09-17T05:04:52.481366+0000","last_fullsized":"2024-09-17T05:04:52.481366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:58.840632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466570+0000","last_change":"2024-09-17T05:04:52.466570+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466570+0000","last_undegraded":"2024-09-17T05:04:52.466570+0000","last_fullsized":"2024-09-17T05:04:52.466570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490600+0000","last_change":"2024-09-17T05:04:52.487079+0000","last_active":"2024-09-17T05:04:52.490600+0000","last_peered":"2024-09-17T05:04:52.490600+0000","last_clean":"2024-09-17T05:04:52.490600+0000","last_became_active":"2024-09-17T05:04:52.486771+0000","last_became_peered":"2024-09-17T05:04:52.486771+0000","last_unstale":"2024-09-17T05:04:52.490600+0000","last_undegraded":"2024-09-17T05:04:52.490600+0000","last_fullsized":"2024-09-17T05:04:52.490600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:51.755302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.498399+0000","last_change":"2024-09-17T05:04:53.496286+0000","last_active":"2024-09-17T05:04:53.498399+0000","last_peered":"2024-09-17T05:04:53.498399+0000","last_clean":"2024-09-17T05:04:53.498399+0000","last_became_active":"2024-09-17T05:04:53.495971+0000","last_became_peered":"2024-09-17T05:04:53.495971+0000","last_unstale":"2024-09-17T05:04:53.498399+0000","last_undegraded":"2024-09-17T05:04:53.498399+0000","last_fullsized":"2024-09-17T05:04:53.498399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:38.614768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484328+0000","last_change":"2024-09-17T05:04:52.482615+0000","last_active":"2024-09-17T05:04:52.484328+0000","last_peered":"2024-09-17T05:04:52.484328+0000","last_clean":"2024-09-17T05:04:52.484328+0000","last_became_active":"2024-09-17T05:04:52.482267+0000","last_became_peered":"2024-09-17T05:04:52.482267+0000","last_unstale":"2024-09-17T05:04:52.484328+0000","last_undegraded":"2024-09-17T05:04:52.484328+0000","last_fullsized":"2024-09-17T05:04:52.484328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468323+0000","last_change":"2024-09-17T05:04:52.468323+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468323+0000","last_undegraded":"2024-09-17T05:04:52.468323+0000","last_fullsized":"2024-09-17T05:04:52.468323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.495357+0000","last_change":"2024-09-17T05:04:52.476398+0000","last_active":"2024-09-17T05:04:53.495357+0000","last_peered":"2024-09-17T05:04:53.495357+0000","last_clean":"2024-09-17T05:04:53.495357+0000","last_became_active":"2024-09-17T05:04:52.476002+0000","last_became_peered":"2024-09-17T05:04:52.476002+0000","last_unstale":"2024-09-17T05:04:53.495357+0000","last_undegraded":"2024-09-17T05:04:53.495357+0000","last_fullsized":"2024-09-17T05:04:53.495357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:23.106184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459964+0000","last_change":"2024-09-17T05:04:52.459964+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459964+0000","last_undegraded":"2024-09-17T05:04:52.459964+0000","last_fullsized":"2024-09-17T05:04:52.459964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485583+0000","last_change":"2024-09-17T05:04:52.482041+0000","last_active":"2024-09-17T05:04:52.485583+0000","last_peered":"2024-09-17T05:04:52.485583+0000","last_clean":"2024-09-17T05:04:52.485583+0000","last_became_active":"2024-09-17T05:04:52.481707+0000","last_became_peered":"2024-09-17T05:04:52.481707+0000","last_unstale":"2024-09-17T05:04:52.485583+0000","last_undegraded":"2024-09-17T05:04:52.485583+0000","last_fullsized":"2024-09-17T05:04:52.485583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:32.340896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467432+0000","last_change":"2024-09-17T05:04:52.467432+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467432+0000","last_undegraded":"2024-09-17T05:04:52.467432+0000","last_fullsized":"2024-09-17T05:04:52.467432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484343+0000","last_change":"2024-09-17T05:04:52.480980+0000","last_active":"2024-09-17T05:04:52.484343+0000","last_peered":"2024-09-17T05:04:52.484343+0000","last_clean":"2024-09-17T05:04:52.484343+0000","last_became_active":"2024-09-17T05:04:52.477680+0000","last_became_peered":"2024-09-17T05:04:52.477680+0000","last_unstale":"2024-09-17T05:04:52.484343+0000","last_undegraded":"2024-09-17T05:04:52.484343+0000","last_fullsized":"2024-09-17T05:04:52.484343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:06.060774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463355+0000","last_change":"2024-09-17T05:04:52.463355+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463355+0000","last_undegraded":"2024-09-17T05:04:52.463355+0000","last_fullsized":"2024-09-17T05:04:52.463355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479996+0000","last_change":"2024-09-17T05:04:52.478749+0000","last_active":"2024-09-17T05:04:52.479996+0000","last_peered":"2024-09-17T05:04:52.479996+0000","last_clean":"2024-09-17T05:04:52.479996+0000","last_became_active":"2024-09-17T05:04:52.478315+0000","last_became_peered":"2024-09-17T05:04:52.478315+0000","last_unstale":"2024-09-17T05:04:52.479996+0000","last_undegraded":"2024-09-17T05:04:52.479996+0000","last_fullsized":"2024-09-17T05:04:52.479996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:51.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478423+0000","last_change":"2024-09-17T05:04:52.478423+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478423+0000","last_undegraded":"2024-09-17T05:04:52.478423+0000","last_fullsized":"2024-09-17T05:04:52.478423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485541+0000","last_change":"2024-09-17T05:04:52.480538+0000","last_active":"2024-09-17T05:04:52.485541+0000","last_peered":"2024-09-17T05:04:52.485541+0000","last_clean":"2024-09-17T05:04:52.485541+0000","last_became_active":"2024-09-17T05:04:52.480130+0000","last_became_peered":"2024-09-17T05:04:52.480130+0000","last_unstale":"2024-09-17T05:04:52.485541+0000","last_undegraded":"2024-09-17T05:04:52.485541+0000","last_fullsized":"2024-09-17T05:04:52.485541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:06.575023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.470865+0000","last_change":"2024-09-17T05:04:52.470865+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.470865+0000","last_undegraded":"2024-09-17T05:04:52.470865+0000","last_fullsized":"2024-09-17T05:04:52.470865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.486751+0000","last_change":"2024-09-17T05:04:52.484495+0000","last_active":"2024-09-17T05:04:52.486751+0000","last_peered":"2024-09-17T05:04:52.486751+0000","last_clean":"2024-09-17T05:04:52.486751+0000","last_became_active":"2024-09-17T05:04:52.483963+0000","last_became_peered":"2024-09-17T05:04:52.483963+0000","last_unstale":"2024-09-17T05:04:52.486751+0000","last_undegraded":"2024-09-17T05:04:52.486751+0000","last_fullsized":"2024-09-17T05:04:52.486751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:04.344757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459224+0000","last_change":"2024-09-17T05:04:52.459224+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459224+0000","last_undegraded":"2024-09-17T05:04:52.459224+0000","last_fullsized":"2024-09-17T05:04:52.459224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490846+0000","last_change":"2024-09-17T05:04:52.486319+0000","last_active":"2024-09-17T05:04:52.490846+0000","last_peered":"2024-09-17T05:04:52.490846+0000","last_clean":"2024-09-17T05:04:52.490846+0000","last_became_active":"2024-09-17T05:04:52.486037+0000","last_became_peered":"2024-09-17T05:04:52.486037+0000","last_unstale":"2024-09-17T05:04:52.490846+0000","last_undegraded":"2024-09-17T05:04:52.490846+0000","last_fullsized":"2024-09-17T05:04:52.490846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:36.202849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466161+0000","last_change":"2024-09-17T05:04:52.466161+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466161+0000","last_undegraded":"2024-09-17T05:04:52.466161+0000","last_fullsized":"2024-09-17T05:04:52.466161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479955+0000","last_change":"2024-09-17T05:04:52.478046+0000","last_active":"2024-09-17T05:04:52.479955+0000","last_peered":"2024-09-17T05:04:52.479955+0000","last_clean":"2024-09-17T05:04:52.479955+0000","last_became_active":"2024-09-17T05:04:52.477554+0000","last_became_peered":"2024-09-17T05:04:52.477554+0000","last_unstale":"2024-09-17T05:04:52.479955+0000","last_undegraded":"2024-09-17T05:04:52.479955+0000","last_fullsized":"2024-09-17T05:04:52.479955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:31.329683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463789+0000","last_change":"2024-09-17T05:04:52.463789+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463789+0000","last_undegraded":"2024-09-17T05:04:52.463789+0000","last_fullsized":"2024-09-17T05:04:52.463789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485667+0000","last_change":"2024-09-17T05:04:52.481898+0000","last_active":"2024-09-17T05:04:52.485667+0000","last_peered":"2024-09-17T05:04:52.485667+0000","last_clean":"2024-09-17T05:04:52.485667+0000","last_became_active":"2024-09-17T05:04:52.481562+0000","last_became_peered":"2024-09-17T05:04:52.481562+0000","last_unstale":"2024-09-17T05:04:52.485667+0000","last_undegraded":"2024-09-17T05:04:52.485667+0000","last_fullsized":"2024-09-17T05:04:52.485667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466746+0000","last_change":"2024-09-17T05:04:52.466746+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466746+0000","last_undegraded":"2024-09-17T05:04:52.466746+0000","last_fullsized":"2024-09-17T05:04:52.466746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483399+0000","last_change":"2024-09-17T05:04:52.479084+0000","last_active":"2024-09-17T05:04:52.483399+0000","last_peered":"2024-09-17T05:04:52.483399+0000","last_clean":"2024-09-17T05:04:52.483399+0000","last_became_active":"2024-09-17T05:04:52.478401+0000","last_became_peered":"2024-09-17T05:04:52.478401+0000","last_unstale":"2024-09-17T05:04:52.483399+0000","last_undegraded":"2024-09-17T05:04:52.483399+0000","last_fullsized":"2024-09-17T05:04:52.483399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:08.731176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485456+0000","last_change":"2024-09-17T05:04:53.480121+0000","last_active":"2024-09-17T05:04:53.485456+0000","last_peered":"2024-09-17T05:04:53.485456+0000","last_clean":"2024-09-17T05:04:53.485456+0000","last_became_active":"2024-09-17T05:04:53.479707+0000","last_became_peered":"2024-09-17T05:04:53.479707+0000","last_unstale":"2024-09-17T05:04:53.485456+0000","last_undegraded":"2024-09-17T05:04:53.485456+0000","last_fullsized":"2024-09-17T05:04:53.485456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:54.547283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483105+0000","last_change":"2024-09-17T05:04:52.479663+0000","last_active":"2024-09-17T05:04:52.483105+0000","last_peered":"2024-09-17T05:04:52.483105+0000","last_clean":"2024-09-17T05:04:52.483105+0000","last_became_active":"2024-09-17T05:04:52.479313+0000","last_became_peered":"2024-09-17T05:04:52.479313+0000","last_unstale":"2024-09-17T05:04:52.483105+0000","last_undegraded":"2024-09-17T05:04:52.483105+0000","last_fullsized":"2024-09-17T05:04:52.483105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:56.767884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.489610+0000","last_change":"2024-09-17T05:04:53.483761+0000","last_active":"2024-09-17T05:04:53.489610+0000","last_peered":"2024-09-17T05:04:53.489610+0000","last_clean":"2024-09-17T05:04:53.489610+0000","last_became_active":"2024-09-17T05:04:53.483036+0000","last_became_peered":"2024-09-17T05:04:53.483036+0000","last_unstale":"2024-09-17T05:04:53.489610+0000","last_undegraded":"2024-09-17T05:04:53.489610+0000","last_fullsized":"2024-09-17T05:04:53.489610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:05.593222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490673+0000","last_change":"2024-09-17T05:04:52.489855+0000","last_active":"2024-09-17T05:04:52.490673+0000","last_peered":"2024-09-17T05:04:52.490673+0000","last_clean":"2024-09-17T05:04:52.490673+0000","last_became_active":"2024-09-17T05:04:52.487437+0000","last_became_peered":"2024-09-17T05:04:52.487437+0000","last_unstale":"2024-09-17T05:04:52.490673+0000","last_undegraded":"2024-09-17T05:04:52.490673+0000","last_fullsized":"2024-09-17T05:04:52.490673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:39.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477891+0000","last_change":"2024-09-17T05:04:52.477891+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477891+0000","last_undegraded":"2024-09-17T05:04:52.477891+0000","last_fullsized":"2024-09-17T05:04:52.477891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490880+0000","last_change":"2024-09-17T05:04:52.489186+0000","last_active":"2024-09-17T05:04:52.490880+0000","last_peered":"2024-09-17T05:04:52.490880+0000","last_clean":"2024-09-17T05:04:52.490880+0000","last_became_active":"2024-09-17T05:04:52.484432+0000","last_became_peered":"2024-09-17T05:04:52.484432+0000","last_unstale":"2024-09-17T05:04:52.490880+0000","last_undegraded":"2024-09-17T05:04:52.490880+0000","last_fullsized":"2024-09-17T05:04:52.490880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.803390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462678+0000","last_change":"2024-09-17T05:04:52.462678+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462678+0000","last_undegraded":"2024-09-17T05:04:52.462678+0000","last_fullsized":"2024-09-17T05:04:52.462678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488159+0000","last_change":"2024-09-17T05:04:52.486351+0000","last_active":"2024-09-17T05:04:52.488159+0000","last_peered":"2024-09-17T05:04:52.488159+0000","last_clean":"2024-09-17T05:04:52.488159+0000","last_became_active":"2024-09-17T05:04:52.486015+0000","last_became_peered":"2024-09-17T05:04:52.486015+0000","last_unstale":"2024-09-17T05:04:52.488159+0000","last_undegraded":"2024-09-17T05:04:52.488159+0000","last_fullsized":"2024-09-17T05:04:52.488159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.544703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461102+0000","last_change":"2024-09-17T05:04:52.461102+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461102+0000","last_undegraded":"2024-09-17T05:04:52.461102+0000","last_fullsized":"2024-09-17T05:04:52.461102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488049+0000","last_change":"2024-09-17T05:04:52.484413+0000","last_active":"2024-09-17T05:04:52.488049+0000","last_peered":"2024-09-17T05:04:52.488049+0000","last_clean":"2024-09-17T05:04:52.488049+0000","last_became_active":"2024-09-17T05:04:52.483851+0000","last_became_peered":"2024-09-17T05:04:52.483851+0000","last_unstale":"2024-09-17T05:04:52.488049+0000","last_undegraded":"2024-09-17T05:04:52.488049+0000","last_fullsized":"2024-09-17T05:04:52.488049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:23.830173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469110+0000","last_change":"2024-09-17T05:04:52.469110+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469110+0000","last_undegraded":"2024-09-17T05:04:52.469110+0000","last_fullsized":"2024-09-17T05:04:52.469110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478334+0000","last_change":"2024-09-17T05:04:52.475068+0000","last_active":"2024-09-17T05:04:52.478334+0000","last_peered":"2024-09-17T05:04:52.478334+0000","last_clean":"2024-09-17T05:04:52.478334+0000","last_became_active":"2024-09-17T05:04:52.474737+0000","last_became_peered":"2024-09-17T05:04:52.474737+0000","last_unstale":"2024-09-17T05:04:52.478334+0000","last_undegraded":"2024-09-17T05:04:52.478334+0000","last_fullsized":"2024-09-17T05:04:52.478334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:29.910066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463931+0000","last_change":"2024-09-17T05:04:52.463931+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463931+0000","last_undegraded":"2024-09-17T05:04:52.463931+0000","last_fullsized":"2024-09-17T05:04:52.463931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485373+0000","last_change":"2024-09-17T05:04:52.479150+0000","last_active":"2024-09-17T05:04:52.485373+0000","last_peered":"2024-09-17T05:04:52.485373+0000","last_clean":"2024-09-17T05:04:52.485373+0000","last_became_active":"2024-09-17T05:04:52.476295+0000","last_became_peered":"2024-09-17T05:04:52.476295+0000","last_unstale":"2024-09-17T05:04:52.485373+0000","last_undegraded":"2024-09-17T05:04:52.485373+0000","last_fullsized":"2024-09-17T05:04:52.485373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:28.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466916+0000","last_change":"2024-09-17T05:04:52.466916+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466916+0000","last_undegraded":"2024-09-17T05:04:52.466916+0000","last_fullsized":"2024-09-17T05:04:52.466916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.483299+0000","last_change":"2024-09-17T05:04:52.482044+0000","last_active":"2024-09-17T05:04:53.483299+0000","last_peered":"2024-09-17T05:04:53.483299+0000","last_clean":"2024-09-17T05:04:53.483299+0000","last_became_active":"2024-09-17T05:04:52.481677+0000","last_became_peered":"2024-09-17T05:04:52.481677+0000","last_unstale":"2024-09-17T05:04:53.483299+0000","last_undegraded":"2024-09-17T05:04:53.483299+0000","last_fullsized":"2024-09-17T05:04:53.483299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:07.449960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463327+0000","last_change":"2024-09-17T05:04:52.463327+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463327+0000","last_undegraded":"2024-09-17T05:04:52.463327+0000","last_fullsized":"2024-09-17T05:04:52.463327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.492052+0000","last_change":"2024-09-17T05:04:52.491484+0000","last_active":"2024-09-17T05:04:52.492052+0000","last_peered":"2024-09-17T05:04:52.492052+0000","last_clean":"2024-09-17T05:04:52.492052+0000","last_became_active":"2024-09-17T05:04:52.490491+0000","last_became_peered":"2024-09-17T05:04:52.490491+0000","last_unstale":"2024-09-17T05:04:52.492052+0000","last_undegraded":"2024-09-17T05:04:52.492052+0000","last_fullsized":"2024-09-17T05:04:52.492052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:39.450191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485201+0000","last_change":"2024-09-17T05:04:53.480166+0000","last_active":"2024-09-17T05:04:53.485201+0000","last_peered":"2024-09-17T05:04:53.485201+0000","last_clean":"2024-09-17T05:04:53.485201+0000","last_became_active":"2024-09-17T05:04:53.479853+0000","last_became_peered":"2024-09-17T05:04:53.479853+0000","last_unstale":"2024-09-17T05:04:53.485201+0000","last_undegraded":"2024-09-17T05:04:53.485201+0000","last_fullsized":"2024-09-17T05:04:53.485201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:22.807028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.491140+0000","last_change":"2024-09-17T05:04:52.489527+0000","last_active":"2024-09-17T05:04:52.491140+0000","last_peered":"2024-09-17T05:04:52.491140+0000","last_clean":"2024-09-17T05:04:52.491140+0000","last_became_active":"2024-09-17T05:04:52.487078+0000","last_became_peered":"2024-09-17T05:04:52.487078+0000","last_unstale":"2024-09-17T05:04:52.491140+0000","last_undegraded":"2024-09-17T05:04:52.491140+0000","last_fullsized":"2024-09-17T05:04:52.491140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:25.546253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.502905+0000","last_change":"2024-09-17T05:04:53.499221+0000","last_active":"2024-09-17T05:04:53.502905+0000","last_peered":"2024-09-17T05:04:53.502905+0000","last_clean":"2024-09-17T05:04:53.502905+0000","last_became_active":"2024-09-17T05:04:53.498898+0000","last_became_peered":"2024-09-17T05:04:53.498898+0000","last_unstale":"2024-09-17T05:04:53.502905+0000","last_undegraded":"2024-09-17T05:04:53.502905+0000","last_fullsized":"2024-09-17T05:04:53.502905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:25.140293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483295+0000","last_change":"2024-09-17T05:04:52.480615+0000","last_active":"2024-09-17T05:04:52.483295+0000","last_peered":"2024-09-17T05:04:52.483295+0000","last_clean":"2024-09-17T05:04:52.483295+0000","last_became_active":"2024-09-17T05:04:52.479433+0000","last_became_peered":"2024-09-17T05:04:52.479433+0000","last_unstale":"2024-09-17T05:04:52.483295+0000","last_undegraded":"2024-09-17T05:04:52.483295+0000","last_fullsized":"2024-09-17T05:04:52.483295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462471+0000","last_change":"2024-09-17T05:04:52.462471+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462471+0000","last_undegraded":"2024-09-17T05:04:52.462471+0000","last_fullsized":"2024-09-17T05:04:52.462471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489406+0000","last_change":"2024-09-17T05:04:52.488025+0000","last_active":"2024-09-17T05:04:52.489406+0000","last_peered":"2024-09-17T05:04:52.489406+0000","last_clean":"2024-09-17T05:04:52.489406+0000","last_became_active":"2024-09-17T05:04:52.484280+0000","last_became_peered":"2024-09-17T05:04:52.484280+0000","last_unstale":"2024-09-17T05:04:52.489406+0000","last_undegraded":"2024-09-17T05:04:52.489406+0000","last_fullsized":"2024-09-17T05:04:52.489406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:10.565964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.458957+0000","last_change":"2024-09-17T05:04:52.458957+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.458957+0000","last_undegraded":"2024-09-17T05:04:52.458957+0000","last_fullsized":"2024-09-17T05:04:52.458957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478710+0000","last_change":"2024-09-17T05:04:52.474880+0000","last_active":"2024-09-17T05:04:52.478710+0000","last_peered":"2024-09-17T05:04:52.478710+0000","last_clean":"2024-09-17T05:04:52.478710+0000","last_became_active":"2024-09-17T05:04:52.474563+0000","last_became_peered":"2024-09-17T05:04:52.474563+0000","last_unstale":"2024-09-17T05:04:52.478710+0000","last_undegraded":"2024-09-17T05:04:52.478710+0000","last_fullsized":"2024-09-17T05:04:52.478710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:25.901198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466397+0000","last_change":"2024-09-17T05:04:52.466397+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466397+0000","last_undegraded":"2024-09-17T05:04:52.466397+0000","last_fullsized":"2024-09-17T05:04:52.466397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476614+0000","last_change":"2024-09-17T05:04:52.474440+0000","last_active":"2024-09-17T05:04:52.476614+0000","last_peered":"2024-09-17T05:04:52.476614+0000","last_clean":"2024-09-17T05:04:52.476614+0000","last_became_active":"2024-09-17T05:04:52.474097+0000","last_became_peered":"2024-09-17T05:04:52.474097+0000","last_unstale":"2024-09-17T05:04:52.476614+0000","last_undegraded":"2024-09-17T05:04:52.476614+0000","last_fullsized":"2024-09-17T05:04:52.476614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:16.647876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463352+0000","last_change":"2024-09-17T05:04:52.463352+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463352+0000","last_undegraded":"2024-09-17T05:04:52.463352+0000","last_fullsized":"2024-09-17T05:04:52.463352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.468511+0000","last_change":"2024-09-17T05:04:52.466652+0000","last_active":"2024-09-17T05:04:52.468511+0000","last_peered":"2024-09-17T05:04:52.468511+0000","last_clean":"2024-09-17T05:04:52.468511+0000","last_became_active":"2024-09-17T05:04:52.466240+0000","last_became_peered":"2024-09-17T05:04:52.466240+0000","last_unstale":"2024-09-17T05:04:52.468511+0000","last_undegraded":"2024-09-17T05:04:52.468511+0000","last_fullsized":"2024-09-17T05:04:52.468511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:25.791675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459124+0000","last_change":"2024-09-17T05:04:52.459124+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459124+0000","last_undegraded":"2024-09-17T05:04:52.459124+0000","last_fullsized":"2024-09-17T05:04:52.459124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476563+0000","last_change":"2024-09-17T05:04:52.472431+0000","last_active":"2024-09-17T05:04:52.476563+0000","last_peered":"2024-09-17T05:04:52.476563+0000","last_clean":"2024-09-17T05:04:52.476563+0000","last_became_active":"2024-09-17T05:04:52.472066+0000","last_became_peered":"2024-09-17T05:04:52.472066+0000","last_unstale":"2024-09-17T05:04:52.476563+0000","last_undegraded":"2024-09-17T05:04:52.476563+0000","last_fullsized":"2024-09-17T05:04:52.476563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:25.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463436+0000","last_change":"2024-09-17T05:04:52.463436+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463436+0000","last_undegraded":"2024-09-17T05:04:52.463436+0000","last_fullsized":"2024-09-17T05:04:52.463436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490829+0000","last_change":"2024-09-17T05:04:52.486302+0000","last_active":"2024-09-17T05:04:52.490829+0000","last_peered":"2024-09-17T05:04:52.490829+0000","last_clean":"2024-09-17T05:04:52.490829+0000","last_became_active":"2024-09-17T05:04:52.486002+0000","last_became_peered":"2024-09-17T05:04:52.486002+0000","last_unstale":"2024-09-17T05:04:52.490829+0000","last_undegraded":"2024-09-17T05:04:52.490829+0000","last_fullsized":"2024-09-17T05:04:52.490829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.458028+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:52.458028+0000","last_peered":"2024-09-17T05:04:52.458028+0000","last_clean":"2024-09-17T05:04:52.458028+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:52.458028+0000","last_undegraded":"2024-09-17T05:04:52.458028+0000","last_fullsized":"2024-09-17T05:04:52.458028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1040384,"data_stored":470209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36948,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334892,"statfs":{"total":96636764160,"available":96598929408,"internally_reserved":0,"allocated":1634304,"data_stored":1060577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":1067744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:04:58.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:04:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 -- 172.21.15.16:0/2056582026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7006ce90 msgr2=0x7f7b700cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 --2- 172.21.15.16:0/2056582026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7006ce90 0x7f7b700cf320 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7b700ce5c0 tx=0x7f7b6401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 -- 172.21.15.16:0/2056582026 shutdown_connections 2024-09-17T05:04:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 --2- 172.21.15.16:0/2056582026 >> v2:172.21.15.104:3300/0 conn(0x7f7b700cf860 0x7f7b7015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 --2- 172.21.15.16:0/2056582026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b7006ce90 0x7f7b700cf320 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 -- 172.21.15.16:0/2056582026 >> 172.21.15.16:0/2056582026 conn(0x7f7b700d0860 msgr2=0x7f7b700d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 -- 172.21.15.16:0/2056582026 shutdown_connections 2024-09-17T05:04:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.296+0000 7f7b77a00640 1 -- 172.21.15.16:0/2056582026 wait complete. 2024-09-17T05:04:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 Processor -- start 2024-09-17T05:04:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 -- start start 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b7006ce90 0x7f7b701f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 0x7f7b70144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b7015f580 0x7f7b7015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701689e0 con 0x7f7b700cf860 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b700b5a80 con 0x7f7b7015f580 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b77a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7b70168b60 con 0x7f7b7006ce90 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 0x7f7b70144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 0x7f7b70144fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41242/0 (socket says 172.21.15.16:41242) 2024-09-17T05:04:58.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 -- 172.21.15.16:0/4005129079 learned_addr learned my addr 172.21.15.16:0/4005129079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:58.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b75a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b7015f580 0x7f7b7015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b75000640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b7006ce90 0x7f7b701f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 -- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b7006ce90 msgr2=0x7f7b701f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b7006ce90 0x7f7b701f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 -- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b7015f580 msgr2=0x7f7b7015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b7015f580 0x7f7b7015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 -- 172.21.15.16:0/4005129079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b701f9480 con 0x7f7b700cf860 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.300+0000 7f7b6fe00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 0x7f7b70144fc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e7c0 tx=0x7f7b5c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c00ccc0 con 0x7f7b700cf860 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b5c00ce80 con 0x7f7b700cf860 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c005280 con 0x7f7b700cf860 2024-09-17T05:04:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b701f9760 con 0x7f7b700cf860 2024-09-17T05:04:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b70145600 con 0x7f7b700cf860 2024-09-17T05:04:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b701f91f0 con 0x7f7b700cf860 2024-09-17T05:04:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7b5c005420 con 0x7f7b700cf860 2024-09-17T05:04:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b6cc00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b50073200 0x7f7b50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.304+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f7b5c094cf0 con 0x7f7b700cf860 2024-09-17T05:04:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.308+0000 7f7b75000640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b50073200 0x7f7b50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.308+0000 7f7b75000640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b50073200 0x7f7b50075710 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7b640af000 tx=0x7f7b6401b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:58.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.312+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7b5c05f490 con 0x7f7b700cf860 2024-09-17T05:04:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.480+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7b7015cb00 con 0x7f7b50073200 2024-09-17T05:04:58.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b6cc00640 1 -- 172.21.15.16:0/4005129079 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247353 (secure 0 0 0) 0x7f7b7015cb00 con 0x7f7b50073200 2024-09-17T05:04:58.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:04:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b50073200 msgr2=0x7f7b50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b50073200 0x7f7b50075710 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7b640af000 tx=0x7f7b6401b100 comp rx=0 tx=0).stop 2024-09-17T05:04:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 msgr2=0x7f7b70144fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 0x7f7b70144fc0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e7c0 tx=0x7f7b5c00ec80 comp rx=0 tx=0).stop 2024-09-17T05:04:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 shutdown_connections 2024-09-17T05:04:58.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b50073200 0x7f7b50075710 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b7015f580 0x7f7b7015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b700cf860 0x7f7b70144fc0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 --2- 172.21.15.16:0/4005129079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7b7006ce90 0x7f7b701f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 >> 172.21.15.16:0/4005129079 conn(0x7f7b700d0860 msgr2=0x7f7b70157050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 shutdown_connections 2024-09-17T05:04:58.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.488+0000 7f7b77a00640 1 -- 172.21.15.16:0/4005129079 wait complete. 2024-09-17T05:04:58.514 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-17T05:04:57.520432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267088,"kb_used_data":9240,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707632,"statfs":{"total":773094113280,"available":772820615168,"internally_reserved":0,"allocated":9461760,"data_stored":4916240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467906+0000","last_change":"2024-09-17T05:04:52.467906+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467906+0000","last_undegraded":"2024-09-17T05:04:52.467906+0000","last_fullsized":"2024-09-17T05:04:52.467906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464969+0000","last_change":"2024-09-17T05:04:52.464969+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464969+0000","last_undegraded":"2024-09-17T05:04:52.464969+0000","last_fullsized":"2024-09-17T05:04:52.464969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468043+0000","last_change":"2024-09-17T05:04:52.468043+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468043+0000","last_undegraded":"2024-09-17T05:04:52.468043+0000","last_fullsized":"2024-09-17T05:04:52.468043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.472514+0000","last_change":"2024-09-17T05:04:52.472514+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.472514+0000","last_undegraded":"2024-09-17T05:04:52.472514+0000","last_fullsized":"2024-09-17T05:04:52.472514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465690+0000","last_change":"2024-09-17T05:04:52.465690+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465690+0000","last_undegraded":"2024-09-17T05:04:52.465690+0000","last_fullsized":"2024-09-17T05:04:52.465690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460417+0000","last_change":"2024-09-17T05:04:52.460417+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460417+0000","last_undegraded":"2024-09-17T05:04:52.460417+0000","last_fullsized":"2024-09-17T05:04:52.460417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468944+0000","last_change":"2024-09-17T05:04:52.468944+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468944+0000","last_undegraded":"2024-09-17T05:04:52.468944+0000","last_fullsized":"2024-09-17T05:04:52.468944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462576+0000","last_change":"2024-09-17T05:04:52.462576+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462576+0000","last_undegraded":"2024-09-17T05:04:52.462576+0000","last_fullsized":"2024-09-17T05:04:52.462576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.475893+0000","last_change":"2024-09-17T05:04:52.475893+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.475893+0000","last_undegraded":"2024-09-17T05:04:52.475893+0000","last_fullsized":"2024-09-17T05:04:52.475893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469663+0000","last_change":"2024-09-17T05:04:52.469663+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469663+0000","last_undegraded":"2024-09-17T05:04:52.469663+0000","last_fullsized":"2024-09-17T05:04:52.469663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478888+0000","last_change":"2024-09-17T05:04:52.478888+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478888+0000","last_undegraded":"2024-09-17T05:04:52.478888+0000","last_fullsized":"2024-09-17T05:04:52.478888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469566+0000","last_change":"2024-09-17T05:04:52.469566+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469566+0000","last_undegraded":"2024-09-17T05:04:52.469566+0000","last_fullsized":"2024-09-17T05:04:52.469566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.491308+0000","last_change":"2024-09-17T05:04:53.488884+0000","last_active":"2024-09-17T05:04:53.491308+0000","last_peered":"2024-09-17T05:04:53.491308+0000","last_clean":"2024-09-17T05:04:53.491308+0000","last_became_active":"2024-09-17T05:04:53.488567+0000","last_became_peered":"2024-09-17T05:04:53.488567+0000","last_unstale":"2024-09-17T05:04:53.491308+0000","last_undegraded":"2024-09-17T05:04:53.491308+0000","last_fullsized":"2024-09-17T05:04:53.491308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:16.452112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469017+0000","last_change":"2024-09-17T05:04:52.469017+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469017+0000","last_undegraded":"2024-09-17T05:04:52.469017+0000","last_fullsized":"2024-09-17T05:04:52.469017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467045+0000","last_change":"2024-09-17T05:04:52.467045+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467045+0000","last_undegraded":"2024-09-17T05:04:52.467045+0000","last_fullsized":"2024-09-17T05:04:52.467045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462060+0000","last_change":"2024-09-17T05:04:52.462060+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462060+0000","last_undegraded":"2024-09-17T05:04:52.462060+0000","last_fullsized":"2024-09-17T05:04:52.462060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459791+0000","last_change":"2024-09-17T05:04:52.459791+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459791+0000","last_undegraded":"2024-09-17T05:04:52.459791+0000","last_fullsized":"2024-09-17T05:04:52.459791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469183+0000","last_change":"2024-09-17T05:04:52.469183+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469183+0000","last_undegraded":"2024-09-17T05:04:52.469183+0000","last_fullsized":"2024-09-17T05:04:52.469183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465817+0000","last_change":"2024-09-17T05:04:52.465817+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465817+0000","last_undegraded":"2024-09-17T05:04:52.465817+0000","last_fullsized":"2024-09-17T05:04:52.465817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459213+0000","last_change":"2024-09-17T05:04:52.459213+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459213+0000","last_undegraded":"2024-09-17T05:04:52.459213+0000","last_fullsized":"2024-09-17T05:04:52.459213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464345+0000","last_change":"2024-09-17T05:04:52.464345+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464345+0000","last_undegraded":"2024-09-17T05:04:52.464345+0000","last_fullsized":"2024-09-17T05:04:52.464345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464433+0000","last_change":"2024-09-17T05:04:52.464433+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464433+0000","last_undegraded":"2024-09-17T05:04:52.464433+0000","last_fullsized":"2024-09-17T05:04:52.464433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464344+0000","last_change":"2024-09-17T05:04:52.464344+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464344+0000","last_undegraded":"2024-09-17T05:04:52.464344+0000","last_fullsized":"2024-09-17T05:04:52.464344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466713+0000","last_change":"2024-09-17T05:04:52.466713+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466713+0000","last_undegraded":"2024-09-17T05:04:52.466713+0000","last_fullsized":"2024-09-17T05:04:52.466713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464434+0000","last_change":"2024-09-17T05:04:52.464434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464434+0000","last_undegraded":"2024-09-17T05:04:52.464434+0000","last_fullsized":"2024-09-17T05:04:52.464434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463434+0000","last_change":"2024-09-17T05:04:52.463434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463434+0000","last_undegraded":"2024-09-17T05:04:52.463434+0000","last_fullsized":"2024-09-17T05:04:52.463434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467567+0000","last_change":"2024-09-17T05:04:52.467567+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467567+0000","last_undegraded":"2024-09-17T05:04:52.467567+0000","last_fullsized":"2024-09-17T05:04:52.467567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.494978+0000","last_change":"2024-09-17T05:04:53.491786+0000","last_active":"2024-09-17T05:04:53.494978+0000","last_peered":"2024-09-17T05:04:53.494978+0000","last_clean":"2024-09-17T05:04:53.494978+0000","last_became_active":"2024-09-17T05:04:53.491402+0000","last_became_peered":"2024-09-17T05:04:53.491402+0000","last_unstale":"2024-09-17T05:04:53.494978+0000","last_undegraded":"2024-09-17T05:04:53.494978+0000","last_fullsized":"2024-09-17T05:04:53.494978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:40.782888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463766+0000","last_change":"2024-09-17T05:04:52.463766+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463766+0000","last_undegraded":"2024-09-17T05:04:52.463766+0000","last_fullsized":"2024-09-17T05:04:52.463766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466787+0000","last_change":"2024-09-17T05:04:52.466787+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466787+0000","last_undegraded":"2024-09-17T05:04:52.466787+0000","last_fullsized":"2024-09-17T05:04:52.466787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464914+0000","last_change":"2024-09-17T05:04:52.464914+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464914+0000","last_undegraded":"2024-09-17T05:04:52.464914+0000","last_fullsized":"2024-09-17T05:04:52.464914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461563+0000","last_change":"2024-09-17T05:04:52.461563+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461563+0000","last_undegraded":"2024-09-17T05:04:52.461563+0000","last_fullsized":"2024-09-17T05:04:52.461563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467712+0000","last_change":"2024-09-17T05:04:52.467712+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467712+0000","last_undegraded":"2024-09-17T05:04:52.467712+0000","last_fullsized":"2024-09-17T05:04:52.467712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476554+0000","last_change":"2024-09-17T05:04:52.474787+0000","last_active":"2024-09-17T05:04:52.476554+0000","last_peered":"2024-09-17T05:04:52.476554+0000","last_clean":"2024-09-17T05:04:52.476554+0000","last_became_active":"2024-09-17T05:04:52.474407+0000","last_became_peered":"2024-09-17T05:04:52.474407+0000","last_unstale":"2024-09-17T05:04:52.476554+0000","last_undegraded":"2024-09-17T05:04:52.476554+0000","last_fullsized":"2024-09-17T05:04:52.476554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:40.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460299+0000","last_change":"2024-09-17T05:04:52.460299+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460299+0000","last_undegraded":"2024-09-17T05:04:52.460299+0000","last_fullsized":"2024-09-17T05:04:52.460299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489502+0000","last_change":"2024-09-17T05:04:52.488780+0000","last_active":"2024-09-17T05:04:52.489502+0000","last_peered":"2024-09-17T05:04:52.489502+0000","last_clean":"2024-09-17T05:04:52.489502+0000","last_became_active":"2024-09-17T05:04:52.488461+0000","last_became_peered":"2024-09-17T05:04:52.488461+0000","last_unstale":"2024-09-17T05:04:52.489502+0000","last_undegraded":"2024-09-17T05:04:52.489502+0000","last_fullsized":"2024-09-17T05:04:52.489502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:13.981678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477383+0000","last_change":"2024-09-17T05:04:52.477383+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477383+0000","last_undegraded":"2024-09-17T05:04:52.477383+0000","last_fullsized":"2024-09-17T05:04:52.477383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.481366+0000","last_change":"2024-09-17T05:04:52.479681+0000","last_active":"2024-09-17T05:04:52.481366+0000","last_peered":"2024-09-17T05:04:52.481366+0000","last_clean":"2024-09-17T05:04:52.481366+0000","last_became_active":"2024-09-17T05:04:52.477648+0000","last_became_peered":"2024-09-17T05:04:52.477648+0000","last_unstale":"2024-09-17T05:04:52.481366+0000","last_undegraded":"2024-09-17T05:04:52.481366+0000","last_fullsized":"2024-09-17T05:04:52.481366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:58.840632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466570+0000","last_change":"2024-09-17T05:04:52.466570+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466570+0000","last_undegraded":"2024-09-17T05:04:52.466570+0000","last_fullsized":"2024-09-17T05:04:52.466570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490600+0000","last_change":"2024-09-17T05:04:52.487079+0000","last_active":"2024-09-17T05:04:52.490600+0000","last_peered":"2024-09-17T05:04:52.490600+0000","last_clean":"2024-09-17T05:04:52.490600+0000","last_became_active":"2024-09-17T05:04:52.486771+0000","last_became_peered":"2024-09-17T05:04:52.486771+0000","last_unstale":"2024-09-17T05:04:52.490600+0000","last_undegraded":"2024-09-17T05:04:52.490600+0000","last_fullsized":"2024-09-17T05:04:52.490600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:51.755302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.498399+0000","last_change":"2024-09-17T05:04:53.496286+0000","last_active":"2024-09-17T05:04:53.498399+0000","last_peered":"2024-09-17T05:04:53.498399+0000","last_clean":"2024-09-17T05:04:53.498399+0000","last_became_active":"2024-09-17T05:04:53.495971+0000","last_became_peered":"2024-09-17T05:04:53.495971+0000","last_unstale":"2024-09-17T05:04:53.498399+0000","last_undegraded":"2024-09-17T05:04:53.498399+0000","last_fullsized":"2024-09-17T05:04:53.498399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:38.614768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484328+0000","last_change":"2024-09-17T05:04:52.482615+0000","last_active":"2024-09-17T05:04:52.484328+0000","last_peered":"2024-09-17T05:04:52.484328+0000","last_clean":"2024-09-17T05:04:52.484328+0000","last_became_active":"2024-09-17T05:04:52.482267+0000","last_became_peered":"2024-09-17T05:04:52.482267+0000","last_unstale":"2024-09-17T05:04:52.484328+0000","last_undegraded":"2024-09-17T05:04:52.484328+0000","last_fullsized":"2024-09-17T05:04:52.484328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468323+0000","last_change":"2024-09-17T05:04:52.468323+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468323+0000","last_undegraded":"2024-09-17T05:04:52.468323+0000","last_fullsized":"2024-09-17T05:04:52.468323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.495357+0000","last_change":"2024-09-17T05:04:52.476398+0000","last_active":"2024-09-17T05:04:53.495357+0000","last_peered":"2024-09-17T05:04:53.495357+0000","last_clean":"2024-09-17T05:04:53.495357+0000","last_became_active":"2024-09-17T05:04:52.476002+0000","last_became_peered":"2024-09-17T05:04:52.476002+0000","last_unstale":"2024-09-17T05:04:53.495357+0000","last_undegraded":"2024-09-17T05:04:53.495357+0000","last_fullsized":"2024-09-17T05:04:53.495357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:23.106184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459964+0000","last_change":"2024-09-17T05:04:52.459964+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459964+0000","last_undegraded":"2024-09-17T05:04:52.459964+0000","last_fullsized":"2024-09-17T05:04:52.459964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485583+0000","last_change":"2024-09-17T05:04:52.482041+0000","last_active":"2024-09-17T05:04:52.485583+0000","last_peered":"2024-09-17T05:04:52.485583+0000","last_clean":"2024-09-17T05:04:52.485583+0000","last_became_active":"2024-09-17T05:04:52.481707+0000","last_became_peered":"2024-09-17T05:04:52.481707+0000","last_unstale":"2024-09-17T05:04:52.485583+0000","last_undegraded":"2024-09-17T05:04:52.485583+0000","last_fullsized":"2024-09-17T05:04:52.485583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:32.340896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467432+0000","last_change":"2024-09-17T05:04:52.467432+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467432+0000","last_undegraded":"2024-09-17T05:04:52.467432+0000","last_fullsized":"2024-09-17T05:04:52.467432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484343+0000","last_change":"2024-09-17T05:04:52.480980+0000","last_active":"2024-09-17T05:04:52.484343+0000","last_peered":"2024-09-17T05:04:52.484343+0000","last_clean":"2024-09-17T05:04:52.484343+0000","last_became_active":"2024-09-17T05:04:52.477680+0000","last_became_peered":"2024-09-17T05:04:52.477680+0000","last_unstale":"2024-09-17T05:04:52.484343+0000","last_undegraded":"2024-09-17T05:04:52.484343+0000","last_fullsized":"2024-09-17T05:04:52.484343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:06.060774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463355+0000","last_change":"2024-09-17T05:04:52.463355+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463355+0000","last_undegraded":"2024-09-17T05:04:52.463355+0000","last_fullsized":"2024-09-17T05:04:52.463355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479996+0000","last_change":"2024-09-17T05:04:52.478749+0000","last_active":"2024-09-17T05:04:52.479996+0000","last_peered":"2024-09-17T05:04:52.479996+0000","last_clean":"2024-09-17T05:04:52.479996+0000","last_became_active":"2024-09-17T05:04:52.478315+0000","last_became_peered":"2024-09-17T05:04:52.478315+0000","last_unstale":"2024-09-17T05:04:52.479996+0000","last_undegraded":"2024-09-17T05:04:52.479996+0000","last_fullsized":"2024-09-17T05:04:52.479996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:51.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478423+0000","last_change":"2024-09-17T05:04:52.478423+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478423+0000","last_undegraded":"2024-09-17T05:04:52.478423+0000","last_fullsized":"2024-09-17T05:04:52.478423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485541+0000","last_change":"2024-09-17T05:04:52.480538+0000","last_active":"2024-09-17T05:04:52.485541+0000","last_peered":"2024-09-17T05:04:52.485541+0000","last_clean":"2024-09-17T05:04:52.485541+0000","last_became_active":"2024-09-17T05:04:52.480130+0000","last_became_peered":"2024-09-17T05:04:52.480130+0000","last_unstale":"2024-09-17T05:04:52.485541+0000","last_undegraded":"2024-09-17T05:04:52.485541+0000","last_fullsized":"2024-09-17T05:04:52.485541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:06.575023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.470865+0000","last_change":"2024-09-17T05:04:52.470865+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.470865+0000","last_undegraded":"2024-09-17T05:04:52.470865+0000","last_fullsized":"2024-09-17T05:04:52.470865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.486751+0000","last_change":"2024-09-17T05:04:52.484495+0000","last_active":"2024-09-17T05:04:52.486751+0000","last_peered":"2024-09-17T05:04:52.486751+0000","last_clean":"2024-09-17T05:04:52.486751+0000","last_became_active":"2024-09-17T05:04:52.483963+0000","last_became_peered":"2024-09-17T05:04:52.483963+0000","last_unstale":"2024-09-17T05:04:52.486751+0000","last_undegraded":"2024-09-17T05:04:52.486751+0000","last_fullsized":"2024-09-17T05:04:52.486751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:04.344757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459224+0000","last_change":"2024-09-17T05:04:52.459224+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459224+0000","last_undegraded":"2024-09-17T05:04:52.459224+0000","last_fullsized":"2024-09-17T05:04:52.459224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490846+0000","last_change":"2024-09-17T05:04:52.486319+0000","last_active":"2024-09-17T05:04:52.490846+0000","last_peered":"2024-09-17T05:04:52.490846+0000","last_clean":"2024-09-17T05:04:52.490846+0000","last_became_active":"2024-09-17T05:04:52.486037+0000","last_became_peered":"2024-09-17T05:04:52.486037+0000","last_unstale":"2024-09-17T05:04:52.490846+0000","last_undegraded":"2024-09-17T05:04:52.490846+0000","last_fullsized":"2024-09-17T05:04:52.490846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:36.202849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466161+0000","last_change":"2024-09-17T05:04:52.466161+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466161+0000","last_undegraded":"2024-09-17T05:04:52.466161+0000","last_fullsized":"2024-09-17T05:04:52.466161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479955+0000","last_change":"2024-09-17T05:04:52.478046+0000","last_active":"2024-09-17T05:04:52.479955+0000","last_peered":"2024-09-17T05:04:52.479955+0000","last_clean":"2024-09-17T05:04:52.479955+0000","last_became_active":"2024-09-17T05:04:52.477554+0000","last_became_peered":"2024-09-17T05:04:52.477554+0000","last_unstale":"2024-09-17T05:04:52.479955+0000","last_undegraded":"2024-09-17T05:04:52.479955+0000","last_fullsized":"2024-09-17T05:04:52.479955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:31.329683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463789+0000","last_change":"2024-09-17T05:04:52.463789+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463789+0000","last_undegraded":"2024-09-17T05:04:52.463789+0000","last_fullsized":"2024-09-17T05:04:52.463789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485667+0000","last_change":"2024-09-17T05:04:52.481898+0000","last_active":"2024-09-17T05:04:52.485667+0000","last_peered":"2024-09-17T05:04:52.485667+0000","last_clean":"2024-09-17T05:04:52.485667+0000","last_became_active":"2024-09-17T05:04:52.481562+0000","last_became_peered":"2024-09-17T05:04:52.481562+0000","last_unstale":"2024-09-17T05:04:52.485667+0000","last_undegraded":"2024-09-17T05:04:52.485667+0000","last_fullsized":"2024-09-17T05:04:52.485667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466746+0000","last_change":"2024-09-17T05:04:52.466746+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466746+0000","last_undegraded":"2024-09-17T05:04:52.466746+0000","last_fullsized":"2024-09-17T05:04:52.466746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483399+0000","last_change":"2024-09-17T05:04:52.479084+0000","last_active":"2024-09-17T05:04:52.483399+0000","last_peered":"2024-09-17T05:04:52.483399+0000","last_clean":"2024-09-17T05:04:52.483399+0000","last_became_active":"2024-09-17T05:04:52.478401+0000","last_became_peered":"2024-09-17T05:04:52.478401+0000","last_unstale":"2024-09-17T05:04:52.483399+0000","last_undegraded":"2024-09-17T05:04:52.483399+0000","last_fullsized":"2024-09-17T05:04:52.483399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:08.731176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485456+0000","last_change":"2024-09-17T05:04:53.480121+0000","last_active":"2024-09-17T05:04:53.485456+0000","last_peered":"2024-09-17T05:04:53.485456+0000","last_clean":"2024-09-17T05:04:53.485456+0000","last_became_active":"2024-09-17T05:04:53.479707+0000","last_became_peered":"2024-09-17T05:04:53.479707+0000","last_unstale":"2024-09-17T05:04:53.485456+0000","last_undegraded":"2024-09-17T05:04:53.485456+0000","last_fullsized":"2024-09-17T05:04:53.485456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:54.547283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483105+0000","last_change":"2024-09-17T05:04:52.479663+0000","last_active":"2024-09-17T05:04:52.483105+0000","last_peered":"2024-09-17T05:04:52.483105+0000","last_clean":"2024-09-17T05:04:52.483105+0000","last_became_active":"2024-09-17T05:04:52.479313+0000","last_became_peered":"2024-09-17T05:04:52.479313+0000","last_unstale":"2024-09-17T05:04:52.483105+0000","last_undegraded":"2024-09-17T05:04:52.483105+0000","last_fullsized":"2024-09-17T05:04:52.483105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:56.767884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.489610+0000","last_change":"2024-09-17T05:04:53.483761+0000","last_active":"2024-09-17T05:04:53.489610+0000","last_peered":"2024-09-17T05:04:53.489610+0000","last_clean":"2024-09-17T05:04:53.489610+0000","last_became_active":"2024-09-17T05:04:53.483036+0000","last_became_peered":"2024-09-17T05:04:53.483036+0000","last_unstale":"2024-09-17T05:04:53.489610+0000","last_undegraded":"2024-09-17T05:04:53.489610+0000","last_fullsized":"2024-09-17T05:04:53.489610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:05.593222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490673+0000","last_change":"2024-09-17T05:04:52.489855+0000","last_active":"2024-09-17T05:04:52.490673+0000","last_peered":"2024-09-17T05:04:52.490673+0000","last_clean":"2024-09-17T05:04:52.490673+0000","last_became_active":"2024-09-17T05:04:52.487437+0000","last_became_peered":"2024-09-17T05:04:52.487437+0000","last_unstale":"2024-09-17T05:04:52.490673+0000","last_undegraded":"2024-09-17T05:04:52.490673+0000","last_fullsized":"2024-09-17T05:04:52.490673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:39.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477891+0000","last_change":"2024-09-17T05:04:52.477891+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477891+0000","last_undegraded":"2024-09-17T05:04:52.477891+0000","last_fullsized":"2024-09-17T05:04:52.477891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490880+0000","last_change":"2024-09-17T05:04:52.489186+0000","last_active":"2024-09-17T05:04:52.490880+0000","last_peered":"2024-09-17T05:04:52.490880+0000","last_clean":"2024-09-17T05:04:52.490880+0000","last_became_active":"2024-09-17T05:04:52.484432+0000","last_became_peered":"2024-09-17T05:04:52.484432+0000","last_unstale":"2024-09-17T05:04:52.490880+0000","last_undegraded":"2024-09-17T05:04:52.490880+0000","last_fullsized":"2024-09-17T05:04:52.490880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.803390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462678+0000","last_change":"2024-09-17T05:04:52.462678+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462678+0000","last_undegraded":"2024-09-17T05:04:52.462678+0000","last_fullsized":"2024-09-17T05:04:52.462678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488159+0000","last_change":"2024-09-17T05:04:52.486351+0000","last_active":"2024-09-17T05:04:52.488159+0000","last_peered":"2024-09-17T05:04:52.488159+0000","last_clean":"2024-09-17T05:04:52.488159+0000","last_became_active":"2024-09-17T05:04:52.486015+0000","last_became_peered":"2024-09-17T05:04:52.486015+0000","last_unstale":"2024-09-17T05:04:52.488159+0000","last_undegraded":"2024-09-17T05:04:52.488159+0000","last_fullsized":"2024-09-17T05:04:52.488159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.544703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461102+0000","last_change":"2024-09-17T05:04:52.461102+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461102+0000","last_undegraded":"2024-09-17T05:04:52.461102+0000","last_fullsized":"2024-09-17T05:04:52.461102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488049+0000","last_change":"2024-09-17T05:04:52.484413+0000","last_active":"2024-09-17T05:04:52.488049+0000","last_peered":"2024-09-17T05:04:52.488049+0000","last_clean":"2024-09-17T05:04:52.488049+0000","last_became_active":"2024-09-17T05:04:52.483851+0000","last_became_peered":"2024-09-17T05:04:52.483851+0000","last_unstale":"2024-09-17T05:04:52.488049+0000","last_undegraded":"2024-09-17T05:04:52.488049+0000","last_fullsized":"2024-09-17T05:04:52.488049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:23.830173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469110+0000","last_change":"2024-09-17T05:04:52.469110+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469110+0000","last_undegraded":"2024-09-17T05:04:52.469110+0000","last_fullsized":"2024-09-17T05:04:52.469110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478334+0000","last_change":"2024-09-17T05:04:52.475068+0000","last_active":"2024-09-17T05:04:52.478334+0000","last_peered":"2024-09-17T05:04:52.478334+0000","last_clean":"2024-09-17T05:04:52.478334+0000","last_became_active":"2024-09-17T05:04:52.474737+0000","last_became_peered":"2024-09-17T05:04:52.474737+0000","last_unstale":"2024-09-17T05:04:52.478334+0000","last_undegraded":"2024-09-17T05:04:52.478334+0000","last_fullsized":"2024-09-17T05:04:52.478334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:29.910066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463931+0000","last_change":"2024-09-17T05:04:52.463931+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463931+0000","last_undegraded":"2024-09-17T05:04:52.463931+0000","last_fullsized":"2024-09-17T05:04:52.463931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485373+0000","last_change":"2024-09-17T05:04:52.479150+0000","last_active":"2024-09-17T05:04:52.485373+0000","last_peered":"2024-09-17T05:04:52.485373+0000","last_clean":"2024-09-17T05:04:52.485373+0000","last_became_active":"2024-09-17T05:04:52.476295+0000","last_became_peered":"2024-09-17T05:04:52.476295+0000","last_unstale":"2024-09-17T05:04:52.485373+0000","last_undegraded":"2024-09-17T05:04:52.485373+0000","last_fullsized":"2024-09-17T05:04:52.485373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:28.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466916+0000","last_change":"2024-09-17T05:04:52.466916+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466916+0000","last_undegraded":"2024-09-17T05:04:52.466916+0000","last_fullsized":"2024-09-17T05:04:52.466916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.483299+0000","last_change":"2024-09-17T05:04:52.482044+0000","last_active":"2024-09-17T05:04:53.483299+0000","last_peered":"2024-09-17T05:04:53.483299+0000","last_clean":"2024-09-17T05:04:53.483299+0000","last_became_active":"2024-09-17T05:04:52.481677+0000","last_became_peered":"2024-09-17T05:04:52.481677+0000","last_unstale":"2024-09-17T05:04:53.483299+0000","last_undegraded":"2024-09-17T05:04:53.483299+0000","last_fullsized":"2024-09-17T05:04:53.483299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:07.449960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463327+0000","last_change":"2024-09-17T05:04:52.463327+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463327+0000","last_undegraded":"2024-09-17T05:04:52.463327+0000","last_fullsized":"2024-09-17T05:04:52.463327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.492052+0000","last_change":"2024-09-17T05:04:52.491484+0000","last_active":"2024-09-17T05:04:52.492052+0000","last_peered":"2024-09-17T05:04:52.492052+0000","last_clean":"2024-09-17T05:04:52.492052+0000","last_became_active":"2024-09-17T05:04:52.490491+0000","last_became_peered":"2024-09-17T05:04:52.490491+0000","last_unstale":"2024-09-17T05:04:52.492052+0000","last_undegraded":"2024-09-17T05:04:52.492052+0000","last_fullsized":"2024-09-17T05:04:52.492052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:39.450191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485201+0000","last_change":"2024-09-17T05:04:53.480166+0000","last_active":"2024-09-17T05:04:53.485201+0000","last_peered":"2024-09-17T05:04:53.485201+0000","last_clean":"2024-09-17T05:04:53.485201+0000","last_became_active":"2024-09-17T05:04:53.479853+0000","last_became_peered":"2024-09-17T05:04:53.479853+0000","last_unstale":"2024-09-17T05:04:53.485201+0000","last_undegraded":"2024-09-17T05:04:53.485201+0000","last_fullsized":"2024-09-17T05:04:53.485201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:22.807028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.491140+0000","last_change":"2024-09-17T05:04:52.489527+0000","last_active":"2024-09-17T05:04:52.491140+0000","last_peered":"2024-09-17T05:04:52.491140+0000","last_clean":"2024-09-17T05:04:52.491140+0000","last_became_active":"2024-09-17T05:04:52.487078+0000","last_became_peered":"2024-09-17T05:04:52.487078+0000","last_unstale":"2024-09-17T05:04:52.491140+0000","last_undegraded":"2024-09-17T05:04:52.491140+0000","last_fullsized":"2024-09-17T05:04:52.491140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:25.546253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.502905+0000","last_change":"2024-09-17T05:04:53.499221+0000","last_active":"2024-09-17T05:04:53.502905+0000","last_peered":"2024-09-17T05:04:53.502905+0000","last_clean":"2024-09-17T05:04:53.502905+0000","last_became_active":"2024-09-17T05:04:53.498898+0000","last_became_peered":"2024-09-17T05:04:53.498898+0000","last_unstale":"2024-09-17T05:04:53.502905+0000","last_undegraded":"2024-09-17T05:04:53.502905+0000","last_fullsized":"2024-09-17T05:04:53.502905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:25.140293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483295+0000","last_change":"2024-09-17T05:04:52.480615+0000","last_active":"2024-09-17T05:04:52.483295+0000","last_peered":"2024-09-17T05:04:52.483295+0000","last_clean":"2024-09-17T05:04:52.483295+0000","last_became_active":"2024-09-17T05:04:52.479433+0000","last_became_peered":"2024-09-17T05:04:52.479433+0000","last_unstale":"2024-09-17T05:04:52.483295+0000","last_undegraded":"2024-09-17T05:04:52.483295+0000","last_fullsized":"2024-09-17T05:04:52.483295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462471+0000","last_change":"2024-09-17T05:04:52.462471+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462471+0000","last_undegraded":"2024-09-17T05:04:52.462471+0000","last_fullsized":"2024-09-17T05:04:52.462471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489406+0000","last_change":"2024-09-17T05:04:52.488025+0000","last_active":"2024-09-17T05:04:52.489406+0000","last_peered":"2024-09-17T05:04:52.489406+0000","last_clean":"2024-09-17T05:04:52.489406+0000","last_became_active":"2024-09-17T05:04:52.484280+0000","last_became_peered":"2024-09-17T05:04:52.484280+0000","last_unstale":"2024-09-17T05:04:52.489406+0000","last_undegraded":"2024-09-17T05:04:52.489406+0000","last_fullsized":"2024-09-17T05:04:52.489406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:10.565964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.458957+0000","last_change":"2024-09-17T05:04:52.458957+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.458957+0000","last_undegraded":"2024-09-17T05:04:52.458957+0000","last_fullsized":"2024-09-17T05:04:52.458957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478710+0000","last_change":"2024-09-17T05:04:52.474880+0000","last_active":"2024-09-17T05:04:52.478710+0000","last_peered":"2024-09-17T05:04:52.478710+0000","last_clean":"2024-09-17T05:04:52.478710+0000","last_became_active":"2024-09-17T05:04:52.474563+0000","last_became_peered":"2024-09-17T05:04:52.474563+0000","last_unstale":"2024-09-17T05:04:52.478710+0000","last_undegraded":"2024-09-17T05:04:52.478710+0000","last_fullsized":"2024-09-17T05:04:52.478710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:25.901198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466397+0000","last_change":"2024-09-17T05:04:52.466397+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466397+0000","last_undegraded":"2024-09-17T05:04:52.466397+0000","last_fullsized":"2024-09-17T05:04:52.466397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476614+0000","last_change":"2024-09-17T05:04:52.474440+0000","last_active":"2024-09-17T05:04:52.476614+0000","last_peered":"2024-09-17T05:04:52.476614+0000","last_clean":"2024-09-17T05:04:52.476614+0000","last_became_active":"2024-09-17T05:04:52.474097+0000","last_became_peered":"2024-09-17T05:04:52.474097+0000","last_unstale":"2024-09-17T05:04:52.476614+0000","last_undegraded":"2024-09-17T05:04:52.476614+0000","last_fullsized":"2024-09-17T05:04:52.476614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:16.647876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463352+0000","last_change":"2024-09-17T05:04:52.463352+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463352+0000","last_undegraded":"2024-09-17T05:04:52.463352+0000","last_fullsized":"2024-09-17T05:04:52.463352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.468511+0000","last_change":"2024-09-17T05:04:52.466652+0000","last_active":"2024-09-17T05:04:52.468511+0000","last_peered":"2024-09-17T05:04:52.468511+0000","last_clean":"2024-09-17T05:04:52.468511+0000","last_became_active":"2024-09-17T05:04:52.466240+0000","last_became_peered":"2024-09-17T05:04:52.466240+0000","last_unstale":"2024-09-17T05:04:52.468511+0000","last_undegraded":"2024-09-17T05:04:52.468511+0000","last_fullsized":"2024-09-17T05:04:52.468511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:25.791675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459124+0000","last_change":"2024-09-17T05:04:52.459124+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459124+0000","last_undegraded":"2024-09-17T05:04:52.459124+0000","last_fullsized":"2024-09-17T05:04:52.459124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476563+0000","last_change":"2024-09-17T05:04:52.472431+0000","last_active":"2024-09-17T05:04:52.476563+0000","last_peered":"2024-09-17T05:04:52.476563+0000","last_clean":"2024-09-17T05:04:52.476563+0000","last_became_active":"2024-09-17T05:04:52.472066+0000","last_became_peered":"2024-09-17T05:04:52.472066+0000","last_unstale":"2024-09-17T05:04:52.476563+0000","last_undegraded":"2024-09-17T05:04:52.476563+0000","last_fullsized":"2024-09-17T05:04:52.476563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:25.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463436+0000","last_change":"2024-09-17T05:04:52.463436+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463436+0000","last_undegraded":"2024-09-17T05:04:52.463436+0000","last_fullsized":"2024-09-17T05:04:52.463436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490829+0000","last_change":"2024-09-17T05:04:52.486302+0000","last_active":"2024-09-17T05:04:52.490829+0000","last_peered":"2024-09-17T05:04:52.490829+0000","last_clean":"2024-09-17T05:04:52.490829+0000","last_became_active":"2024-09-17T05:04:52.486002+0000","last_became_peered":"2024-09-17T05:04:52.486002+0000","last_unstale":"2024-09-17T05:04:52.490829+0000","last_undegraded":"2024-09-17T05:04:52.490829+0000","last_fullsized":"2024-09-17T05:04:52.490829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.458028+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:52.458028+0000","last_peered":"2024-09-17T05:04:52.458028+0000","last_clean":"2024-09-17T05:04:52.458028+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:52.458028+0000","last_undegraded":"2024-09-17T05:04:52.458028+0000","last_fullsized":"2024-09-17T05:04:52.458028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1040384,"data_stored":470209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36948,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334892,"statfs":{"total":96636764160,"available":96598929408,"internally_reserved":0,"allocated":1634304,"data_stored":1060577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":1067744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:04:58.518 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3f is not active+clean 2024-09-17T05:04:58.518 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.467906+0000', 'last_change': '2024-09-17T05:04:52.467906+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.467906+0000', 'last_undegraded': '2024-09-17T05:04:52.467906+0000', 'last_fullsized': '2024-09-17T05:04:52.467906+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.518 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3e is not active+clean 2024-09-17T05:04:58.518 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.464969+0000', 'last_change': '2024-09-17T05:04:52.464969+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.464969+0000', 'last_undegraded': '2024-09-17T05:04:52.464969+0000', 'last_fullsized': '2024-09-17T05:04:52.464969+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.518 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3d is not active+clean 2024-09-17T05:04:58.518 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.468043+0000', 'last_change': '2024-09-17T05:04:52.468043+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.468043+0000', 'last_undegraded': '2024-09-17T05:04:52.468043+0000', 'last_fullsized': '2024-09-17T05:04:52.468043+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3c is not active+clean 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.472514+0000', 'last_change': '2024-09-17T05:04:52.472514+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.472514+0000', 'last_undegraded': '2024-09-17T05:04:52.472514+0000', 'last_fullsized': '2024-09-17T05:04:52.472514+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3b is not active+clean 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.465690+0000', 'last_change': '2024-09-17T05:04:52.465690+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.465690+0000', 'last_undegraded': '2024-09-17T05:04:52.465690+0000', 'last_fullsized': '2024-09-17T05:04:52.465690+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3a is not active+clean 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.460417+0000', 'last_change': '2024-09-17T05:04:52.460417+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.460417+0000', 'last_undegraded': '2024-09-17T05:04:52.460417+0000', 'last_fullsized': '2024-09-17T05:04:52.460417+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:PG 14.39 is not active+clean 2024-09-17T05:04:58.519 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.468944+0000', 'last_change': '2024-09-17T05:04:52.468944+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.468944+0000', 'last_undegraded': '2024-09-17T05:04:52.468944+0000', 'last_fullsized': '2024-09-17T05:04:52.468944+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:PG 14.38 is not active+clean 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.462576+0000', 'last_change': '2024-09-17T05:04:52.462576+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.462576+0000', 'last_undegraded': '2024-09-17T05:04:52.462576+0000', 'last_fullsized': '2024-09-17T05:04:52.462576+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:PG 14.37 is not active+clean 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.475893+0000', 'last_change': '2024-09-17T05:04:52.475893+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.475893+0000', 'last_undegraded': '2024-09-17T05:04:52.475893+0000', 'last_fullsized': '2024-09-17T05:04:52.475893+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:PG 14.36 is not active+clean 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.469663+0000', 'last_change': '2024-09-17T05:04:52.469663+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.469663+0000', 'last_undegraded': '2024-09-17T05:04:52.469663+0000', 'last_fullsized': '2024-09-17T05:04:52.469663+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:PG 14.35 is not active+clean 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.478888+0000', 'last_change': '2024-09-17T05:04:52.478888+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.478888+0000', 'last_undegraded': '2024-09-17T05:04:52.478888+0000', 'last_fullsized': '2024-09-17T05:04:52.478888+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:PG 14.34 is not active+clean 2024-09-17T05:04:58.520 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.469566+0000', 'last_change': '2024-09-17T05:04:52.469566+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.469566+0000', 'last_undegraded': '2024-09-17T05:04:52.469566+0000', 'last_fullsized': '2024-09-17T05:04:52.469566+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:PG 14.32 is not active+clean 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.469017+0000', 'last_change': '2024-09-17T05:04:52.469017+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.469017+0000', 'last_undegraded': '2024-09-17T05:04:52.469017+0000', 'last_fullsized': '2024-09-17T05:04:52.469017+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:PG 14.31 is not active+clean 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.467045+0000', 'last_change': '2024-09-17T05:04:52.467045+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.467045+0000', 'last_undegraded': '2024-09-17T05:04:52.467045+0000', 'last_fullsized': '2024-09-17T05:04:52.467045+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:PG 14.30 is not active+clean 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.462060+0000', 'last_change': '2024-09-17T05:04:52.462060+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.462060+0000', 'last_undegraded': '2024-09-17T05:04:52.462060+0000', 'last_fullsized': '2024-09-17T05:04:52.462060+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2f is not active+clean 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.459791+0000', 'last_change': '2024-09-17T05:04:52.459791+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.459791+0000', 'last_undegraded': '2024-09-17T05:04:52.459791+0000', 'last_fullsized': '2024-09-17T05:04:52.459791+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2e is not active+clean 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.469183+0000', 'last_change': '2024-09-17T05:04:52.469183+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.469183+0000', 'last_undegraded': '2024-09-17T05:04:52.469183+0000', 'last_fullsized': '2024-09-17T05:04:52.469183+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2d is not active+clean 2024-09-17T05:04:58.521 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.465817+0000', 'last_change': '2024-09-17T05:04:52.465817+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.465817+0000', 'last_undegraded': '2024-09-17T05:04:52.465817+0000', 'last_fullsized': '2024-09-17T05:04:52.465817+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2c is not active+clean 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.459213+0000', 'last_change': '2024-09-17T05:04:52.459213+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.459213+0000', 'last_undegraded': '2024-09-17T05:04:52.459213+0000', 'last_fullsized': '2024-09-17T05:04:52.459213+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2b is not active+clean 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.464345+0000', 'last_change': '2024-09-17T05:04:52.464345+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.464345+0000', 'last_undegraded': '2024-09-17T05:04:52.464345+0000', 'last_fullsized': '2024-09-17T05:04:52.464345+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2a is not active+clean 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.464433+0000', 'last_change': '2024-09-17T05:04:52.464433+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.464433+0000', 'last_undegraded': '2024-09-17T05:04:52.464433+0000', 'last_fullsized': '2024-09-17T05:04:52.464433+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:PG 14.29 is not active+clean 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.464344+0000', 'last_change': '2024-09-17T05:04:52.464344+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.464344+0000', 'last_undegraded': '2024-09-17T05:04:52.464344+0000', 'last_fullsized': '2024-09-17T05:04:52.464344+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:PG 14.28 is not active+clean 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466713+0000', 'last_change': '2024-09-17T05:04:52.466713+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466713+0000', 'last_undegraded': '2024-09-17T05:04:52.466713+0000', 'last_fullsized': '2024-09-17T05:04:52.466713+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.522 INFO:tasks.ceph.ceph_manager.ceph:PG 14.27 is not active+clean 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.464434+0000', 'last_change': '2024-09-17T05:04:52.464434+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.464434+0000', 'last_undegraded': '2024-09-17T05:04:52.464434+0000', 'last_fullsized': '2024-09-17T05:04:52.464434+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:PG 14.26 is not active+clean 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463434+0000', 'last_change': '2024-09-17T05:04:52.463434+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463434+0000', 'last_undegraded': '2024-09-17T05:04:52.463434+0000', 'last_fullsized': '2024-09-17T05:04:52.463434+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:PG 14.25 is not active+clean 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.467567+0000', 'last_change': '2024-09-17T05:04:52.467567+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.467567+0000', 'last_undegraded': '2024-09-17T05:04:52.467567+0000', 'last_fullsized': '2024-09-17T05:04:52.467567+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:PG 14.23 is not active+clean 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463766+0000', 'last_change': '2024-09-17T05:04:52.463766+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463766+0000', 'last_undegraded': '2024-09-17T05:04:52.463766+0000', 'last_fullsized': '2024-09-17T05:04:52.463766+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:PG 14.22 is not active+clean 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466787+0000', 'last_change': '2024-09-17T05:04:52.466787+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466787+0000', 'last_undegraded': '2024-09-17T05:04:52.466787+0000', 'last_fullsized': '2024-09-17T05:04:52.466787+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:PG 14.21 is not active+clean 2024-09-17T05:04:58.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.464914+0000', 'last_change': '2024-09-17T05:04:52.464914+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.464914+0000', 'last_undegraded': '2024-09-17T05:04:52.464914+0000', 'last_fullsized': '2024-09-17T05:04:52.464914+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:PG 14.20 is not active+clean 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.461563+0000', 'last_change': '2024-09-17T05:04:52.461563+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.461563+0000', 'last_undegraded': '2024-09-17T05:04:52.461563+0000', 'last_fullsized': '2024-09-17T05:04:52.461563+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1c is not active+clean 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.467712+0000', 'last_change': '2024-09-17T05:04:52.467712+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.467712+0000', 'last_undegraded': '2024-09-17T05:04:52.467712+0000', 'last_fullsized': '2024-09-17T05:04:52.467712+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1d is not active+clean 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.460299+0000', 'last_change': '2024-09-17T05:04:52.460299+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.460299+0000', 'last_undegraded': '2024-09-17T05:04:52.460299+0000', 'last_fullsized': '2024-09-17T05:04:52.460299+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1e is not active+clean 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.477383+0000', 'last_change': '2024-09-17T05:04:52.477383+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.477383+0000', 'last_undegraded': '2024-09-17T05:04:52.477383+0000', 'last_fullsized': '2024-09-17T05:04:52.477383+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1f is not active+clean 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466570+0000', 'last_change': '2024-09-17T05:04:52.466570+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466570+0000', 'last_undegraded': '2024-09-17T05:04:52.466570+0000', 'last_fullsized': '2024-09-17T05:04:52.466570+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.524 INFO:tasks.ceph.ceph_manager.ceph:PG 14.19 is not active+clean 2024-09-17T05:04:58.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.468323+0000', 'last_change': '2024-09-17T05:04:52.468323+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.468323+0000', 'last_undegraded': '2024-09-17T05:04:52.468323+0000', 'last_fullsized': '2024-09-17T05:04:52.468323+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.525 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1a is not active+clean 2024-09-17T05:04:58.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.459964+0000', 'last_change': '2024-09-17T05:04:52.459964+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.459964+0000', 'last_undegraded': '2024-09-17T05:04:52.459964+0000', 'last_fullsized': '2024-09-17T05:04:52.459964+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.525 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1b is not active+clean 2024-09-17T05:04:58.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.467432+0000', 'last_change': '2024-09-17T05:04:52.467432+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.467432+0000', 'last_undegraded': '2024-09-17T05:04:52.467432+0000', 'last_fullsized': '2024-09-17T05:04:52.467432+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.525 INFO:tasks.ceph.ceph_manager.ceph:PG 14.14 is not active+clean 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463355+0000', 'last_change': '2024-09-17T05:04:52.463355+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463355+0000', 'last_undegraded': '2024-09-17T05:04:52.463355+0000', 'last_fullsized': '2024-09-17T05:04:52.463355+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:PG 14.15 is not active+clean 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.478423+0000', 'last_change': '2024-09-17T05:04:52.478423+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.478423+0000', 'last_undegraded': '2024-09-17T05:04:52.478423+0000', 'last_fullsized': '2024-09-17T05:04:52.478423+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:PG 14.16 is not active+clean 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.470865+0000', 'last_change': '2024-09-17T05:04:52.470865+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.470865+0000', 'last_undegraded': '2024-09-17T05:04:52.470865+0000', 'last_fullsized': '2024-09-17T05:04:52.470865+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:PG 14.17 is not active+clean 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.459224+0000', 'last_change': '2024-09-17T05:04:52.459224+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.459224+0000', 'last_undegraded': '2024-09-17T05:04:52.459224+0000', 'last_fullsized': '2024-09-17T05:04:52.459224+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:PG 14.10 is not active+clean 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466161+0000', 'last_change': '2024-09-17T05:04:52.466161+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466161+0000', 'last_undegraded': '2024-09-17T05:04:52.466161+0000', 'last_fullsized': '2024-09-17T05:04:52.466161+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.526 INFO:tasks.ceph.ceph_manager.ceph:PG 14.11 is not active+clean 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463789+0000', 'last_change': '2024-09-17T05:04:52.463789+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463789+0000', 'last_undegraded': '2024-09-17T05:04:52.463789+0000', 'last_fullsized': '2024-09-17T05:04:52.463789+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:PG 14.12 is not active+clean 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466746+0000', 'last_change': '2024-09-17T05:04:52.466746+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466746+0000', 'last_undegraded': '2024-09-17T05:04:52.466746+0000', 'last_fullsized': '2024-09-17T05:04:52.466746+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:PG 14.d is not active+clean 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.477891+0000', 'last_change': '2024-09-17T05:04:52.477891+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.477891+0000', 'last_undegraded': '2024-09-17T05:04:52.477891+0000', 'last_fullsized': '2024-09-17T05:04:52.477891+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:PG 14.e is not active+clean 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.462678+0000', 'last_change': '2024-09-17T05:04:52.462678+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.462678+0000', 'last_undegraded': '2024-09-17T05:04:52.462678+0000', 'last_fullsized': '2024-09-17T05:04:52.462678+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:PG 14.8 is not active+clean 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.461102+0000', 'last_change': '2024-09-17T05:04:52.461102+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.461102+0000', 'last_undegraded': '2024-09-17T05:04:52.461102+0000', 'last_fullsized': '2024-09-17T05:04:52.461102+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.527 INFO:tasks.ceph.ceph_manager.ceph:PG 14.9 is not active+clean 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.469110+0000', 'last_change': '2024-09-17T05:04:52.469110+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.469110+0000', 'last_undegraded': '2024-09-17T05:04:52.469110+0000', 'last_fullsized': '2024-09-17T05:04:52.469110+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:PG 14.a is not active+clean 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463931+0000', 'last_change': '2024-09-17T05:04:52.463931+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463931+0000', 'last_undegraded': '2024-09-17T05:04:52.463931+0000', 'last_fullsized': '2024-09-17T05:04:52.463931+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:PG 14.b is not active+clean 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466916+0000', 'last_change': '2024-09-17T05:04:52.466916+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466916+0000', 'last_undegraded': '2024-09-17T05:04:52.466916+0000', 'last_fullsized': '2024-09-17T05:04:52.466916+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:PG 14.4 is not active+clean 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463327+0000', 'last_change': '2024-09-17T05:04:52.463327+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463327+0000', 'last_undegraded': '2024-09-17T05:04:52.463327+0000', 'last_fullsized': '2024-09-17T05:04:52.463327+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:PG 14.7 is not active+clean 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.462471+0000', 'last_change': '2024-09-17T05:04:52.462471+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.462471+0000', 'last_undegraded': '2024-09-17T05:04:52.462471+0000', 'last_fullsized': '2024-09-17T05:04:52.462471+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:PG 14.0 is not active+clean 2024-09-17T05:04:58.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.458957+0000', 'last_change': '2024-09-17T05:04:52.458957+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.458957+0000', 'last_undegraded': '2024-09-17T05:04:52.458957+0000', 'last_fullsized': '2024-09-17T05:04:52.458957+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:PG 14.1 is not active+clean 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.466397+0000', 'last_change': '2024-09-17T05:04:52.466397+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.466397+0000', 'last_undegraded': '2024-09-17T05:04:52.466397+0000', 'last_fullsized': '2024-09-17T05:04:52.466397+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:PG 14.2 is not active+clean 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463352+0000', 'last_change': '2024-09-17T05:04:52.463352+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463352+0000', 'last_undegraded': '2024-09-17T05:04:52.463352+0000', 'last_fullsized': '2024-09-17T05:04:52.463352+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:PG 14.3 is not active+clean 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.459124+0000', 'last_change': '2024-09-17T05:04:52.459124+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.459124+0000', 'last_undegraded': '2024-09-17T05:04:52.459124+0000', 'last_fullsized': '2024-09-17T05:04:52.459124+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:PG 14.f is not active+clean 2024-09-17T05:04:58.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:04:52.463436+0000', 'last_change': '2024-09-17T05:04:52.463436+0000', 'last_active': '2024-09-17T05:04:52.447941+0000', 'last_peered': '2024-09-17T05:04:52.447941+0000', 'last_clean': '2024-09-17T05:04:52.447941+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:04:52.463436+0000', 'last_undegraded': '2024-09-17T05:04:52.463436+0000', 'last_fullsized': '2024-09-17T05:04:52.463436+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'last_clean_scrub_stamp': '2024-09-17T05:04:52.447941+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:04:58.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:04:58.685 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-17T05:04:58.680+0000 7fc50b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:04:58.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 -- 172.21.15.16:0/1261082354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40d1c90 msgr2=0x7fa8d415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 --2- 172.21.15.16:0/1261082354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40d1c90 0x7fa8d415cab0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8d41604f0 tx=0x7fa8bc01c8c0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 -- 172.21.15.16:0/1261082354 shutdown_connections 2024-09-17T05:04:58.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 --2- 172.21.15.16:0/1261082354 >> v2:172.21.15.104:3300/0 conn(0x7fa8d40c9440 0x7fa8d40c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 --2- 172.21.15.16:0/1261082354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40d1c90 0x7fa8d415cab0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 -- 172.21.15.16:0/1261082354 >> 172.21.15.16:0/1261082354 conn(0x7fa8d40d2270 msgr2=0x7fa8d40d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 -- 172.21.15.16:0/1261082354 shutdown_connections 2024-09-17T05:04:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 -- 172.21.15.16:0/1261082354 wait complete. 2024-09-17T05:04:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 Processor -- start 2024-09-17T05:04:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.716+0000 7fa8dac00640 1 -- start start 2024-09-17T05:04:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 0x7fa8d4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8d40cc8c0 0x7fa8d41527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8d40b5940 0x7fa8d41fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4169f00 con 0x7fa8d40c9440 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4169d80 con 0x7fa8d40cc8c0 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d416a080 con 0x7fa8d40b5940 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 0x7fa8d4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 0x7fa8d4152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41266/0 (socket says 172.21.15.16:41266) 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 -- 172.21.15.16:0/3354676238 learned_addr learned my addr 172.21.15.16:0/3354676238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8cbe00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8d40cc8c0 0x7fa8d41527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d8c00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8d40b5940 0x7fa8d41fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 -- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8d40b5940 msgr2=0x7fa8d41fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8d40b5940 0x7fa8d41fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 -- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8d40cc8c0 msgr2=0x7fa8d41527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8d40cc8c0 0x7fa8d41527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 -- 172.21.15.16:0/3354676238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d40b1670 con 0x7fa8d40c9440 2024-09-17T05:04:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d8c00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8d40b5940 0x7fa8d41fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:04:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d3e00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 0x7fa8d4152260 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c390 tx=0x7fa8c400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40052d0 con 0x7fa8d40c9440 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8c4007020 con 0x7fa8d40c9440 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4007470 con 0x7fa8d40c9440 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d40b10f0 con 0x7fa8d40c9440 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d40b1390 con 0x7fa8d40c9440 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.720+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d40c9850 con 0x7fa8d40c9440 2024-09-17T05:04:58.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.724+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8c40067e0 con 0x7fa8d40c9440 2024-09-17T05:04:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.724+0000 7fa8d0c00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa8b0073230 0x7fa8b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.724+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fa8c4093d90 con 0x7fa8d40c9440 2024-09-17T05:04:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.724+0000 7fa8cbe00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa8b0073230 0x7fa8b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.724+0000 7fa8cbe00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa8b0073230 0x7fa8b0075740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0af000 tx=0x7fa8bc0a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.728+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8c405e530 con 0x7fa8d40c9440 2024-09-17T05:04:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.964+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa8d40d0c00 con 0x7fa8d40c9440 2024-09-17T05:04:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8d0c00640 1 -- 172.21.15.16:0/3354676238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1496 (secure 0 0 0) 0x7fa8c4062f60 con 0x7fa8d40c9440 2024-09-17T05:04:58.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:58.970 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":256,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1726549257,"num_in_osds":8,"osd_in_since":1726549246,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":57},{"state_name":"active+clean","count":40}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":273498112,"bytes_avail":772820615168,"bytes_total":773094113280,"inactive_pgs_ratio":0.58762884140014648},"fsmap":{"epoch":52,"btime":"2024-09-17T05:04:56:488386+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6429}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T05:04:05.508046+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa8b0073230 msgr2=0x7fa8b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa8b0073230 0x7fa8b0075740 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0af000 tx=0x7fa8bc0a7040 comp rx=0 tx=0).stop 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 msgr2=0x7fa8d4152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 0x7fa8d4152260 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400c390 tx=0x7fa8c400c850 comp rx=0 tx=0).stop 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 shutdown_connections 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa8b0073230 0x7fa8b0075740 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa8d40b5940 0x7fa8d41fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8d40cc8c0 0x7fa8d41527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 --2- 172.21.15.16:0/3354676238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d40c9440 0x7fa8d4152260 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:58.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 >> 172.21.15.16:0/3354676238 conn(0x7fa8d40d2270 msgr2=0x7fa8d40d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:58.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 shutdown_connections 2024-09-17T05:04:58.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:58.968+0000 7fa8dac00640 1 -- 172.21.15.16:0/3354676238 wait complete. 2024-09-17T05:04:58.992 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:04:58.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 -- 172.21.15.16:0/1917359374 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 msgr2=0x7f14fc0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 --2- 172.21.15.16:0/1917359374 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 0x7f14fc0b3990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f14e40097f0 tx=0x7f14e401c5b0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 -- 172.21.15.16:0/1917359374 shutdown_connections 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 --2- 172.21.15.16:0/1917359374 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 0x7f14fc0b3990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 -- 172.21.15.16:0/1917359374 >> 172.21.15.16:0/1917359374 conn(0x7f14fc06f900 msgr2=0x7f14fc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 -- 172.21.15.16:0/1917359374 shutdown_connections 2024-09-17T05:04:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.148+0000 7f1501800640 1 -- 172.21.15.16:0/1917359374 wait complete. 2024-09-17T05:04:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 Processor -- start 2024-09-17T05:04:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 -- start start 2024-09-17T05:04:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14fc0b3ed0 0x7f14fc0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 0x7f14fc1417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 0x7f14fc143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc16b0c0 con 0x7f14fc15d600 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc16af40 con 0x7f14fc0b3ed0 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc16b240 con 0x7f14fc0cf560 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 0x7f14fc143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 0x7f14fc143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41274/0 (socket says 172.21.15.16:41274) 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 -- 172.21.15.16:0/618735674 learned_addr learned my addr 172.21.15.16:0/618735674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14f3e00640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 0x7f14fc1417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14faa00640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14fc0b3ed0 0x7f14fc0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:59.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 -- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 msgr2=0x7f14fc1417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:59.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 0x7f14fc1417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 -- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14fc0b3ed0 msgr2=0x7f14fc0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14fc0b3ed0 0x7f14fc0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 -- 172.21.15.16:0/618735674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc143d40 con 0x7f14fc15d600 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14fb400640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 0x7f14fc143180 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c7a0 tx=0x7f14f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4017070 con 0x7f14fc15d600 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14f4005070 con 0x7f14fc15d600 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40054a0 con 0x7f14fc15d600 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc144020 con 0x7f14fc15d600 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.152+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1412c0 con 0x7f14fc15d600 2024-09-17T05:04:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.156+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc0b3990 con 0x7f14fc15d600 2024-09-17T05:04:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.156+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f14f401a070 con 0x7f14fc15d600 2024-09-17T05:04:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.156+0000 7f14f2a00640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14d8073230 0x7f14d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:04:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.156+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f14f4093a40 con 0x7f14fc15d600 2024-09-17T05:04:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.156+0000 7f14faa00640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14d8073230 0x7f14d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:04:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.160+0000 7f14faa00640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14d8073230 0x7f14d8075740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f14ec007fd0 tx=0x7f14ec009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:04:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.160+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14f405e1e0 con 0x7f14fc15d600 2024-09-17T05:04:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.332+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14fc0b3530 con 0x7f14d8073230 2024-09-17T05:04:59.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.336+0000 7f14f2a00640 1 -- 172.21.15.16:0/618735674 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247353 (secure 0 0 0) 0x7f14fc0b3530 con 0x7f14d8073230 2024-09-17T05:04:59.343 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:04:59.344 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14d8073230 msgr2=0x7f14d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14d8073230 0x7f14d8075740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f14ec007fd0 tx=0x7f14ec009040 comp rx=0 tx=0).stop 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 msgr2=0x7f14fc143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 0x7f14fc143180 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f14f400c7a0 tx=0x7f14f400cc60 comp rx=0 tx=0).stop 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 shutdown_connections 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f14d8073230 0x7f14d8075740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14fc15d600 0x7f14fc143180 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f14fc0cf560 0x7f14fc1417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 --2- 172.21.15.16:0/618735674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14fc0b3ed0 0x7f14fc0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 >> 172.21.15.16:0/618735674 conn(0x7f14fc06f900 msgr2=0x7f14fc15e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 shutdown_connections 2024-09-17T05:04:59.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:04:59.340+0000 7f1501800640 1 -- 172.21.15.16:0/618735674 wait complete. 2024-09-17T05:04:59.363 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-17T05:04:57.520432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267088,"kb_used_data":9240,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707632,"statfs":{"total":773094113280,"available":772820615168,"internally_reserved":0,"allocated":9461760,"data_stored":4916240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001254"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467906+0000","last_change":"2024-09-17T05:04:52.467906+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467906+0000","last_undegraded":"2024-09-17T05:04:52.467906+0000","last_fullsized":"2024-09-17T05:04:52.467906+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464969+0000","last_change":"2024-09-17T05:04:52.464969+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464969+0000","last_undegraded":"2024-09-17T05:04:52.464969+0000","last_fullsized":"2024-09-17T05:04:52.464969+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468043+0000","last_change":"2024-09-17T05:04:52.468043+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468043+0000","last_undegraded":"2024-09-17T05:04:52.468043+0000","last_fullsized":"2024-09-17T05:04:52.468043+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.472514+0000","last_change":"2024-09-17T05:04:52.472514+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.472514+0000","last_undegraded":"2024-09-17T05:04:52.472514+0000","last_fullsized":"2024-09-17T05:04:52.472514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465690+0000","last_change":"2024-09-17T05:04:52.465690+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465690+0000","last_undegraded":"2024-09-17T05:04:52.465690+0000","last_fullsized":"2024-09-17T05:04:52.465690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460417+0000","last_change":"2024-09-17T05:04:52.460417+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460417+0000","last_undegraded":"2024-09-17T05:04:52.460417+0000","last_fullsized":"2024-09-17T05:04:52.460417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468944+0000","last_change":"2024-09-17T05:04:52.468944+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468944+0000","last_undegraded":"2024-09-17T05:04:52.468944+0000","last_fullsized":"2024-09-17T05:04:52.468944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462576+0000","last_change":"2024-09-17T05:04:52.462576+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462576+0000","last_undegraded":"2024-09-17T05:04:52.462576+0000","last_fullsized":"2024-09-17T05:04:52.462576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.475893+0000","last_change":"2024-09-17T05:04:52.475893+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.475893+0000","last_undegraded":"2024-09-17T05:04:52.475893+0000","last_fullsized":"2024-09-17T05:04:52.475893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469663+0000","last_change":"2024-09-17T05:04:52.469663+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469663+0000","last_undegraded":"2024-09-17T05:04:52.469663+0000","last_fullsized":"2024-09-17T05:04:52.469663+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478888+0000","last_change":"2024-09-17T05:04:52.478888+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478888+0000","last_undegraded":"2024-09-17T05:04:52.478888+0000","last_fullsized":"2024-09-17T05:04:52.478888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469566+0000","last_change":"2024-09-17T05:04:52.469566+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469566+0000","last_undegraded":"2024-09-17T05:04:52.469566+0000","last_fullsized":"2024-09-17T05:04:52.469566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.491308+0000","last_change":"2024-09-17T05:04:53.488884+0000","last_active":"2024-09-17T05:04:53.491308+0000","last_peered":"2024-09-17T05:04:53.491308+0000","last_clean":"2024-09-17T05:04:53.491308+0000","last_became_active":"2024-09-17T05:04:53.488567+0000","last_became_peered":"2024-09-17T05:04:53.488567+0000","last_unstale":"2024-09-17T05:04:53.491308+0000","last_undegraded":"2024-09-17T05:04:53.491308+0000","last_fullsized":"2024-09-17T05:04:53.491308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:16.452112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469017+0000","last_change":"2024-09-17T05:04:52.469017+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469017+0000","last_undegraded":"2024-09-17T05:04:52.469017+0000","last_fullsized":"2024-09-17T05:04:52.469017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467045+0000","last_change":"2024-09-17T05:04:52.467045+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467045+0000","last_undegraded":"2024-09-17T05:04:52.467045+0000","last_fullsized":"2024-09-17T05:04:52.467045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462060+0000","last_change":"2024-09-17T05:04:52.462060+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462060+0000","last_undegraded":"2024-09-17T05:04:52.462060+0000","last_fullsized":"2024-09-17T05:04:52.462060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459791+0000","last_change":"2024-09-17T05:04:52.459791+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459791+0000","last_undegraded":"2024-09-17T05:04:52.459791+0000","last_fullsized":"2024-09-17T05:04:52.459791+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469183+0000","last_change":"2024-09-17T05:04:52.469183+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469183+0000","last_undegraded":"2024-09-17T05:04:52.469183+0000","last_fullsized":"2024-09-17T05:04:52.469183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.465817+0000","last_change":"2024-09-17T05:04:52.465817+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.465817+0000","last_undegraded":"2024-09-17T05:04:52.465817+0000","last_fullsized":"2024-09-17T05:04:52.465817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459213+0000","last_change":"2024-09-17T05:04:52.459213+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459213+0000","last_undegraded":"2024-09-17T05:04:52.459213+0000","last_fullsized":"2024-09-17T05:04:52.459213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464345+0000","last_change":"2024-09-17T05:04:52.464345+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464345+0000","last_undegraded":"2024-09-17T05:04:52.464345+0000","last_fullsized":"2024-09-17T05:04:52.464345+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464433+0000","last_change":"2024-09-17T05:04:52.464433+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464433+0000","last_undegraded":"2024-09-17T05:04:52.464433+0000","last_fullsized":"2024-09-17T05:04:52.464433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464344+0000","last_change":"2024-09-17T05:04:52.464344+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464344+0000","last_undegraded":"2024-09-17T05:04:52.464344+0000","last_fullsized":"2024-09-17T05:04:52.464344+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466713+0000","last_change":"2024-09-17T05:04:52.466713+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466713+0000","last_undegraded":"2024-09-17T05:04:52.466713+0000","last_fullsized":"2024-09-17T05:04:52.466713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464434+0000","last_change":"2024-09-17T05:04:52.464434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464434+0000","last_undegraded":"2024-09-17T05:04:52.464434+0000","last_fullsized":"2024-09-17T05:04:52.464434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463434+0000","last_change":"2024-09-17T05:04:52.463434+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463434+0000","last_undegraded":"2024-09-17T05:04:52.463434+0000","last_fullsized":"2024-09-17T05:04:52.463434+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467567+0000","last_change":"2024-09-17T05:04:52.467567+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467567+0000","last_undegraded":"2024-09-17T05:04:52.467567+0000","last_fullsized":"2024-09-17T05:04:52.467567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.494978+0000","last_change":"2024-09-17T05:04:53.491786+0000","last_active":"2024-09-17T05:04:53.494978+0000","last_peered":"2024-09-17T05:04:53.494978+0000","last_clean":"2024-09-17T05:04:53.494978+0000","last_became_active":"2024-09-17T05:04:53.491402+0000","last_became_peered":"2024-09-17T05:04:53.491402+0000","last_unstale":"2024-09-17T05:04:53.494978+0000","last_undegraded":"2024-09-17T05:04:53.494978+0000","last_fullsized":"2024-09-17T05:04:53.494978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:40.782888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463766+0000","last_change":"2024-09-17T05:04:52.463766+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463766+0000","last_undegraded":"2024-09-17T05:04:52.463766+0000","last_fullsized":"2024-09-17T05:04:52.463766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466787+0000","last_change":"2024-09-17T05:04:52.466787+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466787+0000","last_undegraded":"2024-09-17T05:04:52.466787+0000","last_fullsized":"2024-09-17T05:04:52.466787+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.464914+0000","last_change":"2024-09-17T05:04:52.464914+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.464914+0000","last_undegraded":"2024-09-17T05:04:52.464914+0000","last_fullsized":"2024-09-17T05:04:52.464914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461563+0000","last_change":"2024-09-17T05:04:52.461563+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461563+0000","last_undegraded":"2024-09-17T05:04:52.461563+0000","last_fullsized":"2024-09-17T05:04:52.461563+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467712+0000","last_change":"2024-09-17T05:04:52.467712+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467712+0000","last_undegraded":"2024-09-17T05:04:52.467712+0000","last_fullsized":"2024-09-17T05:04:52.467712+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476554+0000","last_change":"2024-09-17T05:04:52.474787+0000","last_active":"2024-09-17T05:04:52.476554+0000","last_peered":"2024-09-17T05:04:52.476554+0000","last_clean":"2024-09-17T05:04:52.476554+0000","last_became_active":"2024-09-17T05:04:52.474407+0000","last_became_peered":"2024-09-17T05:04:52.474407+0000","last_unstale":"2024-09-17T05:04:52.476554+0000","last_undegraded":"2024-09-17T05:04:52.476554+0000","last_fullsized":"2024-09-17T05:04:52.476554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:40.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.460299+0000","last_change":"2024-09-17T05:04:52.460299+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.460299+0000","last_undegraded":"2024-09-17T05:04:52.460299+0000","last_fullsized":"2024-09-17T05:04:52.460299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489502+0000","last_change":"2024-09-17T05:04:52.488780+0000","last_active":"2024-09-17T05:04:52.489502+0000","last_peered":"2024-09-17T05:04:52.489502+0000","last_clean":"2024-09-17T05:04:52.489502+0000","last_became_active":"2024-09-17T05:04:52.488461+0000","last_became_peered":"2024-09-17T05:04:52.488461+0000","last_unstale":"2024-09-17T05:04:52.489502+0000","last_undegraded":"2024-09-17T05:04:52.489502+0000","last_fullsized":"2024-09-17T05:04:52.489502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:13.981678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477383+0000","last_change":"2024-09-17T05:04:52.477383+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477383+0000","last_undegraded":"2024-09-17T05:04:52.477383+0000","last_fullsized":"2024-09-17T05:04:52.477383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.481366+0000","last_change":"2024-09-17T05:04:52.479681+0000","last_active":"2024-09-17T05:04:52.481366+0000","last_peered":"2024-09-17T05:04:52.481366+0000","last_clean":"2024-09-17T05:04:52.481366+0000","last_became_active":"2024-09-17T05:04:52.477648+0000","last_became_peered":"2024-09-17T05:04:52.477648+0000","last_unstale":"2024-09-17T05:04:52.481366+0000","last_undegraded":"2024-09-17T05:04:52.481366+0000","last_fullsized":"2024-09-17T05:04:52.481366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:58.840632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466570+0000","last_change":"2024-09-17T05:04:52.466570+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466570+0000","last_undegraded":"2024-09-17T05:04:52.466570+0000","last_fullsized":"2024-09-17T05:04:52.466570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490600+0000","last_change":"2024-09-17T05:04:52.487079+0000","last_active":"2024-09-17T05:04:52.490600+0000","last_peered":"2024-09-17T05:04:52.490600+0000","last_clean":"2024-09-17T05:04:52.490600+0000","last_became_active":"2024-09-17T05:04:52.486771+0000","last_became_peered":"2024-09-17T05:04:52.486771+0000","last_unstale":"2024-09-17T05:04:52.490600+0000","last_undegraded":"2024-09-17T05:04:52.490600+0000","last_fullsized":"2024-09-17T05:04:52.490600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:51.755302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.498399+0000","last_change":"2024-09-17T05:04:53.496286+0000","last_active":"2024-09-17T05:04:53.498399+0000","last_peered":"2024-09-17T05:04:53.498399+0000","last_clean":"2024-09-17T05:04:53.498399+0000","last_became_active":"2024-09-17T05:04:53.495971+0000","last_became_peered":"2024-09-17T05:04:53.495971+0000","last_unstale":"2024-09-17T05:04:53.498399+0000","last_undegraded":"2024-09-17T05:04:53.498399+0000","last_fullsized":"2024-09-17T05:04:53.498399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:38.614768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484328+0000","last_change":"2024-09-17T05:04:52.482615+0000","last_active":"2024-09-17T05:04:52.484328+0000","last_peered":"2024-09-17T05:04:52.484328+0000","last_clean":"2024-09-17T05:04:52.484328+0000","last_became_active":"2024-09-17T05:04:52.482267+0000","last_became_peered":"2024-09-17T05:04:52.482267+0000","last_unstale":"2024-09-17T05:04:52.484328+0000","last_undegraded":"2024-09-17T05:04:52.484328+0000","last_fullsized":"2024-09-17T05:04:52.484328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.468323+0000","last_change":"2024-09-17T05:04:52.468323+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.468323+0000","last_undegraded":"2024-09-17T05:04:52.468323+0000","last_fullsized":"2024-09-17T05:04:52.468323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.495357+0000","last_change":"2024-09-17T05:04:52.476398+0000","last_active":"2024-09-17T05:04:53.495357+0000","last_peered":"2024-09-17T05:04:53.495357+0000","last_clean":"2024-09-17T05:04:53.495357+0000","last_became_active":"2024-09-17T05:04:52.476002+0000","last_became_peered":"2024-09-17T05:04:52.476002+0000","last_unstale":"2024-09-17T05:04:53.495357+0000","last_undegraded":"2024-09-17T05:04:53.495357+0000","last_fullsized":"2024-09-17T05:04:53.495357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:23.106184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459964+0000","last_change":"2024-09-17T05:04:52.459964+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459964+0000","last_undegraded":"2024-09-17T05:04:52.459964+0000","last_fullsized":"2024-09-17T05:04:52.459964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485583+0000","last_change":"2024-09-17T05:04:52.482041+0000","last_active":"2024-09-17T05:04:52.485583+0000","last_peered":"2024-09-17T05:04:52.485583+0000","last_clean":"2024-09-17T05:04:52.485583+0000","last_became_active":"2024-09-17T05:04:52.481707+0000","last_became_peered":"2024-09-17T05:04:52.481707+0000","last_unstale":"2024-09-17T05:04:52.485583+0000","last_undegraded":"2024-09-17T05:04:52.485583+0000","last_fullsized":"2024-09-17T05:04:52.485583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:04:32.340896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.467432+0000","last_change":"2024-09-17T05:04:52.467432+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.467432+0000","last_undegraded":"2024-09-17T05:04:52.467432+0000","last_fullsized":"2024-09-17T05:04:52.467432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.484343+0000","last_change":"2024-09-17T05:04:52.480980+0000","last_active":"2024-09-17T05:04:52.484343+0000","last_peered":"2024-09-17T05:04:52.484343+0000","last_clean":"2024-09-17T05:04:52.484343+0000","last_became_active":"2024-09-17T05:04:52.477680+0000","last_became_peered":"2024-09-17T05:04:52.477680+0000","last_unstale":"2024-09-17T05:04:52.484343+0000","last_undegraded":"2024-09-17T05:04:52.484343+0000","last_fullsized":"2024-09-17T05:04:52.484343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:06:06.060774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463355+0000","last_change":"2024-09-17T05:04:52.463355+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463355+0000","last_undegraded":"2024-09-17T05:04:52.463355+0000","last_fullsized":"2024-09-17T05:04:52.463355+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479996+0000","last_change":"2024-09-17T05:04:52.478749+0000","last_active":"2024-09-17T05:04:52.479996+0000","last_peered":"2024-09-17T05:04:52.479996+0000","last_clean":"2024-09-17T05:04:52.479996+0000","last_became_active":"2024-09-17T05:04:52.478315+0000","last_became_peered":"2024-09-17T05:04:52.478315+0000","last_unstale":"2024-09-17T05:04:52.479996+0000","last_undegraded":"2024-09-17T05:04:52.479996+0000","last_fullsized":"2024-09-17T05:04:52.479996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:51.474713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.478423+0000","last_change":"2024-09-17T05:04:52.478423+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.478423+0000","last_undegraded":"2024-09-17T05:04:52.478423+0000","last_fullsized":"2024-09-17T05:04:52.478423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485541+0000","last_change":"2024-09-17T05:04:52.480538+0000","last_active":"2024-09-17T05:04:52.485541+0000","last_peered":"2024-09-17T05:04:52.485541+0000","last_clean":"2024-09-17T05:04:52.485541+0000","last_became_active":"2024-09-17T05:04:52.480130+0000","last_became_peered":"2024-09-17T05:04:52.480130+0000","last_unstale":"2024-09-17T05:04:52.485541+0000","last_undegraded":"2024-09-17T05:04:52.485541+0000","last_fullsized":"2024-09-17T05:04:52.485541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:06.575023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.470865+0000","last_change":"2024-09-17T05:04:52.470865+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.470865+0000","last_undegraded":"2024-09-17T05:04:52.470865+0000","last_fullsized":"2024-09-17T05:04:52.470865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.486751+0000","last_change":"2024-09-17T05:04:52.484495+0000","last_active":"2024-09-17T05:04:52.486751+0000","last_peered":"2024-09-17T05:04:52.486751+0000","last_clean":"2024-09-17T05:04:52.486751+0000","last_became_active":"2024-09-17T05:04:52.483963+0000","last_became_peered":"2024-09-17T05:04:52.483963+0000","last_unstale":"2024-09-17T05:04:52.486751+0000","last_undegraded":"2024-09-17T05:04:52.486751+0000","last_fullsized":"2024-09-17T05:04:52.486751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:04.344757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459224+0000","last_change":"2024-09-17T05:04:52.459224+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459224+0000","last_undegraded":"2024-09-17T05:04:52.459224+0000","last_fullsized":"2024-09-17T05:04:52.459224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490846+0000","last_change":"2024-09-17T05:04:52.486319+0000","last_active":"2024-09-17T05:04:52.490846+0000","last_peered":"2024-09-17T05:04:52.490846+0000","last_clean":"2024-09-17T05:04:52.490846+0000","last_became_active":"2024-09-17T05:04:52.486037+0000","last_became_peered":"2024-09-17T05:04:52.486037+0000","last_unstale":"2024-09-17T05:04:52.490846+0000","last_undegraded":"2024-09-17T05:04:52.490846+0000","last_fullsized":"2024-09-17T05:04:52.490846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:36.202849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466161+0000","last_change":"2024-09-17T05:04:52.466161+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466161+0000","last_undegraded":"2024-09-17T05:04:52.466161+0000","last_fullsized":"2024-09-17T05:04:52.466161+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.479955+0000","last_change":"2024-09-17T05:04:52.478046+0000","last_active":"2024-09-17T05:04:52.479955+0000","last_peered":"2024-09-17T05:04:52.479955+0000","last_clean":"2024-09-17T05:04:52.479955+0000","last_became_active":"2024-09-17T05:04:52.477554+0000","last_became_peered":"2024-09-17T05:04:52.477554+0000","last_unstale":"2024-09-17T05:04:52.479955+0000","last_undegraded":"2024-09-17T05:04:52.479955+0000","last_fullsized":"2024-09-17T05:04:52.479955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:04:31.329683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463789+0000","last_change":"2024-09-17T05:04:52.463789+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463789+0000","last_undegraded":"2024-09-17T05:04:52.463789+0000","last_fullsized":"2024-09-17T05:04:52.463789+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485667+0000","last_change":"2024-09-17T05:04:52.481898+0000","last_active":"2024-09-17T05:04:52.485667+0000","last_peered":"2024-09-17T05:04:52.485667+0000","last_clean":"2024-09-17T05:04:52.485667+0000","last_became_active":"2024-09-17T05:04:52.481562+0000","last_became_peered":"2024-09-17T05:04:52.481562+0000","last_unstale":"2024-09-17T05:04:52.485667+0000","last_undegraded":"2024-09-17T05:04:52.485667+0000","last_fullsized":"2024-09-17T05:04:52.485667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466746+0000","last_change":"2024-09-17T05:04:52.466746+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466746+0000","last_undegraded":"2024-09-17T05:04:52.466746+0000","last_fullsized":"2024-09-17T05:04:52.466746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483399+0000","last_change":"2024-09-17T05:04:52.479084+0000","last_active":"2024-09-17T05:04:52.483399+0000","last_peered":"2024-09-17T05:04:52.483399+0000","last_clean":"2024-09-17T05:04:52.483399+0000","last_became_active":"2024-09-17T05:04:52.478401+0000","last_became_peered":"2024-09-17T05:04:52.478401+0000","last_unstale":"2024-09-17T05:04:52.483399+0000","last_undegraded":"2024-09-17T05:04:52.483399+0000","last_fullsized":"2024-09-17T05:04:52.483399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:08.731176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485456+0000","last_change":"2024-09-17T05:04:53.480121+0000","last_active":"2024-09-17T05:04:53.485456+0000","last_peered":"2024-09-17T05:04:53.485456+0000","last_clean":"2024-09-17T05:04:53.485456+0000","last_became_active":"2024-09-17T05:04:53.479707+0000","last_became_peered":"2024-09-17T05:04:53.479707+0000","last_unstale":"2024-09-17T05:04:53.485456+0000","last_undegraded":"2024-09-17T05:04:53.485456+0000","last_fullsized":"2024-09-17T05:04:53.485456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:54.547283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483105+0000","last_change":"2024-09-17T05:04:52.479663+0000","last_active":"2024-09-17T05:04:52.483105+0000","last_peered":"2024-09-17T05:04:52.483105+0000","last_clean":"2024-09-17T05:04:52.483105+0000","last_became_active":"2024-09-17T05:04:52.479313+0000","last_became_peered":"2024-09-17T05:04:52.479313+0000","last_unstale":"2024-09-17T05:04:52.483105+0000","last_undegraded":"2024-09-17T05:04:52.483105+0000","last_fullsized":"2024-09-17T05:04:52.483105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:56.767884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.489610+0000","last_change":"2024-09-17T05:04:53.483761+0000","last_active":"2024-09-17T05:04:53.489610+0000","last_peered":"2024-09-17T05:04:53.489610+0000","last_clean":"2024-09-17T05:04:53.489610+0000","last_became_active":"2024-09-17T05:04:53.483036+0000","last_became_peered":"2024-09-17T05:04:53.483036+0000","last_unstale":"2024-09-17T05:04:53.489610+0000","last_undegraded":"2024-09-17T05:04:53.489610+0000","last_fullsized":"2024-09-17T05:04:53.489610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:10:05.593222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490673+0000","last_change":"2024-09-17T05:04:52.489855+0000","last_active":"2024-09-17T05:04:52.490673+0000","last_peered":"2024-09-17T05:04:52.490673+0000","last_clean":"2024-09-17T05:04:52.490673+0000","last_became_active":"2024-09-17T05:04:52.487437+0000","last_became_peered":"2024-09-17T05:04:52.487437+0000","last_unstale":"2024-09-17T05:04:52.490673+0000","last_undegraded":"2024-09-17T05:04:52.490673+0000","last_fullsized":"2024-09-17T05:04:52.490673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:39.032030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.477891+0000","last_change":"2024-09-17T05:04:52.477891+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.477891+0000","last_undegraded":"2024-09-17T05:04:52.477891+0000","last_fullsized":"2024-09-17T05:04:52.477891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490880+0000","last_change":"2024-09-17T05:04:52.489186+0000","last_active":"2024-09-17T05:04:52.490880+0000","last_peered":"2024-09-17T05:04:52.490880+0000","last_clean":"2024-09-17T05:04:52.490880+0000","last_became_active":"2024-09-17T05:04:52.484432+0000","last_became_peered":"2024-09-17T05:04:52.484432+0000","last_unstale":"2024-09-17T05:04:52.490880+0000","last_undegraded":"2024-09-17T05:04:52.490880+0000","last_fullsized":"2024-09-17T05:04:52.490880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.803390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462678+0000","last_change":"2024-09-17T05:04:52.462678+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462678+0000","last_undegraded":"2024-09-17T05:04:52.462678+0000","last_fullsized":"2024-09-17T05:04:52.462678+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488159+0000","last_change":"2024-09-17T05:04:52.486351+0000","last_active":"2024-09-17T05:04:52.488159+0000","last_peered":"2024-09-17T05:04:52.488159+0000","last_clean":"2024-09-17T05:04:52.488159+0000","last_became_active":"2024-09-17T05:04:52.486015+0000","last_became_peered":"2024-09-17T05:04:52.486015+0000","last_unstale":"2024-09-17T05:04:52.488159+0000","last_undegraded":"2024-09-17T05:04:52.488159+0000","last_fullsized":"2024-09-17T05:04:52.488159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.544703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.461102+0000","last_change":"2024-09-17T05:04:52.461102+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.461102+0000","last_undegraded":"2024-09-17T05:04:52.461102+0000","last_fullsized":"2024-09-17T05:04:52.461102+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.488049+0000","last_change":"2024-09-17T05:04:52.484413+0000","last_active":"2024-09-17T05:04:52.488049+0000","last_peered":"2024-09-17T05:04:52.488049+0000","last_clean":"2024-09-17T05:04:52.488049+0000","last_became_active":"2024-09-17T05:04:52.483851+0000","last_became_peered":"2024-09-17T05:04:52.483851+0000","last_unstale":"2024-09-17T05:04:52.488049+0000","last_undegraded":"2024-09-17T05:04:52.488049+0000","last_fullsized":"2024-09-17T05:04:52.488049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:23.830173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.469110+0000","last_change":"2024-09-17T05:04:52.469110+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.469110+0000","last_undegraded":"2024-09-17T05:04:52.469110+0000","last_fullsized":"2024-09-17T05:04:52.469110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478334+0000","last_change":"2024-09-17T05:04:52.475068+0000","last_active":"2024-09-17T05:04:52.478334+0000","last_peered":"2024-09-17T05:04:52.478334+0000","last_clean":"2024-09-17T05:04:52.478334+0000","last_became_active":"2024-09-17T05:04:52.474737+0000","last_became_peered":"2024-09-17T05:04:52.474737+0000","last_unstale":"2024-09-17T05:04:52.478334+0000","last_undegraded":"2024-09-17T05:04:52.478334+0000","last_fullsized":"2024-09-17T05:04:52.478334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:29.910066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463931+0000","last_change":"2024-09-17T05:04:52.463931+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463931+0000","last_undegraded":"2024-09-17T05:04:52.463931+0000","last_fullsized":"2024-09-17T05:04:52.463931+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.485373+0000","last_change":"2024-09-17T05:04:52.479150+0000","last_active":"2024-09-17T05:04:52.485373+0000","last_peered":"2024-09-17T05:04:52.485373+0000","last_clean":"2024-09-17T05:04:52.485373+0000","last_became_active":"2024-09-17T05:04:52.476295+0000","last_became_peered":"2024-09-17T05:04:52.476295+0000","last_unstale":"2024-09-17T05:04:52.485373+0000","last_undegraded":"2024-09-17T05:04:52.485373+0000","last_fullsized":"2024-09-17T05:04:52.485373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:28.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466916+0000","last_change":"2024-09-17T05:04:52.466916+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466916+0000","last_undegraded":"2024-09-17T05:04:52.466916+0000","last_fullsized":"2024-09-17T05:04:52.466916+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.483299+0000","last_change":"2024-09-17T05:04:52.482044+0000","last_active":"2024-09-17T05:04:53.483299+0000","last_peered":"2024-09-17T05:04:53.483299+0000","last_clean":"2024-09-17T05:04:53.483299+0000","last_became_active":"2024-09-17T05:04:52.481677+0000","last_became_peered":"2024-09-17T05:04:52.481677+0000","last_unstale":"2024-09-17T05:04:53.483299+0000","last_undegraded":"2024-09-17T05:04:53.483299+0000","last_fullsized":"2024-09-17T05:04:53.483299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:07.449960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463327+0000","last_change":"2024-09-17T05:04:52.463327+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463327+0000","last_undegraded":"2024-09-17T05:04:52.463327+0000","last_fullsized":"2024-09-17T05:04:52.463327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.492052+0000","last_change":"2024-09-17T05:04:52.491484+0000","last_active":"2024-09-17T05:04:52.492052+0000","last_peered":"2024-09-17T05:04:52.492052+0000","last_clean":"2024-09-17T05:04:52.492052+0000","last_became_active":"2024-09-17T05:04:52.490491+0000","last_became_peered":"2024-09-17T05:04:52.490491+0000","last_unstale":"2024-09-17T05:04:52.492052+0000","last_undegraded":"2024-09-17T05:04:52.492052+0000","last_fullsized":"2024-09-17T05:04:52.492052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:39.450191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.485201+0000","last_change":"2024-09-17T05:04:53.480166+0000","last_active":"2024-09-17T05:04:53.485201+0000","last_peered":"2024-09-17T05:04:53.485201+0000","last_clean":"2024-09-17T05:04:53.485201+0000","last_became_active":"2024-09-17T05:04:53.479853+0000","last_became_peered":"2024-09-17T05:04:53.479853+0000","last_unstale":"2024-09-17T05:04:53.485201+0000","last_undegraded":"2024-09-17T05:04:53.485201+0000","last_fullsized":"2024-09-17T05:04:53.485201+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:06:22.807028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.491140+0000","last_change":"2024-09-17T05:04:52.489527+0000","last_active":"2024-09-17T05:04:52.491140+0000","last_peered":"2024-09-17T05:04:52.491140+0000","last_clean":"2024-09-17T05:04:52.491140+0000","last_became_active":"2024-09-17T05:04:52.487078+0000","last_became_peered":"2024-09-17T05:04:52.487078+0000","last_unstale":"2024-09-17T05:04:52.491140+0000","last_undegraded":"2024-09-17T05:04:52.491140+0000","last_fullsized":"2024-09-17T05:04:52.491140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:51:25.546253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T05:04:53.502905+0000","last_change":"2024-09-17T05:04:53.499221+0000","last_active":"2024-09-17T05:04:53.502905+0000","last_peered":"2024-09-17T05:04:53.502905+0000","last_clean":"2024-09-17T05:04:53.502905+0000","last_became_active":"2024-09-17T05:04:53.498898+0000","last_became_peered":"2024-09-17T05:04:53.498898+0000","last_unstale":"2024-09-17T05:04:53.502905+0000","last_undegraded":"2024-09-17T05:04:53.502905+0000","last_fullsized":"2024-09-17T05:04:53.502905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:25.140293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.483295+0000","last_change":"2024-09-17T05:04:52.480615+0000","last_active":"2024-09-17T05:04:52.483295+0000","last_peered":"2024-09-17T05:04:52.483295+0000","last_clean":"2024-09-17T05:04:52.483295+0000","last_became_active":"2024-09-17T05:04:52.479433+0000","last_became_peered":"2024-09-17T05:04:52.479433+0000","last_unstale":"2024-09-17T05:04:52.483295+0000","last_undegraded":"2024-09-17T05:04:52.483295+0000","last_fullsized":"2024-09-17T05:04:52.483295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:04.596504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.462471+0000","last_change":"2024-09-17T05:04:52.462471+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.462471+0000","last_undegraded":"2024-09-17T05:04:52.462471+0000","last_fullsized":"2024-09-17T05:04:52.462471+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.489406+0000","last_change":"2024-09-17T05:04:52.488025+0000","last_active":"2024-09-17T05:04:52.489406+0000","last_peered":"2024-09-17T05:04:52.489406+0000","last_clean":"2024-09-17T05:04:52.489406+0000","last_became_active":"2024-09-17T05:04:52.484280+0000","last_became_peered":"2024-09-17T05:04:52.484280+0000","last_unstale":"2024-09-17T05:04:52.489406+0000","last_undegraded":"2024-09-17T05:04:52.489406+0000","last_fullsized":"2024-09-17T05:04:52.489406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:10.565964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.458957+0000","last_change":"2024-09-17T05:04:52.458957+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.458957+0000","last_undegraded":"2024-09-17T05:04:52.458957+0000","last_fullsized":"2024-09-17T05:04:52.458957+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.478710+0000","last_change":"2024-09-17T05:04:52.474880+0000","last_active":"2024-09-17T05:04:52.478710+0000","last_peered":"2024-09-17T05:04:52.478710+0000","last_clean":"2024-09-17T05:04:52.478710+0000","last_became_active":"2024-09-17T05:04:52.474563+0000","last_became_peered":"2024-09-17T05:04:52.474563+0000","last_unstale":"2024-09-17T05:04:52.478710+0000","last_undegraded":"2024-09-17T05:04:52.478710+0000","last_fullsized":"2024-09-17T05:04:52.478710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:25.901198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.466397+0000","last_change":"2024-09-17T05:04:52.466397+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.466397+0000","last_undegraded":"2024-09-17T05:04:52.466397+0000","last_fullsized":"2024-09-17T05:04:52.466397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476614+0000","last_change":"2024-09-17T05:04:52.474440+0000","last_active":"2024-09-17T05:04:52.476614+0000","last_peered":"2024-09-17T05:04:52.476614+0000","last_clean":"2024-09-17T05:04:52.476614+0000","last_became_active":"2024-09-17T05:04:52.474097+0000","last_became_peered":"2024-09-17T05:04:52.474097+0000","last_unstale":"2024-09-17T05:04:52.476614+0000","last_undegraded":"2024-09-17T05:04:52.476614+0000","last_fullsized":"2024-09-17T05:04:52.476614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:16.647876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463352+0000","last_change":"2024-09-17T05:04:52.463352+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463352+0000","last_undegraded":"2024-09-17T05:04:52.463352+0000","last_fullsized":"2024-09-17T05:04:52.463352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.468511+0000","last_change":"2024-09-17T05:04:52.466652+0000","last_active":"2024-09-17T05:04:52.468511+0000","last_peered":"2024-09-17T05:04:52.468511+0000","last_clean":"2024-09-17T05:04:52.468511+0000","last_became_active":"2024-09-17T05:04:52.466240+0000","last_became_peered":"2024-09-17T05:04:52.466240+0000","last_unstale":"2024-09-17T05:04:52.468511+0000","last_undegraded":"2024-09-17T05:04:52.468511+0000","last_fullsized":"2024-09-17T05:04:52.468511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:25.791675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.459124+0000","last_change":"2024-09-17T05:04:52.459124+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.459124+0000","last_undegraded":"2024-09-17T05:04:52.459124+0000","last_fullsized":"2024-09-17T05:04:52.459124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.476563+0000","last_change":"2024-09-17T05:04:52.472431+0000","last_active":"2024-09-17T05:04:52.476563+0000","last_peered":"2024-09-17T05:04:52.476563+0000","last_clean":"2024-09-17T05:04:52.476563+0000","last_became_active":"2024-09-17T05:04:52.472066+0000","last_became_peered":"2024-09-17T05:04:52.472066+0000","last_unstale":"2024-09-17T05:04:52.476563+0000","last_undegraded":"2024-09-17T05:04:52.476563+0000","last_fullsized":"2024-09-17T05:04:52.476563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:25.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-09-17T05:04:52.463436+0000","last_change":"2024-09-17T05:04:52.463436+0000","last_active":"2024-09-17T05:04:52.447941+0000","last_peered":"2024-09-17T05:04:52.447941+0000","last_clean":"2024-09-17T05:04:52.447941+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:04:52.463436+0000","last_undegraded":"2024-09-17T05:04:52.463436+0000","last_fullsized":"2024-09-17T05:04:52.463436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.490829+0000","last_change":"2024-09-17T05:04:52.486302+0000","last_active":"2024-09-17T05:04:52.490829+0000","last_peered":"2024-09-17T05:04:52.490829+0000","last_clean":"2024-09-17T05:04:52.490829+0000","last_became_active":"2024-09-17T05:04:52.486002+0000","last_became_peered":"2024-09-17T05:04:52.486002+0000","last_unstale":"2024-09-17T05:04:52.490829+0000","last_undegraded":"2024-09-17T05:04:52.490829+0000","last_fullsized":"2024-09-17T05:04:52.490829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:17.637328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":233,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T05:04:52.458028+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:52.458028+0000","last_peered":"2024-09-17T05:04:52.458028+0000","last_clean":"2024-09-17T05:04:52.458028+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:52.458028+0000","last_undegraded":"2024-09-17T05:04:52.458028+0000","last_fullsized":"2024-09-17T05:04:52.458028+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335472,"statfs":{"total":96636764160,"available":96599523328,"internally_reserved":0,"allocated":1040384,"data_stored":470209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36948,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334892,"statfs":{"total":96636764160,"available":96598929408,"internally_reserved":0,"allocated":1634304,"data_stored":1060577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1028096,"data_stored":463542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":1067744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:05:00.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:05:00.604 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:05:02.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 -- 172.21.15.16:0/3785076558 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c073240 msgr2=0x7f2a5c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 --2- 172.21.15.16:0/3785076558 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c073240 0x7f2a5c073650 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009c80 tx=0x7f2a5401cd10 comp rx=0 tx=0).stop 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 -- 172.21.15.16:0/3785076558 shutdown_connections 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 --2- 172.21.15.16:0/3785076558 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c073240 0x7f2a5c073650 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 --2- 172.21.15.16:0/3785076558 >> v2:172.21.15.104:3300/0 conn(0x7f2a5c0b2f50 0x7f2a5c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 -- 172.21.15.16:0/3785076558 >> 172.21.15.16:0/3785076558 conn(0x7f2a5c06f900 msgr2=0x7f2a5c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 -- 172.21.15.16:0/3785076558 shutdown_connections 2024-09-17T05:05:02.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.536+0000 7f2a62c00640 1 -- 172.21.15.16:0/3785076558 wait complete. 2024-09-17T05:05:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 Processor -- start 2024-09-17T05:05:02.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 -- start start 2024-09-17T05:05:02.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 0x7f2a5c147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a5c0b2f50 0x7f2a5c165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c15c0a0 0x7f2a5c165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c168c90 con 0x7f2a5c073240 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c168b10 con 0x7f2a5c0b2f50 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a62c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c168e10 con 0x7f2a5c15c0a0 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 0x7f2a5c147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 0x7f2a5c147930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41284/0 (socket says 172.21.15.16:41284) 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 -- 172.21.15.16:0/1731682585 learned_addr learned my addr 172.21.15.16:0/1731682585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a60c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c15c0a0 0x7f2a5c165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a53e00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a5c0b2f50 0x7f2a5c165030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 -- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c15c0a0 msgr2=0x7f2a5c165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c15c0a0 0x7f2a5c165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 -- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a5c0b2f50 msgr2=0x7f2a5c165030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a5c0b2f50 0x7f2a5c165030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 -- 172.21.15.16:0/1731682585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0ce000 con 0x7f2a5c073240 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a5be00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 0x7f2a5c147930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e9a0 tx=0x7f2a4c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00cd80 con 0x7f2a5c073240 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.540+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a4c004020 con 0x7f2a5c073240 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c0ce2e0 con 0x7f2a5c073240 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c004450 con 0x7f2a5c073240 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c0ce8a0 con 0x7f2a5c073240 2024-09-17T05:05:02.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c073650 con 0x7f2a5c073240 2024-09-17T05:05:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2a4c003600 con 0x7f2a5c073240 2024-09-17T05:05:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a58c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2a38073230 0x7f2a38075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f2a4c094bd0 con 0x7f2a5c073240 2024-09-17T05:05:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.544+0000 7f2a53e00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2a38073230 0x7f2a38075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.548+0000 7f2a53e00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2a38073230 0x7f2a38075740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15bfb0 tx=0x7f2a44009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:02.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.548+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2a4c05f400 con 0x7f2a5c073240 2024-09-17T05:05:02.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.720+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a5c0cc7d0 con 0x7f2a38073230 2024-09-17T05:05:02.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.728+0000 7f2a58c00640 1 -- 172.21.15.16:0/1731682585 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252063 (secure 0 0 0) 0x7f2a5c0cc7d0 con 0x7f2a38073230 2024-09-17T05:05:02.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:02.734 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2a38073230 msgr2=0x7f2a38075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2a38073230 0x7f2a38075740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15bfb0 tx=0x7f2a44009040 comp rx=0 tx=0).stop 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 msgr2=0x7f2a5c147930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 0x7f2a5c147930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e9a0 tx=0x7f2a4c00ee60 comp rx=0 tx=0).stop 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 shutdown_connections 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2a38073230 0x7f2a38075740 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a5c15c0a0 0x7f2a5c165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a5c0b2f50 0x7f2a5c165030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 --2- 172.21.15.16:0/1731682585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a5c073240 0x7f2a5c147930 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 >> 172.21.15.16:0/1731682585 conn(0x7f2a5c06f900 msgr2=0x7f2a5c06d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:02.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 shutdown_connections 2024-09-17T05:05:02.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.732+0000 7f2a62c00640 1 -- 172.21.15.16:0/1731682585 wait complete. 2024-09-17T05:05:02.755 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-17T05:05:01.521634+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267384,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754707336,"statfs":{"total":773094113280,"available":772820312064,"internally_reserved":0,"allocated":9773056,"data_stored":5058100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.063852"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.946018+0000","last_change":"2024-09-17T05:04:53.483724+0000","last_active":"2024-09-17T05:04:54.946018+0000","last_peered":"2024-09-17T05:04:54.946018+0000","last_clean":"2024-09-17T05:04:54.946018+0000","last_became_active":"2024-09-17T05:04:53.483418+0000","last_became_peered":"2024-09-17T05:04:53.483418+0000","last_unstale":"2024-09-17T05:04:54.946018+0000","last_undegraded":"2024-09-17T05:04:54.946018+0000","last_fullsized":"2024-09-17T05:04:54.946018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:43.110744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469679+0000","last_change":"2024-09-17T05:04:53.491092+0000","last_active":"2024-09-17T05:04:55.469679+0000","last_peered":"2024-09-17T05:04:55.469679+0000","last_clean":"2024-09-17T05:04:55.469679+0000","last_became_active":"2024-09-17T05:04:53.490786+0000","last_became_peered":"2024-09-17T05:04:53.490786+0000","last_unstale":"2024-09-17T05:04:55.469679+0000","last_undegraded":"2024-09-17T05:04:55.469679+0000","last_fullsized":"2024-09-17T05:04:55.469679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:16.112684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.485314+0000","last_change":"2024-09-17T05:04:53.489195+0000","last_active":"2024-09-17T05:04:54.485314+0000","last_peered":"2024-09-17T05:04:54.485314+0000","last_clean":"2024-09-17T05:04:54.485314+0000","last_became_active":"2024-09-17T05:04:53.488890+0000","last_became_peered":"2024-09-17T05:04:53.488890+0000","last_unstale":"2024-09-17T05:04:54.485314+0000","last_undegraded":"2024-09-17T05:04:54.485314+0000","last_fullsized":"2024-09-17T05:04:54.485314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:21.267310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484570+0000","last_change":"2024-09-17T05:04:53.481654+0000","last_active":"2024-09-17T05:04:54.484570+0000","last_peered":"2024-09-17T05:04:54.484570+0000","last_clean":"2024-09-17T05:04:54.484570+0000","last_became_active":"2024-09-17T05:04:53.481288+0000","last_became_peered":"2024-09-17T05:04:53.481288+0000","last_unstale":"2024-09-17T05:04:54.484570+0000","last_undegraded":"2024-09-17T05:04:54.484570+0000","last_fullsized":"2024-09-17T05:04:54.484570+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:19.806854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469701+0000","last_change":"2024-09-17T05:04:53.493645+0000","last_active":"2024-09-17T05:04:55.469701+0000","last_peered":"2024-09-17T05:04:55.469701+0000","last_clean":"2024-09-17T05:04:55.469701+0000","last_became_active":"2024-09-17T05:04:53.492689+0000","last_became_peered":"2024-09-17T05:04:53.492689+0000","last_unstale":"2024-09-17T05:04:55.469701+0000","last_undegraded":"2024-09-17T05:04:55.469701+0000","last_fullsized":"2024-09-17T05:04:55.469701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:37.599417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483352+0000","last_change":"2024-09-17T05:04:53.478412+0000","last_active":"2024-09-17T05:04:54.483352+0000","last_peered":"2024-09-17T05:04:54.483352+0000","last_clean":"2024-09-17T05:04:54.483352+0000","last_became_active":"2024-09-17T05:04:53.478067+0000","last_became_peered":"2024-09-17T05:04:53.478067+0000","last_unstale":"2024-09-17T05:04:54.483352+0000","last_undegraded":"2024-09-17T05:04:54.483352+0000","last_fullsized":"2024-09-17T05:04:54.483352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:21.782360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482024+0000","last_change":"2024-09-17T05:04:53.483950+0000","last_active":"2024-09-17T05:04:54.482024+0000","last_peered":"2024-09-17T05:04:54.482024+0000","last_clean":"2024-09-17T05:04:54.482024+0000","last_became_active":"2024-09-17T05:04:53.483604+0000","last_became_peered":"2024-09-17T05:04:53.483604+0000","last_unstale":"2024-09-17T05:04:54.482024+0000","last_undegraded":"2024-09-17T05:04:54.482024+0000","last_fullsized":"2024-09-17T05:04:54.482024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:49:21.349043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483071+0000","last_change":"2024-09-17T05:04:53.466318+0000","last_active":"2024-09-17T05:04:54.483071+0000","last_peered":"2024-09-17T05:04:54.483071+0000","last_clean":"2024-09-17T05:04:54.483071+0000","last_became_active":"2024-09-17T05:04:53.465948+0000","last_became_peered":"2024-09-17T05:04:53.465948+0000","last_unstale":"2024-09-17T05:04:54.483071+0000","last_undegraded":"2024-09-17T05:04:54.483071+0000","last_fullsized":"2024-09-17T05:04:54.483071+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:52:45.757579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482556+0000","last_change":"2024-09-17T05:04:53.497544+0000","last_active":"2024-09-17T05:04:54.482556+0000","last_peered":"2024-09-17T05:04:54.482556+0000","last_clean":"2024-09-17T05:04:54.482556+0000","last_became_active":"2024-09-17T05:04:53.497213+0000","last_became_peered":"2024-09-17T05:04:53.497213+0000","last_unstale":"2024-09-17T05:04:54.482556+0000","last_undegraded":"2024-09-17T05:04:54.482556+0000","last_fullsized":"2024-09-17T05:04:54.482556+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:41.899609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483868+0000","last_change":"2024-09-17T05:04:53.497328+0000","last_active":"2024-09-17T05:04:54.483868+0000","last_peered":"2024-09-17T05:04:54.483868+0000","last_clean":"2024-09-17T05:04:54.483868+0000","last_became_active":"2024-09-17T05:04:53.497024+0000","last_became_peered":"2024-09-17T05:04:53.497024+0000","last_unstale":"2024-09-17T05:04:54.483868+0000","last_undegraded":"2024-09-17T05:04:54.483868+0000","last_fullsized":"2024-09-17T05:04:54.483868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:52.758766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482184+0000","last_change":"2024-09-17T05:04:53.483195+0000","last_active":"2024-09-17T05:04:54.482184+0000","last_peered":"2024-09-17T05:04:54.482184+0000","last_clean":"2024-09-17T05:04:54.482184+0000","last_became_active":"2024-09-17T05:04:53.482843+0000","last_became_peered":"2024-09-17T05:04:53.482843+0000","last_unstale":"2024-09-17T05:04:54.482184+0000","last_undegraded":"2024-09-17T05:04:54.482184+0000","last_fullsized":"2024-09-17T05:04:54.482184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:43.094484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484869+0000","last_change":"2024-09-17T05:04:53.495595+0000","last_active":"2024-09-17T05:04:54.484869+0000","last_peered":"2024-09-17T05:04:54.484869+0000","last_clean":"2024-09-17T05:04:54.484869+0000","last_became_active":"2024-09-17T05:04:53.490342+0000","last_became_peered":"2024-09-17T05:04:53.490342+0000","last_unstale":"2024-09-17T05:04:54.484869+0000","last_undegraded":"2024-09-17T05:04:54.484869+0000","last_fullsized":"2024-09-17T05:04:54.484869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:44.714435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672312+0000","last_change":"2024-09-17T05:04:53.488884+0000","last_active":"2024-09-17T05:04:55.672312+0000","last_peered":"2024-09-17T05:04:55.672312+0000","last_clean":"2024-09-17T05:04:55.672312+0000","last_became_active":"2024-09-17T05:04:53.488567+0000","last_became_peered":"2024-09-17T05:04:53.488567+0000","last_unstale":"2024-09-17T05:04:55.672312+0000","last_undegraded":"2024-09-17T05:04:55.672312+0000","last_fullsized":"2024-09-17T05:04:55.672312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:22.661150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.671968+0000","last_change":"2024-09-17T05:04:53.500800+0000","last_active":"2024-09-17T05:04:55.671968+0000","last_peered":"2024-09-17T05:04:55.671968+0000","last_clean":"2024-09-17T05:04:55.671968+0000","last_became_active":"2024-09-17T05:04:53.499589+0000","last_became_peered":"2024-09-17T05:04:53.499589+0000","last_unstale":"2024-09-17T05:04:55.671968+0000","last_undegraded":"2024-09-17T05:04:55.671968+0000","last_fullsized":"2024-09-17T05:04:55.671968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:40.035494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482868+0000","last_change":"2024-09-17T05:04:53.486001+0000","last_active":"2024-09-17T05:04:54.482868+0000","last_peered":"2024-09-17T05:04:54.482868+0000","last_clean":"2024-09-17T05:04:54.482868+0000","last_became_active":"2024-09-17T05:04:53.483580+0000","last_became_peered":"2024-09-17T05:04:53.483580+0000","last_unstale":"2024-09-17T05:04:54.482868+0000","last_undegraded":"2024-09-17T05:04:54.482868+0000","last_fullsized":"2024-09-17T05:04:54.482868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:29.242635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482483+0000","last_change":"2024-09-17T05:04:53.487680+0000","last_active":"2024-09-17T05:04:54.482483+0000","last_peered":"2024-09-17T05:04:54.482483+0000","last_clean":"2024-09-17T05:04:54.482483+0000","last_became_active":"2024-09-17T05:04:53.487312+0000","last_became_peered":"2024-09-17T05:04:53.487312+0000","last_unstale":"2024-09-17T05:04:54.482483+0000","last_undegraded":"2024-09-17T05:04:54.482483+0000","last_fullsized":"2024-09-17T05:04:54.482483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:42:57.107973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484346+0000","last_change":"2024-09-17T05:04:53.479527+0000","last_active":"2024-09-17T05:04:54.484346+0000","last_peered":"2024-09-17T05:04:54.484346+0000","last_clean":"2024-09-17T05:04:54.484346+0000","last_became_active":"2024-09-17T05:04:53.479215+0000","last_became_peered":"2024-09-17T05:04:53.479215+0000","last_unstale":"2024-09-17T05:04:54.484346+0000","last_undegraded":"2024-09-17T05:04:54.484346+0000","last_fullsized":"2024-09-17T05:04:54.484346+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:22.770497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484217+0000","last_change":"2024-09-17T05:04:53.490595+0000","last_active":"2024-09-17T05:04:54.484217+0000","last_peered":"2024-09-17T05:04:54.484217+0000","last_clean":"2024-09-17T05:04:54.484217+0000","last_became_active":"2024-09-17T05:04:53.490061+0000","last_became_peered":"2024-09-17T05:04:53.490061+0000","last_unstale":"2024-09-17T05:04:54.484217+0000","last_undegraded":"2024-09-17T05:04:54.484217+0000","last_fullsized":"2024-09-17T05:04:54.484217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:14.985142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672893+0000","last_change":"2024-09-17T05:04:53.483406+0000","last_active":"2024-09-17T05:04:55.672893+0000","last_peered":"2024-09-17T05:04:55.672893+0000","last_clean":"2024-09-17T05:04:55.672893+0000","last_became_active":"2024-09-17T05:04:53.482547+0000","last_became_peered":"2024-09-17T05:04:53.482547+0000","last_unstale":"2024-09-17T05:04:55.672893+0000","last_undegraded":"2024-09-17T05:04:55.672893+0000","last_fullsized":"2024-09-17T05:04:55.672893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:16.515570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483256+0000","last_change":"2024-09-17T05:04:53.478511+0000","last_active":"2024-09-17T05:04:54.483256+0000","last_peered":"2024-09-17T05:04:54.483256+0000","last_clean":"2024-09-17T05:04:54.483256+0000","last_became_active":"2024-09-17T05:04:53.478013+0000","last_became_peered":"2024-09-17T05:04:53.478013+0000","last_unstale":"2024-09-17T05:04:54.483256+0000","last_undegraded":"2024-09-17T05:04:54.483256+0000","last_fullsized":"2024-09-17T05:04:54.483256+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:02:49.487048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469755+0000","last_change":"2024-09-17T05:04:53.493228+0000","last_active":"2024-09-17T05:04:55.469755+0000","last_peered":"2024-09-17T05:04:55.469755+0000","last_clean":"2024-09-17T05:04:55.469755+0000","last_became_active":"2024-09-17T05:04:53.492285+0000","last_became_peered":"2024-09-17T05:04:53.492285+0000","last_unstale":"2024-09-17T05:04:55.469755+0000","last_undegraded":"2024-09-17T05:04:55.469755+0000","last_fullsized":"2024-09-17T05:04:55.469755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:15:59.553785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672685+0000","last_change":"2024-09-17T05:04:53.489459+0000","last_active":"2024-09-17T05:04:55.672685+0000","last_peered":"2024-09-17T05:04:55.672685+0000","last_clean":"2024-09-17T05:04:55.672685+0000","last_became_active":"2024-09-17T05:04:53.487553+0000","last_became_peered":"2024-09-17T05:04:53.487553+0000","last_unstale":"2024-09-17T05:04:55.672685+0000","last_undegraded":"2024-09-17T05:04:55.672685+0000","last_fullsized":"2024-09-17T05:04:55.672685+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:08.436991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469205+0000","last_change":"2024-09-17T05:04:53.489332+0000","last_active":"2024-09-17T05:04:55.469205+0000","last_peered":"2024-09-17T05:04:55.469205+0000","last_clean":"2024-09-17T05:04:55.469205+0000","last_became_active":"2024-09-17T05:04:53.489040+0000","last_became_peered":"2024-09-17T05:04:53.489040+0000","last_unstale":"2024-09-17T05:04:55.469205+0000","last_undegraded":"2024-09-17T05:04:55.469205+0000","last_fullsized":"2024-09-17T05:04:55.469205+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:30.913530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945573+0000","last_change":"2024-09-17T05:04:53.491953+0000","last_active":"2024-09-17T05:04:54.945573+0000","last_peered":"2024-09-17T05:04:54.945573+0000","last_clean":"2024-09-17T05:04:54.945573+0000","last_became_active":"2024-09-17T05:04:53.491302+0000","last_became_peered":"2024-09-17T05:04:53.491302+0000","last_unstale":"2024-09-17T05:04:54.945573+0000","last_undegraded":"2024-09-17T05:04:54.945573+0000","last_fullsized":"2024-09-17T05:04:54.945573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:51.026487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469729+0000","last_change":"2024-09-17T05:04:53.498370+0000","last_active":"2024-09-17T05:04:55.469729+0000","last_peered":"2024-09-17T05:04:55.469729+0000","last_clean":"2024-09-17T05:04:55.469729+0000","last_became_active":"2024-09-17T05:04:53.497881+0000","last_became_peered":"2024-09-17T05:04:53.497881+0000","last_unstale":"2024-09-17T05:04:55.469729+0000","last_undegraded":"2024-09-17T05:04:55.469729+0000","last_fullsized":"2024-09-17T05:04:55.469729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:51:08.379177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469958+0000","last_change":"2024-09-17T05:04:53.491043+0000","last_active":"2024-09-17T05:04:55.469958+0000","last_peered":"2024-09-17T05:04:55.469958+0000","last_clean":"2024-09-17T05:04:55.469958+0000","last_became_active":"2024-09-17T05:04:53.490704+0000","last_became_peered":"2024-09-17T05:04:53.490704+0000","last_unstale":"2024-09-17T05:04:55.469958+0000","last_undegraded":"2024-09-17T05:04:55.469958+0000","last_fullsized":"2024-09-17T05:04:55.469958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:16.515570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.485383+0000","last_change":"2024-09-17T05:04:53.487237+0000","last_active":"2024-09-17T05:04:54.485383+0000","last_peered":"2024-09-17T05:04:54.485383+0000","last_clean":"2024-09-17T05:04:54.485383+0000","last_became_active":"2024-09-17T05:04:53.485973+0000","last_became_peered":"2024-09-17T05:04:53.485973+0000","last_unstale":"2024-09-17T05:04:54.485383+0000","last_undegraded":"2024-09-17T05:04:54.485383+0000","last_fullsized":"2024-09-17T05:04:54.485383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:26.874427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672607+0000","last_change":"2024-09-17T05:04:53.491786+0000","last_active":"2024-09-17T05:04:55.672607+0000","last_peered":"2024-09-17T05:04:55.672607+0000","last_clean":"2024-09-17T05:04:55.672607+0000","last_became_active":"2024-09-17T05:04:53.491402+0000","last_became_peered":"2024-09-17T05:04:53.491402+0000","last_unstale":"2024-09-17T05:04:55.672607+0000","last_undegraded":"2024-09-17T05:04:55.672607+0000","last_fullsized":"2024-09-17T05:04:55.672607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:20.817493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.470095+0000","last_change":"2024-09-17T05:04:53.488904+0000","last_active":"2024-09-17T05:04:55.470095+0000","last_peered":"2024-09-17T05:04:55.470095+0000","last_clean":"2024-09-17T05:04:55.470095+0000","last_became_active":"2024-09-17T05:04:53.488597+0000","last_became_peered":"2024-09-17T05:04:53.488597+0000","last_unstale":"2024-09-17T05:04:55.470095+0000","last_undegraded":"2024-09-17T05:04:55.470095+0000","last_fullsized":"2024-09-17T05:04:55.470095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:51.026487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469159+0000","last_change":"2024-09-17T05:04:53.488491+0000","last_active":"2024-09-17T05:04:55.469159+0000","last_peered":"2024-09-17T05:04:55.469159+0000","last_clean":"2024-09-17T05:04:55.469159+0000","last_became_active":"2024-09-17T05:04:53.488019+0000","last_became_peered":"2024-09-17T05:04:53.488019+0000","last_unstale":"2024-09-17T05:04:55.469159+0000","last_undegraded":"2024-09-17T05:04:55.469159+0000","last_fullsized":"2024-09-17T05:04:55.469159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:08.436991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672339+0000","last_change":"2024-09-17T05:04:53.480032+0000","last_active":"2024-09-17T05:04:55.672339+0000","last_peered":"2024-09-17T05:04:55.672339+0000","last_clean":"2024-09-17T05:04:55.672339+0000","last_became_active":"2024-09-17T05:04:53.479695+0000","last_became_peered":"2024-09-17T05:04:53.479695+0000","last_unstale":"2024-09-17T05:04:55.672339+0000","last_undegraded":"2024-09-17T05:04:55.672339+0000","last_fullsized":"2024-09-17T05:04:55.672339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:15.123320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482771+0000","last_change":"2024-09-17T05:04:53.483777+0000","last_active":"2024-09-17T05:04:54.482771+0000","last_peered":"2024-09-17T05:04:54.482771+0000","last_clean":"2024-09-17T05:04:54.482771+0000","last_became_active":"2024-09-17T05:04:53.483469+0000","last_became_peered":"2024-09-17T05:04:53.483469+0000","last_unstale":"2024-09-17T05:04:54.482771+0000","last_undegraded":"2024-09-17T05:04:54.482771+0000","last_fullsized":"2024-09-17T05:04:54.482771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:09.891109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945775+0000","last_change":"2024-09-17T05:04:53.484864+0000","last_active":"2024-09-17T05:04:54.945775+0000","last_peered":"2024-09-17T05:04:54.945775+0000","last_clean":"2024-09-17T05:04:54.945775+0000","last_became_active":"2024-09-17T05:04:53.484538+0000","last_became_peered":"2024-09-17T05:04:53.484538+0000","last_unstale":"2024-09-17T05:04:54.945775+0000","last_undegraded":"2024-09-17T05:04:54.945775+0000","last_fullsized":"2024-09-17T05:04:54.945775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.388958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"86'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.470256+0000","last_change":"2024-09-17T05:04:52.474787+0000","last_active":"2024-09-17T05:04:55.470256+0000","last_peered":"2024-09-17T05:04:55.470256+0000","last_clean":"2024-09-17T05:04:55.470256+0000","last_became_active":"2024-09-17T05:04:52.474407+0000","last_became_peered":"2024-09-17T05:04:52.474407+0000","last_unstale":"2024-09-17T05:04:55.470256+0000","last_undegraded":"2024-09-17T05:04:55.470256+0000","last_fullsized":"2024-09-17T05:04:55.470256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:31.505656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483671+0000","last_change":"2024-09-17T05:04:53.480090+0000","last_active":"2024-09-17T05:04:54.483671+0000","last_peered":"2024-09-17T05:04:54.483671+0000","last_clean":"2024-09-17T05:04:54.483671+0000","last_became_active":"2024-09-17T05:04:53.479559+0000","last_became_peered":"2024-09-17T05:04:53.479559+0000","last_unstale":"2024-09-17T05:04:54.483671+0000","last_undegraded":"2024-09-17T05:04:54.483671+0000","last_fullsized":"2024-09-17T05:04:54.483671+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:53.158365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672187+0000","last_change":"2024-09-17T05:04:52.488780+0000","last_active":"2024-09-17T05:04:55.672187+0000","last_peered":"2024-09-17T05:04:55.672187+0000","last_clean":"2024-09-17T05:04:55.672187+0000","last_became_active":"2024-09-17T05:04:52.488461+0000","last_became_peered":"2024-09-17T05:04:52.488461+0000","last_unstale":"2024-09-17T05:04:55.672187+0000","last_undegraded":"2024-09-17T05:04:55.672187+0000","last_fullsized":"2024-09-17T05:04:55.672187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:38:32.403628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482402+0000","last_change":"2024-09-17T05:04:53.486999+0000","last_active":"2024-09-17T05:04:54.482402+0000","last_peered":"2024-09-17T05:04:54.482402+0000","last_clean":"2024-09-17T05:04:54.482402+0000","last_became_active":"2024-09-17T05:04:53.486644+0000","last_became_peered":"2024-09-17T05:04:53.486644+0000","last_unstale":"2024-09-17T05:04:54.482402+0000","last_undegraded":"2024-09-17T05:04:54.482402+0000","last_fullsized":"2024-09-17T05:04:54.482402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:41.870458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469574+0000","last_change":"2024-09-17T05:04:52.479681+0000","last_active":"2024-09-17T05:04:55.469574+0000","last_peered":"2024-09-17T05:04:55.469574+0000","last_clean":"2024-09-17T05:04:55.469574+0000","last_became_active":"2024-09-17T05:04:52.477648+0000","last_became_peered":"2024-09-17T05:04:52.477648+0000","last_unstale":"2024-09-17T05:04:55.469574+0000","last_undegraded":"2024-09-17T05:04:55.469574+0000","last_fullsized":"2024-09-17T05:04:55.469574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:51.755302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484638+0000","last_change":"2024-09-17T05:04:53.498651+0000","last_active":"2024-09-17T05:04:54.484638+0000","last_peered":"2024-09-17T05:04:54.484638+0000","last_clean":"2024-09-17T05:04:54.484638+0000","last_became_active":"2024-09-17T05:04:53.498319+0000","last_became_peered":"2024-09-17T05:04:53.498319+0000","last_unstale":"2024-09-17T05:04:54.484638+0000","last_undegraded":"2024-09-17T05:04:54.484638+0000","last_fullsized":"2024-09-17T05:04:54.484638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:03:24.166369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672124+0000","last_change":"2024-09-17T05:04:52.487079+0000","last_active":"2024-09-17T05:04:55.672124+0000","last_peered":"2024-09-17T05:04:55.672124+0000","last_clean":"2024-09-17T05:04:55.672124+0000","last_became_active":"2024-09-17T05:04:52.486771+0000","last_became_peered":"2024-09-17T05:04:52.486771+0000","last_unstale":"2024-09-17T05:04:55.672124+0000","last_undegraded":"2024-09-17T05:04:55.672124+0000","last_fullsized":"2024-09-17T05:04:55.672124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:52.622308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672247+0000","last_change":"2024-09-17T05:04:53.496286+0000","last_active":"2024-09-17T05:04:55.672247+0000","last_peered":"2024-09-17T05:04:55.672247+0000","last_clean":"2024-09-17T05:04:55.672247+0000","last_became_active":"2024-09-17T05:04:53.495971+0000","last_became_peered":"2024-09-17T05:04:53.495971+0000","last_unstale":"2024-09-17T05:04:55.672247+0000","last_undegraded":"2024-09-17T05:04:55.672247+0000","last_fullsized":"2024-09-17T05:04:55.672247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:12:23.793483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672760+0000","last_change":"2024-09-17T05:04:52.482615+0000","last_active":"2024-09-17T05:04:55.672760+0000","last_peered":"2024-09-17T05:04:55.672760+0000","last_clean":"2024-09-17T05:04:55.672760+0000","last_became_active":"2024-09-17T05:04:52.482267+0000","last_became_peered":"2024-09-17T05:04:52.482267+0000","last_unstale":"2024-09-17T05:04:55.672760+0000","last_undegraded":"2024-09-17T05:04:55.672760+0000","last_fullsized":"2024-09-17T05:04:55.672760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:38.962359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482093+0000","last_change":"2024-09-17T05:04:53.491419+0000","last_active":"2024-09-17T05:04:54.482093+0000","last_peered":"2024-09-17T05:04:54.482093+0000","last_clean":"2024-09-17T05:04:54.482093+0000","last_became_active":"2024-09-17T05:04:53.491007+0000","last_became_peered":"2024-09-17T05:04:53.491007+0000","last_unstale":"2024-09-17T05:04:54.482093+0000","last_undegraded":"2024-09-17T05:04:54.482093+0000","last_fullsized":"2024-09-17T05:04:54.482093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:30.614105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672207+0000","last_change":"2024-09-17T05:04:52.476398+0000","last_active":"2024-09-17T05:04:55.672207+0000","last_peered":"2024-09-17T05:04:55.672207+0000","last_clean":"2024-09-17T05:04:55.672207+0000","last_became_active":"2024-09-17T05:04:52.476002+0000","last_became_peered":"2024-09-17T05:04:52.476002+0000","last_unstale":"2024-09-17T05:04:55.672207+0000","last_undegraded":"2024-09-17T05:04:55.672207+0000","last_fullsized":"2024-09-17T05:04:55.672207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:23.106184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483466+0000","last_change":"2024-09-17T05:04:53.486587+0000","last_active":"2024-09-17T05:04:54.483466+0000","last_peered":"2024-09-17T05:04:54.483466+0000","last_clean":"2024-09-17T05:04:54.483466+0000","last_became_active":"2024-09-17T05:04:53.486247+0000","last_became_peered":"2024-09-17T05:04:53.486247+0000","last_unstale":"2024-09-17T05:04:54.483466+0000","last_undegraded":"2024-09-17T05:04:54.483466+0000","last_fullsized":"2024-09-17T05:04:54.483466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:34.395731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483580+0000","last_change":"2024-09-17T05:04:52.482041+0000","last_active":"2024-09-17T05:04:54.483580+0000","last_peered":"2024-09-17T05:04:54.483580+0000","last_clean":"2024-09-17T05:04:54.483580+0000","last_became_active":"2024-09-17T05:04:52.481707+0000","last_became_peered":"2024-09-17T05:04:52.481707+0000","last_unstale":"2024-09-17T05:04:54.483580+0000","last_undegraded":"2024-09-17T05:04:54.483580+0000","last_fullsized":"2024-09-17T05:04:54.483580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:25.791675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945908+0000","last_change":"2024-09-17T05:04:53.485123+0000","last_active":"2024-09-17T05:04:54.945908+0000","last_peered":"2024-09-17T05:04:54.945908+0000","last_clean":"2024-09-17T05:04:54.945908+0000","last_became_active":"2024-09-17T05:04:53.484796+0000","last_became_peered":"2024-09-17T05:04:53.484796+0000","last_unstale":"2024-09-17T05:04:54.945908+0000","last_undegraded":"2024-09-17T05:04:54.945908+0000","last_fullsized":"2024-09-17T05:04:54.945908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:38.867601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469151+0000","last_change":"2024-09-17T05:04:52.480980+0000","last_active":"2024-09-17T05:04:55.469151+0000","last_peered":"2024-09-17T05:04:55.469151+0000","last_clean":"2024-09-17T05:04:55.469151+0000","last_became_active":"2024-09-17T05:04:52.477680+0000","last_became_peered":"2024-09-17T05:04:52.477680+0000","last_unstale":"2024-09-17T05:04:55.469151+0000","last_undegraded":"2024-09-17T05:04:55.469151+0000","last_fullsized":"2024-09-17T05:04:55.469151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:01.240703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469442+0000","last_change":"2024-09-17T05:04:53.487329+0000","last_active":"2024-09-17T05:04:55.469442+0000","last_peered":"2024-09-17T05:04:55.469442+0000","last_clean":"2024-09-17T05:04:55.469442+0000","last_became_active":"2024-09-17T05:04:53.487011+0000","last_became_peered":"2024-09-17T05:04:53.487011+0000","last_unstale":"2024-09-17T05:04:55.469442+0000","last_undegraded":"2024-09-17T05:04:55.469442+0000","last_fullsized":"2024-09-17T05:04:55.469442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:15.123320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.470347+0000","last_change":"2024-09-17T05:04:52.478749+0000","last_active":"2024-09-17T05:04:55.470347+0000","last_peered":"2024-09-17T05:04:55.470347+0000","last_clean":"2024-09-17T05:04:55.470347+0000","last_became_active":"2024-09-17T05:04:52.478315+0000","last_became_peered":"2024-09-17T05:04:52.478315+0000","last_unstale":"2024-09-17T05:04:55.470347+0000","last_undegraded":"2024-09-17T05:04:55.470347+0000","last_fullsized":"2024-09-17T05:04:55.470347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.544703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482470+0000","last_change":"2024-09-17T05:04:53.488719+0000","last_active":"2024-09-17T05:04:54.482470+0000","last_peered":"2024-09-17T05:04:54.482470+0000","last_clean":"2024-09-17T05:04:54.482470+0000","last_became_active":"2024-09-17T05:04:53.488333+0000","last_became_peered":"2024-09-17T05:04:53.488333+0000","last_unstale":"2024-09-17T05:04:54.482470+0000","last_undegraded":"2024-09-17T05:04:54.482470+0000","last_fullsized":"2024-09-17T05:04:54.482470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:21:48.243620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484324+0000","last_change":"2024-09-17T05:04:52.480538+0000","last_active":"2024-09-17T05:04:54.484324+0000","last_peered":"2024-09-17T05:04:54.484324+0000","last_clean":"2024-09-17T05:04:54.484324+0000","last_became_active":"2024-09-17T05:04:52.480130+0000","last_became_peered":"2024-09-17T05:04:52.480130+0000","last_unstale":"2024-09-17T05:04:54.484324+0000","last_undegraded":"2024-09-17T05:04:54.484324+0000","last_fullsized":"2024-09-17T05:04:54.484324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:25.263416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.946032+0000","last_change":"2024-09-17T05:04:53.502754+0000","last_active":"2024-09-17T05:04:54.946032+0000","last_peered":"2024-09-17T05:04:54.946032+0000","last_clean":"2024-09-17T05:04:54.946032+0000","last_became_active":"2024-09-17T05:04:53.502357+0000","last_became_peered":"2024-09-17T05:04:53.502357+0000","last_unstale":"2024-09-17T05:04:54.946032+0000","last_undegraded":"2024-09-17T05:04:54.946032+0000","last_fullsized":"2024-09-17T05:04:54.946032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:22.071743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"86'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482723+0000","last_change":"2024-09-17T05:04:52.484495+0000","last_active":"2024-09-17T05:04:54.482723+0000","last_peered":"2024-09-17T05:04:54.482723+0000","last_clean":"2024-09-17T05:04:54.482723+0000","last_became_active":"2024-09-17T05:04:52.483963+0000","last_became_peered":"2024-09-17T05:04:52.483963+0000","last_unstale":"2024-09-17T05:04:54.482723+0000","last_undegraded":"2024-09-17T05:04:54.482723+0000","last_fullsized":"2024-09-17T05:04:54.482723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:42.107281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484692+0000","last_change":"2024-09-17T05:04:53.466409+0000","last_active":"2024-09-17T05:04:54.484692+0000","last_peered":"2024-09-17T05:04:54.484692+0000","last_clean":"2024-09-17T05:04:54.484692+0000","last_became_active":"2024-09-17T05:04:53.466055+0000","last_became_peered":"2024-09-17T05:04:53.466055+0000","last_unstale":"2024-09-17T05:04:54.484692+0000","last_undegraded":"2024-09-17T05:04:54.484692+0000","last_fullsized":"2024-09-17T05:04:54.484692+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:30.126273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"86'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.946014+0000","last_change":"2024-09-17T05:04:52.486319+0000","last_active":"2024-09-17T05:04:54.946014+0000","last_peered":"2024-09-17T05:04:54.946014+0000","last_clean":"2024-09-17T05:04:54.946014+0000","last_became_active":"2024-09-17T05:04:52.486037+0000","last_became_peered":"2024-09-17T05:04:52.486037+0000","last_unstale":"2024-09-17T05:04:54.946014+0000","last_undegraded":"2024-09-17T05:04:54.946014+0000","last_fullsized":"2024-09-17T05:04:54.946014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:38.962359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945549+0000","last_change":"2024-09-17T05:04:53.491589+0000","last_active":"2024-09-17T05:04:54.945549+0000","last_peered":"2024-09-17T05:04:54.945549+0000","last_clean":"2024-09-17T05:04:54.945549+0000","last_became_active":"2024-09-17T05:04:53.490866+0000","last_became_peered":"2024-09-17T05:04:53.490866+0000","last_unstale":"2024-09-17T05:04:54.945549+0000","last_undegraded":"2024-09-17T05:04:54.945549+0000","last_fullsized":"2024-09-17T05:04:54.945549+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:52.935773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.470106+0000","last_change":"2024-09-17T05:04:52.478046+0000","last_active":"2024-09-17T05:04:55.470106+0000","last_peered":"2024-09-17T05:04:55.470106+0000","last_clean":"2024-09-17T05:04:55.470106+0000","last_became_active":"2024-09-17T05:04:52.477554+0000","last_became_peered":"2024-09-17T05:04:52.477554+0000","last_unstale":"2024-09-17T05:04:55.470106+0000","last_undegraded":"2024-09-17T05:04:55.470106+0000","last_fullsized":"2024-09-17T05:04:55.470106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:05:18.803390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469128+0000","last_change":"2024-09-17T05:04:53.487456+0000","last_active":"2024-09-17T05:04:55.469128+0000","last_peered":"2024-09-17T05:04:55.469128+0000","last_clean":"2024-09-17T05:04:55.469128+0000","last_became_active":"2024-09-17T05:04:53.487169+0000","last_became_peered":"2024-09-17T05:04:53.487169+0000","last_unstale":"2024-09-17T05:04:55.469128+0000","last_undegraded":"2024-09-17T05:04:55.469128+0000","last_fullsized":"2024-09-17T05:04:55.469128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:09.062653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945874+0000","last_change":"2024-09-17T05:04:52.481898+0000","last_active":"2024-09-17T05:04:54.945874+0000","last_peered":"2024-09-17T05:04:54.945874+0000","last_clean":"2024-09-17T05:04:54.945874+0000","last_became_active":"2024-09-17T05:04:52.481562+0000","last_became_peered":"2024-09-17T05:04:52.481562+0000","last_unstale":"2024-09-17T05:04:54.945874+0000","last_undegraded":"2024-09-17T05:04:54.945874+0000","last_fullsized":"2024-09-17T05:04:54.945874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:18.224822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945666+0000","last_change":"2024-09-17T05:04:53.494544+0000","last_active":"2024-09-17T05:04:54.945666+0000","last_peered":"2024-09-17T05:04:54.945666+0000","last_clean":"2024-09-17T05:04:54.945666+0000","last_became_active":"2024-09-17T05:04:53.494204+0000","last_became_peered":"2024-09-17T05:04:53.494204+0000","last_unstale":"2024-09-17T05:04:54.945666+0000","last_undegraded":"2024-09-17T05:04:54.945666+0000","last_fullsized":"2024-09-17T05:04:54.945666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:09.891109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.945689+0000","last_change":"2024-09-17T05:04:52.479084+0000","last_active":"2024-09-17T05:04:54.945689+0000","last_peered":"2024-09-17T05:04:54.945689+0000","last_clean":"2024-09-17T05:04:54.945689+0000","last_became_active":"2024-09-17T05:04:52.478401+0000","last_became_peered":"2024-09-17T05:04:52.478401+0000","last_unstale":"2024-09-17T05:04:54.945689+0000","last_undegraded":"2024-09-17T05:04:54.945689+0000","last_fullsized":"2024-09-17T05:04:54.945689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:54:33.062565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672268+0000","last_change":"2024-09-17T05:04:53.480121+0000","last_active":"2024-09-17T05:04:55.672268+0000","last_peered":"2024-09-17T05:04:55.672268+0000","last_clean":"2024-09-17T05:04:55.672268+0000","last_became_active":"2024-09-17T05:04:53.479707+0000","last_became_peered":"2024-09-17T05:04:53.479707+0000","last_unstale":"2024-09-17T05:04:55.672268+0000","last_undegraded":"2024-09-17T05:04:55.672268+0000","last_fullsized":"2024-09-17T05:04:55.672268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:31.480151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482666+0000","last_change":"2024-09-17T05:04:52.479663+0000","last_active":"2024-09-17T05:04:54.482666+0000","last_peered":"2024-09-17T05:04:54.482666+0000","last_clean":"2024-09-17T05:04:54.482666+0000","last_became_active":"2024-09-17T05:04:52.479313+0000","last_became_peered":"2024-09-17T05:04:52.479313+0000","last_unstale":"2024-09-17T05:04:54.482666+0000","last_undegraded":"2024-09-17T05:04:54.482666+0000","last_fullsized":"2024-09-17T05:04:54.482666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:19.084321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672872+0000","last_change":"2024-09-17T05:04:53.483761+0000","last_active":"2024-09-17T05:04:55.672872+0000","last_peered":"2024-09-17T05:04:55.672872+0000","last_clean":"2024-09-17T05:04:55.672872+0000","last_became_active":"2024-09-17T05:04:53.483036+0000","last_became_peered":"2024-09-17T05:04:53.483036+0000","last_unstale":"2024-09-17T05:04:55.672872+0000","last_undegraded":"2024-09-17T05:04:55.672872+0000","last_fullsized":"2024-09-17T05:04:55.672872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:13.787717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672948+0000","last_change":"2024-09-17T05:04:52.489855+0000","last_active":"2024-09-17T05:04:55.672948+0000","last_peered":"2024-09-17T05:04:55.672948+0000","last_clean":"2024-09-17T05:04:55.672948+0000","last_became_active":"2024-09-17T05:04:52.487437+0000","last_became_peered":"2024-09-17T05:04:52.487437+0000","last_unstale":"2024-09-17T05:04:55.672948+0000","last_undegraded":"2024-09-17T05:04:55.672948+0000","last_fullsized":"2024-09-17T05:04:55.672948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:01.648572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482601+0000","last_change":"2024-09-17T05:04:53.479393+0000","last_active":"2024-09-17T05:04:54.482601+0000","last_peered":"2024-09-17T05:04:54.482601+0000","last_clean":"2024-09-17T05:04:54.482601+0000","last_became_active":"2024-09-17T05:04:53.479054+0000","last_became_peered":"2024-09-17T05:04:53.479054+0000","last_unstale":"2024-09-17T05:04:54.482601+0000","last_undegraded":"2024-09-17T05:04:54.482601+0000","last_fullsized":"2024-09-17T05:04:54.482601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:49.976409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672260+0000","last_change":"2024-09-17T05:04:52.489186+0000","last_active":"2024-09-17T05:04:55.672260+0000","last_peered":"2024-09-17T05:04:55.672260+0000","last_clean":"2024-09-17T05:04:55.672260+0000","last_became_active":"2024-09-17T05:04:52.484432+0000","last_became_peered":"2024-09-17T05:04:52.484432+0000","last_unstale":"2024-09-17T05:04:55.672260+0000","last_undegraded":"2024-09-17T05:04:55.672260+0000","last_fullsized":"2024-09-17T05:04:55.672260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:00.149032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.470119+0000","last_change":"2024-09-17T05:04:53.481924+0000","last_active":"2024-09-17T05:04:55.470119+0000","last_peered":"2024-09-17T05:04:55.470119+0000","last_clean":"2024-09-17T05:04:55.470119+0000","last_became_active":"2024-09-17T05:04:53.481582+0000","last_became_peered":"2024-09-17T05:04:53.481582+0000","last_unstale":"2024-09-17T05:04:55.470119+0000","last_undegraded":"2024-09-17T05:04:55.470119+0000","last_fullsized":"2024-09-17T05:04:55.470119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:52.935773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672183+0000","last_change":"2024-09-17T05:04:52.486351+0000","last_active":"2024-09-17T05:04:55.672183+0000","last_peered":"2024-09-17T05:04:55.672183+0000","last_clean":"2024-09-17T05:04:55.672183+0000","last_became_active":"2024-09-17T05:04:52.486015+0000","last_became_peered":"2024-09-17T05:04:52.486015+0000","last_unstale":"2024-09-17T05:04:55.672183+0000","last_undegraded":"2024-09-17T05:04:55.672183+0000","last_fullsized":"2024-09-17T05:04:55.672183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:48.366304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484008+0000","last_change":"2024-09-17T05:04:53.487545+0000","last_active":"2024-09-17T05:04:54.484008+0000","last_peered":"2024-09-17T05:04:54.484008+0000","last_clean":"2024-09-17T05:04:54.484008+0000","last_became_active":"2024-09-17T05:04:53.487247+0000","last_became_peered":"2024-09-17T05:04:53.487247+0000","last_unstale":"2024-09-17T05:04:54.484008+0000","last_undegraded":"2024-09-17T05:04:54.484008+0000","last_fullsized":"2024-09-17T05:04:54.484008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:05.668827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482100+0000","last_change":"2024-09-17T05:04:52.484413+0000","last_active":"2024-09-17T05:04:54.482100+0000","last_peered":"2024-09-17T05:04:54.482100+0000","last_clean":"2024-09-17T05:04:54.482100+0000","last_became_active":"2024-09-17T05:04:52.483851+0000","last_became_peered":"2024-09-17T05:04:52.483851+0000","last_unstale":"2024-09-17T05:04:54.482100+0000","last_undegraded":"2024-09-17T05:04:54.482100+0000","last_fullsized":"2024-09-17T05:04:54.482100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:48:34.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.671953+0000","last_change":"2024-09-17T05:04:53.501262+0000","last_active":"2024-09-17T05:04:55.671953+0000","last_peered":"2024-09-17T05:04:55.671953+0000","last_clean":"2024-09-17T05:04:55.671953+0000","last_became_active":"2024-09-17T05:04:53.500888+0000","last_became_peered":"2024-09-17T05:04:53.500888+0000","last_unstale":"2024-09-17T05:04:55.671953+0000","last_undegraded":"2024-09-17T05:04:55.671953+0000","last_fullsized":"2024-09-17T05:04:55.671953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:37.206313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483530+0000","last_change":"2024-09-17T05:04:52.475068+0000","last_active":"2024-09-17T05:04:54.483530+0000","last_peered":"2024-09-17T05:04:54.483530+0000","last_clean":"2024-09-17T05:04:54.483530+0000","last_became_active":"2024-09-17T05:04:52.474737+0000","last_became_peered":"2024-09-17T05:04:52.474737+0000","last_unstale":"2024-09-17T05:04:54.483530+0000","last_undegraded":"2024-09-17T05:04:54.483530+0000","last_fullsized":"2024-09-17T05:04:54.483530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:58:15.109220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.673088+0000","last_change":"2024-09-17T05:04:53.483876+0000","last_active":"2024-09-17T05:04:55.673088+0000","last_peered":"2024-09-17T05:04:55.673088+0000","last_clean":"2024-09-17T05:04:55.673088+0000","last_became_active":"2024-09-17T05:04:53.483524+0000","last_became_peered":"2024-09-17T05:04:53.483524+0000","last_unstale":"2024-09-17T05:04:55.673088+0000","last_undegraded":"2024-09-17T05:04:55.673088+0000","last_fullsized":"2024-09-17T05:04:55.673088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:49:51.026487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.671966+0000","last_change":"2024-09-17T05:04:52.479150+0000","last_active":"2024-09-17T05:04:55.671966+0000","last_peered":"2024-09-17T05:04:55.671966+0000","last_clean":"2024-09-17T05:04:55.671966+0000","last_became_active":"2024-09-17T05:04:52.476295+0000","last_became_peered":"2024-09-17T05:04:52.476295+0000","last_unstale":"2024-09-17T05:04:55.671966+0000","last_undegraded":"2024-09-17T05:04:55.671966+0000","last_fullsized":"2024-09-17T05:04:55.671966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:25.901198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.946177+0000","last_change":"2024-09-17T05:04:53.501606+0000","last_active":"2024-09-17T05:04:54.946177+0000","last_peered":"2024-09-17T05:04:54.946177+0000","last_clean":"2024-09-17T05:04:54.946177+0000","last_became_active":"2024-09-17T05:04:53.493685+0000","last_became_peered":"2024-09-17T05:04:53.493685+0000","last_unstale":"2024-09-17T05:04:54.946177+0000","last_undegraded":"2024-09-17T05:04:54.946177+0000","last_fullsized":"2024-09-17T05:04:54.946177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:22.660080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672512+0000","last_change":"2024-09-17T05:04:52.482044+0000","last_active":"2024-09-17T05:04:55.672512+0000","last_peered":"2024-09-17T05:04:55.672512+0000","last_clean":"2024-09-17T05:04:55.672512+0000","last_became_active":"2024-09-17T05:04:52.481677+0000","last_became_peered":"2024-09-17T05:04:52.481677+0000","last_unstale":"2024-09-17T05:04:55.672512+0000","last_undegraded":"2024-09-17T05:04:55.672512+0000","last_fullsized":"2024-09-17T05:04:55.672512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:07.449960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.673314+0000","last_change":"2024-09-17T05:04:53.493754+0000","last_active":"2024-09-17T05:04:55.673314+0000","last_peered":"2024-09-17T05:04:55.673314+0000","last_clean":"2024-09-17T05:04:55.673314+0000","last_became_active":"2024-09-17T05:04:53.487652+0000","last_became_peered":"2024-09-17T05:04:53.487652+0000","last_unstale":"2024-09-17T05:04:55.673314+0000","last_undegraded":"2024-09-17T05:04:55.673314+0000","last_fullsized":"2024-09-17T05:04:55.673314+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:52.935773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482475+0000","last_change":"2024-09-17T05:04:52.491484+0000","last_active":"2024-09-17T05:04:54.482475+0000","last_peered":"2024-09-17T05:04:54.482475+0000","last_clean":"2024-09-17T05:04:54.482475+0000","last_became_active":"2024-09-17T05:04:52.490491+0000","last_became_peered":"2024-09-17T05:04:52.490491+0000","last_unstale":"2024-09-17T05:04:54.482475+0000","last_undegraded":"2024-09-17T05:04:54.482475+0000","last_fullsized":"2024-09-17T05:04:54.482475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:37.682991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672460+0000","last_change":"2024-09-17T05:04:53.480166+0000","last_active":"2024-09-17T05:04:55.672460+0000","last_peered":"2024-09-17T05:04:55.672460+0000","last_clean":"2024-09-17T05:04:55.672460+0000","last_became_active":"2024-09-17T05:04:53.479853+0000","last_became_peered":"2024-09-17T05:04:53.479853+0000","last_unstale":"2024-09-17T05:04:55.672460+0000","last_undegraded":"2024-09-17T05:04:55.672460+0000","last_fullsized":"2024-09-17T05:04:55.672460+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:40.354751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482836+0000","last_change":"2024-09-17T05:04:52.489527+0000","last_active":"2024-09-17T05:04:54.482836+0000","last_peered":"2024-09-17T05:04:54.482836+0000","last_clean":"2024-09-17T05:04:54.482836+0000","last_became_active":"2024-09-17T05:04:52.487078+0000","last_became_peered":"2024-09-17T05:04:52.487078+0000","last_unstale":"2024-09-17T05:04:54.482836+0000","last_undegraded":"2024-09-17T05:04:54.482836+0000","last_fullsized":"2024-09-17T05:04:54.482836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:32.552368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672705+0000","last_change":"2024-09-17T05:04:53.499221+0000","last_active":"2024-09-17T05:04:55.672705+0000","last_peered":"2024-09-17T05:04:55.672705+0000","last_clean":"2024-09-17T05:04:55.672705+0000","last_became_active":"2024-09-17T05:04:53.498898+0000","last_became_peered":"2024-09-17T05:04:53.498898+0000","last_unstale":"2024-09-17T05:04:55.672705+0000","last_undegraded":"2024-09-17T05:04:55.672705+0000","last_fullsized":"2024-09-17T05:04:55.672705+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:57.034015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672911+0000","last_change":"2024-09-17T05:04:52.480615+0000","last_active":"2024-09-17T05:04:55.672911+0000","last_peered":"2024-09-17T05:04:55.672911+0000","last_clean":"2024-09-17T05:04:55.672911+0000","last_became_active":"2024-09-17T05:04:52.479433+0000","last_became_peered":"2024-09-17T05:04:52.479433+0000","last_unstale":"2024-09-17T05:04:55.672911+0000","last_undegraded":"2024-09-17T05:04:55.672911+0000","last_fullsized":"2024-09-17T05:04:55.672911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:55.687641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.470123+0000","last_change":"2024-09-17T05:04:53.495031+0000","last_active":"2024-09-17T05:04:55.470123+0000","last_peered":"2024-09-17T05:04:55.470123+0000","last_clean":"2024-09-17T05:04:55.470123+0000","last_became_active":"2024-09-17T05:04:53.494692+0000","last_became_peered":"2024-09-17T05:04:53.494692+0000","last_unstale":"2024-09-17T05:04:55.470123+0000","last_undegraded":"2024-09-17T05:04:55.470123+0000","last_fullsized":"2024-09-17T05:04:55.470123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.388958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.482374+0000","last_change":"2024-09-17T05:04:52.488025+0000","last_active":"2024-09-17T05:04:54.482374+0000","last_peered":"2024-09-17T05:04:54.482374+0000","last_clean":"2024-09-17T05:04:54.482374+0000","last_became_active":"2024-09-17T05:04:52.484280+0000","last_became_peered":"2024-09-17T05:04:52.484280+0000","last_unstale":"2024-09-17T05:04:54.482374+0000","last_undegraded":"2024-09-17T05:04:54.482374+0000","last_fullsized":"2024-09-17T05:04:54.482374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:27:45.589589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.484775+0000","last_change":"2024-09-17T05:04:53.500627+0000","last_active":"2024-09-17T05:04:54.484775+0000","last_peered":"2024-09-17T05:04:54.484775+0000","last_clean":"2024-09-17T05:04:54.484775+0000","last_became_active":"2024-09-17T05:04:53.500285+0000","last_became_peered":"2024-09-17T05:04:53.500285+0000","last_unstale":"2024-09-17T05:04:54.484775+0000","last_undegraded":"2024-09-17T05:04:54.484775+0000","last_fullsized":"2024-09-17T05:04:54.484775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:53:59.153467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.946295+0000","last_change":"2024-09-17T05:04:52.474880+0000","last_active":"2024-09-17T05:04:54.946295+0000","last_peered":"2024-09-17T05:04:54.946295+0000","last_clean":"2024-09-17T05:04:54.946295+0000","last_became_active":"2024-09-17T05:04:52.474563+0000","last_became_peered":"2024-09-17T05:04:52.474563+0000","last_unstale":"2024-09-17T05:04:54.946295+0000","last_undegraded":"2024-09-17T05:04:54.946295+0000","last_fullsized":"2024-09-17T05:04:54.946295+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:16.264008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483207+0000","last_change":"2024-09-17T05:04:53.486942+0000","last_active":"2024-09-17T05:04:54.483207+0000","last_peered":"2024-09-17T05:04:54.483207+0000","last_clean":"2024-09-17T05:04:54.483207+0000","last_became_active":"2024-09-17T05:04:53.486561+0000","last_became_peered":"2024-09-17T05:04:53.486561+0000","last_unstale":"2024-09-17T05:04:54.483207+0000","last_undegraded":"2024-09-17T05:04:54.483207+0000","last_fullsized":"2024-09-17T05:04:54.483207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:46.548167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469221+0000","last_change":"2024-09-17T05:04:52.474440+0000","last_active":"2024-09-17T05:04:55.469221+0000","last_peered":"2024-09-17T05:04:55.469221+0000","last_clean":"2024-09-17T05:04:55.469221+0000","last_became_active":"2024-09-17T05:04:52.474097+0000","last_became_peered":"2024-09-17T05:04:52.474097+0000","last_unstale":"2024-09-17T05:04:55.469221+0000","last_undegraded":"2024-09-17T05:04:55.469221+0000","last_fullsized":"2024-09-17T05:04:55.469221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:13.981678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.672948+0000","last_change":"2024-09-17T05:04:53.481055+0000","last_active":"2024-09-17T05:04:55.672948+0000","last_peered":"2024-09-17T05:04:55.672948+0000","last_clean":"2024-09-17T05:04:55.672948+0000","last_became_active":"2024-09-17T05:04:53.480735+0000","last_became_peered":"2024-09-17T05:04:53.480735+0000","last_unstale":"2024-09-17T05:04:55.672948+0000","last_undegraded":"2024-09-17T05:04:55.672948+0000","last_fullsized":"2024-09-17T05:04:55.672948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:26.811075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469626+0000","last_change":"2024-09-17T05:04:52.466652+0000","last_active":"2024-09-17T05:04:55.469626+0000","last_peered":"2024-09-17T05:04:55.469626+0000","last_clean":"2024-09-17T05:04:55.469626+0000","last_became_active":"2024-09-17T05:04:52.466240+0000","last_became_peered":"2024-09-17T05:04:52.466240+0000","last_unstale":"2024-09-17T05:04:55.469626+0000","last_undegraded":"2024-09-17T05:04:55.469626+0000","last_fullsized":"2024-09-17T05:04:55.469626+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:28.239172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.483052+0000","last_change":"2024-09-17T05:04:53.484911+0000","last_active":"2024-09-17T05:04:54.483052+0000","last_peered":"2024-09-17T05:04:54.483052+0000","last_clean":"2024-09-17T05:04:54.483052+0000","last_became_active":"2024-09-17T05:04:53.480325+0000","last_became_peered":"2024-09-17T05:04:53.480325+0000","last_unstale":"2024-09-17T05:04:54.483052+0000","last_undegraded":"2024-09-17T05:04:54.483052+0000","last_fullsized":"2024-09-17T05:04:54.483052+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:22.660080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469390+0000","last_change":"2024-09-17T05:04:52.472431+0000","last_active":"2024-09-17T05:04:55.469390+0000","last_peered":"2024-09-17T05:04:55.469390+0000","last_clean":"2024-09-17T05:04:55.469390+0000","last_became_active":"2024-09-17T05:04:52.472066+0000","last_became_peered":"2024-09-17T05:04:52.472066+0000","last_unstale":"2024-09-17T05:04:55.469390+0000","last_undegraded":"2024-09-17T05:04:55.469390+0000","last_fullsized":"2024-09-17T05:04:55.469390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:50.199400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469993+0000","last_change":"2024-09-17T05:04:53.493243+0000","last_active":"2024-09-17T05:04:55.469993+0000","last_peered":"2024-09-17T05:04:55.469993+0000","last_clean":"2024-09-17T05:04:55.469993+0000","last_became_active":"2024-09-17T05:04:53.492829+0000","last_became_peered":"2024-09-17T05:04:53.492829+0000","last_unstale":"2024-09-17T05:04:55.469993+0000","last_undegraded":"2024-09-17T05:04:55.469993+0000","last_fullsized":"2024-09-17T05:04:55.469993+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:52.447941+0000","last_clean_scrub_stamp":"2024-09-17T05:04:52.447941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:07:24.833636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:54.946221+0000","last_change":"2024-09-17T05:04:52.486302+0000","last_active":"2024-09-17T05:04:54.946221+0000","last_peered":"2024-09-17T05:04:54.946221+0000","last_clean":"2024-09-17T05:04:54.946221+0000","last_became_active":"2024-09-17T05:04:52.486002+0000","last_became_peered":"2024-09-17T05:04:52.486002+0000","last_unstale":"2024-09-17T05:04:54.946221+0000","last_undegraded":"2024-09-17T05:04:54.946221+0000","last_fullsized":"2024-09-17T05:04:54.946221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:04:51.444477+0000","last_clean_scrub_stamp":"2024-09-17T05:04:51.444477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:48:55.687641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":237,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-17T05:04:55.469567+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:04:55.469567+0000","last_peered":"2024-09-17T05:04:55.469567+0000","last_clean":"2024-09-17T05:04:55.469567+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:04:55.469567+0000","last_undegraded":"2024-09-17T05:04:55.469567+0000","last_fullsized":"2024-09-17T05:04:55.469567+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1064960,"data_stored":484068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1085440,"data_stored":485804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1064960,"data_stored":484068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36408,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335432,"statfs":{"total":96636764160,"available":96599482368,"internally_reserved":0,"allocated":1085440,"data_stored":485792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":484672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36980,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334860,"statfs":{"total":96636764160,"available":96598896640,"internally_reserved":0,"allocated":1671168,"data_stored":1075122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1069056,"data_stored":484138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":1074436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:05:02.760 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:05:02.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:05:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc132c00640 1 -- 172.21.15.16:0/3813081371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc110004810 msgr2=0x7fc110004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc132c00640 1 --2- 172.21.15.16:0/3813081371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc110004810 0x7fc110004c20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc114009b60 tx=0x7fc11401d2b0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc130c00640 1 -- 172.21.15.16:0/3813081371 reap_dead start 2024-09-17T05:05:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc132c00640 1 -- 172.21.15.16:0/3813081371 shutdown_connections 2024-09-17T05:05:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc132c00640 1 -- 172.21.15.16:0/3813081371 >> 172.21.15.16:0/3813081371 conn(0x7fc12c06f900 msgr2=0x7fc12c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc132c00640 1 -- 172.21.15.16:0/3813081371 shutdown_connections 2024-09-17T05:05:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.964+0000 7fc132c00640 1 -- 172.21.15.16:0/3813081371 wait complete. 2024-09-17T05:05:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 Processor -- start 2024-09-17T05:05:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 -- start start 2024-09-17T05:05:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc110004810 0x7fc12c1f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 0x7fc12c159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc12c13bbc0 0x7fc12c159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0b42e0 con 0x7fc12c13b7b0 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c0b5280 con 0x7fc110004810 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c0b4460 con 0x7fc12c13bbc0 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 0x7fc12c159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 0x7fc12c159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41296/0 (socket says 172.21.15.16:41296) 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 -- 172.21.15.16:0/1194546789 learned_addr learned my addr 172.21.15.16:0/1194546789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 -- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc12c13bbc0 msgr2=0x7fc12c159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc12c13bbc0 0x7fc12c159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc12be00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc110004810 0x7fc12c1f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 -- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc110004810 msgr2=0x7fc12c1f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc110004810 0x7fc12c1f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:02.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 -- 172.21.15.16:0/1194546789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c1e6640 con 0x7fc12c13b7b0 2024-09-17T05:05:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc12be00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc110004810 0x7fc12c1f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:05:02.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc123e00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 0x7fc12c159320 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc11401d0e0 tx=0x7fc1140055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140bd070 con 0x7fc12c13b7b0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1140b3040 con 0x7fc12c13b7b0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1140b2070 con 0x7fc12c13b7b0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c0af050 con 0x7fc12c13b7b0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.968+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c0af610 con 0x7fc12c13b7b0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.972+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c1e63b0 con 0x7fc12c13b7b0 2024-09-17T05:05:02.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.972+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc114004290 con 0x7fc12c13b7b0 2024-09-17T05:05:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.972+0000 7fc128c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc108073230 0x7fc108075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.972+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fc11412e8c0 con 0x7fc12c13b7b0 2024-09-17T05:05:02.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.972+0000 7fc12be00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc108073230 0x7fc108075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:02.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.976+0000 7fc12be00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc108073230 0x7fc108075740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0036b0 tx=0x7fc11c003600 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:02.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:02.976+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc1140f3db0 con 0x7fc12c13b7b0 2024-09-17T05:05:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.212+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc12c0b4980 con 0x7fc12c13b7b0 2024-09-17T05:05:03.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc128c00640 1 -- 172.21.15.16:0/1194546789 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7fc114016fc0 con 0x7fc12c13b7b0 2024-09-17T05:05:03.218 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:05:03.218 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:05:03.218 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:05:03.218 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:05:03.219 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:05:03.219 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc108073230 msgr2=0x7fc108075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc108073230 0x7fc108075740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0036b0 tx=0x7fc11c003600 comp rx=0 tx=0).stop 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 msgr2=0x7fc12c159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 0x7fc12c159320 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc11401d0e0 tx=0x7fc1140055d0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 shutdown_connections 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc108073230 0x7fc108075740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc12c13bbc0 0x7fc12c159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12c13b7b0 0x7fc12c159320 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 --2- 172.21.15.16:0/1194546789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc110004810 0x7fc12c1f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 >> 172.21.15.16:0/1194546789 conn(0x7fc12c06f900 msgr2=0x7fc12c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:03.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 shutdown_connections 2024-09-17T05:05:03.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.216+0000 7fc132c00640 1 -- 172.21.15.16:0/1194546789 wait complete. 2024-09-17T05:05:03.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:05:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f9630a00640 1 -- 172.21.15.16:0/2086455256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610004d50 msgr2=0x7f96100051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f9630a00640 1 --2- 172.21.15.16:0/2086455256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9610004d50 0x7f96100051a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9624009e90 tx=0x7f962401d590 comp rx=0 tx=0).stop 2024-09-17T05:05:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f962a800640 1 -- 172.21.15.16:0/2086455256 reap_dead start 2024-09-17T05:05:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f9630a00640 1 -- 172.21.15.16:0/2086455256 shutdown_connections 2024-09-17T05:05:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f9630a00640 1 -- 172.21.15.16:0/2086455256 >> 172.21.15.16:0/2086455256 conn(0x7f962c0d0860 msgr2=0x7f962c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f9630a00640 1 -- 172.21.15.16:0/2086455256 shutdown_connections 2024-09-17T05:05:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.396+0000 7f9630a00640 1 -- 172.21.15.16:0/2086455256 wait complete. 2024-09-17T05:05:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 Processor -- start 2024-09-17T05:05:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 -- start start 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9610004d50 0x7f962c1e8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f962c1e8c90 0x7f962c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 0x7f962c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0d0390 con 0x7f962c153ef0 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f962c0b5a80 con 0x7f962c1e8c90 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9630a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f962c0b4640 con 0x7f9610004d50 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 0x7f962c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 0x7f962c155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41310/0 (socket says 172.21.15.16:41310) 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9629e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9610004d50 0x7f962c1e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 -- 172.21.15.16:0/3578058406 learned_addr learned my addr 172.21.15.16:0/3578058406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f9623e00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f962c1e8c90 0x7f962c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 -- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9610004d50 msgr2=0x7f962c1e8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9610004d50 0x7f962c1e8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 -- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f962c1e8c90 msgr2=0x7f962c1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f962c1e8c90 0x7f962c1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 -- 172.21.15.16:0/3578058406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962c154d10 con 0x7f962c153ef0 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.400+0000 7f962a800640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 0x7f962c155a90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9624009e90 tx=0x7f9624004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624009a90 con 0x7f962c153ef0 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96240a9510 con 0x7f962c153ef0 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96240a9d30 con 0x7f962c153ef0 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962c1f62c0 con 0x7f962c153ef0 2024-09-17T05:05:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962c155290 con 0x7f962c153ef0 2024-09-17T05:05:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f962c0cf320 con 0x7f962c153ef0 2024-09-17T05:05:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96240a7040 con 0x7f962c153ef0 2024-09-17T05:05:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9622000640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9608073200 0x7f9608075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f962412fba0 con 0x7f962c153ef0 2024-09-17T05:05:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.404+0000 7f9629e00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9608073200 0x7f9608075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.408+0000 7f9629e00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9608073200 0x7f9608075710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f961c003ff0 tx=0x7f961c001250 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:03.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.408+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f962412fe20 con 0x7f962c153ef0 2024-09-17T05:05:03.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.620+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f962c1f64f0 con 0x7f962c153ef0 2024-09-17T05:05:03.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.620+0000 7f9622000640 1 -- 172.21.15.16:0/3578058406 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5228 (secure 0 0 0) 0x7f96240ca080 con 0x7f962c153ef0 2024-09-17T05:05:03.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:03.626 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"btime":"2024-09-17T05:04:56:488386+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:04:55.907544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:05:03.626 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 52 2024-09-17T05:05:03.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9608073200 msgr2=0x7f9608075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9608073200 0x7f9608075710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f961c003ff0 tx=0x7f961c001250 comp rx=0 tx=0).stop 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 msgr2=0x7f962c155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 0x7f962c155a90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9624009e90 tx=0x7f9624004f30 comp rx=0 tx=0).stop 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 shutdown_connections 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9608073200 0x7f9608075710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f962c153ef0 0x7f962c155a90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f962c1e8c90 0x7f962c1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 --2- 172.21.15.16:0/3578058406 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9610004d50 0x7f962c1e8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 >> 172.21.15.16:0/3578058406 conn(0x7f962c0d0860 msgr2=0x7f962c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 shutdown_connections 2024-09-17T05:05:03.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:03.624+0000 7f9630a00640 1 -- 172.21.15.16:0/3578058406 wait complete. 2024-09-17T05:05:04.049 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:04:52.887943+0000', 'modified': '2024-09-17T05:04:55.907544+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6429}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6429': {'gid': 6429, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6834/1846453187', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6832', 'nonce': 1846453187}, {'type': 'v1', 'addr': '172.21.15.16:6834', 'nonce': 1846453187}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6429, 'qdb_cluster': [6429]} 2024-09-17T05:05:04.050 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:05:04.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:05:04.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 -- 172.21.15.16:0/1021408667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 msgr2=0x7f6c6c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 --2- 172.21.15.16:0/1021408667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c6c00b9d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c610 tx=0x7f6c8001f9c0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c87e00640 1 -- 172.21.15.16:0/1021408667 reap_dead start 2024-09-17T05:05:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 -- 172.21.15.16:0/1021408667 shutdown_connections 2024-09-17T05:05:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 -- 172.21.15.16:0/1021408667 >> 172.21.15.16:0/1021408667 conn(0x7f6c880b2dd0 msgr2=0x7f6c880b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 -- 172.21.15.16:0/1021408667 shutdown_connections 2024-09-17T05:05:04.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 -- 172.21.15.16:0/1021408667 wait complete. 2024-09-17T05:05:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 Processor -- start 2024-09-17T05:05:04.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.212+0000 7f6c8e000640 1 -- start start 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c88150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c880b4ae0 0x7f6c880b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c880cb740 0x7f6c880cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880cd390 con 0x7f6c6c0051b0 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880cd210 con 0x7f6c880b4ae0 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6c880cd510 con 0x7f6c880cb740 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c88150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c88150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41326/0 (socket says 172.21.15.16:41326) 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 -- 172.21.15.16:0/259601801 learned_addr learned my addr 172.21.15.16:0/259601801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c880b4ae0 0x7f6c880b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87e00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c880cb740 0x7f6c880cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 -- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c880cb740 msgr2=0x7f6c880cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c880cb740 0x7f6c880cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 -- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c880b4ae0 msgr2=0x7f6c880b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c880b4ae0 0x7f6c880b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 -- 172.21.15.16:0/259601801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c880cc090 con 0x7f6c6c0051b0 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c880b4ae0 0x7f6c880b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87e00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c880cb740 0x7f6c880cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:04.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c87400640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c88150e10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6c78003f00 tx=0x7f6c78001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c7fe00640 1 -- 172.21.15.16:0/259601801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78015070 con 0x7f6c6c0051b0 2024-09-17T05:05:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c7fe00640 1 -- 172.21.15.16:0/259601801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c78011940 con 0x7f6c6c0051b0 2024-09-17T05:05:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c880b5760 con 0x7f6c6c0051b0 2024-09-17T05:05:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c7fe00640 1 -- 172.21.15.16:0/259601801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78011d90 con 0x7f6c6c0051b0 2024-09-17T05:05:04.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.216+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c880ada40 con 0x7f6c6c0051b0 2024-09-17T05:05:04.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 Processor -- start 2024-09-17T05:05:04.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 -- start start 2024-09-17T05:05:04.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c7fe00640 1 -- 172.21.15.16:0/259601801 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6c78010420 con 0x7f6c6c0051b0 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af9754f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973fe950 0x55af9754fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af97550120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af97424410 con 0x55af973e7ca0 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af97361b40 con 0x55af973fe950 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55af97350f70 con 0x55af9700e460 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af97550120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af97550120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41328/0 (socket says 172.21.15.16:41328) 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c7ea00640 1 -- 172.21.15.16:0/3900987243 learned_addr learned my addr 172.21.15.16:0/3900987243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c7fe00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c64073200 0x7f6c64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.220+0000 7f6c87e00640 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af9754f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c87400640 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973fe950 0x55af9754fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7fe00640 1 -- 172.21.15.16:0/259601801 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f6c78095090 con 0x7f6c6c0051b0 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7ea00640 1 -- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 msgr2=0x55af9754f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af9754f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7ea00640 1 -- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973fe950 msgr2=0x55af9754fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973fe950 0x55af9754fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7ea00640 1 -- 172.21.15.16:0/3900987243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af973185b0 con 0x55af973e7ca0 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c87400640 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973fe950 0x55af9754fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c87e00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c64073200 0x7f6c64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af97550120 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009c70 tx=0x7f6c70013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ae7112a673b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c77400640 1 -- 172.21.15.16:0/3900987243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7001c640 con 0x55af973e7ca0 2024-09-17T05:05:04.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c77400640 1 -- 172.21.15.16:0/3900987243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c7001c7e0 con 0x55af973e7ca0 2024-09-17T05:05:04.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c77400640 1 -- 172.21.15.16:0/3900987243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7001cbf0 con 0x55af973e7ca0 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 -- 172.21.15.16:0/3900987243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 msgr2=0x55af97550120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af97550120 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009c70 tx=0x7f6c70013830 comp rx=0 tx=0).stop 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c87e00640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c64073200 0x7f6c64075710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6c800b0000 tx=0x7f6c80006ab0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 -- 172.21.15.16:0/3900987243 shutdown_connections 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af97550120 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973fe950 0x55af9754fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 --2- 172.21.15.16:0/3900987243 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af9754f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 -- 172.21.15.16:0/3900987243 >> 172.21.15.16:0/3900987243 conn(0x55af974d4d90 msgr2=0x55af97504690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 -- 172.21.15.16:0/3900987243 shutdown_connections 2024-09-17T05:05:04.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 -- 172.21.15.16:0/3900987243 wait complete. 2024-09-17T05:05:04.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 -- start start 2024-09-17T05:05:04.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.224+0000 7f6c95733000 1 Processor -- start 2024-09-17T05:05:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af97556920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af97556e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 0x55af975573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af972eb5f0 con 0x55af973fe950 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af97354d80 con 0x55af973e7ca0 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55af972f7520 con 0x55af9700e460 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 10 client.0 _ll_register_callbacks cb 0x55af972fb830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 0x55af975573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 0x55af975573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41342/0 (socket says 172.21.15.16:41342) 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af97556920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af97556e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 -- 172.21.15.16:0/245927112 learned_addr learned my addr 172.21.15.16:0/245927112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 -- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 msgr2=0x55af97556920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af97556920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 -- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 msgr2=0x55af97556e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af97556e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 -- 172.21.15.16:0/245927112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af972ef190 con 0x55af973fe950 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87e00640 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 0x55af975573a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6c80020260 tx=0x7f6c8001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171d9aecd4d668df server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 1 -- 172.21.15.16:0/245927112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c800cd060 con 0x55af973fe950 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af972e9110 con 0x55af973fe950 2024-09-17T05:05:04.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 1 -- 172.21.15.16:0/245927112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f6c80006670 con 0x55af973fe950 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 1 -- 172.21.15.16:0/245927112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6c800d3050 con 0x55af973fe950 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 10 client.6582 fetch_fsmap learned FSMap version 52 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55af973d5f10 con 0x55af973fe950 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 1 -- 172.21.15.16:0/245927112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55af972e9110 con 0x55af973fe950 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 1 -- 172.21.15.16:0/245927112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f6c800d6020 con 0x55af973fe950 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 10 client.6582 fetch_fsmap finished waiting for FSMap version 52 2024-09-17T05:05:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 10 client.6582 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 20 client.6582 populate_metadata read hostname 'smithi016' 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96ff37d0 0x55af97560d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 4 client.6582 mds_command: new command op to 6429 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 --> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af97363a90 con 0x55af96ff37d0 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87400640 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96ff37d0 0x55af97560d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c6be00640 10 client.6582 ms_handle_connect on v2:172.21.15.16:6832/1846453187 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.228+0000 7f6c87400640 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96ff37d0 0x55af97560d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99d28fdc367dd491 server_cookie=d55fd885b98724a in_seq=0 out_seq=0 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c6be00640 1 -- 172.21.15.16:0/245927112 <== mds.0 v2:172.21.15.16:6832/1846453187 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af97363a90 con 0x55af96ff37d0 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c6be00640 10 client.6582 handle_command_reply: tid=0 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 client.6582 shutdown 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 msgr2=0x55af975573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 0x55af975573a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6c80020260 tx=0x7f6c8001c800 comp rx=0 tx=0).stop 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 shutdown_connections 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96ff37d0 0x55af97560d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973fe950 0x55af975573a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af97556e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 --2- 172.21.15.16:0/245927112 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af9700e460 0x55af97556920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c6be00640 0 client.6582 ms_handle_reset on v2:172.21.15.16:6832/1846453187 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 >> 172.21.15.16:0/245927112 conn(0x55af974d4d90 msgr2=0x55af9750b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 shutdown_connections 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 1 -- 172.21.15.16:0/245927112 wait complete. 2024-09-17T05:05:04.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.232+0000 7f6c95733000 20 client.6582 trim_cache size 0 max 16384 2024-09-17T05:05:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 Processor -- start 2024-09-17T05:05:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 -- start start 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af9700e460 0x55af975c1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af975c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88202b20 con 0x55af973e7ca0 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c88201b60 con 0x55af9700e460 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6c88200ec0 con 0x55af973fe950 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af975c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af975c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41356/0 (socket says 172.21.15.16:41356) 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 -- 172.21.15.16:0/2503154378 learned_addr learned my addr 172.21.15.16:0/2503154378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c87400640 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af9700e460 0x55af975c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c87e00640 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 -- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 msgr2=0x55af975c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 -- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af9700e460 msgr2=0x55af975c1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af9700e460 0x55af975c1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 -- 172.21.15.16:0/2503154378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af973acdb0 con 0x55af973e7ca0 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c87400640 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af9700e460 0x55af975c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af975c1b60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6c70014fd0 tx=0x7f6c700046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413d91c95d67489e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c76a00640 1 -- 172.21.15.16:0/2503154378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7002b070 con 0x55af973e7ca0 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c76a00640 1 -- 172.21.15.16:0/2503154378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c70005330 con 0x55af973e7ca0 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.252+0000 7f6c76a00640 1 -- 172.21.15.16:0/2503154378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c700047b0 con 0x55af973e7ca0 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- 172.21.15.16:0/2503154378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 msgr2=0x55af975c1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af975c1b60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6c70014fd0 tx=0x7f6c700046d0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- 172.21.15.16:0/2503154378 shutdown_connections 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af973e7ca0 0x55af975c1b60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- 172.21.15.16:0/2503154378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af9700e460 0x55af975c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- 172.21.15.16:0/2503154378 >> 172.21.15.16:0/2503154378 conn(0x55af974d4d90 msgr2=0x55af9754d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- 172.21.15.16:0/2503154378 shutdown_connections 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- 172.21.15.16:0/2503154378 wait complete. 2024-09-17T05:05:04.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- start start 2024-09-17T05:05:04.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 Processor -- start 2024-09-17T05:05:04.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 0x55af975c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af975c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af97354d80 con 0x55af9700e460 2024-09-17T05:05:04.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55af97360910 con 0x55af973e7ca0 2024-09-17T05:05:04.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55af9731cee0 con 0x55af973fe950 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c95733000 10 client.0 _ll_register_callbacks cb 0x7f6c6c003890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 0x55af975c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 0x55af975c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41372/0 (socket says 172.21.15.16:41372) 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 -- 172.21.15.16:0/55366484 learned_addr learned my addr 172.21.15.16:0/55366484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af975c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87e00640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 -- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 msgr2=0x55af975c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 -- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 msgr2=0x55af975c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af975c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 -- 172.21.15.16:0/55366484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af97317ab0 con 0x55af9700e460 2024-09-17T05:05:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87e00640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c87400640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 0x55af975c55a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6c78003ed0 tx=0x7f6c7801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c6579bc22d9f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.256+0000 7f6c6be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 1 -- 172.21.15.16:0/55366484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78020040 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 1 -- 172.21.15.16:0/55366484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f6c7805bb10 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c70005330 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 1 -- 172.21.15.16:0/55366484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6c78064380 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 10 client.6588 fetch_fsmap learned FSMap version 52 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55af972f6070 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 1 -- 172.21.15.16:0/55366484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c70005330 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 1 -- 172.21.15.16:0/55366484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f6c780970d0 con 0x55af9700e460 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 10 client.6588 fetch_fsmap finished waiting for FSMap version 52 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 10 client.6588 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:05:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 20 client.6588 populate_metadata read hostname 'smithi016' 2024-09-17T05:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96fe1830 0x55af97582640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 4 client.6588 mds_command: new command op to 6429 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 --> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] -- command(tid 0: {"prefix": "status"}) -- 0x55af9735f540 con 0x55af96fe1830 2024-09-17T05:05:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96fe1830 0x55af97582640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c7ea00640 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96fe1830 0x55af97582640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e17e59a0d67e5b0 server_cookie=3b3e9638a6f7578c in_seq=0 out_seq=0 2024-09-17T05:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 10 client.6588 ms_handle_connect on v2:172.21.15.16:6832/1846453187 2024-09-17T05:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 1 -- 172.21.15.16:0/55366484 <== mds.0 v2:172.21.15.16:6832/1846453187 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55af9735f540 con 0x55af96fe1830 2024-09-17T05:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 10 client.6588 handle_command_reply: tid=0 2024-09-17T05:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 client.6588 shutdown 2024-09-17T05:05:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 msgr2=0x55af975c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 0x55af975c55a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6c78003ed0 tx=0x7f6c7801e000 comp rx=0 tx=0).stop 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 shutdown_connections 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:6832/1846453187,v1:172.21.15.16:6834/1846453187] conn(0x55af96fe1830 0x55af97582640 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55af973fe950 0x55af975c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55af973e7ca0 0x55af975c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 --2- 172.21.15.16:0/55366484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af9700e460 0x55af975c55a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c6be00640 0 client.6588 ms_handle_reset on v2:172.21.15.16:6832/1846453187 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 >> 172.21.15.16:0/55366484 conn(0x55af974d4d90 msgr2=0x55af975606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 shutdown_connections 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 1 -- 172.21.15.16:0/55366484 wait complete. 2024-09-17T05:05:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c95733000 20 client.6588 trim_cache size 0 max 16384 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6429, 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.806971627999999, 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 52, 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 87, 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 87, 2024-09-17T05:05:04.268 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.929688200999999 2024-09-17T05:05:04.269 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:05:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c64073200 msgr2=0x7f6c64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c64073200 0x7f6c64075710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6c800b0000 tx=0x7f6c80006ab0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 msgr2=0x7f6c88150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c88150e10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6c78003f00 tx=0x7f6c78001810 comp rx=0 tx=0).stop 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 shutdown_connections 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6c64073200 0x7f6c64075710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6c880cb740 0x7f6c880cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c880b4ae0 0x7f6c880b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 --2- 172.21.15.16:0/259601801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c6c0051b0 0x7f6c88150e10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 >> 172.21.15.16:0/259601801 conn(0x7f6c880b2dd0 msgr2=0x7f6c88073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 shutdown_connections 2024-09-17T05:05:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.260+0000 7f6c8e000640 1 -- 172.21.15.16:0/259601801 wait complete. 2024-09-17T05:05:04.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 -- 172.21.15.16:0/1732034704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f649c004810 msgr2=0x7f649c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 --2- 172.21.15.16:0/1732034704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f649c004810 0x7f649c004c80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f64a0006f10 tx=0x7f64a001ce60 comp rx=0 tx=0).stop 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 -- 172.21.15.16:0/1732034704 shutdown_connections 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 --2- 172.21.15.16:0/1732034704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f649c004810 0x7f649c004c80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 -- 172.21.15.16:0/1732034704 >> 172.21.15.16:0/1732034704 conn(0x7f64b806f900 msgr2=0x7f64b80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 -- 172.21.15.16:0/1732034704 shutdown_connections 2024-09-17T05:05:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.440+0000 7f64bd400640 1 -- 172.21.15.16:0/1732034704 wait complete. 2024-09-17T05:05:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 Processor -- start 2024-09-17T05:05:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 -- start start 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f649c004810 0x7f64b815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 0x7f64b81f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64b80ab190 0x7f64b81fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64b80743e0 con 0x7f64b80aad80 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8074260 con 0x7f649c004810 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f64b815f150 con 0x7f64b80ab190 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 0x7f64b81f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 0x7f64b81f9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41396/0 (socket says 172.21.15.16:41396) 2024-09-17T05:05:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 -- 172.21.15.16:0/1746534897 learned_addr learned my addr 172.21.15.16:0/1746534897 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64b6800640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f649c004810 0x7f64b815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64b7200640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64b80ab190 0x7f64b81fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 -- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64b80ab190 msgr2=0x7f64b81fa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64b80ab190 0x7f64b81fa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 -- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f649c004810 msgr2=0x7f64b815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f649c004810 0x7f64b815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 -- 172.21.15.16:0/1746534897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b81590e0 con 0x7f64b80aad80 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64b7200640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64b80ab190 0x7f64b81fa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64ade00640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 0x7f64b81f9fa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f64a001c920 tx=0x7f64a00a5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0001500 con 0x7f64b80aad80 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64a00016a0 con 0x7f64b80aad80 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b81593c0 con 0x7f64b80aad80 2024-09-17T05:05:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0006310 con 0x7f64b80aad80 2024-09-17T05:05:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.444+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b8159980 con 0x7f64b80aad80 2024-09-17T05:05:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.448+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b8158e50 con 0x7f64b80aad80 2024-09-17T05:05:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.448+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f64a0003600 con 0x7f64b80aad80 2024-09-17T05:05:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.448+0000 7f64af400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6494073200 0x7f6494075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.448+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f64a012dc70 con 0x7f64b80aad80 2024-09-17T05:05:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.452+0000 7f64b6800640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6494073200 0x7f6494075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.452+0000 7f64b6800640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6494073200 0x7f6494075710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f64b806bbd0 tx=0x7f64a80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.452+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f64a0016c10 con 0x7f64b80aad80 2024-09-17T05:05:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f64b815f6b0 con 0x7f64b80aad80 2024-09-17T05:05:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64af400640 1 -- 172.21.15.16:0/1746534897 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f64a00a5df0 con 0x7f64b80aad80 2024-09-17T05:05:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6494073200 msgr2=0x7f6494075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6494073200 0x7f6494075710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f64b806bbd0 tx=0x7f64a80091e0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 msgr2=0x7f64b81f9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 0x7f64b81f9fa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f64a001c920 tx=0x7f64a00a5c00 comp rx=0 tx=0).stop 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 shutdown_connections 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6494073200 0x7f6494075710 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64b80ab190 0x7f64b81fa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b80aad80 0x7f64b81f9fa0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 --2- 172.21.15.16:0/1746534897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f649c004810 0x7f64b815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 >> 172.21.15.16:0/1746534897 conn(0x7f64b806f900 msgr2=0x7f64b80d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 shutdown_connections 2024-09-17T05:05:04.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04.664+0000 7f64bd400640 1 -- 172.21.15.16:0/1746534897 wait complete. 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:05:04.690 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:05:04.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:04.691 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:05:04.700 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:05:04.701 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42188sec preferred_lft 42188sec 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.702 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:05:04.703 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:04.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:04.704 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:05:04.763 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:05:04.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:04.765 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:05:04.822 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:05:04.822 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:05:04.822 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:05:04.823 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:05:04.823 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:05:04.823 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:05:04.823 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:05:04.823 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:05:04.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:05:04.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:05:04.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:05:04.823 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:05:04.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:05:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:05:04.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:04.961 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:05:04.970 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:05:04.970 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:05:04.970 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:05:04.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:04.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:05:05.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:05:05.032 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:05:05.081 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:05:05.081 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:05:05.081 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:05:05.132 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:05:05.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:05.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:05:05.234 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:05:05.234 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:05:05.234 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:05:05.252 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:05:05.252 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:05:05.252 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:05:05.252 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:05:05.252 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:05:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:05:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:05:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:05:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:05:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:05:05.263 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:05:05.263 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:05:05.263 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:05:05.266 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:05:05.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:05.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:05:05.287 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:05:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:05:05.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:05:05.393 INFO:teuthology.orchestra.run.smithi016.stdout:client6373 2024-09-17T05:05:05.393 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:05:05.393 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:05:05.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 -- 172.21.15.16:0/2660414153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 msgr2=0x7f09f81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 --2- 172.21.15.16:0/2660414153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f81535c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009900 tx=0x7f09e801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 -- 172.21.15.16:0/2660414153 shutdown_connections 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 --2- 172.21.15.16:0/2660414153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09f80cc8b0 0x7f09f80cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 --2- 172.21.15.16:0/2660414153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f81535c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 -- 172.21.15.16:0/2660414153 >> 172.21.15.16:0/2660414153 conn(0x7f09f80d2260 msgr2=0x7f09f80d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 -- 172.21.15.16:0/2660414153 shutdown_connections 2024-09-17T05:05:05.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 -- 172.21.15.16:0/2660414153 wait complete. 2024-09-17T05:05:05.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 Processor -- start 2024-09-17T05:05:05.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.604+0000 7f09fd600640 1 -- start start 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09f80cc8b0 0x7f09f80b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09f80d0340 0x7f09f80cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f8152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80b3770 con 0x7f09f80d1880 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80b35f0 con 0x7f09f80d0340 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f09f80b38f0 con 0x7f09f80cc8b0 2024-09-17T05:05:05.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f8152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f8152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41424/0 (socket says 172.21.15.16:41424) 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 -- 172.21.15.16:0/1489791455 learned_addr learned my addr 172.21.15.16:0/1489791455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09ee000640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09f80d0340 0x7f09f80cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f6a00640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09f80cc8b0 0x7f09f80b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 -- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09f80cc8b0 msgr2=0x7f09f80b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09f80cc8b0 0x7f09f80b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 -- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09f80d0340 msgr2=0x7f09f80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09f80d0340 0x7f09f80cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 -- 172.21.15.16:0/1489791455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f815cab0 con 0x7f09f80d1880 2024-09-17T05:05:05.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09f7400640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f8152550 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c6b0 tx=0x7f09f000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:05.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0003b80 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09f0003d20 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0005330 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f815c4f0 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f8155390 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.608+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f80cccc0 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.612+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f09f00054d0 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.612+0000 7f09ef400640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09d4073200 0x7f09d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.612+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f09f0094100 con 0x7f09f80d1880 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.612+0000 7f09f6a00640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09d4073200 0x7f09d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:05.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.612+0000 7f09f6a00640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09d4073200 0x7f09d4075710 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09e80af000 tx=0x7f09e801b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:05.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.616+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f09f8161110 con 0x7f09f80d1880 2024-09-17T05:05:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.828+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f09f806a730 con 0x7f09f80d1880 2024-09-17T05:05:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.828+0000 7f09ef400640 1 -- 172.21.15.16:0/1489791455 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f09f005e8a0 con 0x7f09f80d1880 2024-09-17T05:05:05.832 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09d4073200 msgr2=0x7f09d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09d4073200 0x7f09d4075710 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f09e80af000 tx=0x7f09e801b0f0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 msgr2=0x7f09f8152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f8152550 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f09f000c6b0 tx=0x7f09f000cb70 comp rx=0 tx=0).stop 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 shutdown_connections 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f09d4073200 0x7f09d4075710 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09f80d1880 0x7f09f8152550 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09f80d0340 0x7f09f80cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 --2- 172.21.15.16:0/1489791455 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09f80cc8b0 0x7f09f80b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 >> 172.21.15.16:0/1489791455 conn(0x7f09f80d2260 msgr2=0x7f09f815e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 shutdown_connections 2024-09-17T05:05:05.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:05.832+0000 7f09fd600640 1 -- 172.21.15.16:0/1489791455 wait complete. 2024-09-17T05:05:05.854 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:05:05.854 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:05:05.854 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:05:05.854 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:05:05.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:05:05.855 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:05:05.855 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:05:05.855 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:05:05.855 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:05:05.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:05.855 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:05:05.865 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:05:05.866 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:05:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:05:05.868 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:05:05.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:05.870 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:05:05.932 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:05:05.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:05.934 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:05:05.989 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:05:05.990 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:05:05.990 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:05:05.990 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:05:05.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:05:05.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:05:05.990 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:05:05.990 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:05:06.084 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:05:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:05:06.127 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:05:06.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:06.134 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:05:06.143 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:05:06.143 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:05:06.144 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:05:06.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:06.144 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:05:06.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:05:06.208 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:05:06.263 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:05:06.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:05:06.264 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:05:06.316 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:05:06.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:06.316 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:05:06.416 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:05:06.416 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:05:06.416 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:05:06.436 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:05:06.437 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:05:06.437 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:05:06.437 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:05:06.437 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:05:06.444 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:05:06.445 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:05:06.445 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:05:06.445 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:05:06.445 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:05:06.445 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:05:06.446 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:05:06.446 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:05:06.449 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:05:06.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:05:06.449 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:05:06.470 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:05:06.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:05:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:05:06.566 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:05:06.572 INFO:teuthology.orchestra.run.smithi104.stdout:client6379 2024-09-17T05:05:06.573 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:05:06.573 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:05:06.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-17T05:05:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b2e00640 1 -- 172.21.15.16:0/3219384162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3190004000 msgr2=0x7f3190004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b2e00640 1 --2- 172.21.15.16:0/3219384162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3190004000 0x7f3190004810 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f319800ac50 tx=0x7f319801e060 comp rx=0 tx=0).stop 2024-09-17T05:05:06.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b0e00640 1 -- 172.21.15.16:0/3219384162 reap_dead start 2024-09-17T05:05:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b2e00640 1 -- 172.21.15.16:0/3219384162 shutdown_connections 2024-09-17T05:05:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b2e00640 1 -- 172.21.15.16:0/3219384162 >> 172.21.15.16:0/3219384162 conn(0x7f31ac0b2250 msgr2=0x7f31ac0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b2e00640 1 -- 172.21.15.16:0/3219384162 shutdown_connections 2024-09-17T05:05:06.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.752+0000 7f31b2e00640 1 -- 172.21.15.16:0/3219384162 wait complete. 2024-09-17T05:05:06.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 Processor -- start 2024-09-17T05:05:06.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 -- start start 2024-09-17T05:05:06.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 0x7f31ac0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:06.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 0x7f31ac14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f31ac14f260 0x7f31ac1fc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac0ce960 con 0x7f31ac14c5b0 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac0ce7e0 con 0x7f3190004000 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.756+0000 7f31b2e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f31ac0ceae0 con 0x7f31ac14f260 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 0x7f31ac14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 0x7f31ac0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 0x7f31ac0ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:35974/0 (socket says 172.21.15.16:35974) 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 -- 172.21.15.16:0/1044506855 learned_addr learned my addr 172.21.15.16:0/1044506855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31ab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 0x7f31ac14ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41454/0 (socket says 172.21.15.16:41454) 2024-09-17T05:05:06.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31b0e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f31ac14f260 0x7f31ac1fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 -- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f31ac14f260 msgr2=0x7f31ac1fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f31ac14f260 0x7f31ac1fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 -- 172.21.15.16:0/1044506855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 msgr2=0x7f31ac14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 0x7f31ac14ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 -- 172.21.15.16:0/1044506855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac14f670 con 0x7f3190004000 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31ab400640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 0x7f31ac14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31abe00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 0x7f31ac0ae280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f319c011b10 tx=0x7f319c009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c00fd00 con 0x7f3190004000 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f319c004020 con 0x7f3190004000 2024-09-17T05:05:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac0ae7c0 con 0x7f3190004000 2024-09-17T05:05:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c00fea0 con 0x7f3190004000 2024-09-17T05:05:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac0aef20 con 0x7f3190004000 2024-09-17T05:05:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ac06e3b0 con 0x7f3190004000 2024-09-17T05:05:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.760+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f319c003600 con 0x7f3190004000 2024-09-17T05:05:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.764+0000 7f31a3e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f318c073200 0x7f318c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.764+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f319c094ef0 con 0x7f3190004000 2024-09-17T05:05:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.764+0000 7f31ab400640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f318c073200 0x7f318c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.764+0000 7f31ab400640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f318c073200 0x7f318c075710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f31980ae000 tx=0x7f319801ae00 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.768+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31ac14f670 con 0x7f3190004000 2024-09-17T05:05:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.944+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f31ac075070 con 0x7f3190004000 2024-09-17T05:05:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31a3e00640 1 -- 172.21.15.16:0/1044506855 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v44)=0 v44) ==== 121+0+0 (secure 0 0 0) 0x7f319c004470 con 0x7f3190004000 2024-09-17T05:05:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f318c073200 msgr2=0x7f318c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f318c073200 0x7f318c075710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f31980ae000 tx=0x7f319801ae00 comp rx=0 tx=0).stop 2024-09-17T05:05:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 msgr2=0x7f31ac0ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 0x7f31ac0ae280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f319c011b10 tx=0x7f319c009f20 comp rx=0 tx=0).stop 2024-09-17T05:05:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 shutdown_connections 2024-09-17T05:05:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f318c073200 0x7f318c075710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f31ac14f260 0x7f31ac1fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31ac14c5b0 0x7f31ac14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 --2- 172.21.15.16:0/1044506855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3190004000 0x7f31ac0ae280 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 >> 172.21.15.16:0/1044506855 conn(0x7f31ac0b2250 msgr2=0x7f31ac0cd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 shutdown_connections 2024-09-17T05:05:06.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:06.952+0000 7f31b2e00640 1 -- 172.21.15.16:0/1044506855 wait complete. 2024-09-17T05:05:06.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-09-17T05:05:07.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 -- 172.21.15.16:0/749165424 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac06d5a0 msgr2=0x7f9aac06d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 --2- 172.21.15.16:0/749165424 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac06d5a0 0x7f9aac06d9b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009bf0 tx=0x7f9aa401cbb0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 -- 172.21.15.16:0/749165424 shutdown_connections 2024-09-17T05:05:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 --2- 172.21.15.16:0/749165424 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac06d5a0 0x7f9aac06d9b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 -- 172.21.15.16:0/749165424 >> 172.21.15.16:0/749165424 conn(0x7f9aac0b2250 msgr2=0x7f9aac0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 -- 172.21.15.16:0/749165424 shutdown_connections 2024-09-17T05:05:07.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.136+0000 7f9ab2000640 1 -- 172.21.15.16:0/749165424 wait complete. 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 Processor -- start 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 -- start start 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 0x7f9aac151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9aac0733e0 0x7f9aac13d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac15d850 0x7f9aac1fca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac16acf0 con 0x7f9aac06d5a0 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac16ae70 con 0x7f9aac0733e0 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9aac16aff0 con 0x7f9aac15d850 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 0x7f9aac151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 0x7f9aac151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54822/0 (socket says 172.21.15.16:54822) 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 -- 172.21.15.16:0/49060254 learned_addr learned my addr 172.21.15.16:0/49060254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aa3e00640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9aac0733e0 0x7f9aac13d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aabe00640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac15d850 0x7f9aac1fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 -- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac15d850 msgr2=0x7f9aac1fca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac15d850 0x7f9aac1fca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 -- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9aac0733e0 msgr2=0x7f9aac13d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9aac0733e0 0x7f9aac13d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 -- 172.21.15.16:0/49060254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac14f850 con 0x7f9aac06d5a0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aab400640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 0x7f9aac151d50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c430 tx=0x7f9a9c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c011850 con 0x7f9aac06d5a0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a9c017070 con 0x7f9aac06d5a0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c011da0 con 0x7f9aac06d5a0 2024-09-17T05:05:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac14fb30 con 0x7f9aac06d5a0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.140+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac1fc510 con 0x7f9aac06d5a0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.144+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac15d140 con 0x7f9aac06d5a0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.144+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9a9c01c460 con 0x7f9aac06d5a0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.144+0000 7f9aa3400640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9a88073230 0x7f9a88075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.144+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f9a9c093fd0 con 0x7f9aac06d5a0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.144+0000 7f9aa3e00640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9a88073230 0x7f9a88075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.144+0000 7f9aa3e00640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9a88073230 0x7f9a88075740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0511b0 tx=0x7f9a94009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:07.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.148+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9a9c05e8a0 con 0x7f9aac06d5a0 2024-09-17T05:05:07.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.324+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f9aac15d610 con 0x7f9aac06d5a0 2024-09-17T05:05:07.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.328+0000 7f9aa3400640 1 -- 172.21.15.16:0/49060254 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v45)=0 v45) ==== 119+0+0 (secure 0 0 0) 0x7f9a9c0632d0 con 0x7f9aac06d5a0 2024-09-17T05:05:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9a88073230 msgr2=0x7f9a88075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9a88073230 0x7f9a88075740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0511b0 tx=0x7f9a94009040 comp rx=0 tx=0).stop 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 msgr2=0x7f9aac151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 0x7f9aac151d50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c00c430 tx=0x7f9a9c00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 shutdown_connections 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9a88073230 0x7f9a88075740 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9aac15d850 0x7f9aac1fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9aac0733e0 0x7f9aac13d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 --2- 172.21.15.16:0/49060254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9aac06d5a0 0x7f9aac151d50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 >> 172.21.15.16:0/49060254 conn(0x7f9aac0b2250 msgr2=0x7f9aac0b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 shutdown_connections 2024-09-17T05:05:07.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.332+0000 7f9ab2000640 1 -- 172.21.15.16:0/49060254 wait complete. 2024-09-17T05:05:07.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 -- 172.21.15.16:0/1498641160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074e30 msgr2=0x7efe4815f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 --2- 172.21.15.16:0/1498641160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074e30 0x7efe4815f520 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7efe480c70b0 tx=0x7efe3001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 -- 172.21.15.16:0/1498641160 shutdown_connections 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 --2- 172.21.15.16:0/1498641160 >> v2:172.21.15.104:3300/0 conn(0x7efe4815ba50 0x7efe4815be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 --2- 172.21.15.16:0/1498641160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074e30 0x7efe4815f520 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 --2- 172.21.15.16:0/1498641160 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe48074a20 0x7efe4815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 -- 172.21.15.16:0/1498641160 >> 172.21.15.16:0/1498641160 conn(0x7efe480b2250 msgr2=0x7efe480d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:07.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 -- 172.21.15.16:0/1498641160 shutdown_connections 2024-09-17T05:05:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 -- 172.21.15.16:0/1498641160 wait complete. 2024-09-17T05:05:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 Processor -- start 2024-09-17T05:05:07.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.516+0000 7efe4f800640 1 -- start start 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 0x7efe48157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe48074e30 0x7efe481f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe4815ba50 0x7efe480b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe4806dc20 con 0x7efe48074a20 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe4806daa0 con 0x7efe48074e30 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efe480ce5d0 con 0x7efe4815ba50 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 0x7efe48157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 0x7efe48157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54840/0 (socket says 172.21.15.16:54840) 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 -- 172.21.15.16:0/2673359015 learned_addr learned my addr 172.21.15.16:0/2673359015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4d800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe4815ba50 0x7efe480b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe3fe00640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe48074e30 0x7efe481f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 -- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe4815ba50 msgr2=0x7efe480b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe4815ba50 0x7efe480b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 -- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe48074e30 msgr2=0x7efe481f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe48074e30 0x7efe481f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 -- 172.21.15.16:0/2673359015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe48140550 con 0x7efe48074a20 2024-09-17T05:05:07.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4ce00640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 0x7efe48157740 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7efe3800e740 tx=0x7efe3800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3800ccc0 con 0x7efe48074a20 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe3800ce60 con 0x7efe48074a20 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48140830 con 0x7efe48074a20 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38005280 con 0x7efe48074a20 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe48140df0 con 0x7efe48074a20 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.520+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4815be60 con 0x7efe48074a20 2024-09-17T05:05:07.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.524+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efe38005420 con 0x7efe48074a20 2024-09-17T05:05:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.524+0000 7efe45600640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efe24073230 0x7efe24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.524+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7efe38094c30 con 0x7efe48074a20 2024-09-17T05:05:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.524+0000 7efe3fe00640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efe24073230 0x7efe24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.524+0000 7efe3fe00640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efe24073230 0x7efe24075740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efe300ae000 tx=0x7efe3001b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.528+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efe3805f460 con 0x7efe48074a20 2024-09-17T05:05:07.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.704+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7efe480cab10 con 0x7efe48074a20 2024-09-17T05:05:07.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.708+0000 7efe45600640 1 -- 172.21.15.16:0/2673359015 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v46)=0 v46) ==== 128+0+0 (secure 0 0 0) 0x7efe38063e90 con 0x7efe48074a20 2024-09-17T05:05:07.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efe24073230 msgr2=0x7efe24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efe24073230 0x7efe24075740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efe300ae000 tx=0x7efe3001b100 comp rx=0 tx=0).stop 2024-09-17T05:05:07.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 msgr2=0x7efe48157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 0x7efe48157740 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7efe3800e740 tx=0x7efe3800ec00 comp rx=0 tx=0).stop 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 shutdown_connections 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efe24073230 0x7efe24075740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe4815ba50 0x7efe480b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe48074e30 0x7efe481f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 --2- 172.21.15.16:0/2673359015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe48074a20 0x7efe48157740 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 >> 172.21.15.16:0/2673359015 conn(0x7efe480b2250 msgr2=0x7efe4815ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:07.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 shutdown_connections 2024-09-17T05:05:07.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.712+0000 7efe4f800640 1 -- 172.21.15.16:0/2673359015 wait complete. 2024-09-17T05:05:07.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 -- 172.21.15.16:0/621076990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40b78a0 msgr2=0x7f4ef40b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 --2- 172.21.15.16:0/621076990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40b78a0 0x7f4ef40b7cb0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40d2020 tx=0x7f4ee001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 -- 172.21.15.16:0/621076990 shutdown_connections 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 --2- 172.21.15.16:0/621076990 >> v2:172.21.15.104:3300/0 conn(0x7f4ef40cefa0 0x7f4ef40cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 --2- 172.21.15.16:0/621076990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40b78a0 0x7f4ef40b7cb0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 -- 172.21.15.16:0/621076990 >> 172.21.15.16:0/621076990 conn(0x7f4ef40b4160 msgr2=0x7f4ef40b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 -- 172.21.15.16:0/621076990 shutdown_connections 2024-09-17T05:05:07.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.892+0000 7f4efae00640 1 -- 172.21.15.16:0/621076990 wait complete. 2024-09-17T05:05:07.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 Processor -- start 2024-09-17T05:05:07.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 -- start start 2024-09-17T05:05:07.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef40b78a0 0x7f4ef406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 0x7f4ef40c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ef4162df0 0x7f4ef41516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4144930 con 0x7f4ef40cefa0 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41447b0 con 0x7f4ef40b78a0 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef4144ab0 con 0x7f4ef4162df0 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 0x7f4ef40c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 0x7f4ef40c3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54870/0 (socket says 172.21.15.16:54870) 2024-09-17T05:05:07.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 -- 172.21.15.16:0/1820280531 learned_addr learned my addr 172.21.15.16:0/1820280531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9a00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef40b78a0 0x7f4ef406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4efa400640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ef4162df0 0x7f4ef41516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 -- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ef4162df0 msgr2=0x7f4ef41516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ef4162df0 0x7f4ef41516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 -- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef40b78a0 msgr2=0x7f4ef406dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef40b78a0 0x7f4ef406dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 -- 172.21.15.16:0/1820280531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef40c41c0 con 0x7f4ef40cefa0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9a00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef40b78a0 0x7f4ef406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.896+0000 7f4ef9000640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 0x7f4ef40c3c80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00194d0 tx=0x7f4ee00a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00b2040 con 0x7f4ef40cefa0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ee00042c0 con 0x7f4ef40cefa0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef40715f0 con 0x7f4ef40cefa0 2024-09-17T05:05:07.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0005290 con 0x7f4ef40cefa0 2024-09-17T05:05:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef40719a0 con 0x7f4ef40cefa0 2024-09-17T05:05:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef40b7cb0 con 0x7f4ef40cefa0 2024-09-17T05:05:07.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4ee0005430 con 0x7f4ef40cefa0 2024-09-17T05:05:07.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef1600640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ed4073200 0x7f4ed4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:07.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f4ee012e2e0 con 0x7f4ef40cefa0 2024-09-17T05:05:07.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.900+0000 7f4ef9a00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ed4073200 0x7f4ed4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:07.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.904+0000 7f4ef9a00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ed4073200 0x7f4ed4075710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80016b0 tx=0x7f4ee8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:07.904+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4ee00c4060 con 0x7f4ef40cefa0 2024-09-17T05:05:08.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.080+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f4ef4142990 con 0x7f4ef40cefa0 2024-09-17T05:05:08.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.084+0000 7f4ef1600640 1 -- 172.21.15.16:0/1820280531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v47)=0 v47) ==== 124+0+0 (secure 0 0 0) 0x7f4ee00164b0 con 0x7f4ef40cefa0 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ed4073200 msgr2=0x7f4ed4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ed4073200 0x7f4ed4075710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80016b0 tx=0x7f4ee8009040 comp rx=0 tx=0).stop 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 msgr2=0x7f4ef40c3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 0x7f4ef40c3c80 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00194d0 tx=0x7f4ee00a56f0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 shutdown_connections 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ed4073200 0x7f4ed4075710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ef4162df0 0x7f4ef41516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef40cefa0 0x7f4ef40c3c80 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 --2- 172.21.15.16:0/1820280531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ef40b78a0 0x7f4ef406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 >> 172.21.15.16:0/1820280531 conn(0x7f4ef40b4160 msgr2=0x7f4ef41550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 shutdown_connections 2024-09-17T05:05:08.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.088+0000 7f4efae00640 1 -- 172.21.15.16:0/1820280531 wait complete. 2024-09-17T05:05:08.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 -- 172.21.15.16:0/3796593891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 msgr2=0x7f44400cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 --2- 172.21.15.16:0/3796593891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400cb2c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4438009bc0 tx=0x7f443801cb70 comp rx=0 tx=0).stop 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 -- 172.21.15.16:0/3796593891 shutdown_connections 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 --2- 172.21.15.16:0/3796593891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400cb2c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 --2- 172.21.15.16:0/3796593891 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44400ceea0 0x7f444015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 -- 172.21.15.16:0/3796593891 >> 172.21.15.16:0/3796593891 conn(0x7f44400b2250 msgr2=0x7f44400d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 -- 172.21.15.16:0/3796593891 shutdown_connections 2024-09-17T05:05:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.268+0000 7f4447a00640 1 -- 172.21.15.16:0/3796593891 wait complete. 2024-09-17T05:05:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 Processor -- start 2024-09-17T05:05:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 -- start start 2024-09-17T05:05:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4440074510 0x7f4440148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44400ceea0 0x7f44400c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4440167ba0 con 0x7f4440074920 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f444006d790 con 0x7f44400ceea0 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4440167d20 con 0x7f4440074510 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54892/0 (socket says 172.21.15.16:54892) 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 -- 172.21.15.16:0/1616882032 learned_addr learned my addr 172.21.15.16:0/1616882032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4445000640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4440074510 0x7f4440148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 -- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4440074510 msgr2=0x7f4440148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4445a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44400ceea0 0x7f44400c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4440074510 0x7f4440148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 -- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44400ceea0 msgr2=0x7f44400c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44400ceea0 0x7f44400c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 -- 172.21.15.16:0/1616882032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44401fd6d0 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4445a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44400ceea0 0x7f44400c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443fe00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400c8020 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c760 tx=0x7f442c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c016020 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442c00a8f0 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c00ad40 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.272+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44401524c0 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.276+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401fd3f0 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.276+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44401fd180 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.276+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f442c019020 con 0x7f4440074920 2024-09-17T05:05:08.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.276+0000 7f443cc00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4420073230 0x7f4420075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.276+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f442c0941c0 con 0x7f4440074920 2024-09-17T05:05:08.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.276+0000 7f4445000640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4420073230 0x7f4420075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.280+0000 7f4445000640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4420073230 0x7f4420075740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4434005fd0 tx=0x7f4434005f20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:08.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.280+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f442c020020 con 0x7f4440074920 2024-09-17T05:05:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.456+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f4440152690 con 0x7f4440074920 2024-09-17T05:05:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.456+0000 7f443cc00640 1 -- 172.21.15.16:0/1616882032 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v47) ==== 106+0+4 (secure 0 0 0) 0x7f44400cab40 con 0x7f4440074920 2024-09-17T05:05:08.460 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-17T05:05:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4420073230 msgr2=0x7f4420075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4420073230 0x7f4420075740 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4434005fd0 tx=0x7f4434005f20 comp rx=0 tx=0).stop 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 msgr2=0x7f44400c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400c8020 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f442c00c760 tx=0x7f442c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 shutdown_connections 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4420073230 0x7f4420075740 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44400ceea0 0x7f44400c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4440074920 0x7f44400c8020 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 --2- 172.21.15.16:0/1616882032 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4440074510 0x7f4440148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 >> 172.21.15.16:0/1616882032 conn(0x7f44400b2250 msgr2=0x7f44400b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 shutdown_connections 2024-09-17T05:05:08.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.460+0000 7f4447a00640 1 -- 172.21.15.16:0/1616882032 wait complete. 2024-09-17T05:05:08.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 -- 172.21.15.16:0/1893246384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c003a60 msgr2=0x7ff20c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 --2- 172.21.15.16:0/1893246384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c003a60 0x7ff20c003ed0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff21800c2e0 tx=0x7ff21801cec0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 -- 172.21.15.16:0/1893246384 shutdown_connections 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 --2- 172.21.15.16:0/1893246384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20c003a60 0x7ff20c003ed0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 -- 172.21.15.16:0/1893246384 >> 172.21.15.16:0/1893246384 conn(0x7ff2280b2250 msgr2=0x7ff2280d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 -- 172.21.15.16:0/1893246384 shutdown_connections 2024-09-17T05:05:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 -- 172.21.15.16:0/1893246384 wait complete. 2024-09-17T05:05:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 Processor -- start 2024-09-17T05:05:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.644+0000 7ff22d600640 1 -- start start 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c003a60 0x7ff2281f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22814d630 0x7ff2281fa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 0x7ff2280c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff22815c750 con 0x7ff228154ac0 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff22815c5d0 con 0x7ff22814d630 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff22814ddf0 con 0x7ff20c003a60 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 0x7ff2280c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 0x7ff2280c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54904/0 (socket says 172.21.15.16:54904) 2024-09-17T05:05:08.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 -- 172.21.15.16:0/1380449244 learned_addr learned my addr 172.21.15.16:0/1380449244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff226a00640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c003a60 0x7ff2281f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff21e000640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22814d630 0x7ff2281fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 -- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c003a60 msgr2=0x7ff2281f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c003a60 0x7ff2281f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 -- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22814d630 msgr2=0x7ff2281fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22814d630 0x7ff2281fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 -- 172.21.15.16:0/1380449244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2280c6a40 con 0x7ff228154ac0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff227400640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 0x7ff2280c8ce0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff228073480 tx=0x7ff220010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220017070 con 0x7ff228154ac0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff220015070 con 0x7ff228154ac0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2280c8200 con 0x7ff228154ac0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220004d90 con 0x7ff228154ac0 2024-09-17T05:05:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2280c3fb0 con 0x7ff228154ac0 2024-09-17T05:05:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.648+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2280c4210 con 0x7ff228154ac0 2024-09-17T05:05:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.652+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff220003980 con 0x7ff228154ac0 2024-09-17T05:05:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.652+0000 7ff21f400640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff204073200 0x7ff204075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.652+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7ff2200949f0 con 0x7ff228154ac0 2024-09-17T05:05:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.652+0000 7ff226a00640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff204073200 0x7ff204075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.652+0000 7ff226a00640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff204073200 0x7ff204075710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff2180ab000 tx=0x7ff218019d00 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:08.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.656+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff22005f7e0 con 0x7ff228154ac0 2024-09-17T05:05:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7ff2280ce9b0 con 0x7ff228154ac0 2024-09-17T05:05:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff21f400640 1 -- 172.21.15.16:0/1380449244 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v47) ==== 112+0+10 (secure 0 0 0) 0x7ff22005f0e0 con 0x7ff228154ac0 2024-09-17T05:05:08.834 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff204073200 msgr2=0x7ff204075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff204073200 0x7ff204075710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff2180ab000 tx=0x7ff218019d00 comp rx=0 tx=0).stop 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 msgr2=0x7ff2280c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 0x7ff2280c8ce0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff228073480 tx=0x7ff220010f20 comp rx=0 tx=0).stop 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 shutdown_connections 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff204073200 0x7ff204075710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff228154ac0 0x7ff2280c8ce0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff22814d630 0x7ff2281fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 --2- 172.21.15.16:0/1380449244 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff20c003a60 0x7ff2281f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 >> 172.21.15.16:0/1380449244 conn(0x7ff2280b2250 msgr2=0x7ff2280cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 shutdown_connections 2024-09-17T05:05:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08.832+0000 7ff22d600640 1 -- 172.21.15.16:0/1380449244 wait complete. 2024-09-17T05:05:08.856 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:05:08.857 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:05:08.865 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:05:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:05:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:05:08.967 INFO:teuthology.orchestra.run.smithi016.stdout:client6373 2024-09-17T05:05:08.967 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:05:08.967 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:05:08.976 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:05:08.976 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-17T05:05:08.976 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-17T05:05:08.976 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-17T05:05:08.976 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> n = 400 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-17T05:05:08.977 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-17T05:05:08.978 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:05:09.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:05:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 -- 172.21.15.16:0/3280212234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc0003a60 msgr2=0x7fdbc0003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/3280212234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc0003a60 0x7fdbc0003ef0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c2e0 tx=0x7fdbcc01cf20 comp rx=0 tx=0).stop 2024-09-17T05:05:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 -- 172.21.15.16:0/3280212234 shutdown_connections 2024-09-17T05:05:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/3280212234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc0003a60 0x7fdbc0003ef0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 -- 172.21.15.16:0/3280212234 >> 172.21.15.16:0/3280212234 conn(0x7fdbdc06fcf0 msgr2=0x7fdbdc070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 -- 172.21.15.16:0/3280212234 shutdown_connections 2024-09-17T05:05:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.184+0000 7fdbe1c00640 1 -- 172.21.15.16:0/3280212234 wait complete. 2024-09-17T05:05:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.188+0000 7fdbe1c00640 1 Processor -- start 2024-09-17T05:05:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.188+0000 7fdbe1c00640 1 -- start start 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc0003a60 0x7fdbdc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 0x7fdbdc06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbdc15e830 0x7fdbdc06e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc152450 con 0x7fdbdc15e420 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1522d0 con 0x7fdbdc15e830 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdbdc1525d0 con 0x7fdbc0003a60 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 0x7fdbdc06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 0x7fdbdc06dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54916/0 (socket says 172.21.15.16:54916) 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 -- 172.21.15.16:0/823788709 learned_addr learned my addr 172.21.15.16:0/823788709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbdaa00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc0003a60 0x7fdbdc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbdb400640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbdc15e830 0x7fdbdc06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 -- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc0003a60 msgr2=0x7fdbdc0af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc0003a60 0x7fdbdc0af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 -- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbdc15e830 msgr2=0x7fdbdc06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbdc15e830 0x7fdbdc06e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 -- 172.21.15.16:0/823788709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc1f9080 con 0x7fdbdc15e420 2024-09-17T05:05:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbda000640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 0x7fdbdc06dee0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8004da0 tx=0x7fdbc8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80105d0 con 0x7fdbdc15e420 2024-09-17T05:05:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbc8016070 con 0x7fdbdc15e420 2024-09-17T05:05:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8010ae0 con 0x7fdbdc15e420 2024-09-17T05:05:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc1f95b0 con 0x7fdbdc15e420 2024-09-17T05:05:09.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc1f9b70 con 0x7fdbdc15e420 2024-09-17T05:05:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.192+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc1f9320 con 0x7fdbdc15e420 2024-09-17T05:05:09.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.196+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdbc8010c80 con 0x7fdbdc15e420 2024-09-17T05:05:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.196+0000 7fdbd2a00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdbbc073200 0x7fdbbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.196+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fdbc8094a20 con 0x7fdbdc15e420 2024-09-17T05:05:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.196+0000 7fdbdaa00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdbbc073200 0x7fdbbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.196+0000 7fdbdaa00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdbbc073200 0x7fdbbc075710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0ac000 tx=0x7fdbcc019d60 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:09.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.204+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdbc805f250 con 0x7fdbdc15e420 2024-09-17T05:05:09.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.496+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbdc15f030 con 0x7fdbdc15e420 2024-09-17T05:05:09.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.496+0000 7fdbd2a00640 1 -- 172.21.15.16:0/823788709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5228 (secure 0 0 0) 0x7fdbc8063c80 con 0x7fdbdc15e420 2024-09-17T05:05:09.500 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:09.500 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":52,"btime":"2024-09-17T05:04:56:488386+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:04:55.907544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:05:09.500 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 52 2024-09-17T05:05:09.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdbbc073200 msgr2=0x7fdbbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdbbc073200 0x7fdbbc075710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0ac000 tx=0x7fdbcc019d60 comp rx=0 tx=0).stop 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 msgr2=0x7fdbdc06dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 0x7fdbdc06dee0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8004da0 tx=0x7fdbc8001640 comp rx=0 tx=0).stop 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 shutdown_connections 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdbbc073200 0x7fdbbc075710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbdc15e830 0x7fdbdc06e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc15e420 0x7fdbdc06dee0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 --2- 172.21.15.16:0/823788709 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc0003a60 0x7fdbdc0af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 >> 172.21.15.16:0/823788709 conn(0x7fdbdc06fcf0 msgr2=0x7fdbdc151220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 shutdown_connections 2024-09-17T05:05:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.500+0000 7fdbe1c00640 1 -- 172.21.15.16:0/823788709 wait complete. 2024-09-17T05:05:09.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-17T05:05:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-17T05:05:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:09.709 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":6379,"entity":{"name":{"type":"client","num":6379},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2295609657}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.2519708839999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9897161224850053,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6379 v1:192.168.0.1:0/2295609657","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":6373,"entity":{"name":{"type":"client","num":6373},"addr":{"type":"v1","addr":"172.21.15.16:0","nonce":4272962761}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":425,"uptime":4.4319603240000003,"requests_in_flight":400,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":199.50156063455356,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":181.00145425346355,"halflife":1.5},"recall_caps_throttle2o":{"value":148.24709823853814,"halflife":0.5},"session_cache_liveness":{"value":804.31673772565568,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6373 v1:172.21.15.16:0/4272962761","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:05:09.710 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2295609657, "type": "v1" }, "name": { "num": 6379, "type": "client" } }, "id": 6379, "inst": "client.6379 v1:192.168.0.1:0/2295609657", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9897161224850053 }, "state": "open", "uptime": 3.251970884 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 4272962761, "type": "v1" }, "name": { "num": 6373, "type": "client" } }, "id": 6373, "inst": "client.6373 v1:172.21.15.16:0/4272962761", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 199.50156063455356 }, "recall_caps_throttle": { "halflife": 1.5, "value": 181.00145425346355 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 148.24709823853814 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 425, "requests_in_flight": 400, "session_cache_liveness": { "halflife": 300, "value": 804.3167377256557 }, "state": "open", "uptime": 4.431960324 } ] 2024-09-17T05:05:09.710 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-17T05:05:09.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 -- 172.21.15.16:0/2059788123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406da50 msgr2=0x7fb12406ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 --2- 172.21.15.16:0/2059788123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406da50 0x7fb12406ce90 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb124072860 tx=0x7fb11401c880 comp rx=0 tx=0).stop 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 -- 172.21.15.16:0/2059788123 shutdown_connections 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 --2- 172.21.15.16:0/2059788123 >> v2:172.21.15.104:3300/0 conn(0x7fb1240b4980 0x7fb124165360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 --2- 172.21.15.16:0/2059788123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406da50 0x7fb12406ce90 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 -- 172.21.15.16:0/2059788123 >> 172.21.15.16:0/2059788123 conn(0x7fb12406f900 msgr2=0x7fb1240b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 -- 172.21.15.16:0/2059788123 shutdown_connections 2024-09-17T05:05:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 -- 172.21.15.16:0/2059788123 wait complete. 2024-09-17T05:05:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 Processor -- start 2024-09-17T05:05:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.912+0000 7fb12b400640 1 -- start start 2024-09-17T05:05:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb12406da50 0x7fb12413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 0x7fb124157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb124152b10 0x7fb124157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb124168950 con 0x7fb12406de60 2024-09-17T05:05:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb1241687d0 con 0x7fb124152b10 2024-09-17T05:05:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb124168ad0 con 0x7fb12406da50 2024-09-17T05:05:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 0x7fb124157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 0x7fb124157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54944/0 (socket says 172.21.15.16:54944) 2024-09-17T05:05:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 -- 172.21.15.16:0/3191946879 learned_addr learned my addr 172.21.15.16:0/3191946879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb128a00640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb12406da50 0x7fb12413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb129400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb124152b10 0x7fb124157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 -- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb12406da50 msgr2=0x7fb12413e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb12406da50 0x7fb12413e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 -- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb124152b10 msgr2=0x7fb124157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb124152b10 0x7fb124157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 -- 172.21.15.16:0/3191946879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1240ad900 con 0x7fb12406de60 2024-09-17T05:05:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb11be00640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 0x7fb124157310 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b440 tx=0x7fb10c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c015020 con 0x7fb12406de60 2024-09-17T05:05:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb10c011980 con 0x7fb12406de60 2024-09-17T05:05:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1241580e0 con 0x7fb12406de60 2024-09-17T05:05:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011dd0 con 0x7fb12406de60 2024-09-17T05:05:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1240ade80 con 0x7fb12406de60 2024-09-17T05:05:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.916+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb124157e50 con 0x7fb12406de60 2024-09-17T05:05:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.920+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb10c018020 con 0x7fb12406de60 2024-09-17T05:05:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.920+0000 7fb121600640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb100073230 0x7fb100075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.920+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fb10c0942d0 con 0x7fb12406de60 2024-09-17T05:05:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.920+0000 7fb128a00640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb100073230 0x7fb100075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.920+0000 7fb128a00640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb100073230 0x7fb100075740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb1140af000 tx=0x7fb1140a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:09.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:09.924+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb10c01f020 con 0x7fb12406de60 2024-09-17T05:05:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.140+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb12406ce90 con 0x7fb12406de60 2024-09-17T05:05:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb121600640 1 -- 172.21.15.16:0/3191946879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fb10c00d040 con 0x7fb12406de60 2024-09-17T05:05:10.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:10.145 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb100073230 msgr2=0x7fb100075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb100073230 0x7fb100075740 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb1140af000 tx=0x7fb1140a7040 comp rx=0 tx=0).stop 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 msgr2=0x7fb124157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 0x7fb124157310 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b440 tx=0x7fb10c00b900 comp rx=0 tx=0).stop 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 shutdown_connections 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb100073230 0x7fb100075740 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb124152b10 0x7fb124157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb12406de60 0x7fb124157310 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 --2- 172.21.15.16:0/3191946879 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb12406da50 0x7fb12413e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 >> 172.21.15.16:0/3191946879 conn(0x7fb12406f900 msgr2=0x7fb1241519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 shutdown_connections 2024-09-17T05:05:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.144+0000 7fb12b400640 1 -- 172.21.15.16:0/3191946879 wait complete. 2024-09-17T05:05:10.168 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:05:10.168 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-17T05:05:10.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 -- 172.21.15.16:0/3809125491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee40b2f50 msgr2=0x7f1ee415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 --2- 172.21.15.16:0/3809125491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee40b2f50 0x7f1ee415bb60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009840 tx=0x7f1ed401c790 comp rx=0 tx=0).stop 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 -- 172.21.15.16:0/3809125491 shutdown_connections 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 --2- 172.21.15.16:0/3809125491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee40b2f50 0x7f1ee415bb60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 -- 172.21.15.16:0/3809125491 >> 172.21.15.16:0/3809125491 conn(0x7f1ee406f900 msgr2=0x7f1ee40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 -- 172.21.15.16:0/3809125491 shutdown_connections 2024-09-17T05:05:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 -- 172.21.15.16:0/3809125491 wait complete. 2024-09-17T05:05:10.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 Processor -- start 2024-09-17T05:05:10.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.324+0000 7f1eea600640 1 -- start start 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ee4073240 0x7f1ee41f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ee40b2f50 0x7f1ee414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 0x7f1ee4151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4169cf0 con 0x7f1ee415c0a0 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4169b70 con 0x7f1ee40b2f50 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee4169e70 con 0x7f1ee4073240 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 0x7f1ee4151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 0x7f1ee4151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54968/0 (socket says 172.21.15.16:54968) 2024-09-17T05:05:10.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 -- 172.21.15.16:0/3294679633 learned_addr learned my addr 172.21.15.16:0/3294679633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3400640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ee4073240 0x7f1ee41f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1edaa00640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ee40b2f50 0x7f1ee414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 -- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ee4073240 msgr2=0x7f1ee41f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ee4073240 0x7f1ee41f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 -- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ee40b2f50 msgr2=0x7f1ee414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ee40b2f50 0x7f1ee414e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 -- 172.21.15.16:0/3294679633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee4159100 con 0x7f1ee415c0a0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1ee3e00640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 0x7f1ee4151640 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c730 tx=0x7f1edc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc003b80 con 0x7f1ee415c0a0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1edc003d40 con 0x7f1ee415c0a0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0052d0 con 0x7f1ee415c0a0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee4158580 con 0x7f1ee415c0a0 2024-09-17T05:05:10.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee4158b40 con 0x7f1ee415c0a0 2024-09-17T05:05:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee41582f0 con 0x7f1ee415c0a0 2024-09-17T05:05:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.328+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1edc005680 con 0x7f1ee415c0a0 2024-09-17T05:05:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.332+0000 7f1edbe00640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1ec0073230 0x7f1ec0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.332+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f1edc0943b0 con 0x7f1ee415c0a0 2024-09-17T05:05:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.332+0000 7f1ee3400640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1ec0073230 0x7f1ec0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:10.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.332+0000 7f1ee3400640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1ec0073230 0x7f1ec0075740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40af000 tx=0x7f1ed401afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:10.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.336+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1edc05f1a0 con 0x7f1ee415c0a0 2024-09-17T05:05:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.544+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1ee406e380 con 0x7f1ee415c0a0 2024-09-17T05:05:10.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1edbe00640 1 -- 172.21.15.16:0/3294679633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f1edc05eaa0 con 0x7f1ee415c0a0 2024-09-17T05:05:10.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:10.550 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1ec0073230 msgr2=0x7f1ec0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1ec0073230 0x7f1ec0075740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40af000 tx=0x7f1ed401afd0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 msgr2=0x7f1ee4151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 0x7f1ee4151640 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00c730 tx=0x7f1edc00cbf0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 shutdown_connections 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1ec0073230 0x7f1ec0075740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ee415c0a0 0x7f1ee4151640 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ee40b2f50 0x7f1ee414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 --2- 172.21.15.16:0/3294679633 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ee4073240 0x7f1ee41f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 >> 172.21.15.16:0/3294679633 conn(0x7f1ee406f900 msgr2=0x7f1ee40cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:10.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 shutdown_connections 2024-09-17T05:05:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:10.548+0000 7f1eea600640 1 -- 172.21.15.16:0/3294679633 wait complete. 2024-09-17T05:05:10.572 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:10.572 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:13.577 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:15.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:15.579 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 -- 172.21.15.16:0/1637901088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab8003a10 msgr2=0x7fbab8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 --2- 172.21.15.16:0/1637901088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbab8003a10 0x7fbab8003e80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbac400d420 tx=0x7fbac401e2c0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad3400640 1 -- 172.21.15.16:0/1637901088 reap_dead start 2024-09-17T05:05:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 -- 172.21.15.16:0/1637901088 shutdown_connections 2024-09-17T05:05:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 -- 172.21.15.16:0/1637901088 >> 172.21.15.16:0/1637901088 conn(0x7fbad406f900 msgr2=0x7fbad40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 -- 172.21.15.16:0/1637901088 shutdown_connections 2024-09-17T05:05:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 -- 172.21.15.16:0/1637901088 wait complete. 2024-09-17T05:05:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 Processor -- start 2024-09-17T05:05:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.744+0000 7fbad9800640 1 -- start start 2024-09-17T05:05:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbab8003a10 0x7fbad413e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:15.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 0x7fbad40ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad4140de0 0x7fbad40cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbad415f030 con 0x7fbad41409d0 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbad415eeb0 con 0x7fbab8003a10 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbad40732f0 con 0x7fbad4140de0 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 0x7fbad40ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 0x7fbad40ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55000/0 (socket says 172.21.15.16:55000) 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 -- 172.21.15.16:0/21854793 learned_addr learned my addr 172.21.15.16:0/21854793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad3400640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad4140de0 0x7fbad40cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad2a00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbab8003a10 0x7fbad413e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 -- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad4140de0 msgr2=0x7fbad40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad4140de0 0x7fbad40cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 -- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbab8003a10 msgr2=0x7fbad413e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbab8003a10 0x7fbad413e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 -- 172.21.15.16:0/21854793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad413e940 con 0x7fbad41409d0 2024-09-17T05:05:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad2a00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbab8003a10 0x7fbad413e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacbe00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 0x7fbad40ce720 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbabc011ab0 tx=0x7fbabc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc01e070 con 0x7fbad41409d0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbabc00fe70 con 0x7fbad41409d0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc0042b0 con 0x7fbad41409d0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad4152240 con 0x7fbad41409d0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad4141800 con 0x7fbad41409d0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.748+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad4141270 con 0x7fbad41409d0 2024-09-17T05:05:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.752+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbabc021070 con 0x7fbad41409d0 2024-09-17T05:05:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.752+0000 7fbacaa00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.752+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fbabc094e50 con 0x7fbad41409d0 2024-09-17T05:05:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.752+0000 7fbad2a00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.752+0000 7fbad2a00640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbac40ae000 tx=0x7fbac40b2040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.756+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbabc05f680 con 0x7fbad41409d0 2024-09-17T05:05:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.964+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbad4073850 con 0x7fbad41409d0 2024-09-17T05:05:15.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.964+0000 7fbacaa00640 1 -- 172.21.15.16:0/21854793 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fbabc0640b0 con 0x7fbad41409d0 2024-09-17T05:05:15.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:15.969 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 msgr2=0x7fbab0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbac40ae000 tx=0x7fbac40b2040 comp rx=0 tx=0).stop 2024-09-17T05:05:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 msgr2=0x7fbad40ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 0x7fbad40ce720 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbabc011ab0 tx=0x7fbabc011f70 comp rx=0 tx=0).stop 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 shutdown_connections 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbab0073200 0x7fbab0075710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbad4140de0 0x7fbad40cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbad41409d0 0x7fbad40ce720 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 --2- 172.21.15.16:0/21854793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbab8003a10 0x7fbad413e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 >> 172.21.15.16:0/21854793 conn(0x7fbad406f900 msgr2=0x7fbad41611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 shutdown_connections 2024-09-17T05:05:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:15.968+0000 7fbad9800640 1 -- 172.21.15.16:0/21854793 wait complete. 2024-09-17T05:05:15.991 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:15.992 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:17.581 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:19.583 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:20.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.160+0000 7f53a9e00640 1 -- 172.21.15.16:0/2493145525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388004e70 msgr2=0x7f53880052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:21.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.160+0000 7f53a9e00640 1 --2- 172.21.15.16:0/2493145525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5388004e70 0x7f53880052c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f53a4072d60 tx=0x7f539c01d110 comp rx=0 tx=0).stop 2024-09-17T05:05:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.160+0000 7f53a3400640 1 -- 172.21.15.16:0/2493145525 reap_dead start 2024-09-17T05:05:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.164+0000 7f53a9e00640 1 -- 172.21.15.16:0/2493145525 shutdown_connections 2024-09-17T05:05:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.164+0000 7f53a9e00640 1 -- 172.21.15.16:0/2493145525 >> 172.21.15.16:0/2493145525 conn(0x7f53a406f900 msgr2=0x7f53a40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.164+0000 7f53a9e00640 1 -- 172.21.15.16:0/2493145525 shutdown_connections 2024-09-17T05:05:21.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.164+0000 7f53a9e00640 1 -- 172.21.15.16:0/2493145525 wait complete. 2024-09-17T05:05:21.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.164+0000 7f53a9e00640 1 Processor -- start 2024-09-17T05:05:21.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.164+0000 7f53a9e00640 1 -- start start 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5388004e70 0x7f53a413e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 0x7f53a41f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a41f3cb0 0x7f53a41f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4073ce0 con 0x7f53a41f4380 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53a4073b60 con 0x7f5388004e70 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f53a4073e60 con 0x7f53a41f3cb0 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 0x7f53a41f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 0x7f53a41f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60186/0 (socket says 172.21.15.16:60186) 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5388004e70 0x7f53a413e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 -- 172.21.15.16:0/3070717581 learned_addr learned my addr 172.21.15.16:0/3070717581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a3400640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a41f3cb0 0x7f53a41f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 -- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a41f3cb0 msgr2=0x7f53a41f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a41f3cb0 0x7f53a41f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 -- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5388004e70 msgr2=0x7f53a413e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5388004e70 0x7f53a413e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 -- 172.21.15.16:0/3070717581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a41e8450 con 0x7f53a41f4380 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2a00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5388004e70 0x7f53a413e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a2000640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 0x7f53a41f3770 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5390003ea0 tx=0x7f5390003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390015070 con 0x7f53a41f4380 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5390011840 con 0x7f53a41f4380 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a415a280 con 0x7f53a41f4380 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390011c50 con 0x7f53a41f4380 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a4141540 con 0x7f53a41f4380 2024-09-17T05:05:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a40b3520 con 0x7f53a41f4380 2024-09-17T05:05:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.168+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5390018070 con 0x7f53a41f4380 2024-09-17T05:05:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.172+0000 7f539b400640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073200 0x7f5384075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.172+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f5390094ff0 con 0x7f53a41f4380 2024-09-17T05:05:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.172+0000 7f53a2a00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073200 0x7f5384075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:21.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.172+0000 7f53a2a00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073200 0x7f5384075710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f53940017c0 tx=0x7f53940067b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:21.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.176+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f539005f7b0 con 0x7f53a41f4380 2024-09-17T05:05:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.384+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f53a40b1be0 con 0x7f53a41f4380 2024-09-17T05:05:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.384+0000 7f539b400640 1 -- 172.21.15.16:0/3070717581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f53900641e0 con 0x7f53a41f4380 2024-09-17T05:05:21.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:21.387 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.384+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073200 msgr2=0x7f5384075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.384+0000 7f53a9e00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073200 0x7f5384075710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f53940017c0 tx=0x7f53940067b0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.384+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 msgr2=0x7f53a41f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.384+0000 7f53a9e00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 0x7f53a41f3770 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5390003ea0 tx=0x7f5390003f90 comp rx=0 tx=0).stop 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 shutdown_connections 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5384073200 0x7f5384075710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53a41f3cb0 0x7f53a41f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53a41f4380 0x7f53a41f3770 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 --2- 172.21.15.16:0/3070717581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5388004e70 0x7f53a413e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:21.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 >> 172.21.15.16:0/3070717581 conn(0x7f53a406f900 msgr2=0x7f53a415e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 shutdown_connections 2024-09-17T05:05:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:21.388+0000 7f53a9e00640 1 -- 172.21.15.16:0/3070717581 wait complete. 2024-09-17T05:05:21.409 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:21.410 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:21.589 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:23.588 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:25.589 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:26.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 -- 172.21.15.16:0/2324384603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 msgr2=0x7fc710167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 --2- 172.21.15.16:0/2324384603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc710167500 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc710072860 tx=0x7fc70801cb80 comp rx=0 tx=0).stop 2024-09-17T05:05:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 -- 172.21.15.16:0/2324384603 shutdown_connections 2024-09-17T05:05:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 --2- 172.21.15.16:0/2324384603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc710167500 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 --2- 172.21.15.16:0/2324384603 >> v2:172.21.15.104:3300/0 conn(0x7fc71015d600 0x7fc7100cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 -- 172.21.15.16:0/2324384603 >> 172.21.15.16:0/2324384603 conn(0x7fc71006f900 msgr2=0x7fc7100b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 -- 172.21.15.16:0/2324384603 shutdown_connections 2024-09-17T05:05:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 -- 172.21.15.16:0/2324384603 wait complete. 2024-09-17T05:05:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 Processor -- start 2024-09-17T05:05:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.576+0000 7fc716000640 1 -- start start 2024-09-17T05:05:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7100cf390 0x7fc7100afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71015d600 0x7fc7100b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc7101fc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc71016a260 con 0x7fc710160880 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc7100b5280 con 0x7fc7100cf390 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc71016a3e0 con 0x7fc71015d600 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc7101fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc7101fc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 -- 172.21.15.16:0/917165096 learned_addr learned my addr 172.21.15.16:0/917165096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70f400640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7100cf390 0x7fc7100afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc707e00640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71015d600 0x7fc7100b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 -- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71015d600 msgr2=0x7fc7100b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71015d600 0x7fc7100b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 -- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7100cf390 msgr2=0x7fc7100afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7100cf390 0x7fc7100afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 -- 172.21.15.16:0/917165096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc710140a80 con 0x7fc710160880 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc707e00640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71015d600 0x7fc7100b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc70fe00640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc7101fc650 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc708009b10 tx=0x7fc708009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080bc070 con 0x7fc710160880 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7080042f0 con 0x7fc710160880 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc710140d60 con 0x7fc710160880 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080b2040 con 0x7fc710160880 2024-09-17T05:05:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc710072e70 con 0x7fc710160880 2024-09-17T05:05:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.580+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7100730d0 con 0x7fc710160880 2024-09-17T05:05:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.584+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7080bf070 con 0x7fc710160880 2024-09-17T05:05:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.584+0000 7fc707400640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc6ec073230 0x7fc6ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.584+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fc70812e160 con 0x7fc710160880 2024-09-17T05:05:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.584+0000 7fc70f400640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc6ec073230 0x7fc6ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.584+0000 7fc70f400640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc6ec073230 0x7fc6ec075740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc7100afb90 tx=0x7fc70000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.588+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc708016770 con 0x7fc710160880 2024-09-17T05:05:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.804+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc71015a530 con 0x7fc710160880 2024-09-17T05:05:26.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.804+0000 7fc707400640 1 -- 172.21.15.16:0/917165096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc7080f8900 con 0x7fc710160880 2024-09-17T05:05:26.807 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:26.807 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc6ec073230 msgr2=0x7fc6ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:26.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc6ec073230 0x7fc6ec075740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc7100afb90 tx=0x7fc70000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 msgr2=0x7fc7101fc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc7101fc650 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc708009b10 tx=0x7fc708009310 comp rx=0 tx=0).stop 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 shutdown_connections 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc6ec073230 0x7fc6ec075740 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc710160880 0x7fc7101fc650 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc71015d600 0x7fc7100b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 --2- 172.21.15.16:0/917165096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7100cf390 0x7fc7100afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 >> 172.21.15.16:0/917165096 conn(0x7fc71006f900 msgr2=0x7fc7100d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 shutdown_connections 2024-09-17T05:05:26.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:26.808+0000 7fc716000640 1 -- 172.21.15.16:0/917165096 wait complete. 2024-09-17T05:05:26.829 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:26.830 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:27.591 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:30.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:05:30.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:05:31.596 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:31.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 -- 172.21.15.16:0/2584480650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 msgr2=0x7f30b015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 --2- 172.21.15.16:0/2584480650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 0x7f30b015bb60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009840 tx=0x7f30a001c6b0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 -- 172.21.15.16:0/2584480650 shutdown_connections 2024-09-17T05:05:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 --2- 172.21.15.16:0/2584480650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 0x7f30b015bb60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 -- 172.21.15.16:0/2584480650 >> 172.21.15.16:0/2584480650 conn(0x7f30b006f900 msgr2=0x7f30b00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 -- 172.21.15.16:0/2584480650 shutdown_connections 2024-09-17T05:05:32.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 -- 172.21.15.16:0/2584480650 wait complete. 2024-09-17T05:05:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 Processor -- start 2024-09-17T05:05:32.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.000+0000 7f30b5200640 1 -- start start 2024-09-17T05:05:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30b0073240 0x7f30b013d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 0x7f30b013fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:32.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 0x7f30b01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30b0169cc0 con 0x7f30b015c0a0 2024-09-17T05:05:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30b0169b40 con 0x7f30b00b2f50 2024-09-17T05:05:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f30b0169e40 con 0x7f30b0073240 2024-09-17T05:05:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 0x7f30b01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 0x7f30b01548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39238/0 (socket says 172.21.15.16:39238) 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 -- 172.21.15.16:0/2263874823 learned_addr learned my addr 172.21.15.16:0/2263874823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ade00640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30b0073240 0x7f30b013d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30a5400640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 0x7f30b013fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 -- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30b0073240 msgr2=0x7f30b013d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30b0073240 0x7f30b013d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 -- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 msgr2=0x7f30b013fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 0x7f30b013fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 -- 172.21.15.16:0/2263874823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b0140c10 con 0x7f30b015c0a0 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30ae800640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 0x7f30b01548a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c3c0 tx=0x7f30a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8017070 con 0x7f30b015c0a0 2024-09-17T05:05:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a80053c0 con 0x7f30b015c0a0 2024-09-17T05:05:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b0140650 con 0x7f30b015c0a0 2024-09-17T05:05:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8003860 con 0x7f30b015c0a0 2024-09-17T05:05:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b0160320 con 0x7f30b015c0a0 2024-09-17T05:05:32.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.004+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b00b1e20 con 0x7f30b015c0a0 2024-09-17T05:05:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.008+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30a801a070 con 0x7f30b015c0a0 2024-09-17T05:05:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.008+0000 7f30a6a00640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f308c073200 0x7f308c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.008+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f30a80936b0 con 0x7f30b015c0a0 2024-09-17T05:05:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.008+0000 7f30ade00640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f308c073200 0x7f308c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:32.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.008+0000 7f30ade00640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f308c073200 0x7f308c075710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f30a00af000 tx=0x7f30a001aef0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:32.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.012+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30a805e530 con 0x7f30b015c0a0 2024-09-17T05:05:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.232+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f30b015bae0 con 0x7f30b015c0a0 2024-09-17T05:05:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.232+0000 7f30a6a00640 1 -- 172.21.15.16:0/2263874823 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f30a805de30 con 0x7f30b015c0a0 2024-09-17T05:05:32.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:32.236 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:32.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f308c073200 msgr2=0x7f308c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f308c073200 0x7f308c075710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f30a00af000 tx=0x7f30a001aef0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 msgr2=0x7f30b01548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 0x7f30b01548a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f30a800c3c0 tx=0x7f30a800c880 comp rx=0 tx=0).stop 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 shutdown_connections 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f308c073200 0x7f308c075710 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30b015c0a0 0x7f30b01548a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30b00b2f50 0x7f30b013fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 --2- 172.21.15.16:0/2263874823 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30b0073240 0x7f30b013d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 >> 172.21.15.16:0/2263874823 conn(0x7f30b006f900 msgr2=0x7f30b006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 shutdown_connections 2024-09-17T05:05:32.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:32.236+0000 7f30b5200640 1 -- 172.21.15.16:0/2263874823 wait complete. 2024-09-17T05:05:32.259 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:32.259 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:37.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 -- 172.21.15.16:0/3012159680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730071210 msgr2=0x7f6730071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 --2- 172.21.15.16:0/3012159680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730071210 0x7f6730071620 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6718009840 tx=0x7f671801c7f0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 -- 172.21.15.16:0/3012159680 shutdown_connections 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 --2- 172.21.15.16:0/3012159680 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730071210 0x7f6730071620 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 --2- 172.21.15.16:0/3012159680 >> v2:172.21.15.104:3300/0 conn(0x7f6730152740 0x7f67300b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 -- 172.21.15.16:0/3012159680 >> 172.21.15.16:0/3012159680 conn(0x7f67300d10f0 msgr2=0x7f67300d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 -- 172.21.15.16:0/3012159680 shutdown_connections 2024-09-17T05:05:37.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.428+0000 7f6735e00640 1 -- 172.21.15.16:0/3012159680 wait complete. 2024-09-17T05:05:37.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 Processor -- start 2024-09-17T05:05:37.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 -- start start 2024-09-17T05:05:37.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 0x7f6730150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:37.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730152740 0x7f673013b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f673015f020 0x7f673015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f673006d8a0 con 0x7f6730071210 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f673006d720 con 0x7f673015f020 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f673006da20 con 0x7f6730152740 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 0x7f6730150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 0x7f6730150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35654/0 (socket says 172.21.15.16:35654) 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 -- 172.21.15.16:0/4254627885 learned_addr learned my addr 172.21.15.16:0/4254627885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6727e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730152740 0x7f673013b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735400640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f673015f020 0x7f673015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 -- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730152740 msgr2=0x7f673013b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730152740 0x7f673013b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 -- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f673015f020 msgr2=0x7f673015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:37.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f673015f020 0x7f673015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6734a00640 1 -- 172.21.15.16:0/4254627885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67301fca00 con 0x7f6730071210 2024-09-17T05:05:37.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.432+0000 7f6735400640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f673015f020 0x7f673015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f6734a00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 0x7f6730150a70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f672000e9a0 tx=0x7f672000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000cd80 con 0x7f6730071210 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6720004020 con 0x7f6730071210 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301fcce0 con 0x7f6730071210 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720004450 con 0x7f6730071210 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301fd2a0 con 0x7f6730071210 2024-09-17T05:05:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6730071620 con 0x7f6730071210 2024-09-17T05:05:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6720003600 con 0x7f6730071210 2024-09-17T05:05:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f672d600640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f670c073200 0x7f670c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.436+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f6720094d70 con 0x7f6730071210 2024-09-17T05:05:37.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.440+0000 7f6727e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f670c073200 0x7f670c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:37.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.440+0000 7f6727e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f670c073200 0x7f670c075710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f67180af000 tx=0x7f671801b030 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.444+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f672005fb70 con 0x7f6730071210 2024-09-17T05:05:37.602 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:37.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6730071dd0 con 0x7f6730071210 2024-09-17T05:05:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f672d600640 1 -- 172.21.15.16:0/4254627885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f672005f470 con 0x7f6730071210 2024-09-17T05:05:37.658 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:37.658 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f670c073200 msgr2=0x7f670c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f670c073200 0x7f670c075710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f67180af000 tx=0x7f671801b030 comp rx=0 tx=0).stop 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 msgr2=0x7f6730150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 0x7f6730150a70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f672000e9a0 tx=0x7f672000ee60 comp rx=0 tx=0).stop 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 shutdown_connections 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f670c073200 0x7f670c075710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f673015f020 0x7f673015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6730152740 0x7f673013b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 --2- 172.21.15.16:0/4254627885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6730071210 0x7f6730150a70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 >> 172.21.15.16:0/4254627885 conn(0x7f67300d10f0 msgr2=0x7f67300d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 shutdown_connections 2024-09-17T05:05:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:37.656+0000 7f6735e00640 1 -- 172.21.15.16:0/4254627885 wait complete. 2024-09-17T05:05:37.680 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:37.680 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:42.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 -- 172.21.15.16:0/3632293819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15f150 msgr2=0x7f8b9c15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 --2- 172.21.15.16:0/3632293819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15f150 0x7f8b9c15f560 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0ccfb0 tx=0x7f8b9401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:05:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 -- 172.21.15.16:0/3632293819 shutdown_connections 2024-09-17T05:05:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 --2- 172.21.15.16:0/3632293819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15f150 0x7f8b9c15f560 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 --2- 172.21.15.16:0/3632293819 >> v2:172.21.15.104:3300/0 conn(0x7f8b9c15c0a0 0x7f8b9c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 --2- 172.21.15.16:0/3632293819 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b9c0b2f50 0x7f8b9c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 -- 172.21.15.16:0/3632293819 >> 172.21.15.16:0/3632293819 conn(0x7f8b9c06f900 msgr2=0x7f8b9c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 -- 172.21.15.16:0/3632293819 shutdown_connections 2024-09-17T05:05:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.848+0000 7f8ba2000640 1 -- 172.21.15.16:0/3632293819 wait complete. 2024-09-17T05:05:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 Processor -- start 2024-09-17T05:05:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 -- start start 2024-09-17T05:05:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b9c0b2f50 0x7f8b9c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:42.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 0x7f8b9c0c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b9c15f150 0x7f8b9c0c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c073f30 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c073db0 con 0x7f8b9c15f150 2024-09-17T05:05:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9c0740b0 con 0x7f8b9c0b2f50 2024-09-17T05:05:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 0x7f8b9c0c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 0x7f8b9c0c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35670/0 (socket says 172.21.15.16:35670) 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 -- 172.21.15.16:0/216604147 learned_addr learned my addr 172.21.15.16:0/216604147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9be00640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b9c15f150 0x7f8b9c0c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9b400640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b9c0b2f50 0x7f8b9c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 -- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b9c0b2f50 msgr2=0x7f8b9c155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b9c0b2f50 0x7f8b9c155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 -- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b9c15f150 msgr2=0x7f8b9c0c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b9c15f150 0x7f8b9c0c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 -- 172.21.15.16:0/216604147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c1fc4c0 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b9aa00640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 0x7f8b9c0c5180 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e740 tx=0x7f8b8800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800ccc0 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b8800ce60 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c1fc7a0 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88005260 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.852+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c1fcd60 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.856+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9c1fc230 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.856+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b88005400 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.856+0000 7f8b93400640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b7c073200 0x7f8b7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:42.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.856+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f8b88094c50 con 0x7f8b9c15c0a0 2024-09-17T05:05:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.856+0000 7f8b9b400640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b7c073200 0x7f8b7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:42.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.856+0000 7f8b9b400640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b7c073200 0x7f8b7c075710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c004610 tx=0x7f8b8c00b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:42.860+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b8805f400 con 0x7f8b9c15c0a0 2024-09-17T05:05:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.072+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8b9c15c4b0 con 0x7f8b9c15c0a0 2024-09-17T05:05:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.072+0000 7f8b93400640 1 -- 172.21.15.16:0/216604147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f8b88063e30 con 0x7f8b9c15c0a0 2024-09-17T05:05:43.075 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:43.075 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b7c073200 msgr2=0x7f8b7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b7c073200 0x7f8b7c075710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c004610 tx=0x7f8b8c00b260 comp rx=0 tx=0).stop 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 msgr2=0x7f8b9c0c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 0x7f8b9c0c5180 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800e740 tx=0x7f8b8800ec00 comp rx=0 tx=0).stop 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 shutdown_connections 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b7c073200 0x7f8b7c075710 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b9c15f150 0x7f8b9c0c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b9c15c0a0 0x7f8b9c0c5180 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 --2- 172.21.15.16:0/216604147 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b9c0b2f50 0x7f8b9c155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 >> 172.21.15.16:0/216604147 conn(0x7f8b9c06f900 msgr2=0x7f8b9c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 shutdown_connections 2024-09-17T05:05:43.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:43.076+0000 7f8ba2000640 1 -- 172.21.15.16:0/216604147 wait complete. 2024-09-17T05:05:43.098 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:43.098 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:43.609 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:47.610 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:48.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:48.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 -- 172.21.15.16:0/2875721425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f388006f120 msgr2=0x7f3880168310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 --2- 172.21.15.16:0/2875721425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f388006f120 0x7f3880168310 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f38800b4b90 tx=0x7f387801ca30 comp rx=0 tx=0).stop 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 -- 172.21.15.16:0/2875721425 shutdown_connections 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 --2- 172.21.15.16:0/2875721425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f388006f120 0x7f3880168310 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 --2- 172.21.15.16:0/2875721425 >> v2:172.21.15.104:3300/0 conn(0x7f38800d0160 0x7f38800d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 -- 172.21.15.16:0/2875721425 >> 172.21.15.16:0/2875721425 conn(0x7f38800cf070 msgr2=0x7f38800cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 -- 172.21.15.16:0/2875721425 shutdown_connections 2024-09-17T05:05:48.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 -- 172.21.15.16:0/2875721425 wait complete. 2024-09-17T05:05:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 Processor -- start 2024-09-17T05:05:48.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.380+0000 7f3886a00640 1 -- start start 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f388006f120 0x7f388014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38800b3f70 0x7f388014e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 0x7f388015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f388016b960 con 0x7f38800d0160 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f388016b7e0 con 0x7f38800b3f70 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f388016bae0 con 0x7f388006f120 2024-09-17T05:05:48.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 0x7f388015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 0x7f388015bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37466/0 (socket says 172.21.15.16:37466) 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 -- 172.21.15.16:0/3662523184 learned_addr learned my addr 172.21.15.16:0/3662523184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f387fe00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f388006f120 0x7f388014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3877400640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38800b3f70 0x7f388014e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 -- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f388006f120 msgr2=0x7f388014de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f388006f120 0x7f388014de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 -- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38800b3f70 msgr2=0x7f388014e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38800b3f70 0x7f388014e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 -- 172.21.15.16:0/3662523184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388015a600 con 0x7f38800d0160 2024-09-17T05:05:48.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3884a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 0x7f388015bf40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3878098620 tx=0x7f3878005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38780b2040 con 0x7f38800d0160 2024-09-17T05:05:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38780bb070 con 0x7f38800d0160 2024-09-17T05:05:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388015a8e0 con 0x7f38800d0160 2024-09-17T05:05:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878004530 con 0x7f38800d0160 2024-09-17T05:05:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3880159830 con 0x7f38800d0160 2024-09-17T05:05:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.384+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388014ec00 con 0x7f38800d0160 2024-09-17T05:05:48.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.388+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f38780046d0 con 0x7f38800d0160 2024-09-17T05:05:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.388+0000 7f387cc00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f385c073200 0x7f385c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:48.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.388+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7f387812ed40 con 0x7f38800d0160 2024-09-17T05:05:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.388+0000 7f387fe00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f385c073200 0x7f385c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:48.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.388+0000 7f387fe00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f385c073200 0x7f385c075710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3870009b40 tx=0x7f38700091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.392+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3878016770 con 0x7f38800d0160 2024-09-17T05:05:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f38800d0570 con 0x7f38800d0160 2024-09-17T05:05:48.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f387cc00640 1 -- 172.21.15.16:0/3662523184 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f38780f9560 con 0x7f38800d0160 2024-09-17T05:05:48.611 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:48.611 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f385c073200 msgr2=0x7f385c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f385c073200 0x7f385c075710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3870009b40 tx=0x7f38700091e0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 msgr2=0x7f388015bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 0x7f388015bf40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3878098620 tx=0x7f3878005a60 comp rx=0 tx=0).stop 2024-09-17T05:05:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 shutdown_connections 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f385c073200 0x7f385c075710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38800d0160 0x7f388015bf40 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38800b3f70 0x7f388014e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 --2- 172.21.15.16:0/3662523184 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f388006f120 0x7f388014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 >> 172.21.15.16:0/3662523184 conn(0x7f38800cf070 msgr2=0x7f38801513e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 shutdown_connections 2024-09-17T05:05:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:48.608+0000 7f3886a00640 1 -- 172.21.15.16:0/3662523184 wait complete. 2024-09-17T05:05:48.633 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:48.633 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:49.612 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:53.617 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:53.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 -- 172.21.15.16:0/2706179329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 msgr2=0x7faa1815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 --2- 172.21.15.16:0/2706179329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815c680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa00009840 tx=0x7faa0001c790 comp rx=0 tx=0).stop 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 -- 172.21.15.16:0/2706179329 shutdown_connections 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 --2- 172.21.15.16:0/2706179329 >> v2:172.21.15.104:3300/0 conn(0x7faa18073240 0x7faa18073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 --2- 172.21.15.16:0/2706179329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815c680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 -- 172.21.15.16:0/2706179329 >> 172.21.15.16:0/2706179329 conn(0x7faa1806f900 msgr2=0x7faa180b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:53.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 -- 172.21.15.16:0/2706179329 shutdown_connections 2024-09-17T05:05:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 -- 172.21.15.16:0/2706179329 wait complete. 2024-09-17T05:05:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 Processor -- start 2024-09-17T05:05:53.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.804+0000 7faa1d800640 1 -- start start 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 0x7faa181f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa180b2f50 0x7faa181f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faa1806e1a0 con 0x7faa18073240 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa1806e020 con 0x7faa180b2f50 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7faa1806e320 con 0x7faa1815c270 2024-09-17T05:05:53.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 0x7faa181f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 0x7faa181f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37482/0 (socket says 172.21.15.16:37482) 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 -- 172.21.15.16:0/1539528187 learned_addr learned my addr 172.21.15.16:0/1539528187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa17400640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa0e000640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa180b2f50 0x7faa181f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 -- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 msgr2=0x7faa1815de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 -- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa180b2f50 msgr2=0x7faa181f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa180b2f50 0x7faa181f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 -- 172.21.15.16:0/1539528187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa181f8050 con 0x7faa18073240 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa17400640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa0e000640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa180b2f50 0x7faa181f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa16a00640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 0x7faa181f72c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faa0800c460 tx=0x7faa0800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08016020 con 0x7faa18073240 2024-09-17T05:05:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa08012940 con 0x7faa18073240 2024-09-17T05:05:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08012d90 con 0x7faa18073240 2024-09-17T05:05:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa181f8330 con 0x7faa18073240 2024-09-17T05:05:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa18150860 con 0x7faa18073240 2024-09-17T05:05:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.808+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa18150650 con 0x7faa18073240 2024-09-17T05:05:53.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.812+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faa08019020 con 0x7faa18073240 2024-09-17T05:05:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.812+0000 7faa0f400640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa9f4073200 0x7fa9f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.812+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7faa08094130 con 0x7faa18073240 2024-09-17T05:05:53.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.812+0000 7faa0e000640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa9f4073200 0x7fa9f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.812+0000 7faa0e000640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa9f4073200 0x7fa9f4075710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa000af000 tx=0x7faa0001afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:53.816+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faa08020030 con 0x7faa18073240 2024-09-17T05:05:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.032+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faa18073650 con 0x7faa18073240 2024-09-17T05:05:54.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.032+0000 7faa0f400640 1 -- 172.21.15.16:0/1539528187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7faa0805efc0 con 0x7faa18073240 2024-09-17T05:05:54.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:54.035 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:54.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa9f4073200 msgr2=0x7fa9f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa9f4073200 0x7fa9f4075710 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa000af000 tx=0x7faa0001afd0 comp rx=0 tx=0).stop 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 msgr2=0x7faa181f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 0x7faa181f72c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7faa0800c460 tx=0x7faa0800c920 comp rx=0 tx=0).stop 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 shutdown_connections 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa9f4073200 0x7fa9f4075710 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faa1815c270 0x7faa1815de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa180b2f50 0x7faa181f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 --2- 172.21.15.16:0/1539528187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa18073240 0x7faa181f72c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 >> 172.21.15.16:0/1539528187 conn(0x7faa1806f900 msgr2=0x7faa181654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 shutdown_connections 2024-09-17T05:05:54.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:54.036+0000 7faa1d800640 1 -- 172.21.15.16:0/1539528187 wait complete. 2024-09-17T05:05:54.057 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:54.057 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:57.621 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:05:59.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:05:59.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc54d600640 1 -- 172.21.15.16:0/376922246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c005360 msgr2=0x7fc52c00b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc54d600640 1 --2- 172.21.15.16:0/376922246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc52c005360 0x7fc52c00b9f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc540001f50 tx=0x7fc54001cc40 comp rx=0 tx=0).stop 2024-09-17T05:05:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc547400640 1 -- 172.21.15.16:0/376922246 reap_dead start 2024-09-17T05:05:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc54d600640 1 -- 172.21.15.16:0/376922246 shutdown_connections 2024-09-17T05:05:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc54d600640 1 -- 172.21.15.16:0/376922246 >> 172.21.15.16:0/376922246 conn(0x7fc54806f900 msgr2=0x7fc5480b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc54d600640 1 -- 172.21.15.16:0/376922246 shutdown_connections 2024-09-17T05:05:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.228+0000 7fc54d600640 1 -- 172.21.15.16:0/376922246 wait complete. 2024-09-17T05:05:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 Processor -- start 2024-09-17T05:05:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 -- start start 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc52c005360 0x7fc54814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5481500c0 0x7fc54815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 0x7fc54815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc548167a70 con 0x7fc5480afa90 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481678f0 con 0x7fc5481500c0 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc54d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc548167bf0 con 0x7fc52c005360 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 0x7fc54815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc546000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5481500c0 0x7fc54815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc546a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc52c005360 0x7fc54814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 0x7fc54815aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35906/0 (socket says 172.21.15.16:35906) 2024-09-17T05:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 -- 172.21.15.16:0/2857995103 learned_addr learned my addr 172.21.15.16:0/2857995103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:05:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc546a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc52c005360 0x7fc54814ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:56952/0 (socket says 172.21.15.16:56952) 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 -- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc52c005360 msgr2=0x7fc54814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc52c005360 0x7fc54814ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 -- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5481500c0 msgr2=0x7fc54815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5481500c0 0x7fc54815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 -- 172.21.15.16:0/2857995103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5481650a0 con 0x7fc5480afa90 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.232+0000 7fc547400640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 0x7fc54815aaa0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc540001f50 tx=0x7fc540003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540001ba0 con 0x7fc5480afa90 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5400b4030 con 0x7fc5480afa90 2024-09-17T05:05:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54814f750 con 0x7fc5480afa90 2024-09-17T05:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540006c50 con 0x7fc5480afa90 2024-09-17T05:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54814f370 con 0x7fc5480afa90 2024-09-17T05:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5480b35b0 con 0x7fc5480afa90 2024-09-17T05:05:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5400063b0 con 0x7fc5480afa90 2024-09-17T05:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc53ea00640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc528073200 0x7fc528075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fc54012e720 con 0x7fc5480afa90 2024-09-17T05:05:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.236+0000 7fc546a00640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc528073200 0x7fc528075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:05:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.240+0000 7fc546a00640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc528073200 0x7fc528075710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc548072860 tx=0x7fc5380091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:05:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.244+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5400169f0 con 0x7fc5480afa90 2024-09-17T05:05:59.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.456+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc54814f920 con 0x7fc5480afa90 2024-09-17T05:05:59.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.456+0000 7fc53ea00640 1 -- 172.21.15.16:0/2857995103 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc5400f8fb0 con 0x7fc5480afa90 2024-09-17T05:05:59.459 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:05:59.459 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:05:59.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc528073200 msgr2=0x7fc528075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc528073200 0x7fc528075710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc548072860 tx=0x7fc5380091e0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 msgr2=0x7fc54815aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 0x7fc54815aaa0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc540001f50 tx=0x7fc540003f40 comp rx=0 tx=0).stop 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 shutdown_connections 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc528073200 0x7fc528075710 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5480afa90 0x7fc54815aaa0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5481500c0 0x7fc54815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 --2- 172.21.15.16:0/2857995103 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc52c005360 0x7fc54814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 >> 172.21.15.16:0/2857995103 conn(0x7fc54806f900 msgr2=0x7fc548151a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 shutdown_connections 2024-09-17T05:05:59.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:05:59.460+0000 7fc54d600640 1 -- 172.21.15.16:0/2857995103 wait complete. 2024-09-17T05:05:59.481 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:05:59.481 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:05:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:00.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:06:00.729 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:06:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:03.627 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:04.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 -- 172.21.15.16:0/3134475559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d8071210 msgr2=0x7f64d8071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 --2- 172.21.15.16:0/3134475559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d8071210 0x7f64d8071620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009840 tx=0x7f64c401c820 comp rx=0 tx=0).stop 2024-09-17T05:06:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 -- 172.21.15.16:0/3134475559 shutdown_connections 2024-09-17T05:06:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 --2- 172.21.15.16:0/3134475559 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64d80cda00 0x7f64d80cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 --2- 172.21.15.16:0/3134475559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d8071210 0x7f64d8071620 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 -- 172.21.15.16:0/3134475559 >> 172.21.15.16:0/3134475559 conn(0x7f64d80d10f0 msgr2=0x7f64d80d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.648+0000 7f64dea00640 1 -- 172.21.15.16:0/3134475559 shutdown_connections 2024-09-17T05:06:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 -- 172.21.15.16:0/3134475559 wait complete. 2024-09-17T05:06:04.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 Processor -- start 2024-09-17T05:06:04.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 -- start start 2024-09-17T05:06:04.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64d8071210 0x7f64d8138a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:04.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d80cda00 0x7f64d813f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:04.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 0x7f64d806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8169dd0 con 0x7f64d815d530 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8169c50 con 0x7f64d80cda00 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.652+0000 7f64dea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f64d8169f50 con 0x7f64d8071210 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 0x7f64d806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 0x7f64d806e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35918/0 (socket says 172.21.15.16:35918) 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 -- 172.21.15.16:0/2393582398 learned_addr learned my addr 172.21.15.16:0/2393582398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64dd600640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64d8071210 0x7f64d8138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64dcc00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d80cda00 0x7f64d813f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 -- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64d8071210 msgr2=0x7f64d8138a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64d8071210 0x7f64d8138a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 -- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d80cda00 msgr2=0x7f64d813f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d80cda00 0x7f64d813f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 -- 172.21.15.16:0/2393582398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d81faee0 con 0x7f64d815d530 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64de000640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 0x7f64d806e340 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ec30 tx=0x7f64d000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0003b80 con 0x7f64d815d530 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d0003e10 con 0x7f64d815d530 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d81f6da0 con 0x7f64d815d530 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0004280 con 0x7f64d815d530 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d81f7360 con 0x7f64d815d530 2024-09-17T05:06:04.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d8071620 con 0x7f64d815d530 2024-09-17T05:06:04.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.656+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f64d0004420 con 0x7f64d815d530 2024-09-17T05:06:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.660+0000 7f64d5600640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f64b8073200 0x7f64b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.660+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f64d00953e0 con 0x7f64d815d530 2024-09-17T05:06:04.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.660+0000 7f64dd600640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f64b8073200 0x7f64b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:04.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.660+0000 7f64dd600640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f64b8073200 0x7f64b8075710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f64cc001400 tx=0x7f64cc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:04.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.664+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f64d0060120 con 0x7f64d815d530 2024-09-17T05:06:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.876+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f64d81f6b30 con 0x7f64d815d530 2024-09-17T05:06:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.876+0000 7f64d5600640 1 -- 172.21.15.16:0/2393582398 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f64d005fa20 con 0x7f64d815d530 2024-09-17T05:06:04.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:04.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f64b8073200 msgr2=0x7f64b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f64b8073200 0x7f64b8075710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f64cc001400 tx=0x7f64cc009040 comp rx=0 tx=0).stop 2024-09-17T05:06:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 msgr2=0x7f64d806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 0x7f64d806e340 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f64d000ec30 tx=0x7f64d000c6a0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 shutdown_connections 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f64b8073200 0x7f64b8075710 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64d815d530 0x7f64d806e340 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64d80cda00 0x7f64d813f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 --2- 172.21.15.16:0/2393582398 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f64d8071210 0x7f64d8138a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 >> 172.21.15.16:0/2393582398 conn(0x7f64d80d10f0 msgr2=0x7f64d80743f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 shutdown_connections 2024-09-17T05:06:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:04.880+0000 7f64dea00640 1 -- 172.21.15.16:0/2393582398 wait complete. 2024-09-17T05:06:04.902 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:04.902 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:05.629 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:09.633 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:09.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 -- 172.21.15.16:0/925898997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff41015e2a0 msgr2=0x7ff41015e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 --2- 172.21.15.16:0/925898997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff41015e2a0 0x7ff41015e6b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff410072860 tx=0x7ff3f801c770 comp rx=0 tx=0).stop 2024-09-17T05:06:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 -- 172.21.15.16:0/925898997 shutdown_connections 2024-09-17T05:06:10.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 --2- 172.21.15.16:0/925898997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff41015e2a0 0x7ff41015e6b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 --2- 172.21.15.16:0/925898997 >> v2:172.21.15.104:3300/0 conn(0x7ff4100748d0 0x7ff410074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 -- 172.21.15.16:0/925898997 >> 172.21.15.16:0/925898997 conn(0x7ff41006f900 msgr2=0x7ff4100b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 -- 172.21.15.16:0/925898997 shutdown_connections 2024-09-17T05:06:10.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 -- 172.21.15.16:0/925898997 wait complete. 2024-09-17T05:06:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 Processor -- start 2024-09-17T05:06:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.072+0000 7ff417a00640 1 -- start start 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4100748d0 0x7ff4101501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 0x7ff4101f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff41015e2a0 0x7ff410153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff41006da30 con 0x7ff4100cf140 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff41006d8b0 con 0x7ff4100748d0 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff41006dbb0 con 0x7ff41015e2a0 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 0x7ff4101f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 0x7ff4101f9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55312/0 (socket says 172.21.15.16:55312) 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 -- 172.21.15.16:0/1369944959 learned_addr learned my addr 172.21.15.16:0/1369944959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff415000640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4100748d0 0x7ff4101501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff415a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff41015e2a0 0x7ff410153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 -- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff41015e2a0 msgr2=0x7ff410153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff41015e2a0 0x7ff410153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 -- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4100748d0 msgr2=0x7ff4101501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4100748d0 0x7ff4101501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 -- 172.21.15.16:0/1369944959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4101544d0 con 0x7ff4100cf140 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff407e00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 0x7ff4101f9820 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff3f801cc50 tx=0x7ff3f80056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80b2040 con 0x7ff4100cf140 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3f80bb070 con 0x7ff4100cf140 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41014f6d0 con 0x7ff4100cf140 2024-09-17T05:06:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8004390 con 0x7ff4100cf140 2024-09-17T05:06:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff410152790 con 0x7ff4100cf140 2024-09-17T05:06:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.076+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff410154240 con 0x7ff4100cf140 2024-09-17T05:06:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.080+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff3f8003600 con 0x7ff4100cf140 2024-09-17T05:06:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.080+0000 7ff40d600640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff3ec073200 0x7ff3ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.080+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7ff3f812e690 con 0x7ff4100cf140 2024-09-17T05:06:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.080+0000 7ff415000640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff3ec073200 0x7ff3ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.080+0000 7ff415000640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff3ec073200 0x7ff3ec075710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff40000afd0 tx=0x7ff40000c040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.084+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff3f80163f0 con 0x7ff4100cf140 2024-09-17T05:06:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.296+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff410152a50 con 0x7ff4100cf140 2024-09-17T05:06:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.296+0000 7ff40d600640 1 -- 172.21.15.16:0/1369944959 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff3f80f94f0 con 0x7ff4100cf140 2024-09-17T05:06:10.299 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:10.299 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.296+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff3ec073200 msgr2=0x7ff3ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.296+0000 7ff417a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff3ec073200 0x7ff3ec075710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff40000afd0 tx=0x7ff40000c040 comp rx=0 tx=0).stop 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.296+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 msgr2=0x7ff4101f9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.296+0000 7ff417a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 0x7ff4101f9820 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff3f801cc50 tx=0x7ff3f80056e0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 shutdown_connections 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff3ec073200 0x7ff3ec075710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff41015e2a0 0x7ff410153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4100cf140 0x7ff4101f9820 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 --2- 172.21.15.16:0/1369944959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4100748d0 0x7ff4101501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 >> 172.21.15.16:0/1369944959 conn(0x7ff41006f900 msgr2=0x7ff4100ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 shutdown_connections 2024-09-17T05:06:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:10.300+0000 7ff417a00640 1 -- 172.21.15.16:0/1369944959 wait complete. 2024-09-17T05:06:10.321 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:10.321 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:13.637 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:15.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 -- 172.21.15.16:0/709817797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806de60 msgr2=0x7fe5080ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 --2- 172.21.15.16:0/709817797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806de60 0x7fe5080ce340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe500009c80 tx=0x7fe50001cda0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 -- 172.21.15.16:0/709817797 shutdown_connections 2024-09-17T05:06:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 --2- 172.21.15.16:0/709817797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806de60 0x7fe5080ce340 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 --2- 172.21.15.16:0/709817797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe5080b4980 0x7fe5080cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 -- 172.21.15.16:0/709817797 >> 172.21.15.16:0/709817797 conn(0x7fe50806f900 msgr2=0x7fe5080b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 -- 172.21.15.16:0/709817797 shutdown_connections 2024-09-17T05:06:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 -- 172.21.15.16:0/709817797 wait complete. 2024-09-17T05:06:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 Processor -- start 2024-09-17T05:06:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.492+0000 7fe50f800640 1 -- start start 2024-09-17T05:06:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 0x7fe50813e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe50806de60 0x7fe5081425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe508140750 0x7fe508140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe508167e40 con 0x7fe50806da50 2024-09-17T05:06:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe508167cc0 con 0x7fe508140750 2024-09-17T05:06:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe508167fc0 con 0x7fe50806de60 2024-09-17T05:06:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 0x7fe50813e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 0x7fe50813e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55334/0 (socket says 172.21.15.16:55334) 2024-09-17T05:06:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 -- 172.21.15.16:0/1239266675 learned_addr learned my addr 172.21.15.16:0/1239266675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe507e00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe50806de60 0x7fe5081425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50d800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe508140750 0x7fe508140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 -- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe50806de60 msgr2=0x7fe5081425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe50806de60 0x7fe5081425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 -- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe508140750 msgr2=0x7fe508140b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe508140750 0x7fe508140b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 -- 172.21.15.16:0/1239266675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe508152a60 con 0x7fe50806da50 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50ce00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 0x7fe50813e9a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b440 tx=0x7fe4fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc015020 con 0x7fe50806da50 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4fc011980 con 0x7fe50806da50 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50813f6b0 con 0x7fe50806da50 2024-09-17T05:06:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc011dd0 con 0x7fe50806da50 2024-09-17T05:06:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50813fc70 con 0x7fe50806da50 2024-09-17T05:06:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.496+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5080cde00 con 0x7fe50806da50 2024-09-17T05:06:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.500+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4fc018020 con 0x7fe50806da50 2024-09-17T05:06:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.500+0000 7fe504c00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4e8073200 0x7fe4e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.500+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fe4fc094500 con 0x7fe50806da50 2024-09-17T05:06:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.500+0000 7fe507e00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4e8073200 0x7fe4e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.500+0000 7fe507e00640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4e8073200 0x7fe4e8075710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe5080af890 tx=0x7fe4f400b3e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.504+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe4fc01f030 con 0x7fe50806da50 2024-09-17T05:06:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.716+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe5081598f0 con 0x7fe50806da50 2024-09-17T05:06:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.716+0000 7fe504c00640 1 -- 172.21.15.16:0/1239266675 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fe4fc00d040 con 0x7fe50806da50 2024-09-17T05:06:15.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:15.721 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4e8073200 msgr2=0x7fe4e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4e8073200 0x7fe4e8075710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe5080af890 tx=0x7fe4f400b3e0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 msgr2=0x7fe50813e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 0x7fe50813e9a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00b440 tx=0x7fe4fc00b900 comp rx=0 tx=0).stop 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 shutdown_connections 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe4e8073200 0x7fe4e8075710 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe508140750 0x7fe508140b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe50806de60 0x7fe5081425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 --2- 172.21.15.16:0/1239266675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe50806da50 0x7fe50813e9a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 >> 172.21.15.16:0/1239266675 conn(0x7fe50806f900 msgr2=0x7fe5081649d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 shutdown_connections 2024-09-17T05:06:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:15.720+0000 7fe50f800640 1 -- 172.21.15.16:0/1239266675 wait complete. 2024-09-17T05:06:15.742 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:15.742 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:17.641 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:20.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:20.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 -- 172.21.15.16:0/4013617490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec073240 msgr2=0x7f6dec073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 --2- 172.21.15.16:0/4013617490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec073240 0x7f6dec073650 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009bc0 tx=0x7f6de401cb10 comp rx=0 tx=0).stop 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 -- 172.21.15.16:0/4013617490 shutdown_connections 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 --2- 172.21.15.16:0/4013617490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec073240 0x7f6dec073650 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 --2- 172.21.15.16:0/4013617490 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6dec15c270 0x7f6dec15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 -- 172.21.15.16:0/4013617490 >> 172.21.15.16:0/4013617490 conn(0x7f6dec06f900 msgr2=0x7f6dec0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 -- 172.21.15.16:0/4013617490 shutdown_connections 2024-09-17T05:06:20.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.911+0000 7f6df1200640 1 -- 172.21.15.16:0/4013617490 wait complete. 2024-09-17T05:06:20.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 Processor -- start 2024-09-17T05:06:20.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 -- start start 2024-09-17T05:06:20.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec073240 0x7f6dec150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:20.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6dec0b2f50 0x7f6dec0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:20.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 0x7f6dec0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:20.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec168d60 con 0x7f6dec15c270 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec168be0 con 0x7f6dec073240 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6dec168ee0 con 0x7f6dec0b2f50 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 0x7f6dec0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 0x7f6dec0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39298/0 (socket says 172.21.15.16:39298) 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 -- 172.21.15.16:0/3600576881 learned_addr learned my addr 172.21.15.16:0/3600576881 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6de9e00640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec073240 0x7f6dec150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6de3e00640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6dec0b2f50 0x7f6dec0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:20.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 -- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6dec0b2f50 msgr2=0x7f6dec0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:20.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6dec0b2f50 0x7f6dec0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 -- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec073240 msgr2=0x7f6dec150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec073240 0x7f6dec150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 -- 172.21.15.16:0/3600576881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec1507d0 con 0x7f6dec15c270 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6dea800640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 0x7f6dec0ce380 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6de401c580 tx=0x7f6de4009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de40b2040 con 0x7f6dec15c270 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6de40bb070 con 0x7f6dec15c270 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec160d30 con 0x7f6dec15c270 2024-09-17T05:06:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4004590 con 0x7f6dec15c270 2024-09-17T05:06:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.915+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1612b0 con 0x7f6dec15c270 2024-09-17T05:06:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.919+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dec164400 con 0x7f6dec15c270 2024-09-17T05:06:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.919+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6de40c0460 con 0x7f6dec15c270 2024-09-17T05:06:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.919+0000 7f6de2000640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dc8073200 0x7f6dc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.919+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f6de412ecc0 con 0x7f6dec15c270 2024-09-17T05:06:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.919+0000 7f6de9e00640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dc8073200 0x7f6dc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:20.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.919+0000 7f6de9e00640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dc8073200 0x7f6dc8075710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007fd0 tx=0x7f6ddc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:20.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:20.923+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6de4135080 con 0x7f6dec15c270 2024-09-17T05:06:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.135+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6dec15de70 con 0x7f6dec15c270 2024-09-17T05:06:21.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.135+0000 7f6de2000640 1 -- 172.21.15.16:0/3600576881 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f6de4016790 con 0x7f6dec15c270 2024-09-17T05:06:21.139 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:21.139 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:21.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dc8073200 msgr2=0x7f6dc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dc8073200 0x7f6dc8075710 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007fd0 tx=0x7f6ddc009040 comp rx=0 tx=0).stop 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 msgr2=0x7f6dec0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 0x7f6dec0ce380 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6de401c580 tx=0x7f6de4009200 comp rx=0 tx=0).stop 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 shutdown_connections 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6dc8073200 0x7f6dc8075710 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dec15c270 0x7f6dec0ce380 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6dec0b2f50 0x7f6dec0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 --2- 172.21.15.16:0/3600576881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dec073240 0x7f6dec150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 >> 172.21.15.16:0/3600576881 conn(0x7f6dec06f900 msgr2=0x7f6dec15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 shutdown_connections 2024-09-17T05:06:21.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:21.139+0000 7f6df1200640 1 -- 172.21.15.16:0/3600576881 wait complete. 2024-09-17T05:06:21.161 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:21.161 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:23.647 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:26.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 -- 172.21.15.16:0/2484634117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb564073240 msgr2=0x7fb564073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 --2- 172.21.15.16:0/2484634117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb564073240 0x7fb564073650 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb564071ce0 tx=0x7fb55c01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 -- 172.21.15.16:0/2484634117 shutdown_connections 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 --2- 172.21.15.16:0/2484634117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb564073240 0x7fb564073650 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 --2- 172.21.15.16:0/2484634117 >> v2:172.21.15.104:3300/0 conn(0x7fb5640b2f50 0x7fb56415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 -- 172.21.15.16:0/2484634117 >> 172.21.15.16:0/2484634117 conn(0x7fb56406f900 msgr2=0x7fb5640b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 -- 172.21.15.16:0/2484634117 shutdown_connections 2024-09-17T05:06:26.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 -- 172.21.15.16:0/2484634117 wait complete. 2024-09-17T05:06:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 Processor -- start 2024-09-17T05:06:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.331+0000 7fb568c00640 1 -- start start 2024-09-17T05:06:26.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb564073240 0x7fb56415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5640b2f50 0x7fb5641509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 0x7fb5640c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb564168d10 con 0x7fb56415c0a0 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb564168b90 con 0x7fb564073240 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb564168e90 con 0x7fb5640b2f50 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 0x7fb5640c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 0x7fb5640c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39326/0 (socket says 172.21.15.16:39326) 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 -- 172.21.15.16:0/2530760211 learned_addr learned my addr 172.21.15.16:0/2530760211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb561e00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb564073240 0x7fb56415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb55be00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5640b2f50 0x7fb5641509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 -- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5640b2f50 msgr2=0x7fb5641509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:26.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5640b2f50 0x7fb5641509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 -- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb564073240 msgr2=0x7fb56415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb564073240 0x7fb56415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 -- 172.21.15.16:0/2530760211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5641f9c30 con 0x7fb56415c0a0 2024-09-17T05:06:26.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb562800640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 0x7fb5640c9ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009bd0 tx=0x7fb55c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0bb070 con 0x7fb56415c0a0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb55c004d40 con 0x7fb56415c0a0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0b2040 con 0x7fb56415c0a0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5640c9400 con 0x7fb56415c0a0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5640c96a0 con 0x7fb56415c0a0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.335+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb564073650 con 0x7fb56415c0a0 2024-09-17T05:06:26.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.339+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb55c0ad850 con 0x7fb56415c0a0 2024-09-17T05:06:26.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.339+0000 7fb55a000640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb540073200 0x7fb540075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:26.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.339+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fb55c12e330 con 0x7fb56415c0a0 2024-09-17T05:06:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.339+0000 7fb561e00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb540073200 0x7fb540075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:26.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.339+0000 7fb561e00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb540073200 0x7fb540075710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5540058d0 tx=0x7fb554005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.343+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb55c0f1db0 con 0x7fb56415c0a0 2024-09-17T05:06:26.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.555+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb5640b12b0 con 0x7fb56415c0a0 2024-09-17T05:06:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.555+0000 7fb55a000640 1 -- 172.21.15.16:0/2530760211 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fb55c016970 con 0x7fb56415c0a0 2024-09-17T05:06:26.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:26.559 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb540073200 msgr2=0x7fb540075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb540073200 0x7fb540075710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb5540058d0 tx=0x7fb554005860 comp rx=0 tx=0).stop 2024-09-17T05:06:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 msgr2=0x7fb5640c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 0x7fb5640c9ea0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009bd0 tx=0x7fb55c005b00 comp rx=0 tx=0).stop 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 shutdown_connections 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb540073200 0x7fb540075710 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb56415c0a0 0x7fb5640c9ea0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb5640b2f50 0x7fb5641509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 --2- 172.21.15.16:0/2530760211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb564073240 0x7fb56415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 >> 172.21.15.16:0/2530760211 conn(0x7fb56406f900 msgr2=0x7fb5641515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 shutdown_connections 2024-09-17T05:06:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:26.559+0000 7fb568c00640 1 -- 172.21.15.16:0/2530760211 wait complete. 2024-09-17T05:06:26.581 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:26.581 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:27.651 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:29.653 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:30.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:06:30.792 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:06:31.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 -- 172.21.15.16:0/65800087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 msgr2=0x7f401c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 --2- 172.21.15.16:0/65800087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c0cf5f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f401c069cd0 tx=0x7f401401cd40 comp rx=0 tx=0).stop 2024-09-17T05:06:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 -- 172.21.15.16:0/65800087 shutdown_connections 2024-09-17T05:06:31.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 --2- 172.21.15.16:0/65800087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c0cf5f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 --2- 172.21.15.16:0/65800087 >> v2:172.21.15.104:3300/0 conn(0x7f401c06f6c0 0x7f401c071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 -- 172.21.15.16:0/65800087 >> 172.21.15.16:0/65800087 conn(0x7f401c073ca0 msgr2=0x7f401c0740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 -- 172.21.15.16:0/65800087 shutdown_connections 2024-09-17T05:06:31.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.751+0000 7f401b200640 1 -- 172.21.15.16:0/65800087 wait complete. 2024-09-17T05:06:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 Processor -- start 2024-09-17T05:06:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 -- start start 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f401c071380 0x7f401c1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c15db20 0x7f401c164380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f401c168920 con 0x7f401c0cf1e0 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f401c1687a0 con 0x7f401c15db20 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f401c168aa0 con 0x7f401c071380 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c14d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43498/0 (socket says 172.21.15.16:43498) 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 -- 172.21.15.16:0/916729792 learned_addr learned my addr 172.21.15.16:0/916729792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4019e00640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f401c071380 0x7f401c1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401a800640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c15db20 0x7f401c164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 -- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f401c071380 msgr2=0x7f401c1f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f401c071380 0x7f401c1f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 -- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c15db20 msgr2=0x7f401c164380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c15db20 0x7f401c164380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 -- 172.21.15.16:0/916729792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c15cd60 con 0x7f401c0cf1e0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4013e00640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c14d9b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f400400d950 tx=0x7f400400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4004017070 con 0x7f401c0cf1e0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4004009cc0 con 0x7f401c0cf1e0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40040106c0 con 0x7f401c0cf1e0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c15d040 con 0x7f401c0cf1e0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.755+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c164ec0 con 0x7f401c0cf1e0 2024-09-17T05:06:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.759+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401c15e6b0 con 0x7f401c0cf1e0 2024-09-17T05:06:31.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.759+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f400401a070 con 0x7f401c0cf1e0 2024-09-17T05:06:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.759+0000 7f4012000640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ff8073230 0x7f3ff8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.759+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f4004094450 con 0x7f401c0cf1e0 2024-09-17T05:06:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.763+0000 7f4019e00640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ff8073230 0x7f3ff8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.763+0000 7f4019e00640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ff8073230 0x7f3ff8075740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f400c0098e0 tx=0x7f400c009060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.767+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f400405ec60 con 0x7f401c0cf1e0 2024-09-17T05:06:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.979+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f401c071790 con 0x7f401c0cf1e0 2024-09-17T05:06:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.979+0000 7f4012000640 1 -- 172.21.15.16:0/916729792 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f4004063690 con 0x7f401c0cf1e0 2024-09-17T05:06:31.983 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:31.983 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ff8073230 msgr2=0x7f3ff8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ff8073230 0x7f3ff8075740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f400c0098e0 tx=0x7f400c009060 comp rx=0 tx=0).stop 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 msgr2=0x7f401c14d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c14d9b0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f400400d950 tx=0x7f400400de10 comp rx=0 tx=0).stop 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 shutdown_connections 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3ff8073230 0x7f3ff8075740 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f401c15db20 0x7f401c164380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0cf1e0 0x7f401c14d9b0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 --2- 172.21.15.16:0/916729792 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f401c071380 0x7f401c1f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 >> 172.21.15.16:0/916729792 conn(0x7f401c073ca0 msgr2=0x7f401c1579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 shutdown_connections 2024-09-17T05:06:31.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:31.983+0000 7f401b200640 1 -- 172.21.15.16:0/916729792 wait complete. 2024-09-17T05:06:32.005 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:32.005 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:37.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 -- 172.21.15.16:0/4206780620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc073240 msgr2=0x7f4acc073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 --2- 172.21.15.16:0/4206780620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc073240 0x7f4acc073650 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009bc0 tx=0x7f4ac401cb10 comp rx=0 tx=0).stop 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 -- 172.21.15.16:0/4206780620 shutdown_connections 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 --2- 172.21.15.16:0/4206780620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc073240 0x7f4acc073650 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 --2- 172.21.15.16:0/4206780620 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 0x7f4acc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 -- 172.21.15.16:0/4206780620 >> 172.21.15.16:0/4206780620 conn(0x7f4acc06f900 msgr2=0x7f4acc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 -- 172.21.15.16:0/4206780620 shutdown_connections 2024-09-17T05:06:37.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.175+0000 7f4ad3200640 1 -- 172.21.15.16:0/4206780620 wait complete. 2024-09-17T05:06:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 Processor -- start 2024-09-17T05:06:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 -- start start 2024-09-17T05:06:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 0x7f4acc14d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 0x7f4acc14dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc15c0a0 0x7f4acc15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc168e00 con 0x7f4acc073240 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc168c80 con 0x7f4acc15c0a0 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4acc168f80 con 0x7f4acc0b2f50 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 0x7f4acc14d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 0x7f4acc14d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55366/0 (socket says 172.21.15.16:55366) 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 -- 172.21.15.16:0/1542216922 learned_addr learned my addr 172.21.15.16:0/1542216922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad1200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc15c0a0 0x7f4acc15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ac3400640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 0x7f4acc14dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 -- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 msgr2=0x7f4acc14dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 0x7f4acc14dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 -- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc15c0a0 msgr2=0x7f4acc15a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc15c0a0 0x7f4acc15a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 -- 172.21.15.16:0/1542216922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc1f8b30 con 0x7f4acc073240 2024-09-17T05:06:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ac3400640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 0x7f4acc14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:06:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4acbe00640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 0x7f4acc14d640 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400e970 tx=0x7f4ab400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:37.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab400cd80 con 0x7f4acc073240 2024-09-17T05:06:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ab4004020 con 0x7f4acc073240 2024-09-17T05:06:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4004470 con 0x7f4acc073240 2024-09-17T05:06:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc165ba0 con 0x7f4acc073240 2024-09-17T05:06:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.179+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc14e880 con 0x7f4acc073240 2024-09-17T05:06:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.183+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4acc1660c0 con 0x7f4acc073240 2024-09-17T05:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.183+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4ab4003600 con 0x7f4acc073240 2024-09-17T05:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.183+0000 7f4ac8c00640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aa8073200 0x7f4aa8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.183+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f4ab4094e70 con 0x7f4acc073240 2024-09-17T05:06:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.183+0000 7f4ac3400640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aa8073200 0x7f4aa8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:37.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.183+0000 7f4ac3400640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aa8073200 0x7f4aa8075710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0b20a0 tx=0x7f4abc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.187+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4acc1660c0 con 0x7f4acc073240 2024-09-17T05:06:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.399+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4acc073650 con 0x7f4acc073240 2024-09-17T05:06:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.399+0000 7f4ac8c00640 1 -- 172.21.15.16:0/1542216922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f4ab405fbb0 con 0x7f4acc073240 2024-09-17T05:06:37.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:37.405 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aa8073200 msgr2=0x7f4aa8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aa8073200 0x7f4aa8075710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4acc0b20a0 tx=0x7f4abc009040 comp rx=0 tx=0).stop 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 msgr2=0x7f4acc14d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 0x7f4acc14d640 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400e970 tx=0x7f4ab400ee30 comp rx=0 tx=0).stop 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 shutdown_connections 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4aa8073200 0x7f4aa8075710 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4acc15c0a0 0x7f4acc15a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4acc0b2f50 0x7f4acc14dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 --2- 172.21.15.16:0/1542216922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc073240 0x7f4acc14d640 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 >> 172.21.15.16:0/1542216922 conn(0x7f4acc06f900 msgr2=0x7f4acc15d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 shutdown_connections 2024-09-17T05:06:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:37.403+0000 7f4ad3200640 1 -- 172.21.15.16:0/1542216922 wait complete. 2024-09-17T05:06:37.427 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:37.427 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:39.663 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:41.666 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:42.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:42.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 -- 172.21.15.16:0/3413236806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105815eec0 msgr2=0x7f105806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:42.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 --2- 172.21.15.16:0/3413236806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105815eec0 0x7f105806dea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1044009900 tx=0x7f104401c7c0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 -- 172.21.15.16:0/3413236806 shutdown_connections 2024-09-17T05:06:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 --2- 172.21.15.16:0/3413236806 >> v2:172.21.15.104:3300/0 conn(0x7f105806e3e0 0x7f10580b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 --2- 172.21.15.16:0/3413236806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105815eec0 0x7f105806dea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 -- 172.21.15.16:0/3413236806 >> 172.21.15.16:0/3413236806 conn(0x7f1058073510 msgr2=0x7f1058073920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 -- 172.21.15.16:0/3413236806 shutdown_connections 2024-09-17T05:06:42.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 -- 172.21.15.16:0/3413236806 wait complete. 2024-09-17T05:06:42.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 Processor -- start 2024-09-17T05:06:42.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.599+0000 7f105ea00640 1 -- start start 2024-09-17T05:06:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105806e3e0 0x7f10580b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1058074540 0x7f10581539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:42.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 0x7f10581fc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1058168eb0 con 0x7f105815eec0 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1058168d30 con 0x7f1058074540 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1058169030 con 0x7f105806e3e0 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 0x7f10581fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 0x7f10581fc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55374/0 (socket says 172.21.15.16:55374) 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 -- 172.21.15.16:0/2655017499 learned_addr learned my addr 172.21.15.16:0/2655017499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105cc00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1058074540 0x7f10581539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105d600640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105806e3e0 0x7f10580b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 -- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105806e3e0 msgr2=0x7f10580b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105806e3e0 0x7f10580b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 -- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1058074540 msgr2=0x7f10581539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1058074540 0x7f10581539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 -- 172.21.15.16:0/2655017499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058158860 con 0x7f105815eec0 2024-09-17T05:06:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105e000640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 0x7f10581fc550 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f105000ec30 tx=0x7f105000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:42.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050003b80 con 0x7f105815eec0 2024-09-17T05:06:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1050003e10 con 0x7f105815eec0 2024-09-17T05:06:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10580b1f10 con 0x7f105815eec0 2024-09-17T05:06:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050004280 con 0x7f105815eec0 2024-09-17T05:06:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10581fc030 con 0x7f105815eec0 2024-09-17T05:06:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.603+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10580b3600 con 0x7f105815eec0 2024-09-17T05:06:42.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.607+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1050004420 con 0x7f105815eec0 2024-09-17T05:06:42.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.607+0000 7f1055600640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1038073200 0x7f1038075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:42.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.607+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f1050095250 con 0x7f105815eec0 2024-09-17T05:06:42.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.607+0000 7f105d600640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1038073200 0x7f1038075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:42.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.607+0000 7f105d600640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1038073200 0x7f1038075710 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f104c007fd0 tx=0x7f104c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:42.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.611+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f105005f940 con 0x7f105815eec0 2024-09-17T05:06:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.823+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1058076550 con 0x7f105815eec0 2024-09-17T05:06:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.823+0000 7f1055600640 1 -- 172.21.15.16:0/2655017499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f1050064370 con 0x7f105815eec0 2024-09-17T05:06:42.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:42.829 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1038073200 msgr2=0x7f1038075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1038073200 0x7f1038075710 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f104c007fd0 tx=0x7f104c009040 comp rx=0 tx=0).stop 2024-09-17T05:06:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 msgr2=0x7f10581fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 0x7f10581fc550 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f105000ec30 tx=0x7f105000c6a0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 shutdown_connections 2024-09-17T05:06:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1038073200 0x7f1038075710 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f105815eec0 0x7f10581fc550 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1058074540 0x7f10581539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 --2- 172.21.15.16:0/2655017499 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f105806e3e0 0x7f10580b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 >> 172.21.15.16:0/2655017499 conn(0x7f1058073510 msgr2=0x7f1058155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 shutdown_connections 2024-09-17T05:06:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:42.827+0000 7f105ea00640 1 -- 172.21.15.16:0/2655017499 wait complete. 2024-09-17T05:06:42.850 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:42.850 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:45.669 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:47.671 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:47.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:48.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f6154a00640 1 -- 172.21.15.16:0/3482823845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6134003a10 msgr2=0x7f6134003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:48.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f6154a00640 1 --2- 172.21.15.16:0/3482823845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6134003a10 0x7f6134003e80 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f614000e4a0 tx=0x7f614001eee0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f614f200640 1 -- 172.21.15.16:0/3482823845 reap_dead start 2024-09-17T05:06:48.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f6154a00640 1 -- 172.21.15.16:0/3482823845 shutdown_connections 2024-09-17T05:06:48.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f6154a00640 1 -- 172.21.15.16:0/3482823845 >> 172.21.15.16:0/3482823845 conn(0x7f61500d10f0 msgr2=0x7f61500d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:48.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f6154a00640 1 -- 172.21.15.16:0/3482823845 shutdown_connections 2024-09-17T05:06:48.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.019+0000 7f6154a00640 1 -- 172.21.15.16:0/3482823845 wait complete. 2024-09-17T05:06:48.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 Processor -- start 2024-09-17T05:06:48.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 -- start start 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6134003a10 0x7f61501f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f615015a640 0x7f615015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 0x7f6150154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f615006d010 con 0x7f6150154ad0 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f615006ce90 con 0x7f615015a640 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6154a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f615006d190 con 0x7f6134003a10 2024-09-17T05:06:48.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 0x7f6150154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:48.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 0x7f6150154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45154/0 (socket says 172.21.15.16:45154) 2024-09-17T05:06:48.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 -- 172.21.15.16:0/607638014 learned_addr learned my addr 172.21.15.16:0/607638014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614e800640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6134003a10 0x7f61501f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f6145e00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f615015a640 0x7f615015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 -- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6134003a10 msgr2=0x7f61501f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6134003a10 0x7f61501f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 -- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f615015a640 msgr2=0x7f615015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f615015a640 0x7f615015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.023+0000 7f614f200640 1 -- 172.21.15.16:0/607638014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6150153a30 con 0x7f6150154ad0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f614f200640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 0x7f6150154ee0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f615015cb30 tx=0x7f6148011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148004b30 con 0x7f6150154ad0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6148004cd0 con 0x7f6150154ad0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6150153d10 con 0x7f6150154ad0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148003be0 con 0x7f6150154ad0 2024-09-17T05:06:48.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501542d0 con 0x7f6150154ad0 2024-09-17T05:06:48.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615006e440 con 0x7f6150154ad0 2024-09-17T05:06:48.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6148004030 con 0x7f6150154ad0 2024-09-17T05:06:48.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6147400640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f612c073200 0x7f612c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:48.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.027+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f6148094d20 con 0x7f6150154ad0 2024-09-17T05:06:48.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.031+0000 7f614e800640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f612c073200 0x7f612c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:48.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.031+0000 7f614e800640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f612c073200 0x7f612c075710 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f61400ad000 tx=0x7f614001bd20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:48.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.035+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f614805fb80 con 0x7f6150154ad0 2024-09-17T05:06:48.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.243+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f61500b5a80 con 0x7f6150154ad0 2024-09-17T05:06:48.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.243+0000 7f6147400640 1 -- 172.21.15.16:0/607638014 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f614805f480 con 0x7f6150154ad0 2024-09-17T05:06:48.250 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:48.250 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f612c073200 msgr2=0x7f612c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f612c073200 0x7f612c075710 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f61400ad000 tx=0x7f614001bd20 comp rx=0 tx=0).stop 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 msgr2=0x7f6150154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 0x7f6150154ee0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f615015cb30 tx=0x7f6148011f40 comp rx=0 tx=0).stop 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 shutdown_connections 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f612c073200 0x7f612c075710 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6150154ad0 0x7f6150154ee0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f615015a640 0x7f615015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 --2- 172.21.15.16:0/607638014 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6134003a10 0x7f61501f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:48.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 >> 172.21.15.16:0/607638014 conn(0x7f61500d10f0 msgr2=0x7f61500d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:48.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 shutdown_connections 2024-09-17T05:06:48.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:48.247+0000 7f6154a00640 1 -- 172.21.15.16:0/607638014 wait complete. 2024-09-17T05:06:48.271 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:48.271 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:51.674 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:53.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.443+0000 7f39b9400640 1 -- 172.21.15.16:0/254358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 msgr2=0x7f39b40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.443+0000 7f39b9400640 1 --2- 172.21.15.16:0/254358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b40cf550 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009bc0 tx=0x7f39ac01cba0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- 172.21.15.16:0/254358865 shutdown_connections 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 --2- 172.21.15.16:0/254358865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b40cf550 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 --2- 172.21.15.16:0/254358865 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39b415e9b0 0x7f39b4074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- 172.21.15.16:0/254358865 >> 172.21.15.16:0/254358865 conn(0x7f39b406f900 msgr2=0x7f39b40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- 172.21.15.16:0/254358865 shutdown_connections 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- 172.21.15.16:0/254358865 wait complete. 2024-09-17T05:06:53.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 Processor -- start 2024-09-17T05:06:53.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- start start 2024-09-17T05:06:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b40748d0 0x7f39b41fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b41f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39b4153020 0x7f39b41eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39b406d520 con 0x7f39b40cf140 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39b406d3a0 con 0x7f39b40748d0 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.447+0000 7f39b9400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f39b406d6a0 con 0x7f39b4153020 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b41f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b41f9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45164/0 (socket says 172.21.15.16:45164) 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 -- 172.21.15.16:0/329657466 learned_addr learned my addr 172.21.15.16:0/329657466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39b3200640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39b4153020 0x7f39b41eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39b2800640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b40748d0 0x7f39b41fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 -- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39b4153020 msgr2=0x7f39b41eeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39b4153020 0x7f39b41eeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 -- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b40748d0 msgr2=0x7f39b41fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b40748d0 0x7f39b41fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 -- 172.21.15.16:0/329657466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b41fca90 con 0x7f39b40cf140 2024-09-17T05:06:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39b2800640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b40748d0 0x7f39b41fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:06:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39a9e00640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b41f9560 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e7c0 tx=0x7f399c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c00ccc0 con 0x7f39b40cf140 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f399c00ce80 con 0x7f39b40cf140 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c005280 con 0x7f39b40cf140 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b41fcd70 con 0x7f39b40cf140 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b41ef470 con 0x7f39b40cf140 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b41f9aa0 con 0x7f39b40cf140 2024-09-17T05:06:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.451+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f399c005420 con 0x7f39b40cf140 2024-09-17T05:06:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.455+0000 7f39ab400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3990073200 0x7f3990075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:53.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.455+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f399c094eb0 con 0x7f39b40cf140 2024-09-17T05:06:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.455+0000 7f39b2800640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3990073200 0x7f3990075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:53.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.455+0000 7f39b2800640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3990073200 0x7f3990075710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f39a4001230 tx=0x7f39a4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.459+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39b41f9aa0 con 0x7f39b40cf140 2024-09-17T05:06:53.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.671+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f39b41fcf40 con 0x7f39b40cf140 2024-09-17T05:06:53.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.671+0000 7f39ab400640 1 -- 172.21.15.16:0/329657466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f399c05fc80 con 0x7f39b40cf140 2024-09-17T05:06:53.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:53.671 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:53.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3990073200 msgr2=0x7f3990075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:53.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3990073200 0x7f3990075710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f39a4001230 tx=0x7f39a4009040 comp rx=0 tx=0).stop 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 msgr2=0x7f39b41f9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b41f9560 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e7c0 tx=0x7f399c00ec80 comp rx=0 tx=0).stop 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 shutdown_connections 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3990073200 0x7f3990075710 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f39b4153020 0x7f39b41eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b40cf140 0x7f39b41f9560 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 --2- 172.21.15.16:0/329657466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39b40748d0 0x7f39b41fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 >> 172.21.15.16:0/329657466 conn(0x7f39b406f900 msgr2=0x7f39b41506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 shutdown_connections 2024-09-17T05:06:53.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:53.675+0000 7f39b9400640 1 -- 172.21.15.16:0/329657466 wait complete. 2024-09-17T05:06:53.676 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:53.695 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:53.695 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:06:58.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:06:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 -- 172.21.15.16:0/57549614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815f080 msgr2=0x7f6f7815f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/57549614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815f080 0x7f6f7815f490 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6f78072860 tx=0x7f6f6001c6e0 comp rx=0 tx=0).stop 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 -- 172.21.15.16:0/57549614 shutdown_connections 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/57549614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815f080 0x7f6f7815f490 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/57549614 >> v2:172.21.15.104:3300/0 conn(0x7f6f78074b00 0x7f6f780b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 -- 172.21.15.16:0/57549614 >> 172.21.15.16:0/57549614 conn(0x7f6f7806f900 msgr2=0x7f6f780b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 -- 172.21.15.16:0/57549614 shutdown_connections 2024-09-17T05:06:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 -- 172.21.15.16:0/57549614 wait complete. 2024-09-17T05:06:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 Processor -- start 2024-09-17T05:06:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.867+0000 7f6f7fa00640 1 -- start start 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f78074b00 0x7f6f78153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 0x7f6f781557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7815f080 0x7f6f781fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78169080 con 0x7f6f7815ba90 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78168f00 con 0x7f6f7815f080 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6f78169200 con 0x7f6f78074b00 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 0x7f6f781557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 0x7f6f781557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55122/0 (socket says 172.21.15.16:55122) 2024-09-17T05:06:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 -- 172.21.15.16:0/1874891691 learned_addr learned my addr 172.21.15.16:0/1874891691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7d000640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f78074b00 0x7f6f78153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7da00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7815f080 0x7f6f781fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 -- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f78074b00 msgr2=0x7f6f78153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f78074b00 0x7f6f78153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 -- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7815f080 msgr2=0x7f6f781fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7815f080 0x7f6f781fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 -- 172.21.15.16:0/1874891691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f78160520 con 0x7f6f7815ba90 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f6fe00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 0x7f6f781557e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6f60000c10 tx=0x7f6f600a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600bb070 con 0x7f6f7815ba90 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f60004470 con 0x7f6f7815ba90 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600b2040 con 0x7f6f7815ba90 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f78160800 con 0x7f6f7815ba90 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f78160dc0 con 0x7f6f7815ba90 2024-09-17T05:06:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.871+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f78160290 con 0x7f6f7815ba90 2024-09-17T05:06:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.875+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6f600be070 con 0x7f6f7815ba90 2024-09-17T05:06:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.875+0000 7f6f75600640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6f54073200 0x7f6f54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:06:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.875+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f6f6012e170 con 0x7f6f7815ba90 2024-09-17T05:06:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.875+0000 7f6f7d000640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6f54073200 0x7f6f54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:06:58.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.875+0000 7f6f7d000640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6f54073200 0x7f6f54075710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800bb50 tx=0x7f6f6800b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:06:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:58.879+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f600163f0 con 0x7f6f7815ba90 2024-09-17T05:06:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.087+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6f780b2f50 con 0x7f6f7815ba90 2024-09-17T05:06:59.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.087+0000 7f6f75600640 1 -- 172.21.15.16:0/1874891691 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f6f600f88f0 con 0x7f6f7815ba90 2024-09-17T05:06:59.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:06:59.088 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:06:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6f54073200 msgr2=0x7f6f54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:59.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6f54073200 0x7f6f54075710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800bb50 tx=0x7f6f6800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 msgr2=0x7f6f781557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 0x7f6f781557e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6f60000c10 tx=0x7f6f600a5620 comp rx=0 tx=0).stop 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 shutdown_connections 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6f54073200 0x7f6f54075710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7815f080 0x7f6f781fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f7815ba90 0x7f6f781557e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 --2- 172.21.15.16:0/1874891691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6f78074b00 0x7f6f78153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 >> 172.21.15.16:0/1874891691 conn(0x7f6f7806f900 msgr2=0x7f6f780cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 shutdown_connections 2024-09-17T05:06:59.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:06:59.091+0000 7f6f7fa00640 1 -- 172.21.15.16:0/1874891691 wait complete. 2024-09-17T05:06:59.110 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:06:59.110 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:06:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:01.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:07:01.156 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:07:01.685 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:03.687 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:04.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:07:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd9200640 1 -- 172.21.15.16:0/1593435221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb8003c20 msgr2=0x7f8cb8004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd9200640 1 --2- 172.21.15.16:0/1593435221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb8003c20 0x7f8cb8004070 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006fc0 tx=0x7f8cc401cf20 comp rx=0 tx=0).stop 2024-09-17T05:07:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd2800640 1 -- 172.21.15.16:0/1593435221 reap_dead start 2024-09-17T05:07:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd9200640 1 -- 172.21.15.16:0/1593435221 shutdown_connections 2024-09-17T05:07:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd9200640 1 -- 172.21.15.16:0/1593435221 >> 172.21.15.16:0/1593435221 conn(0x7f8cd4073e00 msgr2=0x7f8cd4074210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd9200640 1 -- 172.21.15.16:0/1593435221 shutdown_connections 2024-09-17T05:07:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.283+0000 7f8cd9200640 1 -- 172.21.15.16:0/1593435221 wait complete. 2024-09-17T05:07:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 Processor -- start 2024-09-17T05:07:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 -- start start 2024-09-17T05:07:04.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003c20 0x7f8cd41fc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:04.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 0x7f8cd41fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cd41f8150 0x7f8cd41fc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd415b420 con 0x7f8cd41f7d40 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd415b5a0 con 0x7f8cb8003c20 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd9200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd415b720 con 0x7f8cd41f8150 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 0x7f8cd41fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 0x7f8cd41fbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55128/0 (socket says 172.21.15.16:55128) 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8ccbe00640 1 -- 172.21.15.16:0/630937163 learned_addr learned my addr 172.21.15.16:0/630937163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd1e00640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003c20 0x7f8cd41fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.287+0000 7f8cd2800640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cd41f8150 0x7f8cd41fc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8ccbe00640 1 -- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cd41f8150 msgr2=0x7f8cd41fc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8ccbe00640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cd41f8150 0x7f8cd41fc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8ccbe00640 1 -- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003c20 msgr2=0x7f8cd41fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8ccbe00640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003c20 0x7f8cd41fc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8ccbe00640 1 -- 172.21.15.16:0/630937163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd4165980 con 0x7f8cd41f7d40 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8ccbe00640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 0x7f8cd41fbbf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc011b10 tx=0x7f8cbc009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc004460 con 0x7f8cd41f7d40 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cbc00f9b0 con 0x7f8cd41f7d40 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00fdc0 con 0x7f8cd41f7d40 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd4165c60 con 0x7f8cd41f7d40 2024-09-17T05:07:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd415ce50 con 0x7f8cd41f7d40 2024-09-17T05:07:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd41656f0 con 0x7f8cd41f7d40 2024-09-17T05:07:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.291+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8cbc005020 con 0x7f8cd41f7d40 2024-09-17T05:07:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.295+0000 7f8cca000640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8cb0073200 0x7f8cb0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.295+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f8cbc0950f0 con 0x7f8cd41f7d40 2024-09-17T05:07:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.295+0000 7f8cd1e00640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8cb0073200 0x7f8cb0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.295+0000 7f8cd1e00640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8cb0073200 0x7f8cb0075710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40ac000 tx=0x7f8cc4019d60 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.299+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8cbc05f7e0 con 0x7f8cd41f7d40 2024-09-17T05:07:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8cd4074920 con 0x7f8cd41f7d40 2024-09-17T05:07:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cca000640 1 -- 172.21.15.16:0/630937163 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f8cbc064210 con 0x7f8cd41f7d40 2024-09-17T05:07:04.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:04.507 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:07:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8cb0073200 msgr2=0x7f8cb0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8cb0073200 0x7f8cb0075710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40ac000 tx=0x7f8cc4019d60 comp rx=0 tx=0).stop 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 msgr2=0x7f8cd41fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 0x7f8cd41fbbf0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc011b10 tx=0x7f8cbc009e10 comp rx=0 tx=0).stop 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 shutdown_connections 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8cb0073200 0x7f8cb0075710 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cd41f8150 0x7f8cd41fc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cd41f7d40 0x7f8cd41fbbf0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 --2- 172.21.15.16:0/630937163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb8003c20 0x7f8cd41fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 >> 172.21.15.16:0/630937163 conn(0x7f8cd4073e00 msgr2=0x7f8cd41606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.507+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 shutdown_connections 2024-09-17T05:07:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:04.511+0000 7f8cd9200640 1 -- 172.21.15.16:0/630937163 wait complete. 2024-09-17T05:07:04.529 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:07:04.529 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:07:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:09.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 -- 172.21.15.16:0/1060312550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec1608d0 msgr2=0x7f46ec0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 --2- 172.21.15.16:0/1060312550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec1608d0 0x7f46ec0d0640 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009840 tx=0x7f46d401c700 comp rx=0 tx=0).stop 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 -- 172.21.15.16:0/1060312550 shutdown_connections 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 --2- 172.21.15.16:0/1060312550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec1608d0 0x7f46ec0d0640 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 --2- 172.21.15.16:0/1060312550 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f46ec15c6f0 0x7f46ec160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 -- 172.21.15.16:0/1060312550 >> 172.21.15.16:0/1060312550 conn(0x7f46ec0cf380 msgr2=0x7f46ec0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 -- 172.21.15.16:0/1060312550 shutdown_connections 2024-09-17T05:07:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 -- 172.21.15.16:0/1060312550 wait complete. 2024-09-17T05:07:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 Processor -- start 2024-09-17T05:07:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.703+0000 7f46f0e00640 1 -- start start 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec0d0b80 0x7f46ec0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 0x7f46ec0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f46ec1608d0 0x7f46ec145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0b3350 con 0x7f46ec15c6f0 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0b31d0 con 0x7f46ec0d0b80 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec16a530 con 0x7f46ec1608d0 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 0x7f46ec0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 0x7f46ec0cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44142/0 (socket says 172.21.15.16:44142) 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 -- 172.21.15.16:0/889346116 learned_addr learned my addr 172.21.15.16:0/889346116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e9e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec0d0b80 0x7f46ec0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46ea800640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f46ec1608d0 0x7f46ec145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 -- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f46ec1608d0 msgr2=0x7f46ec145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f46ec1608d0 0x7f46ec145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 -- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec0d0b80 msgr2=0x7f46ec0ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec0d0b80 0x7f46ec0ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 -- 172.21.15.16:0/889346116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec06fab0 con 0x7f46ec15c6f0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e1400640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 0x7f46ec0cad10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f46d401cbe0 tx=0x7f46d4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40be070 con 0x7f46ec15c6f0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46d40a8530 con 0x7f46ec15c6f0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec06fd90 con 0x7f46ec15c6f0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40a8cd0 con 0x7f46ec15c6f0 2024-09-17T05:07:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec165d50 con 0x7f46ec15c6f0 2024-09-17T05:07:09.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec160390 con 0x7f46ec15c6f0 2024-09-17T05:07:09.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.707+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46d40c1070 con 0x7f46ec15c6f0 2024-09-17T05:07:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.711+0000 7f46e2a00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f46c8073200 0x7f46c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.711+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f46d412ebf0 con 0x7f46ec15c6f0 2024-09-17T05:07:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.711+0000 7f46e9e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f46c8073200 0x7f46c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.711+0000 7f46e9e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f46c8073200 0x7f46c8075710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ad0 tx=0x7f46dc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.715+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f46d40ac030 con 0x7f46ec15c6f0 2024-09-17T05:07:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.923+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f46ec06f840 con 0x7f46ec15c6f0 2024-09-17T05:07:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46e2a00640 1 -- 172.21.15.16:0/889346116 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f46d4016410 con 0x7f46ec15c6f0 2024-09-17T05:07:09.925 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:09.925 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f46c8073200 msgr2=0x7f46c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f46c8073200 0x7f46c8075710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009ad0 tx=0x7f46dc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 msgr2=0x7f46ec0cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 0x7f46ec0cad10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f46d401cbe0 tx=0x7f46d4005d90 comp rx=0 tx=0).stop 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 shutdown_connections 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f46c8073200 0x7f46c8075710 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f46ec1608d0 0x7f46ec145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46ec15c6f0 0x7f46ec0cad10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 --2- 172.21.15.16:0/889346116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46ec0d0b80 0x7f46ec0ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 >> 172.21.15.16:0/889346116 conn(0x7f46ec0cf380 msgr2=0x7f46ec0b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 shutdown_connections 2024-09-17T05:07:09.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:09.927+0000 7f46f0e00640 1 -- 172.21.15.16:0/889346116 wait complete. 2024-09-17T05:07:09.946 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:07:09.946 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:07:11.695 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:13.697 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:14.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 -- 172.21.15.16:0/1130514988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c0d0b80 msgr2=0x7f7d3c0d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 --2- 172.21.15.16:0/1130514988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c0d0b80 0x7f7d3c0d0f90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c0cfdd0 tx=0x7f7d3401cc60 comp rx=0 tx=0).stop 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 -- 172.21.15.16:0/1130514988 shutdown_connections 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 --2- 172.21.15.16:0/1130514988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c0d0b80 0x7f7d3c0d0f90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 --2- 172.21.15.16:0/1130514988 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c1608d0 0x7f7d3c0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 --2- 172.21.15.16:0/1130514988 >> v2:172.21.15.104:3300/0 conn(0x7f7d3c15c6f0 0x7f7d3c160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 -- 172.21.15.16:0/1130514988 >> 172.21.15.16:0/1130514988 conn(0x7f7d3c0cf380 msgr2=0x7f7d3c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 -- 172.21.15.16:0/1130514988 shutdown_connections 2024-09-17T05:07:15.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.119+0000 7f7d40c00640 1 -- 172.21.15.16:0/1130514988 wait complete. 2024-09-17T05:07:15.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 Processor -- start 2024-09-17T05:07:15.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 -- start start 2024-09-17T05:07:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c0d0b80 0x7f7d3c147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 0x7f7d3c15a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d3c1608d0 0x7f7d3c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0b2c40 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0b2ac0 con 0x7f7d3c1608d0 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d40c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c0b2dc0 con 0x7f7d3c0d0b80 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 0x7f7d3c15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 0x7f7d3c15a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44162/0 (socket says 172.21.15.16:44162) 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 -- 172.21.15.16:0/4147188073 learned_addr learned my addr 172.21.15.16:0/4147188073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39e00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c0d0b80 0x7f7d3c147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d3a800640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d3c1608d0 0x7f7d3c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 -- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c0d0b80 msgr2=0x7f7d3c147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c0d0b80 0x7f7d3c147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 -- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d3c1608d0 msgr2=0x7f7d3c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d3c1608d0 0x7f7d3c15aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 -- 172.21.15.16:0/4147188073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c15ce70 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39e00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c0d0b80 0x7f7d3c147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.123+0000 7f7d39400640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 0x7f7d3c15a990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800d720 tx=0x7f7d2800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28017070 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d28009d20 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280106c0 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c071150 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c15d0e0 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c06a750 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7d2801a070 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d32000640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d1c073230 0x7f7d1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:15.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.127+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f7d28094620 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.131+0000 7f7d39e00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d1c073230 0x7f7d1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:15.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.131+0000 7f7d39e00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d1c073230 0x7f7d1c075740 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00bbd0 tx=0x7f7d2c00b5b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:15.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.135+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d2805f2e0 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.343+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7d3c1594e0 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.343+0000 7f7d32000640 1 -- 172.21.15.16:0/4147188073 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f7d2805ebe0 con 0x7f7d3c15c6f0 2024-09-17T05:07:15.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:15.345 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d1c073230 msgr2=0x7f7d1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d1c073230 0x7f7d1c075740 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00bbd0 tx=0x7f7d2c00b5b0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 msgr2=0x7f7d3c15a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 0x7f7d3c15a990 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800d720 tx=0x7f7d2800dbe0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 shutdown_connections 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d1c073230 0x7f7d1c075740 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d3c1608d0 0x7f7d3c15aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d3c15c6f0 0x7f7d3c15a990 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 --2- 172.21.15.16:0/4147188073 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d3c0d0b80 0x7f7d3c147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 >> 172.21.15.16:0/4147188073 conn(0x7f7d3c0cf380 msgr2=0x7f7d3c0b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 shutdown_connections 2024-09-17T05:07:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:15.347+0000 7f7d40c00640 1 -- 172.21.15.16:0/4147188073 wait complete. 2024-09-17T05:07:15.366 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:07:15.366 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:07:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:20.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:07:20.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 -- 172.21.15.16:0/2537978489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 msgr2=0x7ff3440cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 --2- 172.21.15.16:0/2537978489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3440cee50 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff338009900 tx=0x7ff33801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 -- 172.21.15.16:0/2537978489 shutdown_connections 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 --2- 172.21.15.16:0/2537978489 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 0x7ff344167500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 --2- 172.21.15.16:0/2537978489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3440cee50 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 -- 172.21.15.16:0/2537978489 >> 172.21.15.16:0/2537978489 conn(0x7ff34406f900 msgr2=0x7ff3440b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 -- 172.21.15.16:0/2537978489 shutdown_connections 2024-09-17T05:07:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 -- 172.21.15.16:0/2537978489 wait complete. 2024-09-17T05:07:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 Processor -- start 2024-09-17T05:07:20.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.539+0000 7ff34c200640 1 -- start start 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3440cf390 0x7ff344166ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3441418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 0x7ff34415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3440b5280 con 0x7ff34415d600 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3440b4b90 con 0x7ff3440cf390 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff34416a260 con 0x7ff344160880 2024-09-17T05:07:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3441418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3441418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59088/0 (socket says 172.21.15.16:59088) 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 -- 172.21.15.16:0/1225845429 learned_addr learned my addr 172.21.15.16:0/1225845429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff349800640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3440cf390 0x7ff344166ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 -- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 msgr2=0x7ff34415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34a200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 0x7ff34415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 0x7ff34415bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 -- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3440cf390 msgr2=0x7ff344166ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3440cf390 0x7ff344166ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 -- 172.21.15.16:0/1225845429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3441fc6d0 con 0x7ff34415d600 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff349800640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3440cf390 0x7ff344166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34a200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 0x7ff34415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff348e00640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3441418e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c3c0 tx=0x7ff33000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3300052d0 con 0x7ff34415d600 2024-09-17T05:07:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff330007020 con 0x7ff34415d600 2024-09-17T05:07:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff330007470 con 0x7ff34415d600 2024-09-17T05:07:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3441fc9b0 con 0x7ff34415d600 2024-09-17T05:07:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3441fcf70 con 0x7ff34415d600 2024-09-17T05:07:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.543+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3440cee50 con 0x7ff34415d600 2024-09-17T05:07:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.547+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff3300067e0 con 0x7ff34415d600 2024-09-17T05:07:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.547+0000 7ff341600640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff324073200 0x7ff324075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.547+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7ff330093900 con 0x7ff34415d600 2024-09-17T05:07:20.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.547+0000 7ff349800640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff324073200 0x7ff324075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.547+0000 7ff349800640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff324073200 0x7ff324075710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff3380af000 tx=0x7ff33801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.551+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff33005e6d0 con 0x7ff34415d600 2024-09-17T05:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.759+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff3440b3530 con 0x7ff34415d600 2024-09-17T05:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff341600640 1 -- 172.21.15.16:0/1225845429 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff330001090 con 0x7ff34415d600 2024-09-17T05:07:20.761 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:20.761 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff324073200 msgr2=0x7ff324075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff324073200 0x7ff324075710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff3380af000 tx=0x7ff33801b120 comp rx=0 tx=0).stop 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 msgr2=0x7ff3441418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3441418e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c3c0 tx=0x7ff33000c880 comp rx=0 tx=0).stop 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 shutdown_connections 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff324073200 0x7ff324075710 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff344160880 0x7ff34415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34415d600 0x7ff3441418e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 --2- 172.21.15.16:0/1225845429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3440cf390 0x7ff344166ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 >> 172.21.15.16:0/1225845429 conn(0x7ff34406f900 msgr2=0x7ff3441458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 shutdown_connections 2024-09-17T05:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20.763+0000 7ff34c200640 1 -- 172.21.15.16:0/1225845429 wait complete. 2024-09-17T05:07:20.782 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:07:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:20 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-17T05:07:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:07:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-17T05:07:21.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:07:21.730 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:07:21.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:07:21.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:07:21.955 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:07:21.955 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:21.955 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-17T05:07:21.955 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:21.955 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:21.965 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:21.965 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:22.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:07:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 -- 172.21.15.16:0/293461039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c074a20 msgr2=0x7fb11c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 --2- 172.21.15.16:0/293461039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c074a20 0x7fb11c15efe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0cca90 tx=0x7fb11001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 -- 172.21.15.16:0/293461039 shutdown_connections 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 --2- 172.21.15.16:0/293461039 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb11c15ba50 0x7fb11c15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 --2- 172.21.15.16:0/293461039 >> v2:172.21.15.104:3300/0 conn(0x7fb11c074e30 0x7fb11c15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 --2- 172.21.15.16:0/293461039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c074a20 0x7fb11c15efe0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 -- 172.21.15.16:0/293461039 >> 172.21.15.16:0/293461039 conn(0x7fb11c0b2250 msgr2=0x7fb11c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 -- 172.21.15.16:0/293461039 shutdown_connections 2024-09-17T05:07:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.231+0000 7fb124600640 1 -- 172.21.15.16:0/293461039 wait complete. 2024-09-17T05:07:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 Processor -- start 2024-09-17T05:07:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 -- start start 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb11c074a20 0x7fb11c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c074e30 0x7fb11c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 0x7fb11c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0cf2b0 con 0x7fb11c15ba50 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0cf130 con 0x7fb11c074e30 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb124600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c0cf430 con 0x7fb11c074a20 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb122600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 0x7fb11c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb122600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 0x7fb11c15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59104/0 (socket says 172.21.15.16:59104) 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb122600640 1 -- 172.21.15.16:0/364382650 learned_addr learned my addr 172.21.15.16:0/364382650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb121c00640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb11c074a20 0x7fb11c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.235+0000 7fb121200640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c074e30 0x7fb11c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb122600640 1 -- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb11c074a20 msgr2=0x7fb11c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb122600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb11c074a20 0x7fb11c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb122600640 1 -- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c074e30 msgr2=0x7fb11c14c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb122600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c074e30 0x7fb11c14c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb122600640 1 -- 172.21.15.16:0/364382650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c0b3c90 con 0x7fb11c15ba50 2024-09-17T05:07:22.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb122600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 0x7fb11c15eae0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb11400cab0 tx=0x7fb11400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11400ae20 con 0x7fb11c15ba50 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb11400f560 con 0x7fb11c15ba50 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11400f9b0 con 0x7fb11c15ba50 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c1fc560 con 0x7fb11c15ba50 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c0b37b0 con 0x7fb11c15ba50 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c0b39e0 con 0x7fb11c15ba50 2024-09-17T05:07:22.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.239+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb11400fb50 con 0x7fb11c15ba50 2024-09-17T05:07:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.243+0000 7fb119600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb0fc073200 0x7fb0fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.243+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fb114094e40 con 0x7fb11c15ba50 2024-09-17T05:07:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.243+0000 7fb121c00640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb0fc073200 0x7fb0fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.243+0000 7fb121c00640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb0fc073200 0x7fb0fc075710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb1100af000 tx=0x7fb11001b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.247+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb114014030 con 0x7fb11c15ba50 2024-09-17T05:07:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.431+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb11c15be60 con 0x7fb11c15ba50 2024-09-17T05:07:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.431+0000 7fb119600640 1 -- 172.21.15.16:0/364382650 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) ==== 80+0+0 (secure 0 0 0) 0x7fb11405f650 con 0x7fb11c15ba50 2024-09-17T05:07:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb0fc073200 msgr2=0x7fb0fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb0fc073200 0x7fb0fc075710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb1100af000 tx=0x7fb11001b100 comp rx=0 tx=0).stop 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 msgr2=0x7fb11c15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 0x7fb11c15eae0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb11400cab0 tx=0x7fb11400cf70 comp rx=0 tx=0).stop 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 shutdown_connections 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb0fc073200 0x7fb0fc075710 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c15ba50 0x7fb11c15eae0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb11c074e30 0x7fb11c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 --2- 172.21.15.16:0/364382650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb11c074a20 0x7fb11c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 >> 172.21.15.16:0/364382650 conn(0x7fb11c0b2250 msgr2=0x7fb11c153540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 shutdown_connections 2024-09-17T05:07:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.435+0000 7fb124600640 1 -- 172.21.15.16:0/364382650 wait complete. 2024-09-17T05:07:22.453 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:07:22.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:22.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:07:22.512 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:07:22.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:22.513 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:07:22.529 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:22.529 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:22.529 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:22.544 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:22.544 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:22.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 -- 172.21.15.16:0/1490227051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0003c70 msgr2=0x7f34e00040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1490227051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0003c70 0x7f34e00040c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009b10 tx=0x7f34ec01d250 comp rx=0 tx=0).stop 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34fde00640 1 -- 172.21.15.16:0/1490227051 reap_dead start 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 -- 172.21.15.16:0/1490227051 shutdown_connections 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 -- 172.21.15.16:0/1490227051 >> 172.21.15.16:0/1490227051 conn(0x7f34f80b2250 msgr2=0x7f34f80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 -- 172.21.15.16:0/1490227051 shutdown_connections 2024-09-17T05:07:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 -- 172.21.15.16:0/1490227051 wait complete. 2024-09-17T05:07:22.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 Processor -- start 2024-09-17T05:07:22.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.759+0000 7f34ffe00640 1 -- start start 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0003c70 0x7f34f80af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b11a0 0x7f34f80c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 0x7f34f8155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8167e10 con 0x7f34f81f9a40 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8167c90 con 0x7f34f80b11a0 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f34f8167f90 con 0x7f34e0003c70 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 0x7f34f8155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 0x7f34f8155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59122/0 (socket says 172.21.15.16:59122) 2024-09-17T05:07:22.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 -- 172.21.15.16:0/1605873774 learned_addr learned my addr 172.21.15.16:0/1605873774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fd400640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0003c70 0x7f34f80af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fca00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b11a0 0x7f34f80c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 -- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0003c70 msgr2=0x7f34f80af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0003c70 0x7f34f80af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 -- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b11a0 msgr2=0x7f34f80c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b11a0 0x7f34f80c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 -- 172.21.15.16:0/1605873774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f80b19c0 con 0x7f34f81f9a40 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34fde00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 0x7f34f8155d40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f34f0006fd0 tx=0x7f34f0006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f000fa50 con 0x7f34f81f9a40 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34f000fc10 con 0x7f34f81f9a40 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0011430 con 0x7f34f81f9a40 2024-09-17T05:07:22.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f81fcf90 con 0x7f34f81f9a40 2024-09-17T05:07:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81fd550 con 0x7f34f81f9a40 2024-09-17T05:07:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.763+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f81fcd00 con 0x7f34f81f9a40 2024-09-17T05:07:22.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.767+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f34f00115d0 con 0x7f34f81f9a40 2024-09-17T05:07:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.767+0000 7f34f5600640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d8073200 0x7f34d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.767+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f34f0094cb0 con 0x7f34f81f9a40 2024-09-17T05:07:22.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.767+0000 7f34fd400640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d8073200 0x7f34d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:22.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.767+0000 7f34fd400640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d8073200 0x7f34d8075710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0ad000 tx=0x7f34ec019ff0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:22.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.771+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34f005f430 con 0x7f34f81f9a40 2024-09-17T05:07:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.955+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34f8074340 con 0x7f34f81f9a40 2024-09-17T05:07:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.955+0000 7f34f5600640 1 -- 172.21.15.16:0/1605873774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) ==== 80+0+0 (secure 0 0 0) 0x7f34f0063e60 con 0x7f34f81f9a40 2024-09-17T05:07:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d8073200 msgr2=0x7f34d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d8073200 0x7f34d8075710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0ad000 tx=0x7f34ec019ff0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 msgr2=0x7f34f8155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 0x7f34f8155d40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f34f0006fd0 tx=0x7f34f0006720 comp rx=0 tx=0).stop 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 shutdown_connections 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34d8073200 0x7f34d8075710 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f81f9a40 0x7f34f8155d40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34f80b11a0 0x7f34f80c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 --2- 172.21.15.16:0/1605873774 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0003c70 0x7f34f80af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 >> 172.21.15.16:0/1605873774 conn(0x7f34f80b2250 msgr2=0x7f34f80cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 shutdown_connections 2024-09-17T05:07:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:22.959+0000 7f34ffe00640 1 -- 172.21.15.16:0/1605873774 wait complete. 2024-09-17T05:07:22.977 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:07:22.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:22.977 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:07:23.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:07:23.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:23.004 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:07:23.057 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:23.057 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:23.057 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:23.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:07:23.066 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:23.066 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:23.066 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:23.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:07:23.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:23.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 -- 172.21.15.16:0/787940276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 msgr2=0x7f3f240b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/787940276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f240b4470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3f240ce5c0 tx=0x7f3f1c01cb80 comp rx=0 tx=0).stop 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 -- 172.21.15.16:0/787940276 shutdown_connections 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/787940276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f240b4470 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/787940276 >> v2:172.21.15.104:3300/0 conn(0x7f3f240cfa30 0x7f3f240b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 -- 172.21.15.16:0/787940276 >> 172.21.15.16:0/787940276 conn(0x7f3f240d0860 msgr2=0x7f3f240d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 -- 172.21.15.16:0/787940276 shutdown_connections 2024-09-17T05:07:23.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.267+0000 7f3f2ac00640 1 -- 172.21.15.16:0/787940276 wait complete. 2024-09-17T05:07:23.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 Processor -- start 2024-09-17T05:07:23.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 -- start start 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f2406ce90 0x7f3f241f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f241fbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f240cfa30 0x7f3f2414ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241689e0 con 0x7f3f240b4060 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3f240b5a80 con 0x7f3f240cfa30 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3f24168b60 con 0x7f3f2406ce90 2024-09-17T05:07:23.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f241fbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f241fbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59136/0 (socket says 172.21.15.16:59136) 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 -- 172.21.15.16:0/2876332726 learned_addr learned my addr 172.21.15.16:0/2876332726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f23e00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f2406ce90 0x7f3f241f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f28c00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f240cfa30 0x7f3f2414ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 -- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f2406ce90 msgr2=0x7f3f241f9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f2406ce90 0x7f3f241f9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 -- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f240cfa30 msgr2=0x7f3f2414ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f240cfa30 0x7f3f2414ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 -- 172.21.15.16:0/2876332726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f241f85c0 con 0x7f3f240b4060 2024-09-17T05:07:23.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f1be00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f241fbbb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0058d0 tx=0x7f3f0c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c017070 con 0x7f3f240b4060 2024-09-17T05:07:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f0c009d20 con 0x7f3f240b4060 2024-09-17T05:07:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0106c0 con 0x7f3f240b4060 2024-09-17T05:07:23.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241f9580 con 0x7f3f240b4060 2024-09-17T05:07:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.271+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241fc950 con 0x7f3f240b4060 2024-09-17T05:07:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.275+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f241fc3e0 con 0x7f3f240b4060 2024-09-17T05:07:23.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.275+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3f0c01a070 con 0x7f3f240b4060 2024-09-17T05:07:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.275+0000 7f3f20c00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f00073200 0x7f3f00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.275+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f3f0c094720 con 0x7f3f240b4060 2024-09-17T05:07:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.275+0000 7f3f23e00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f00073200 0x7f3f00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.275+0000 7f3f23e00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f00073200 0x7f3f00075710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f70 tx=0x7f3f14009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.279+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f241f9580 con 0x7f3f240b4060 2024-09-17T05:07:23.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f240b4470 con 0x7f3f240b4060 2024-09-17T05:07:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f20c00640 1 -- 172.21.15.16:0/2876332726 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5229 (secure 0 0 0) 0x7f3f241f85c0 con 0x7f3f240b4060 2024-09-17T05:07:23.490 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:23.490 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-17T05:06:21:602844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:06:21.512654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":25,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:07:23.490 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-17T05:07:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f00073200 msgr2=0x7f3f00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f00073200 0x7f3f00075710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3f14000f70 tx=0x7f3f14009040 comp rx=0 tx=0).stop 2024-09-17T05:07:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 msgr2=0x7f3f241fbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f241fbbb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0058d0 tx=0x7f3f0c00dde0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 shutdown_connections 2024-09-17T05:07:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f00073200 0x7f3f00075710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3f240cfa30 0x7f3f2414ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f240b4060 0x7f3f241fbbb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 --2- 172.21.15.16:0/2876332726 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3f2406ce90 0x7f3f241f9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 >> 172.21.15.16:0/2876332726 conn(0x7f3f240d0860 msgr2=0x7f3f240cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 shutdown_connections 2024-09-17T05:07:23.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.491+0000 7f3f2ac00640 1 -- 172.21.15.16:0/2876332726 wait complete. 2024-09-17T05:07:23.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:23.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 -- 172.21.15.16:0/1300145983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 msgr2=0x7fe7fc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 --2- 172.21.15.16:0/1300145983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 0x7fe7fc15c4b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8009840 tx=0x7fe7e801c6b0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 -- 172.21.15.16:0/1300145983 shutdown_connections 2024-09-17T05:07:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 --2- 172.21.15.16:0/1300145983 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 0x7fe7fc15c4b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 --2- 172.21.15.16:0/1300145983 >> v2:172.21.15.104:3300/0 conn(0x7fe7fc0b2f50 0x7fe7fc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 -- 172.21.15.16:0/1300145983 >> 172.21.15.16:0/1300145983 conn(0x7fe7fc06f900 msgr2=0x7fe7fc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 -- 172.21.15.16:0/1300145983 shutdown_connections 2024-09-17T05:07:23.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 -- 172.21.15.16:0/1300145983 wait complete. 2024-09-17T05:07:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 Processor -- start 2024-09-17T05:07:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.675+0000 7fe804600640 1 -- start start 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 0x7fe7fc145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 0x7fe7fc1fa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7fc15f150 0x7fe7fc1fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc169180 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc169000 con 0x7fe7fc15f150 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe7fc169300 con 0x7fe7fc15c0a0 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 0x7fe7fc145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 0x7fe7fc145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59142/0 (socket says 172.21.15.16:59142) 2024-09-17T05:07:23.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 -- 172.21.15.16:0/856493156 learned_addr learned my addr 172.21.15.16:0/856493156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801200640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 0x7fe7fc1fa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe802600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7fc15f150 0x7fe7fc1fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 -- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 msgr2=0x7fe7fc1fa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 0x7fe7fc1fa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 -- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7fc15f150 msgr2=0x7fe7fc1fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7fc15f150 0x7fe7fc1fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 -- 172.21.15.16:0/856493156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc160e10 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe802600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7fc15f150 0x7fe7fc1fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe801c00640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 0x7fe7fc145630 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e7c0 tx=0x7fe7f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000ccc0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7f000ce80 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc161bb0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0005280 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc1610d0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.679+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7fc15bb60 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.683+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe7f0005420 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.683+0000 7fe7f9600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:23.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.683+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fe7f0094de0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.683+0000 7fe801200640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:23.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.683+0000 7fe801200640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80af000 tx=0x7fe7e801aef0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:23.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.687+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7f005fbb0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7fc06a7c0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe7f9600640 1 -- 172.21.15.16:0/856493156 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5229 (secure 0 0 0) 0x7fe7f005f4b0 con 0x7fe7fc0b2f50 2024-09-17T05:07:23.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:23.900 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-17T05:06:21:602844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:06:21.512654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":25,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 msgr2=0x7fe7dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80af000 tx=0x7fe7e801aef0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 msgr2=0x7fe7fc145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 0x7fe7fc145630 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e7c0 tx=0x7fe7f000ec80 comp rx=0 tx=0).stop 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 shutdown_connections 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe7dc073230 0x7fe7dc075740 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7fc15f150 0x7fe7fc1fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe7fc15c0a0 0x7fe7fc1fa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 --2- 172.21.15.16:0/856493156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc0b2f50 0x7fe7fc145630 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.899+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 >> 172.21.15.16:0/856493156 conn(0x7fe7fc06f900 msgr2=0x7fe7fc165950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.903+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 shutdown_connections 2024-09-17T05:07:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:23.903+0000 7fe804600640 1 -- 172.21.15.16:0/856493156 wait complete. 2024-09-17T05:07:23.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:24.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 -- 172.21.15.16:0/1396844763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb8160880 msgr2=0x7feeb8167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 --2- 172.21.15.16:0/1396844763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb8160880 0x7feeb8167500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009c80 tx=0x7feeb001cda0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 -- 172.21.15.16:0/1396844763 shutdown_connections 2024-09-17T05:07:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 --2- 172.21.15.16:0/1396844763 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb8160880 0x7feeb8167500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 --2- 172.21.15.16:0/1396844763 >> v2:172.21.15.104:3300/0 conn(0x7feeb815d600 0x7feeb80cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 -- 172.21.15.16:0/1396844763 >> 172.21.15.16:0/1396844763 conn(0x7feeb806f900 msgr2=0x7feeb80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 -- 172.21.15.16:0/1396844763 shutdown_connections 2024-09-17T05:07:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.079+0000 7feebee00640 1 -- 172.21.15.16:0/1396844763 wait complete. 2024-09-17T05:07:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 Processor -- start 2024-09-17T05:07:24.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 -- start start 2024-09-17T05:07:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb80cf390 0x7feeb80adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 0x7feeb81f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb8160880 0x7feeb815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80b4b40 con 0x7feeb815d600 2024-09-17T05:07:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80b5280 con 0x7feeb8160880 2024-09-17T05:07:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7feeb816a260 con 0x7feeb80cf390 2024-09-17T05:07:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 0x7feeb81f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 0x7feeb81f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59158/0 (socket says 172.21.15.16:59158) 2024-09-17T05:07:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb80cf390 0x7feeb80adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 -- 172.21.15.16:0/2022534292 learned_addr learned my addr 172.21.15.16:0/2022534292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:24.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebce00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb8160880 0x7feeb815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 -- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb80cf390 msgr2=0x7feeb80adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb80cf390 0x7feeb80adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 -- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb8160880 msgr2=0x7feeb815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb8160880 0x7feeb815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 -- 172.21.15.16:0/2022534292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb815b660 con 0x7feeb815d600 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeb7e00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb80cf390 0x7feeb80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeafe00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 0x7feeb81f7420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7feea000c460 tx=0x7feea000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea00052d0 con 0x7feeb815d600 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb815b380 con 0x7feeb815d600 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.083+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea0005470 con 0x7feeb815d600 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0012870 con 0x7feeb815d600 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb8142d10 con 0x7feeb815d600 2024-09-17T05:07:24.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb815b0f0 con 0x7feeb815d600 2024-09-17T05:07:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feea0012a10 con 0x7feeb815d600 2024-09-17T05:07:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feeb4c00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee94073230 0x7fee94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7feea0093e90 con 0x7feeb815d600 2024-09-17T05:07:24.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.087+0000 7feeb7e00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee94073230 0x7fee94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.091+0000 7feeb7e00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee94073230 0x7fee94075740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feeb80afb90 tx=0x7feea80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:24.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.091+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feea005e5c0 con 0x7feeb815d600 2024-09-17T05:07:24.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.267+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feeb80cee50 con 0x7feeb815d600 2024-09-17T05:07:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feeb4c00640 1 -- 172.21.15.16:0/2022534292 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17210 (secure 0 0 0) 0x7feea0062ff0 con 0x7feeb815d600 2024-09-17T05:07:24.270 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:24.270 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:05:50.194845+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:50.645738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:51.850045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.a","mappings":[{"from":7,"to":0}]},{"pgid":"14.14","mappings":[{"from":1,"to":0}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee94073230 msgr2=0x7fee94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee94073230 0x7fee94075740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feeb80afb90 tx=0x7feea80091e0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 msgr2=0x7feeb81f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 0x7feeb81f7420 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7feea000c460 tx=0x7feea000c920 comp rx=0 tx=0).stop 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 shutdown_connections 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee94073230 0x7fee94075740 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb8160880 0x7feeb815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feeb815d600 0x7feeb81f7420 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 --2- 172.21.15.16:0/2022534292 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feeb80cf390 0x7feeb80adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 >> 172.21.15.16:0/2022534292 conn(0x7feeb806f900 msgr2=0x7feeb80cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 shutdown_connections 2024-09-17T05:07:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.271+0000 7feebee00640 1 -- 172.21.15.16:0/2022534292 wait complete. 2024-09-17T05:07:24.292 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:07:24.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 -- 172.21.15.16:0/3669142245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 msgr2=0x7f6b7c0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 --2- 172.21.15.16:0/3669142245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c0b40d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c06e490 tx=0x7f6b6801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 -- 172.21.15.16:0/3669142245 shutdown_connections 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 --2- 172.21.15.16:0/3669142245 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 0x7f6b7c0b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 --2- 172.21.15.16:0/3669142245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c0b40d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 -- 172.21.15.16:0/3669142245 >> 172.21.15.16:0/3669142245 conn(0x7f6b7c0d10c0 msgr2=0x7f6b7c0d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 -- 172.21.15.16:0/3669142245 shutdown_connections 2024-09-17T05:07:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.451+0000 7f6b84200640 1 -- 172.21.15.16:0/3669142245 wait complete. 2024-09-17T05:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 Processor -- start 2024-09-17T05:07:24.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 -- start start 2024-09-17T05:07:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c1fc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 0x7f6b7c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b7c152540 0x7f6b7c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c169280 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c169100 con 0x7f6b7c152540 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7c169400 con 0x7f6b7c06ebb0 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c1fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c1fc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59178/0 (socket says 172.21.15.16:59178) 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 -- 172.21.15.16:0/264027514 learned_addr learned my addr 172.21.15.16:0/264027514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b82200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b7c152540 0x7f6b7c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b80e00640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 0x7f6b7c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 -- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 msgr2=0x7f6b7c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 0x7f6b7c158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 -- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b7c152540 msgr2=0x7f6b7c15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b7c152540 0x7f6b7c15d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 -- 172.21.15.16:0/264027514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c1fcaf0 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b82200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b7c152540 0x7f6b7c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b80e00640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 0x7f6b7c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b81800640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c1fc5b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c3f0 tx=0x7f6b7000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70017070 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b700053c0 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c1fcf70 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003860 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.455+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c15cb80 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.459+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c0cd2e0 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.459+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b7001a070 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.459+0000 7f6b79600640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b5c073230 0x7f6b5c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.459+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f6b70093780 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.459+0000 7f6b80e00640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b5c073230 0x7f6b5c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.463+0000 7f6b80e00640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b5c073230 0x7f6b5c075740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6b680af000 tx=0x7f6b6801b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:24.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.463+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b7005e550 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b7c06dc10 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b79600640 1 -- 172.21.15.16:0/264027514 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5229 (secure 0 0 0) 0x7f6b7005de50 con 0x7f6b7c06e7a0 2024-09-17T05:07:24.677 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:24.677 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-17T05:06:21:602844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:06:21.512654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":25,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:07:24.678 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b5c073230 msgr2=0x7f6b5c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b5c073230 0x7f6b5c075740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6b680af000 tx=0x7f6b6801b120 comp rx=0 tx=0).stop 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 msgr2=0x7f6b7c1fc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c1fc5b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c3f0 tx=0x7f6b7000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 shutdown_connections 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b5c073230 0x7f6b5c075740 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b7c152540 0x7f6b7c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b7c06ebb0 0x7f6b7c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 --2- 172.21.15.16:0/264027514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c06e7a0 0x7f6b7c1fc5b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 >> 172.21.15.16:0/264027514 conn(0x7f6b7c0d10c0 msgr2=0x7f6b7c0cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 shutdown_connections 2024-09-17T05:07:24.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.679+0000 7f6b84200640 1 -- 172.21.15.16:0/264027514 wait complete. 2024-09-17T05:07:24.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 -- 172.21.15.16:0/4291840765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec15d600 msgr2=0x7fdcec0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 --2- 172.21.15.16:0/4291840765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec15d600 0x7fdcec0cee50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009840 tx=0x7fdce001c820 comp rx=0 tx=0).stop 2024-09-17T05:07:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 -- 172.21.15.16:0/4291840765 shutdown_connections 2024-09-17T05:07:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 --2- 172.21.15.16:0/4291840765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec15d600 0x7fdcec0cee50 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 -- 172.21.15.16:0/4291840765 >> 172.21.15.16:0/4291840765 conn(0x7fdcec06f900 msgr2=0x7fdcec0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:24.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 -- 172.21.15.16:0/4291840765 shutdown_connections 2024-09-17T05:07:24.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 -- 172.21.15.16:0/4291840765 wait complete. 2024-09-17T05:07:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 Processor -- start 2024-09-17T05:07:24.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.859+0000 7fdcf4600640 1 -- start start 2024-09-17T05:07:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 0x7fdcec139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdcec15d600 0x7fdcec141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcec160880 0x7fdcec142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec16b440 con 0x7fdcec0cf390 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec0b5280 con 0x7fdcec160880 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec16b5c0 con 0x7fdcec15d600 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 0x7fdcec139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 0x7fdcec139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59194/0 (socket says 172.21.15.16:59194) 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 -- 172.21.15.16:0/2057112675 learned_addr learned my addr 172.21.15.16:0/2057112675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf2600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcec160880 0x7fdcec142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1200640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdcec15d600 0x7fdcec141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 -- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdcec15d600 msgr2=0x7fdcec141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdcec15d600 0x7fdcec141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 -- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcec160880 msgr2=0x7fdcec142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:24.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcec160880 0x7fdcec142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 -- 172.21.15.16:0/2057112675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec13e090 con 0x7fdcec0cf390 2024-09-17T05:07:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf1c00640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 0x7fdcec139e80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009790 tx=0x7fdce00a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:24.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00bb070 con 0x7fdcec0cf390 2024-09-17T05:07:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce0004470 con 0x7fdcec0cf390 2024-09-17T05:07:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec13e370 con 0x7fdcec0cf390 2024-09-17T05:07:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00b2040 con 0x7fdcec0cf390 2024-09-17T05:07:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec1586c0 con 0x7fdcec0cf390 2024-09-17T05:07:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.863+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec0cee50 con 0x7fdcec0cf390 2024-09-17T05:07:24.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.867+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdce00be070 con 0x7fdcec0cf390 2024-09-17T05:07:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.867+0000 7fdce9600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdccc073200 0x7fdccc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:24.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.867+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fdce012dec0 con 0x7fdcec0cf390 2024-09-17T05:07:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.867+0000 7fdcf1200640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdccc073200 0x7fdccc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:24.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.867+0000 7fdcf1200640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdccc073200 0x7fdccc075710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdcec168920 tx=0x7fdcd8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:24.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:24.871+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdce0016410 con 0x7fdcec0cf390 2024-09-17T05:07:25.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.087+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcec0b3530 con 0x7fdcec0cf390 2024-09-17T05:07:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.087+0000 7fdce9600640 1 -- 172.21.15.16:0/2057112675 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5229 (secure 0 0 0) 0x7fdce00f86d0 con 0x7fdcec0cf390 2024-09-17T05:07:25.087 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:25.087 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":56,"btime":"2024-09-17T05:06:21:602844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:04:52.887943+0000","modified":"2024-09-17T05:06:21.512654+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":25,"addr":"172.21.15.16:6834/1846453187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1846453187},{"type":"v1","addr":"172.21.15.16:6834","nonce":1846453187}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6429,"qdb_cluster":[6429]},"id":6}]} 2024-09-17T05:07:25.087 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 56 2024-09-17T05:07:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdccc073200 msgr2=0x7fdccc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdccc073200 0x7fdccc075710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdcec168920 tx=0x7fdcd8009040 comp rx=0 tx=0).stop 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 msgr2=0x7fdcec139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 0x7fdcec139e80 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009790 tx=0x7fdce00a5650 comp rx=0 tx=0).stop 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 shutdown_connections 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdccc073200 0x7fdccc075710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcec160880 0x7fdcec142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdcec15d600 0x7fdcec141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 --2- 172.21.15.16:0/2057112675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcec0cf390 0x7fdcec139e80 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 >> 172.21.15.16:0/2057112675 conn(0x7fdcec06f900 msgr2=0x7fdcec0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 shutdown_connections 2024-09-17T05:07:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.091+0000 7fdcf4600640 1 -- 172.21.15.16:0/2057112675 wait complete. 2024-09-17T05:07:25.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 -- 172.21.15.16:0/4079296055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc188073240 msgr2=0x7fc188073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 --2- 172.21.15.16:0/4079296055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc188073240 0x7fc188073650 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc180009c80 tx=0x7fc18001cd10 comp rx=0 tx=0).stop 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 -- 172.21.15.16:0/4079296055 shutdown_connections 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 --2- 172.21.15.16:0/4079296055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc188073240 0x7fc188073650 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 --2- 172.21.15.16:0/4079296055 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc1880b2f50 0x7fc18815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 -- 172.21.15.16:0/4079296055 >> 172.21.15.16:0/4079296055 conn(0x7fc18806f900 msgr2=0x7fc1880b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.267+0000 7fc18e200640 1 -- 172.21.15.16:0/4079296055 shutdown_connections 2024-09-17T05:07:25.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 -- 172.21.15.16:0/4079296055 wait complete. 2024-09-17T05:07:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 Processor -- start 2024-09-17T05:07:25.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 -- start start 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 0x7fc188162db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1880b2f50 0x7fc188164820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc18815c0a0 0x7fc1881fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc188168d60 con 0x7fc1880b2f50 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc188168be0 con 0x7fc188073240 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.271+0000 7fc18e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc188168ee0 con 0x7fc18815c0a0 2024-09-17T05:07:25.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc187400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 0x7fc188162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc187400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 0x7fc188162db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:38138/0 (socket says 172.21.15.16:38138) 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1880b2f50 0x7fc188164820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc187400640 1 -- 172.21.15.16:0/1407770976 learned_addr learned my addr 172.21.15.16:0/1407770976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc187e00640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc18815c0a0 0x7fc1881fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 -- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc18815c0a0 msgr2=0x7fc1881fc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc18815c0a0 0x7fc1881fc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 -- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 msgr2=0x7fc188162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 0x7fc188162db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 -- 172.21.15.16:0/1407770976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc188165550 con 0x7fc1880b2f50 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc187400640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 0x7fc188162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17fe00640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1880b2f50 0x7fc188164820 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc1880722a0 tx=0x7fc17000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170017070 con 0x7fc1880b2f50 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1700053c0 con 0x7fc1880b2f50 2024-09-17T05:07:25.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc188165830 con 0x7fc1880b2f50 2024-09-17T05:07:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170003840 con 0x7fc1880b2f50 2024-09-17T05:07:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc188165df0 con 0x7fc1880b2f50 2024-09-17T05:07:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1881652c0 con 0x7fc1880b2f50 2024-09-17T05:07:25.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.275+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc17001a070 con 0x7fc1880b2f50 2024-09-17T05:07:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.279+0000 7fc17f400640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc164073230 0x7fc164075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.279+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fc1700939d0 con 0x7fc1880b2f50 2024-09-17T05:07:25.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.279+0000 7fc187400640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc164073230 0x7fc164075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.279+0000 7fc187400640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc164073230 0x7fc164075740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc178005fd0 tx=0x7fc17800b390 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:25.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.283+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc170021020 con 0x7fc1880b2f50 2024-09-17T05:07:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.455+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc188073650 con 0x7fc1880b2f50 2024-09-17T05:07:25.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.455+0000 7fc17f400640 1 -- 172.21.15.16:0/1407770976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+17210 (secure 0 0 0) 0x7fc17005e1d0 con 0x7fc1880b2f50 2024-09-17T05:07:25.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:25.457 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:05:50.194845+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:04:50.645738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T05:04:51.850045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.18","mappings":[{"from":6,"to":4}]},{"pgid":"14.a","mappings":[{"from":7,"to":0}]},{"pgid":"14.14","mappings":[{"from":1,"to":0}]},{"pgid":"14.22","mappings":[{"from":1,"to":0}]},{"pgid":"14.25","mappings":[{"from":7,"to":3}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc164073230 msgr2=0x7fc164075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc164073230 0x7fc164075740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc178005fd0 tx=0x7fc17800b390 comp rx=0 tx=0).stop 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1880b2f50 msgr2=0x7fc188164820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1880b2f50 0x7fc188164820 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc1880722a0 tx=0x7fc17000c820 comp rx=0 tx=0).stop 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 shutdown_connections 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc164073230 0x7fc164075740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc18815c0a0 0x7fc1881fc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1880b2f50 0x7fc188164820 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 --2- 172.21.15.16:0/1407770976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc188073240 0x7fc188162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 >> 172.21.15.16:0/1407770976 conn(0x7fc18806f900 msgr2=0x7fc18815c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 shutdown_connections 2024-09-17T05:07:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.459+0000 7fc18e200640 1 -- 172.21.15.16:0/1407770976 wait complete. 2024-09-17T05:07:25.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:07:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 -- 172.21.15.16:0/663675521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e00cf700 msgr2=0x7f19e015ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 --2- 172.21.15.16:0/663675521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e00cf700 0x7f19e015ba30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f19e00726d0 tx=0x7f19c801c800 comp rx=0 tx=0).stop 2024-09-17T05:07:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 -- 172.21.15.16:0/663675521 shutdown_connections 2024-09-17T05:07:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 --2- 172.21.15.16:0/663675521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e00cf700 0x7f19e015ba30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 --2- 172.21.15.16:0/663675521 >> v2:172.21.15.104:3300/0 conn(0x7f19e0152270 0x7f19e00cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 -- 172.21.15.16:0/663675521 >> 172.21.15.16:0/663675521 conn(0x7f19e00d05a0 msgr2=0x7f19e006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 -- 172.21.15.16:0/663675521 shutdown_connections 2024-09-17T05:07:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 -- 172.21.15.16:0/663675521 wait complete. 2024-09-17T05:07:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 Processor -- start 2024-09-17T05:07:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.643+0000 7f19df200640 1 -- start start 2024-09-17T05:07:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e00cf700 0x7f19e0136830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e0152270 0x7f19e014b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19e015bf70 0x7f19e0155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19e00747d0 con 0x7f19e0152270 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0074650 con 0x7f19e00cf700 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f19e0074950 con 0x7f19e015bf70 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e0152270 0x7f19e014b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e0152270 0x7f19e014b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59212/0 (socket says 172.21.15.16:59212) 2024-09-17T05:07:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 -- 172.21.15.16:0/691722350 learned_addr learned my addr 172.21.15.16:0/691722350 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19dde00640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e00cf700 0x7f19e0136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 -- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19e015bf70 msgr2=0x7f19e0155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19de800640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19e015bf70 0x7f19e0155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19e015bf70 0x7f19e0155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 -- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e00cf700 msgr2=0x7f19e0136830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e00cf700 0x7f19e0136830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 -- 172.21.15.16:0/691722350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e0148990 con 0x7f19e0152270 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19dde00640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19e00cf700 0x7f19e0136830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19de800640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19e015bf70 0x7f19e0155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d5400640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e0152270 0x7f19e014b960 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f19c801cce0 tx=0x7f19c8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80b2040 con 0x7f19e0152270 2024-09-17T05:07:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19c80bb070 con 0x7f19e0152270 2024-09-17T05:07:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003600 con 0x7f19e0152270 2024-09-17T05:07:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 -- 172.21.15.16:0/691722350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e0148c00 con 0x7f19e0152270 2024-09-17T05:07:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 -- 172.21.15.16:0/691722350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e014bea0 con 0x7f19e0152270 2024-09-17T05:07:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.647+0000 7f19df200640 1 -- 172.21.15.16:0/691722350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e015ba30 con 0x7f19e0152270 2024-09-17T05:07:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.651+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19c80037a0 con 0x7f19e0152270 2024-09-17T05:07:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.651+0000 7f19d6a00640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.651+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7f19c812f340 con 0x7f19e0152270 2024-09-17T05:07:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.651+0000 7f19dde00640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.651+0000 7f19dde00640 1 --2- 172.21.15.16:0/691722350 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19bc073200 0x7f19bc075710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009ea0 tx=0x7f19d0009500 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:25.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.655+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19c80163f0 con 0x7f19e0152270 2024-09-17T05:07:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.867+0000 7f19df200640 1 -- 172.21.15.16:0/691722350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f19e00b4b70 con 0x7f19e0152270 2024-09-17T05:07:25.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:25.867+0000 7f19d6a00640 1 -- 172.21.15.16:0/691722350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v57) ==== 316+0+0 (secure 0 0 0) 0x7f19c80fa190 con 0x7f19e0152270 2024-09-17T05:07:25.869 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-17T05:07:25.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:07:25.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 -- 172.21.15.16:0/1486997935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 msgr2=0x7fc8e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 --2- 172.21.15.16:0/1486997935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc8e800b9d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc9040ce370 tx=0x7fc8fc01c910 comp rx=0 tx=0).stop 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 -- 172.21.15.16:0/1486997935 shutdown_connections 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 --2- 172.21.15.16:0/1486997935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc8e800b9d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 -- 172.21.15.16:0/1486997935 >> 172.21.15.16:0/1486997935 conn(0x7fc9040b4160 msgr2=0x7fc9040b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 -- 172.21.15.16:0/1486997935 shutdown_connections 2024-09-17T05:07:26.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.039+0000 7fc903e00640 1 -- 172.21.15.16:0/1486997935 wait complete. 2024-09-17T05:07:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 Processor -- start 2024-09-17T05:07:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 -- start start 2024-09-17T05:07:26.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc9041f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9040cb7f0 0x7fc90414b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9040b5f00 0x7fc9040b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040d0470 con 0x7fc8e8005360 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9040d02f0 con 0x7fc9040cb7f0 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc9040d05f0 con 0x7fc9040b5f00 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc9041f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc9041f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59224/0 (socket says 172.21.15.16:59224) 2024-09-17T05:07:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 -- 172.21.15.16:0/2142045 learned_addr learned my addr 172.21.15.16:0/2142045 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 -- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9040b5f00 msgr2=0x7fc9040b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9040b5f00 0x7fc9040b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 -- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9040cb7f0 msgr2=0x7fc90414b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9040cb7f0 0x7fc90414b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 -- 172.21.15.16:0/2142045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9040b6850 con 0x7fc8e8005360 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc902a00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc9041f6180 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4011a80 tx=0x7fc8f4011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400fe70 con 0x7fc8e8005360 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8f4016070 con 0x7fc8e8005360 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4012690 con 0x7fc8e8005360 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90406fab0 con 0x7fc8e8005360 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.043+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9040ca140 con 0x7fc8e8005360 2024-09-17T05:07:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.047+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90406ffd0 con 0x7fc8e8005360 2024-09-17T05:07:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.047+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc8f4018030 con 0x7fc8e8005360 2024-09-17T05:07:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.047+0000 7fc8fb400640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8e0073200 0x7fc8e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.047+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6275+0+0 (secure 0 0 0) 0x7fc8f4094ec0 con 0x7fc8e8005360 2024-09-17T05:07:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.047+0000 7fc8fbe00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8e0073200 0x7fc8e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.047+0000 7fc8fbe00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8e0073200 0x7fc8e0075710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0036b0 tx=0x7fc8ec003600 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.051+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc90406ffd0 con 0x7fc8e8005360 2024-09-17T05:07:26.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.263+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc90415c850 con 0x7fc8e8005360 2024-09-17T05:07:26.262 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:07:26.263+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:07:26.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.263+0000 7fc8fb400640 1 -- 172.21.15.16:0/2142045 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 191+0+0 (secure 0 0 0) 0x7fc9040b6850 con 0x7fc8e8005360 2024-09-17T05:07:26.264 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:07:26.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8e0073200 msgr2=0x7fc8e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8e0073200 0x7fc8e0075710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0036b0 tx=0x7fc8ec003600 comp rx=0 tx=0).stop 2024-09-17T05:07:26.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 msgr2=0x7fc9041f6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc9041f6180 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4011a80 tx=0x7fc8f4011f40 comp rx=0 tx=0).stop 2024-09-17T05:07:26.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 shutdown_connections 2024-09-17T05:07:26.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc8e0073200 0x7fc8e0075710 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0036b0 tx=0x7fc8ec003600 comp rx=0 tx=0).stop 2024-09-17T05:07:26.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9040b5f00 0x7fc9040b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9040cb7f0 0x7fc90414b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 --2- 172.21.15.16:0/2142045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8e8005360 0x7fc9041f6180 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 >> 172.21.15.16:0/2142045 conn(0x7fc9040b4160 msgr2=0x7fc9040cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:26.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 shutdown_connections 2024-09-17T05:07:26.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.267+0000 7fc903e00640 1 -- 172.21.15.16:0/2142045 wait complete. 2024-09-17T05:07:26.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:07:26.406 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:07:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 -- 172.21.15.16:0/1232979402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 msgr2=0x7fbb20004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 --2- 172.21.15.16:0/1232979402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb20004070 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c006f90 tx=0x7fbb2c01cee0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb41c00640 1 -- 172.21.15.16:0/1232979402 reap_dead start 2024-09-17T05:07:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 -- 172.21.15.16:0/1232979402 shutdown_connections 2024-09-17T05:07:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 -- 172.21.15.16:0/1232979402 >> 172.21.15.16:0/1232979402 conn(0x7fbb3c0b2250 msgr2=0x7fbb3c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 -- 172.21.15.16:0/1232979402 shutdown_connections 2024-09-17T05:07:26.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 -- 172.21.15.16:0/1232979402 wait complete. 2024-09-17T05:07:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 Processor -- start 2024-09-17T05:07:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 -- start start 2024-09-17T05:07:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.459+0000 7fbb43c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb3c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb3c0c4d60 0x7fbb3c1f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb3c152030 0x7fbb3c1fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c167b20 con 0x7fbb20003c20 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c1679a0 con 0x7fbb3c152030 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c167ca0 con 0x7fbb3c0c4d60 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb3c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb3c0cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59258/0 (socket says 172.21.15.16:59258) 2024-09-17T05:07:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 -- 172.21.15.16:0/2187882417 learned_addr learned my addr 172.21.15.16:0/2187882417 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb3c0c4d60 0x7fbb3c1f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 -- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb3c0c4d60 msgr2=0x7fbb3c1f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb3c152030 0x7fbb3c1fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb3c0c4d60 0x7fbb3c1f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 -- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb3c152030 msgr2=0x7fbb3c1fc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb3c152030 0x7fbb3c1fc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 -- 172.21.15.16:0/2187882417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c1f8fd0 con 0x7fbb20003c20 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb3c0c4d60 0x7fbb3c1f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb3c152030 0x7fbb3c1fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb41200640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb3c0cd460 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c01c9a0 tx=0x7fbb2c006c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0a5b70 con 0x7fbb20003c20 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb2c0a5d10 con 0x7fbb20003c20 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0062d0 con 0x7fbb20003c20 2024-09-17T05:07:26.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c1f92b0 con 0x7fbb20003c20 2024-09-17T05:07:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1fbf80 con 0x7fbb20003c20 2024-09-17T05:07:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c1fbd70 con 0x7fbb20003c20 2024-09-17T05:07:26.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.463+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb2c003620 con 0x7fbb20003c20 2024-09-17T05:07:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.467+0000 7fbb38c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb1c073200 0x7fbb1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.467+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6361+0+0 (secure 0 0 0) 0x7fbb2c12e240 con 0x7fbb20003c20 2024-09-17T05:07:26.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.467+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb1c073200 0x7fbb1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:26.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.467+0000 7fbb3be00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb1c073200 0x7fbb1c075710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbb28000c10 tx=0x7fbb28006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:26.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.467+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb2c0bf030 con 0x7fbb20003c20 2024-09-17T05:07:26.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:26.683+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbb3c15efe0 con 0x7fbb20003c20 2024-09-17T05:07:27.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.267+0000 7fbb38c00640 1 -- 172.21.15.16:0/2187882417 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7fbb2c016c90 con 0x7fbb20003c20 2024-09-17T05:07:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb1c073200 msgr2=0x7fbb1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb1c073200 0x7fbb1c075710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbb28000c10 tx=0x7fbb28006b20 comp rx=0 tx=0).stop 2024-09-17T05:07:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 msgr2=0x7fbb3c0cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb3c0cd460 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c01c9a0 tx=0x7fbb2c006c70 comp rx=0 tx=0).stop 2024-09-17T05:07:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 shutdown_connections 2024-09-17T05:07:27.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbb1c073200 0x7fbb1c075710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb3c152030 0x7fbb3c1fc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbb3c0c4d60 0x7fbb3c1f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 --2- 172.21.15.16:0/2187882417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb20003c20 0x7fbb3c0cd460 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 >> 172.21.15.16:0/2187882417 conn(0x7fbb3c0b2250 msgr2=0x7fbb3c159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 shutdown_connections 2024-09-17T05:07:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.271+0000 7fbb43c00640 1 -- 172.21.15.16:0/2187882417 wait complete. 2024-09-17T05:07:27.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:07:27.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 -- 172.21.15.16:0/609246409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06d2a0 msgr2=0x7f1fec15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:27.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 --2- 172.21.15.16:0/609246409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06d2a0 0x7f1fec15b770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400acd0 tx=0x7f1fe401de60 comp rx=0 tx=0).stop 2024-09-17T05:07:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 -- 172.21.15.16:0/609246409 shutdown_connections 2024-09-17T05:07:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 --2- 172.21.15.16:0/609246409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06d2a0 0x7f1fec15b770 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 -- 172.21.15.16:0/609246409 >> 172.21.15.16:0/609246409 conn(0x7f1fec0d0050 msgr2=0x7f1fec0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 -- 172.21.15.16:0/609246409 shutdown_connections 2024-09-17T05:07:27.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 -- 172.21.15.16:0/609246409 wait complete. 2024-09-17T05:07:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 Processor -- start 2024-09-17T05:07:27.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.463+0000 7f1ff4400640 1 -- start start 2024-09-17T05:07:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06ce90 0x7f1fec0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1fec06d2a0 0x7f1fec15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 0x7f1fec15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec168fd0 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec168e50 con 0x7f1fec06ce90 2024-09-17T05:07:27.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1fec169150 con 0x7f1fec06d2a0 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 0x7f1fec15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 0x7f1fec15dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47750/0 (socket says 172.21.15.16:47750) 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 -- 172.21.15.16:0/1411758655 learned_addr learned my addr 172.21.15.16:0/1411758655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff1a00640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06ce90 0x7f1fec0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff1000640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1fec06d2a0 0x7f1fec15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 -- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1fec06d2a0 msgr2=0x7f1fec15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1fec06d2a0 0x7f1fec15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 -- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06ce90 msgr2=0x7f1fec0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06ce90 0x7f1fec0cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:27.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 -- 172.21.15.16:0/1411758655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec1503a0 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff2400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 0x7f1fec15dc80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1fe401aa70 tx=0x7f1fe4005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:27.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40bd070 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fe4004130 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec0ccb00 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40b3040 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec0cd0c0 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.467+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fec150110 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.471+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1fe4004540 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.471+0000 7f1fe9600640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1fcc073200 0x7f1fcc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.471+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6361+0+0 (secure 0 0 0) 0x7f1fe412e500 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.471+0000 7f1ff1a00640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1fcc073200 0x7f1fcc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:27.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.471+0000 7f1ff1a00640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1fcc073200 0x7f1fcc075710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007fd0 tx=0x7f1fe0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.475+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1fe4017ae0 con 0x7f1fec0cc0f0 2024-09-17T05:07:27.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:27.651+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1fec0705e0 con 0x7f1fec0cc0f0 2024-09-17T05:07:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.275+0000 7f1fe9600640 1 -- 172.21.15.16:0/1411758655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f1fe4004830 con 0x7f1fec0cc0f0 2024-09-17T05:07:28.273 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:07:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1fcc073200 msgr2=0x7f1fcc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1fcc073200 0x7f1fcc075710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0007fd0 tx=0x7f1fe0009040 comp rx=0 tx=0).stop 2024-09-17T05:07:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 msgr2=0x7f1fec15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:28.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 0x7f1fec15dc80 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1fe401aa70 tx=0x7f1fe4005a60 comp rx=0 tx=0).stop 2024-09-17T05:07:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 shutdown_connections 2024-09-17T05:07:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1fcc073200 0x7f1fcc075710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fec0cc0f0 0x7f1fec15dc80 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1fec06d2a0 0x7f1fec15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 --2- 172.21.15.16:0/1411758655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fec06ce90 0x7f1fec0cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 >> 172.21.15.16:0/1411758655 conn(0x7f1fec0d0050 msgr2=0x7f1fec0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 shutdown_connections 2024-09-17T05:07:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.279+0000 7f1ff4400640 1 -- 172.21.15.16:0/1411758655 wait complete. 2024-09-17T05:07:28.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.471+0000 7f7a6aa00640 1 -- 172.21.15.16:0/1271246770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 msgr2=0x7f7a640cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.471+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/1271246770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a640cec30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7a64074c60 tx=0x7f7a4c01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 -- 172.21.15.16:0/1271246770 shutdown_connections 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/1271246770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a640cec30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/1271246770 >> v2:172.21.15.104:3300/0 conn(0x7f7a640cfdf0 0x7f7a640d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 -- 172.21.15.16:0/1271246770 >> 172.21.15.16:0/1271246770 conn(0x7f7a640715e0 msgr2=0x7f7a640719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:28.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 -- 172.21.15.16:0/1271246770 shutdown_connections 2024-09-17T05:07:28.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 -- 172.21.15.16:0/1271246770 wait complete. 2024-09-17T05:07:28.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 Processor -- start 2024-09-17T05:07:28.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 -- start start 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a640cf170 0x7f7a6415b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a640cfdf0 0x7f7a64155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.475+0000 7f7a6aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a64143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a6aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64169dc0 con 0x7f7a640d1dc0 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a6aa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64169c40 con 0x7f7a640cfdf0 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a6aa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7a64169f40 con 0x7f7a640cf170 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a64143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a64143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47770/0 (socket says 172.21.15.16:47770) 2024-09-17T05:07:28.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 -- 172.21.15.16:0/3654186885 learned_addr learned my addr 172.21.15.16:0/3654186885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a5be00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a640cfdf0 0x7f7a64155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a63e00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a640cf170 0x7f7a6415b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 -- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a640cf170 msgr2=0x7f7a6415b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a640cf170 0x7f7a6415b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 -- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a640cfdf0 msgr2=0x7f7a64155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a640cfdf0 0x7f7a64155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 -- 172.21.15.16:0/3654186885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a64144420 con 0x7f7a640d1dc0 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a63e00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a640cf170 0x7f7a6415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a68a00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a64143840 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00c760 tx=0x7f7a5c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c003db0 con 0x7f7a640d1dc0 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a5c00ce10 con 0x7f7a640d1dc0 2024-09-17T05:07:28.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0052d0 con 0x7f7a640d1dc0 2024-09-17T05:07:28.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a64154e80 con 0x7f7a640d1dc0 2024-09-17T05:07:28.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a64155440 con 0x7f7a640d1dc0 2024-09-17T05:07:28.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.479+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a640d1880 con 0x7f7a640d1dc0 2024-09-17T05:07:28.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.483+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a5c005470 con 0x7f7a640d1dc0 2024-09-17T05:07:28.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.483+0000 7f7a60c00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7a40073230 0x7f7a40075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:28.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.483+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5894+0+0 (secure 0 0 0) 0x7f7a5c0940c0 con 0x7f7a640d1dc0 2024-09-17T05:07:28.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.483+0000 7f7a63e00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7a40073230 0x7f7a40075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:28.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.483+0000 7f7a63e00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7a40073230 0x7f7a40075740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009b90 tx=0x7f7a540091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.487+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7a640d0200 con 0x7f7a640d1dc0 2024-09-17T05:07:28.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:28.671+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7a64144140 con 0x7f7a640d1dc0 2024-09-17T05:07:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.275+0000 7f7a60c00640 1 -- 172.21.15.16:0/3654186885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f7a5c05ef80 con 0x7f7a640d1dc0 2024-09-17T05:07:29.275 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:07:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7a40073230 msgr2=0x7f7a40075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7a40073230 0x7f7a40075740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009b90 tx=0x7f7a540091e0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 msgr2=0x7f7a64143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a64143840 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00c760 tx=0x7f7a5c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 shutdown_connections 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7a40073230 0x7f7a40075740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a640d1dc0 0x7f7a64143840 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a640cfdf0 0x7f7a64155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 --2- 172.21.15.16:0/3654186885 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7a640cf170 0x7f7a6415b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 >> 172.21.15.16:0/3654186885 conn(0x7f7a640715e0 msgr2=0x7f7a6415dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 shutdown_connections 2024-09-17T05:07:29.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.279+0000 7f7a6aa00640 1 -- 172.21.15.16:0/3654186885 wait complete. 2024-09-17T05:07:29.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:07:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 -- 172.21.15.16:0/4155831509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f90074a20 msgr2=0x7f9f9015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 --2- 172.21.15.16:0/4155831509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f90074a20 0x7f9f9015efe0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9f90072860 tx=0x7f9f8001c8a0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 -- 172.21.15.16:0/4155831509 shutdown_connections 2024-09-17T05:07:29.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 --2- 172.21.15.16:0/4155831509 >> v2:172.21.15.104:3300/0 conn(0x7f9f9015f520 0x7f9f9015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 --2- 172.21.15.16:0/4155831509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f90074a20 0x7f9f9015efe0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 -- 172.21.15.16:0/4155831509 >> 172.21.15.16:0/4155831509 conn(0x7f9f900b2250 msgr2=0x7f9f900d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 -- 172.21.15.16:0/4155831509 shutdown_connections 2024-09-17T05:07:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 -- 172.21.15.16:0/4155831509 wait complete. 2024-09-17T05:07:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 Processor -- start 2024-09-17T05:07:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.471+0000 7f9f97c00640 1 -- start start 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f90074a20 0x7f9f900b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f90074e30 0x7f9f9014def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 0x7f9f9014e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90168e20 con 0x7f9f9015f520 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f90168ca0 con 0x7f9f90074a20 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9f90168fa0 con 0x7f9f90074e30 2024-09-17T05:07:29.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 0x7f9f9014e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 0x7f9f9014e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47782/0 (socket says 172.21.15.16:47782) 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 -- 172.21.15.16:0/448540413 learned_addr learned my addr 172.21.15.16:0/448540413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95200640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f90074a20 0x7f9f900b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f87e00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f90074e30 0x7f9f9014def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 -- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f90074e30 msgr2=0x7f9f9014def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f90074e30 0x7f9f9014def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 -- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f90074a20 msgr2=0x7f9f900b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f90074a20 0x7f9f900b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 -- 172.21.15.16:0/448540413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f900c3b90 con 0x7f9f9015f520 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f95c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 0x7f9f9014e430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b9e0 tx=0x7f9f8800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88019070 con 0x7f9f9015f520 2024-09-17T05:07:29.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f8800cc30 con 0x7f9f9015f520 2024-09-17T05:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f880146a0 con 0x7f9f9015f520 2024-09-17T05:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f90153db0 con 0x7f9f9015f520 2024-09-17T05:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.475+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f90154330 con 0x7f9f9015f520 2024-09-17T05:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.479+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9014e9f0 con 0x7f9f9015f520 2024-09-17T05:07:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.479+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f8801c070 con 0x7f9f9015f520 2024-09-17T05:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.479+0000 7f9f8d600640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f6c073200 0x7f9f6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.479+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f9f88093d20 con 0x7f9f9015f520 2024-09-17T05:07:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.479+0000 7f9f95200640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f6c073200 0x7f9f6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.479+0000 7f9f95200640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f6c073200 0x7f9f6c075710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9f800af000 tx=0x7f9f8001b0e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.483+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9f8800ce20 con 0x7f9f9015f520 2024-09-17T05:07:29.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.659+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9f900c3d60 con 0x7f9f9015f520 2024-09-17T05:07:29.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.663+0000 7f9f8d600640 1 -- 172.21.15.16:0/448540413 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v48) ==== 71+0+0 (secure 0 0 0) 0x7f9f8805e8f0 con 0x7f9f9015f520 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.663+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f6c073200 msgr2=0x7f9f6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.663+0000 7f9f97c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f6c073200 0x7f9f6c075710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9f800af000 tx=0x7f9f8001b0e0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.663+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 msgr2=0x7f9f9014e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.663+0000 7f9f97c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 0x7f9f9014e430 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b9e0 tx=0x7f9f8800bea0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 shutdown_connections 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f6c073200 0x7f9f6c075710 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9015f520 0x7f9f9014e430 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f90074e30 0x7f9f9014def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 --2- 172.21.15.16:0/448540413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f90074a20 0x7f9f900b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 >> 172.21.15.16:0/448540413 conn(0x7f9f900b2250 msgr2=0x7f9f9015e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 shutdown_connections 2024-09-17T05:07:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.667+0000 7f9f97c00640 1 -- 172.21.15.16:0/448540413 wait complete. 2024-09-17T05:07:29.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-09-17T05:07:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7e00640 1 -- 172.21.15.16:0/2026968697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0039f0 msgr2=0x7f1a8c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/2026968697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a8c0039f0 0x7f1a8c003e20 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1aa806b3c0 tx=0x7f1a9801cf20 comp rx=0 tx=0).stop 2024-09-17T05:07:29.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7400640 1 -- 172.21.15.16:0/2026968697 reap_dead start 2024-09-17T05:07:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7e00640 1 -- 172.21.15.16:0/2026968697 shutdown_connections 2024-09-17T05:07:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7e00640 1 -- 172.21.15.16:0/2026968697 >> 172.21.15.16:0/2026968697 conn(0x7f1aa8073ca0 msgr2=0x7f1aa80740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7e00640 1 -- 172.21.15.16:0/2026968697 shutdown_connections 2024-09-17T05:07:29.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.843+0000 7f1aa7e00640 1 -- 172.21.15.16:0/2026968697 wait complete. 2024-09-17T05:07:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 Processor -- start 2024-09-17T05:07:29.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 -- start start 2024-09-17T05:07:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c0039f0 0x7f1aa81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 0x7f1aa8144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1aa8161320 0x7f1aa8161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa80cfe80 con 0x7f1aa81f8980 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa80cfd00 con 0x7f1a8c0039f0 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa80d0000 con 0x7f1aa8161320 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1a9fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 0x7f1aa8144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1a9fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 0x7f1aa8144520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47802/0 (socket says 172.21.15.16:47802) 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1a9fe00640 1 -- 172.21.15.16:0/4105239785 learned_addr learned my addr 172.21.15.16:0/4105239785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa7400640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1aa8161320 0x7f1aa8161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.847+0000 7f1aa6a00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c0039f0 0x7f1aa81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9fe00640 1 -- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1aa8161320 msgr2=0x7f1aa8161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9fe00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1aa8161320 0x7f1aa8161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9fe00640 1 -- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c0039f0 msgr2=0x7f1aa81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9fe00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c0039f0 0x7f1aa81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9fe00640 1 -- 172.21.15.16:0/4105239785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa81572d0 con 0x7f1aa81f8980 2024-09-17T05:07:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9fe00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 0x7f1aa8144520 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1a90011b10 tx=0x7f1a90009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9000fe40 con 0x7f1aa81f8980 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a90004050 con 0x7f1aa81f8980 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90004460 con 0x7f1aa81f8980 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa81575b0 con 0x7f1aa81f8980 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa8144000 con 0x7f1aa81f8980 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa806efb0 con 0x7f1aa81f8980 2024-09-17T05:07:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1a90005020 con 0x7f1aa81f8980 2024-09-17T05:07:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.851+0000 7f1a9ea00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1a84073200 0x7f1a84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.855+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f1a90094c90 con 0x7f1aa81f8980 2024-09-17T05:07:29.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.855+0000 7f1aa6a00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1a84073200 0x7f1a84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.855+0000 7f1aa6a00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1a84073200 0x7f1a84075710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a980ab000 tx=0x7f1a98019d60 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:29.859+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1a9005f7c0 con 0x7f1aa81f8980 2024-09-17T05:07:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.091+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f1aa80cd2e0 con 0x7f1aa81f8980 2024-09-17T05:07:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.291+0000 7f1a9ea00640 1 -- 172.21.15.16:0/4105239785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v316) ==== 160+0+0 (secure 0 0 0) 0x7f1a900641f0 con 0x7f1aa81f8980 2024-09-17T05:07:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1a84073200 msgr2=0x7f1a84075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1a84073200 0x7f1a84075710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1a980ab000 tx=0x7f1a98019d60 comp rx=0 tx=0).stop 2024-09-17T05:07:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 msgr2=0x7f1aa8144520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 0x7f1aa8144520 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1a90011b10 tx=0x7f1a90009e10 comp rx=0 tx=0).stop 2024-09-17T05:07:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 shutdown_connections 2024-09-17T05:07:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1a84073200 0x7f1a84075710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1aa8161320 0x7f1aa8161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1aa81f8980 0x7f1aa8144520 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 --2- 172.21.15.16:0/4105239785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a8c0039f0 0x7f1aa81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 >> 172.21.15.16:0/4105239785 conn(0x7f1aa8073ca0 msgr2=0x7f1aa80b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 shutdown_connections 2024-09-17T05:07:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.295+0000 7f1aa7e00640 1 -- 172.21.15.16:0/4105239785 wait complete. 2024-09-17T05:07:30.315 INFO:tasks.cephfs_test_runner:When a client has inodes pinned in its cache (open files), that the MDS ... ok 2024-09-17T05:07:30.315 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) When a client does not advance its oldest tid, the MDS should notice that 2024-09-17T05:07:30.315 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:07:30.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 -- 172.21.15.16:0/2013774282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd434160880 msgr2=0x7fd434167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 --2- 172.21.15.16:0/2013774282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd434160880 0x7fd434167500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009840 tx=0x7fd42c01c740 comp rx=0 tx=0).stop 2024-09-17T05:07:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 -- 172.21.15.16:0/2013774282 shutdown_connections 2024-09-17T05:07:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 --2- 172.21.15.16:0/2013774282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd434160880 0x7fd434167500 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 --2- 172.21.15.16:0/2013774282 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd43415d600 0x7fd4340cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 -- 172.21.15.16:0/2013774282 >> 172.21.15.16:0/2013774282 conn(0x7fd43406f900 msgr2=0x7fd4340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 -- 172.21.15.16:0/2013774282 shutdown_connections 2024-09-17T05:07:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.475+0000 7fd438e00640 1 -- 172.21.15.16:0/2013774282 wait complete. 2024-09-17T05:07:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 Processor -- start 2024-09-17T05:07:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 -- start start 2024-09-17T05:07:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd4340cf390 0x7fd4341526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd43415d600 0x7fd434150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 0x7fd4341511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd43416a260 con 0x7fd434160880 2024-09-17T05:07:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd4340b5280 con 0x7fd43415d600 2024-09-17T05:07:30.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd438e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd43416a3e0 con 0x7fd4340cf390 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd432800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 0x7fd4341511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd432800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 0x7fd4341511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47814/0 (socket says 172.21.15.16:47814) 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd432800640 1 -- 172.21.15.16:0/3484568140 learned_addr learned my addr 172.21.15.16:0/3484568140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd431e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd4340cf390 0x7fd4341526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd429400640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd43415d600 0x7fd434150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd432800640 1 -- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd4340cf390 msgr2=0x7fd4341526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.479+0000 7fd432800640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd4340cf390 0x7fd4341526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd432800640 1 -- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd43415d600 msgr2=0x7fd434150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd432800640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd43415d600 0x7fd434150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd432800640 1 -- 172.21.15.16:0/3484568140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4341fc9e0 con 0x7fd434160880 2024-09-17T05:07:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd432800640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 0x7fd4341511c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd42c01c570 tx=0x7fd42c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0ac070 con 0x7fd434160880 2024-09-17T05:07:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd42c004030 con 0x7fd434160880 2024-09-17T05:07:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c004440 con 0x7fd434160880 2024-09-17T05:07:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341560e0 con 0x7fd434160880 2024-09-17T05:07:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd434155b20 con 0x7fd434160880 2024-09-17T05:07:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4340cee50 con 0x7fd434160880 2024-09-17T05:07:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.483+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd42c0af070 con 0x7fd434160880 2024-09-17T05:07:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.487+0000 7fd42aa00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd410073230 0x7fd410075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.487+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7fd42c12dbb0 con 0x7fd434160880 2024-09-17T05:07:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.487+0000 7fd431e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd410073230 0x7fd410075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.487+0000 7fd431e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd410073230 0x7fd410075740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4340afb90 tx=0x7fd4240091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.491+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd42c016330 con 0x7fd434160880 2024-09-17T05:07:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.667+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4340b3530 con 0x7fd434160880 2024-09-17T05:07:30.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.667+0000 7fd42aa00640 1 -- 172.21.15.16:0/3484568140 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13294 (secure 0 0 0) 0x7fd42c0f8650 con 0x7fd434160880 2024-09-17T05:07:30.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:30.667 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":92,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:29.276216+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/1846453187":"2024-09-18T05:07:26.265138+0000","172.21.15.16:6832/1846453187":"2024-09-18T05:07:26.265138+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd410073230 msgr2=0x7fd410075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd410073230 0x7fd410075740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4340afb90 tx=0x7fd4240091e0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 msgr2=0x7fd4341511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 0x7fd4341511c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd42c01c570 tx=0x7fd42c0052d0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 shutdown_connections 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd410073230 0x7fd410075740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd434160880 0x7fd4341511c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd43415d600 0x7fd434150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 --2- 172.21.15.16:0/3484568140 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd4340cf390 0x7fd4341526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 >> 172.21.15.16:0/3484568140 conn(0x7fd43406f900 msgr2=0x7fd4340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 shutdown_connections 2024-09-17T05:07:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.671+0000 7fd438e00640 1 -- 172.21.15.16:0/3484568140 wait complete. 2024-09-17T05:07:30.690 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:07:30.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:07:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 -- 172.21.15.16:0/890588111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c06ce90 msgr2=0x7fde0c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 --2- 172.21.15.16:0/890588111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c06ce90 0x7fde0c06d2a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fde04009c80 tx=0x7fde0401cce0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 -- 172.21.15.16:0/890588111 shutdown_connections 2024-09-17T05:07:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 --2- 172.21.15.16:0/890588111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c06ce90 0x7fde0c06d2a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 --2- 172.21.15.16:0/890588111 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0c070630 0x7fde0c0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 -- 172.21.15.16:0/890588111 >> 172.21.15.16:0/890588111 conn(0x7fde0c0cd980 msgr2=0x7fde0c0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 -- 172.21.15.16:0/890588111 shutdown_connections 2024-09-17T05:07:30.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.851+0000 7fde12800640 1 -- 172.21.15.16:0/890588111 wait complete. 2024-09-17T05:07:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 Processor -- start 2024-09-17T05:07:30.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 -- start start 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde0c06ce90 0x7fde0c14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 0x7fde0c14d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0c0c46d0 0x7fde0c14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c15d1a0 con 0x7fde0c0cb350 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c15d020 con 0x7fde0c06ce90 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde12800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c15d320 con 0x7fde0c0c46d0 2024-09-17T05:07:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde0aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 0x7fde0c14d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde0aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 0x7fde0c14d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47826/0 (socket says 172.21.15.16:47826) 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde0aa00640 1 -- 172.21.15.16:0/4293180159 learned_addr learned my addr 172.21.15.16:0/4293180159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde0be00640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0c0c46d0 0x7fde0c14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.855+0000 7fde0b400640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde0c06ce90 0x7fde0c14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde0aa00640 1 -- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0c0c46d0 msgr2=0x7fde0c14fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde0aa00640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0c0c46d0 0x7fde0c14fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde0aa00640 1 -- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde0c06ce90 msgr2=0x7fde0c14a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde0aa00640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde0c06ce90 0x7fde0c14a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde0aa00640 1 -- 172.21.15.16:0/4293180159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c14be90 con 0x7fde0c0cb350 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde0aa00640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 0x7fde0c14d710 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fddf800d750 tx=0x7fddf800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8017070 con 0x7fde0c0cb350 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf8009d20 con 0x7fde0c0cb350 2024-09-17T05:07:30.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c14f290 con 0x7fde0c0cb350 2024-09-17T05:07:30.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf800de60 con 0x7fde0c0cb350 2024-09-17T05:07:30.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c14f850 con 0x7fde0c0cb350 2024-09-17T05:07:30.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0c14bc00 con 0x7fde0c0cb350 2024-09-17T05:07:30.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fddf8004280 con 0x7fde0c0cb350 2024-09-17T05:07:30.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.859+0000 7fde03400640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddec073230 0x7fddec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:30.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.863+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7fddf80942c0 con 0x7fde0c0cb350 2024-09-17T05:07:30.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.863+0000 7fde0b400640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddec073230 0x7fddec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:30.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.863+0000 7fde0b400640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddec073230 0x7fddec075740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fddfc007fd0 tx=0x7fddfc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:30.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:30.867+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fddf8020020 con 0x7fde0c0cb350 2024-09-17T05:07:31.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.047+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fde0c14f080 con 0x7fde0c0cb350 2024-09-17T05:07:31.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde03400640 1 -- 172.21.15.16:0/4293180159 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fddf805ee90 con 0x7fde0c0cb350 2024-09-17T05:07:31.049 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:07:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddec073230 msgr2=0x7fddec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddec073230 0x7fddec075740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fddfc007fd0 tx=0x7fddfc009040 comp rx=0 tx=0).stop 2024-09-17T05:07:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 msgr2=0x7fde0c14d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 0x7fde0c14d710 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fddf800d750 tx=0x7fddf800dc10 comp rx=0 tx=0).stop 2024-09-17T05:07:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 shutdown_connections 2024-09-17T05:07:31.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddec073230 0x7fddec075740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0c0c46d0 0x7fde0c14fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0c0cb350 0x7fde0c14d710 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 --2- 172.21.15.16:0/4293180159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde0c06ce90 0x7fde0c14a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 >> 172.21.15.16:0/4293180159 conn(0x7fde0c0cd980 msgr2=0x7fde0c14c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 shutdown_connections 2024-09-17T05:07:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.051+0000 7fde12800640 1 -- 172.21.15.16:0/4293180159 wait complete. 2024-09-17T05:07:31.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-09-17T05:07:31.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:07:31.207 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:07:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 -- 172.21.15.16:0/1504248002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 msgr2=0x7f36bc0c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 --2- 172.21.15.16:0/1504248002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc0c9850 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0c92f0 tx=0x7f36b401cd80 comp rx=0 tx=0).stop 2024-09-17T05:07:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 -- 172.21.15.16:0/1504248002 shutdown_connections 2024-09-17T05:07:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 --2- 172.21.15.16:0/1504248002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc0c9850 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 --2- 172.21.15.16:0/1504248002 >> v2:172.21.15.104:3300/0 conn(0x7f36bc0cc8c0 0x7f36bc15cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 -- 172.21.15.16:0/1504248002 >> 172.21.15.16:0/1504248002 conn(0x7f36bc0d2270 msgr2=0x7f36bc0d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 -- 172.21.15.16:0/1504248002 shutdown_connections 2024-09-17T05:07:31.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 -- 172.21.15.16:0/1504248002 wait complete. 2024-09-17T05:07:31.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 Processor -- start 2024-09-17T05:07:31.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.231+0000 7f36c1000640 1 -- start start 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f36bc0cc8c0 0x7f36bc14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc0d1580 0x7f36bc139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc169eb0 con 0x7f36bc0c9440 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc169d30 con 0x7f36bc0d1580 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f36bc16a030 con 0x7f36bc0cc8c0 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47848/0 (socket says 172.21.15.16:47848) 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 -- 172.21.15.16:0/1886274805 learned_addr learned my addr 172.21.15.16:0/1886274805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:31.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b1400640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f36bc0cc8c0 0x7f36bc14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36ba800640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc0d1580 0x7f36bc139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 -- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f36bc0cc8c0 msgr2=0x7f36bc14dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f36bc0cc8c0 0x7f36bc14dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 -- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc0d1580 msgr2=0x7f36bc139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc0d1580 0x7f36bc139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 -- 172.21.15.16:0/1886274805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc071aa0 con 0x7f36bc0c9440 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36ba800640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc0d1580 0x7f36bc139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b1400640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f36bc0cc8c0 0x7f36bc14dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b9e00640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc15a260 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00e790 tx=0x7f36ac00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00eea0 con 0x7f36bc0c9440 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36ac00cce0 con 0x7f36bc0c9440 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc1591c0 con 0x7f36bc0c9440 2024-09-17T05:07:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004290 con 0x7f36bc0c9440 2024-09-17T05:07:31.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.235+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc159780 con 0x7f36bc0c9440 2024-09-17T05:07:31.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.239+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc14e500 con 0x7f36bc0c9440 2024-09-17T05:07:31.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.239+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36ac003600 con 0x7f36bc0c9440 2024-09-17T05:07:31.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.239+0000 7f36b2a00640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3698073230 0x7f3698075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:31.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.239+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f36ac094aa0 con 0x7f36bc0c9440 2024-09-17T05:07:31.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.239+0000 7f36b1400640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3698073230 0x7f3698075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:31.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.239+0000 7f36b1400640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3698073230 0x7f3698075740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0d0700 tx=0x7f36a4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.243+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36bc14e500 con 0x7f36bc0c9440 2024-09-17T05:07:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:31.483+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f36bc0c9850 con 0x7f36bc0c9440 2024-09-17T05:07:32.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.299+0000 7f36b2a00640 1 -- 172.21.15.16:0/1886274805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v318) ==== 153+0+0 (secure 0 0 0) 0x7f36ac05fb90 con 0x7f36bc0c9440 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3698073230 msgr2=0x7f3698075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3698073230 0x7f3698075740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0d0700 tx=0x7f36a4009040 comp rx=0 tx=0).stop 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 msgr2=0x7f36bc15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc15a260 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00e790 tx=0x7f36ac00ec50 comp rx=0 tx=0).stop 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 shutdown_connections 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3698073230 0x7f3698075740 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc0d1580 0x7f36bc139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f36bc0cc8c0 0x7f36bc14dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 --2- 172.21.15.16:0/1886274805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36bc0c9440 0x7f36bc15a260 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 >> 172.21.15.16:0/1886274805 conn(0x7f36bc0d2270 msgr2=0x7f36bc153820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 shutdown_connections 2024-09-17T05:07:32.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.303+0000 7f36c1000640 1 -- 172.21.15.16:0/1886274805 wait complete. 2024-09-17T05:07:32.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:07:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 -- 172.21.15.16:0/3180128356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 msgr2=0x7f920015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 --2- 172.21.15.16:0/3180128356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f920015c4b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009900 tx=0x7f91f001c850 comp rx=0 tx=0).stop 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 -- 172.21.15.16:0/3180128356 shutdown_connections 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 --2- 172.21.15.16:0/3180128356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f920015c4b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 --2- 172.21.15.16:0/3180128356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f92000b2f50 0x7f920015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 -- 172.21.15.16:0/3180128356 >> 172.21.15.16:0/3180128356 conn(0x7f920006f900 msgr2=0x7f92000b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 -- 172.21.15.16:0/3180128356 shutdown_connections 2024-09-17T05:07:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.491+0000 7f9208800640 1 -- 172.21.15.16:0/3180128356 wait complete. 2024-09-17T05:07:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 Processor -- start 2024-09-17T05:07:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 -- start start 2024-09-17T05:07:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92000b2f50 0x7f9200158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f9200143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f920015f150 0x7f920015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9200169120 con 0x7f920015c0a0 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9200168fa0 con 0x7f92000b2f50 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9208800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f92001692a0 con 0x7f920015f150 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f9200143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f9200143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47864/0 (socket says 172.21.15.16:47864) 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 -- 172.21.15.16:0/3796115597 learned_addr learned my addr 172.21.15.16:0/3796115597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205e00640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92000b2f50 0x7f9200158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9206800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f920015f150 0x7f920015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 -- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f920015f150 msgr2=0x7f920015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f920015f150 0x7f920015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 -- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92000b2f50 msgr2=0x7f9200158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92000b2f50 0x7f9200158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 -- 172.21.15.16:0/3796115597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9200143ea0 con 0x7f920015c0a0 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.495+0000 7f9205400640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f9200143960 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f91f00b0040 tx=0x7f91f00a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f00b2040 con 0x7f920015c0a0 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91f00044b0 con 0x7f920015c0a0 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920015ae20 con 0x7f920015c0a0 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0005280 con 0x7f920015c0a0 2024-09-17T05:07:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920015b3e0 con 0x7f920015c0a0 2024-09-17T05:07:32.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92001587f0 con 0x7f920015c0a0 2024-09-17T05:07:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91f0005420 con 0x7f920015c0a0 2024-09-17T05:07:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f91fa000640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91e4073200 0x7f91e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f91f00bd070 con 0x7f920015c0a0 2024-09-17T05:07:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.499+0000 7f9205e00640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91e4073200 0x7f91e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.503+0000 7f9205e00640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91e4073200 0x7f91e4075710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f91f4004630 tx=0x7f91f400b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:32.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.507+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f91f00164d0 con 0x7f920015c0a0 2024-09-17T05:07:32.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.679+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f920015bb60 con 0x7f920015c0a0 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f91fa000640 1 -- 172.21.15.16:0/3796115597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v48) ==== 59+0+2294 (secure 0 0 0) 0x7f91f00f8f60 con 0x7f920015c0a0 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:07:32.681 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:07:32.682 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:07:32.683 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:07:32.684 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:07:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91e4073200 msgr2=0x7f91e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91e4073200 0x7f91e4075710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f91f4004630 tx=0x7f91f400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 msgr2=0x7f9200143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f9200143960 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f91f00b0040 tx=0x7f91f00a5860 comp rx=0 tx=0).stop 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 shutdown_connections 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f91e4073200 0x7f91e4075710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f920015f150 0x7f920015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f920015c0a0 0x7f9200143960 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 --2- 172.21.15.16:0/3796115597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92000b2f50 0x7f9200158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 >> 172.21.15.16:0/3796115597 conn(0x7f920006f900 msgr2=0x7f920015f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 shutdown_connections 2024-09-17T05:07:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.683+0000 7f9208800640 1 -- 172.21.15.16:0/3796115597 wait complete. 2024-09-17T05:07:32.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:07:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 -- 172.21.15.16:0/2625438803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074a20 msgr2=0x7f959c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 --2- 172.21.15.16:0/2625438803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074a20 0x7f959c15efe0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f959c0ce2c0 tx=0x7f958c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 -- 172.21.15.16:0/2625438803 shutdown_connections 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 --2- 172.21.15.16:0/2625438803 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f959c15f520 0x7f959c15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 --2- 172.21.15.16:0/2625438803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074a20 0x7f959c15efe0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 -- 172.21.15.16:0/2625438803 >> 172.21.15.16:0/2625438803 conn(0x7f959c0b2250 msgr2=0x7f959c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 -- 172.21.15.16:0/2625438803 shutdown_connections 2024-09-17T05:07:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 -- 172.21.15.16:0/2625438803 wait complete. 2024-09-17T05:07:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 Processor -- start 2024-09-17T05:07:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.867+0000 7f95a1400640 1 -- start start 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f959c074a20 0x7f959c1fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 0x7f959c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f959c15f520 0x7f959c153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f959c168e80 con 0x7f959c074e30 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f959c168d00 con 0x7f959c074a20 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f959c169000 con 0x7f959c15f520 2024-09-17T05:07:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 0x7f959c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 0x7f959c0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47886/0 (socket says 172.21.15.16:47886) 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 -- 172.21.15.16:0/2861519451 learned_addr learned my addr 172.21.15.16:0/2861519451 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f959b200640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f959c15f520 0x7f959c153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f959a800640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f959c074a20 0x7f959c1fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 -- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f959c15f520 msgr2=0x7f959c153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f959c15f520 0x7f959c153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 -- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f959c074a20 msgr2=0x7f959c1fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f959c074a20 0x7f959c1fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 -- 172.21.15.16:0/2861519451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c1fc400 con 0x7f959c074e30 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f959a800640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f959c074a20 0x7f959c1fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9591e00640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 0x7f959c0ccc80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f958400c3a0 tx=0x7f958400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584017070 con 0x7f959c074e30 2024-09-17T05:07:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95840053e0 con 0x7f959c074e30 2024-09-17T05:07:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9584003880 con 0x7f959c074e30 2024-09-17T05:07:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c1fc6e0 con 0x7f959c074e30 2024-09-17T05:07:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c1fcca0 con 0x7f959c074e30 2024-09-17T05:07:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c15efe0 con 0x7f959c074e30 2024-09-17T05:07:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.871+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f958401a070 con 0x7f959c074e30 2024-09-17T05:07:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.875+0000 7f9593400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9578073200 0x7f9578075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.875+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f95840934a0 con 0x7f959c074e30 2024-09-17T05:07:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.875+0000 7f959a800640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9578073200 0x7f9578075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.875+0000 7f959a800640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9578073200 0x7f9578075710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f958c0af000 tx=0x7f958c01b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:32.879+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f958405e620 con 0x7f959c074e30 2024-09-17T05:07:33.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.055+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f959c152270 con 0x7f959c074e30 2024-09-17T05:07:33.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.059+0000 7f9593400640 1 -- 172.21.15.16:0/2861519451 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v49)=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f958405df20 con 0x7f959c074e30 2024-09-17T05:07:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9578073200 msgr2=0x7f9578075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9578073200 0x7f9578075710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f958c0af000 tx=0x7f958c01b120 comp rx=0 tx=0).stop 2024-09-17T05:07:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 msgr2=0x7f959c0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 0x7f959c0ccc80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f958400c3a0 tx=0x7f958400c860 comp rx=0 tx=0).stop 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 shutdown_connections 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9578073200 0x7f9578075710 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f959c15f520 0x7f959c153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f959c074e30 0x7f959c0ccc80 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 --2- 172.21.15.16:0/2861519451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f959c074a20 0x7f959c1fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 >> 172.21.15.16:0/2861519451 conn(0x7f959c0b2250 msgr2=0x7f959c0cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 shutdown_connections 2024-09-17T05:07:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.063+0000 7f95a1400640 1 -- 172.21.15.16:0/2861519451 wait complete. 2024-09-17T05:07:33.081 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:07:33.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:07:33.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:07:33.267 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:07:33.267 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:33.267 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:33.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:07:33.276 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:33.277 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:33.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:07:33.286 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:33.286 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:33.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:07:33.329 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:33.329 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:33.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:07:33.338 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:07:33.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3f200640 1 -- 172.21.15.16:0/1697520654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c004810 msgr2=0x7f7f1c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3f200640 1 --2- 172.21.15.16:0/1697520654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c004810 0x7f7f1c004c20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009b40 tx=0x7f7f2001d1d0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3d200640 1 -- 172.21.15.16:0/1697520654 reap_dead start 2024-09-17T05:07:33.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3f200640 1 -- 172.21.15.16:0/1697520654 shutdown_connections 2024-09-17T05:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3f200640 1 -- 172.21.15.16:0/1697520654 >> 172.21.15.16:0/1697520654 conn(0x7f7f380d0860 msgr2=0x7f7f380d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3f200640 1 -- 172.21.15.16:0/1697520654 shutdown_connections 2024-09-17T05:07:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.535+0000 7f7f3f200640 1 -- 172.21.15.16:0/1697520654 wait complete. 2024-09-17T05:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 Processor -- start 2024-09-17T05:07:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 -- start start 2024-09-17T05:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f1c004810 0x7f7f380c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f38155070 0x7f7f381f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 0x7f7f381f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7f380cfe80 con 0x7f7f38155480 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7f38073f30 con 0x7f7f38155070 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7f380d0000 con 0x7f7f1c004810 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 0x7f7f381f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 0x7f7f381f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47904/0 (socket says 172.21.15.16:47904) 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 -- 172.21.15.16:0/305673462 learned_addr learned my addr 172.21.15.16:0/305673462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f37e00640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f1c004810 0x7f7f380c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f2fe00640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f38155070 0x7f7f381f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 -- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f1c004810 msgr2=0x7f7f380c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f1c004810 0x7f7f380c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 -- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f38155070 msgr2=0x7f7f381f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f38155070 0x7f7f381f6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 -- 172.21.15.16:0/305673462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f380c5d40 con 0x7f7f38155480 2024-09-17T05:07:33.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f2fe00640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f38155070 0x7f7f381f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f3d200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 0x7f7f381f6a60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7f38072820 tx=0x7f7f30006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.539+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f28001480 con 0x7f7f38155480 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f30017070 con 0x7f7f38155480 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000fce0 con 0x7f7f38155480 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f380ac0d0 con 0x7f7f38155480 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f380ac690 con 0x7f7f38155480 2024-09-17T05:07:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f380cef00 con 0x7f7f38155480 2024-09-17T05:07:33.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7f3001c460 con 0x7f7f38155480 2024-09-17T05:07:33.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f34c00640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f14073200 0x7f7f14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f7f30094670 con 0x7f7f38155480 2024-09-17T05:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.543+0000 7f7f37e00640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f14073200 0x7f7f14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:33.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.547+0000 7f7f37e00640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f14073200 0x7f7f14075710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006fd0 tx=0x7f7f28006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:33.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.547+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7f3005f870 con 0x7f7f38155480 2024-09-17T05:07:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f3806dec0 con 0x7f7f38155480 2024-09-17T05:07:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f34c00640 1 -- 172.21.15.16:0/305673462 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3914 (secure 0 0 0) 0x7f7f3005f170 con 0x7f7f38155480 2024-09-17T05:07:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:33.758 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":59,"btime":"2024-09-17T05:07:27:267305+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5687,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/4194951369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":4194951369},{"type":"v1","addr":"172.21.15.16:6835","nonce":4194951369}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/802276404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":802276404},{"type":"v1","addr":"172.21.15.104:6837","nonce":802276404}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2977254500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2977254500},{"type":"v1","addr":"172.21.15.104:6835","nonce":2977254500}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2016225883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2016225883},{"type":"v1","addr":"172.21.15.16:6834","nonce":2016225883}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-17T05:07:33.758 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 59 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f14073200 msgr2=0x7f7f14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f14073200 0x7f7f14075710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7f28006fd0 tx=0x7f7f28006b20 comp rx=0 tx=0).stop 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 msgr2=0x7f7f381f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 0x7f7f381f6a60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f7f38072820 tx=0x7f7f30006660 comp rx=0 tx=0).stop 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 shutdown_connections 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7f14073200 0x7f7f14075710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f38155480 0x7f7f381f6a60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7f38155070 0x7f7f381f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 --2- 172.21.15.16:0/305673462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7f1c004810 0x7f7f380c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 >> 172.21.15.16:0/305673462 conn(0x7f7f380d0860 msgr2=0x7f7f38073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:33.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 shutdown_connections 2024-09-17T05:07:33.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:33.759+0000 7f7f3f200640 1 -- 172.21.15.16:0/305673462 wait complete. 2024-09-17T05:07:33.780 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:07:33.780 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:07:33.780 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:07:33.780 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:07:33.781 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:07:33.781 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:07:33.781 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:07:33.781 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:07:33.781 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:07:33.781 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:07:33.782 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:07:33.782 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:07:33.782 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:07:33.782 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:07:33.782 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:07:33.782 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:07:33.793 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:07:33.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:07:33.798 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:07:33.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:07:33.799 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:07:33.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:07:33.801 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:07:33.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:07:33.843 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:07:33.843 DEBUG:teuthology.parallel:result is None 2024-09-17T05:07:33.844 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:07:33.845 DEBUG:teuthology.parallel:result is None 2024-09-17T05:07:33.846 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:07:33.846 DEBUG:teuthology.parallel:result is None 2024-09-17T05:07:33.848 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:07:33.848 DEBUG:teuthology.parallel:result is None 2024-09-17T05:07:33.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:07:33.945 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:07:33.946 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:07:33.948 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:07:33.949 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:07:34.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2731157145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc06fe40 msgr2=0x7f8dbc072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2731157145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc06fe40 0x7f8dbc072c20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009840 tx=0x7f8da801c790 comp rx=0 tx=0).stop 2024-09-17T05:07:34.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2731157145 shutdown_connections 2024-09-17T05:07:34.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2731157145 >> v2:172.21.15.104:3300/0 conn(0x7f8dbc0b34c0 0x7f8dbc0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2731157145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc06fe40 0x7f8dbc072c20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2731157145 >> 172.21.15.16:0/2731157145 conn(0x7f8dbc0d05a0 msgr2=0x7f8dbc06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2731157145 shutdown_connections 2024-09-17T05:07:34.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.015+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2731157145 wait complete. 2024-09-17T05:07:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 Processor -- start 2024-09-17T05:07:34.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 -- start start 2024-09-17T05:07:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dbc0b34c0 0x7f8dbc1f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 0x7f8dbc140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1030 0x7f8dbc0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc165720 con 0x7f8dbc0b34c0 2024-09-17T05:07:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc0b4710 con 0x7f8dbc0b1030 2024-09-17T05:07:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8dbc1658a0 con 0x7f8dbc15bc10 2024-09-17T05:07:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dbc0b34c0 0x7f8dbc1f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 0x7f8dbc140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 0x7f8dbc140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:57468/0 (socket says 172.21.15.16:57468) 2024-09-17T05:07:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 -- 172.21.15.16:0/2627993492 learned_addr learned my addr 172.21.15.16:0/2627993492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2400640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1030 0x7f8dbc0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 -- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1030 msgr2=0x7f8dbc0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1030 0x7f8dbc0b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 -- 172.21.15.16:0/2627993492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dbc0b34c0 msgr2=0x7f8dbc1f9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dbc0b34c0 0x7f8dbc1f9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 -- 172.21.15.16:0/2627993492 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc1fa200 con 0x7f8dbc15bc10 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1a00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dbc0b34c0 0x7f8dbc1f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc1000640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 0x7f8dbc140cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009790 tx=0x7f8da80a6740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8004340 con 0x7f8dbc15bc10 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8da80bd070 con 0x7f8dbc15bc10 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc1411f0 con 0x7f8dbc15bc10 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8003860 con 0x7f8dbc15bc10 2024-09-17T05:07:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.019+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc1500f0 con 0x7f8dbc15bc10 2024-09-17T05:07:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.023+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc072c20 con 0x7f8dbc15bc10 2024-09-17T05:07:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.023+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8da8003a00 con 0x7f8dbc15bc10 2024-09-17T05:07:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.023+0000 7f8db9600640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8d9c073200 0x7f8d9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.023+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5335+0+0 (secure 0 0 0) 0x7f8da812efb0 con 0x7f8dbc15bc10 2024-09-17T05:07:34.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.023+0000 7f8dc1a00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8d9c073200 0x7f8d9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.027+0000 7f8dc1a00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8d9c073200 0x7f8d9c075710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8db0007fd0 tx=0x7f8db0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:34.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.027+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8da80c7020 con 0x7f8dbc15bc10 2024-09-17T05:07:34.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.211+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8dbc14edb0 con 0x7f8dbc15bc10 2024-09-17T05:07:34.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.323+0000 7f8db9600640 1 -- 172.21.15.16:0/2627993492 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) ==== 97+0+0 (secure 0 0 0) 0x7f8dbc1fa200 con 0x7f8dbc15bc10 2024-09-17T05:07:34.324 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8d9c073200 msgr2=0x7f8d9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8d9c073200 0x7f8d9c075710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8db0007fd0 tx=0x7f8db0009040 comp rx=0 tx=0).stop 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 msgr2=0x7f8dbc140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 0x7f8dbc140cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009790 tx=0x7f8da80a6740 comp rx=0 tx=0).stop 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 shutdown_connections 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8d9c073200 0x7f8d9c075710 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0b1030 0x7f8dbc0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8dbc15bc10 0x7f8dbc140cb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 --2- 172.21.15.16:0/2627993492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dbc0b34c0 0x7f8dbc1f9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 >> 172.21.15.16:0/2627993492 conn(0x7f8dbc0d05a0 msgr2=0x7f8dbc06de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 shutdown_connections 2024-09-17T05:07:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.327+0000 7f8dc2e00640 1 -- 172.21.15.16:0/2627993492 wait complete. 2024-09-17T05:07:34.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:07:34.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.507+0000 7f5e64c00640 1 -- 172.21.15.16:0/2727044247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 msgr2=0x7f5e4400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.507+0000 7f5e64c00640 1 --2- 172.21.15.16:0/2727044247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e4400b9d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5e601561d0 tx=0x7f5e5801d4f0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.507+0000 7f5e5e800640 1 -- 172.21.15.16:0/2727044247 reap_dead start 2024-09-17T05:07:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.507+0000 7f5e64c00640 1 -- 172.21.15.16:0/2727044247 shutdown_connections 2024-09-17T05:07:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.507+0000 7f5e64c00640 1 -- 172.21.15.16:0/2727044247 >> 172.21.15.16:0/2727044247 conn(0x7f5e6006f900 msgr2=0x7f5e600b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 -- 172.21.15.16:0/2727044247 shutdown_connections 2024-09-17T05:07:34.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 -- 172.21.15.16:0/2727044247 wait complete. 2024-09-17T05:07:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 Processor -- start 2024-09-17T05:07:34.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 -- start start 2024-09-17T05:07:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e6013dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e60140f50 0x7f5e60154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e60155190 0x7f5e601f8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60073ce0 con 0x7f5e44005330 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e60073b60 con 0x7f5e60155190 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.511+0000 7f5e64c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5e60073e60 con 0x7f5e60140f50 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e6013dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e60140f50 0x7f5e60154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e6013dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47946/0 (socket says 172.21.15.16:47946) 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e57e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e60140f50 0x7f5e60154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:57498/0 (socket says 172.21.15.16:57498) 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 -- 172.21.15.16:0/3447375956 learned_addr learned my addr 172.21.15.16:0/3447375956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5e800640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e60155190 0x7f5e601f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 -- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e60140f50 msgr2=0x7f5e60154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e60140f50 0x7f5e60154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 -- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e60155190 msgr2=0x7f5e601f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e60155190 0x7f5e601f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 -- 172.21.15.16:0/3447375956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e601555a0 con 0x7f5e44005330 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e5de00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e6013dd20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5e500017e0 tx=0x7f5e50003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50015070 con 0x7f5e44005330 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e5000f960 con 0x7f5e44005330 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000fd90 con 0x7f5e44005330 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e601e65c0 con 0x7f5e44005330 2024-09-17T05:07:34.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e601e6b80 con 0x7f5e44005330 2024-09-17T05:07:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e60074330 con 0x7f5e44005330 2024-09-17T05:07:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.515+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e50011430 con 0x7f5e44005330 2024-09-17T05:07:34.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.519+0000 7f5e56000640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e3c073230 0x7f5e3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.519+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f5e50094d40 con 0x7f5e44005330 2024-09-17T05:07:34.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.519+0000 7f5e57e00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e3c073230 0x7f5e3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.519+0000 7f5e57e00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e3c073230 0x7f5e3c075740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5e601f87a0 tx=0x7f5e48006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:34.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.523+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e601555a0 con 0x7f5e44005330 2024-09-17T05:07:34.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.763+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5e6013e610 con 0x7f5e44005330 2024-09-17T05:07:34.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.763+0000 7f5e56000640 1 -- 172.21.15.16:0/3447375956 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f5e5005f990 con 0x7f5e44005330 2024-09-17T05:07:34.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:34.763 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:07:34.763 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:07:34.763 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:07:34.764 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:07:34.765 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.766 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:07:34.767 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.768 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:07:34.769 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:07:34.771 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:07:34.772 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.773 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:07:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.776 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:34.778 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:07:34.778 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e3c073230 msgr2=0x7f5e3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e3c073230 0x7f5e3c075740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5e601f87a0 tx=0x7f5e48006b20 comp rx=0 tx=0).stop 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 msgr2=0x7f5e6013dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e6013dd20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5e500017e0 tx=0x7f5e50003a90 comp rx=0 tx=0).stop 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 shutdown_connections 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e3c073230 0x7f5e3c075740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e60155190 0x7f5e601f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e60140f50 0x7f5e60154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 --2- 172.21.15.16:0/3447375956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e44005330 0x7f5e6013dd20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 >> 172.21.15.16:0/3447375956 conn(0x7f5e6006f900 msgr2=0x7f5e600ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 shutdown_connections 2024-09-17T05:07:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.767+0000 7f5e64c00640 1 -- 172.21.15.16:0/3447375956 wait complete. 2024-09-17T05:07:34.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:07:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 -- 172.21.15.16:0/3100683915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c680 msgr2=0x7f0928073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 --2- 172.21.15.16:0/3100683915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c680 0x7f0928073950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0920009bc0 tx=0x7f092001cb10 comp rx=0 tx=0).stop 2024-09-17T05:07:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 -- 172.21.15.16:0/3100683915 shutdown_connections 2024-09-17T05:07:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 --2- 172.21.15.16:0/3100683915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c680 0x7f0928073950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 -- 172.21.15.16:0/3100683915 >> 172.21.15.16:0/3100683915 conn(0x7f092806f900 msgr2=0x7f09280b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 -- 172.21.15.16:0/3100683915 shutdown_connections 2024-09-17T05:07:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 -- 172.21.15.16:0/3100683915 wait complete. 2024-09-17T05:07:34.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 Processor -- start 2024-09-17T05:07:34.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.947+0000 7f092e400640 1 -- start start 2024-09-17T05:07:34.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09280b2f50 0x7f09281f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c270 0x7f09281475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 0x7f092814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0928169b60 con 0x7f092815c680 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09281699e0 con 0x7f092815c270 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0928169ce0 con 0x7f09280b2f50 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 0x7f092814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 0x7f092814a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47948/0 (socket says 172.21.15.16:47948) 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 -- 172.21.15.16:0/2233506504 learned_addr learned my addr 172.21.15.16:0/2233506504 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f091fe00640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c270 0x7f09281475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09280b2f50 0x7f09281f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 -- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09280b2f50 msgr2=0x7f09281f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09280b2f50 0x7f09281f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 -- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c270 msgr2=0x7f09281475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c270 0x7f09281475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 -- 172.21.15.16:0/2233506504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092814c7f0 con 0x7f092815c680 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09280b2f50 0x7f09281f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f0927e00640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 0x7f092814a790 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0920019720 tx=0x7f0920009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200b2040 con 0x7f092815c680 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0920004d40 con 0x7f092815c680 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920004260 con 0x7f092815c680 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09281661f0 con 0x7f092815c680 2024-09-17T05:07:34.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09281667b0 con 0x7f092815c680 2024-09-17T05:07:34.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.951+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092806dab0 con 0x7f092815c680 2024-09-17T05:07:34.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.955+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f09200be060 con 0x7f092815c680 2024-09-17T05:07:34.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.955+0000 7f091f400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0904073200 0x7f0904075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:34.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.955+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f092012dca0 con 0x7f092815c680 2024-09-17T05:07:34.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.955+0000 7f0927400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0904073200 0x7f0904075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:34.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.955+0000 7f0927400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0904073200 0x7f0904075710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0918000f70 tx=0x7f0918009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:34.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:34.959+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0920016790 con 0x7f092815c680 2024-09-17T05:07:35.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f092806d7d0 con 0x7f092815c680 2024-09-17T05:07:35.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f091f400640 1 -- 172.21.15.16:0/2233506504 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) ==== 84+0+65 (secure 0 0 0) 0x7f09200f8f10 con 0x7f092815c680 2024-09-17T05:07:35.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:35.133 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:07:35.134 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:07:35.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:07:35.134 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:07:35.134 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:07:35.134 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:07:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0904073200 msgr2=0x7f0904075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0904073200 0x7f0904075710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0918000f70 tx=0x7f0918009040 comp rx=0 tx=0).stop 2024-09-17T05:07:35.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 msgr2=0x7f092814a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 0x7f092814a790 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0920019720 tx=0x7f0920009200 comp rx=0 tx=0).stop 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 shutdown_connections 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0904073200 0x7f0904075710 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f092815c680 0x7f092814a790 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092815c270 0x7f09281475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 --2- 172.21.15.16:0/2233506504 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09280b2f50 0x7f09281f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 >> 172.21.15.16:0/2233506504 conn(0x7f092806f900 msgr2=0x7f09280cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 shutdown_connections 2024-09-17T05:07:35.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.135+0000 7f092e400640 1 -- 172.21.15.16:0/2233506504 wait complete. 2024-09-17T05:07:35.156 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:07:35.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:07:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 -- 172.21.15.16:0/2107561770 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56940d1c70 msgr2=0x7f569415c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 --2- 172.21.15.16:0/2107561770 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56940d1c70 0x7f569415c7d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56800098a0 tx=0x7f568001c7b0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 -- 172.21.15.16:0/2107561770 shutdown_connections 2024-09-17T05:07:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 --2- 172.21.15.16:0/2107561770 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56940d1c70 0x7f569415c7d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 -- 172.21.15.16:0/2107561770 >> 172.21.15.16:0/2107561770 conn(0x7f56940d22a0 msgr2=0x7f56940d26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:35.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 -- 172.21.15.16:0/2107561770 shutdown_connections 2024-09-17T05:07:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 -- 172.21.15.16:0/2107561770 wait complete. 2024-09-17T05:07:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 Processor -- start 2024-09-17T05:07:35.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.323+0000 7f569a400640 1 -- start start 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 0x7f5694153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56940cd270 0x7f56941578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5694157e10 0x7f5694150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f569416b250 con 0x7f56940cb0f0 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f569416b0d0 con 0x7f56940cd270 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f569416b3d0 con 0x7f5694157e10 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 0x7f5694153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 0x7f5694153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47960/0 (socket says 172.21.15.16:47960) 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 -- 172.21.15.16:0/1904588270 learned_addr learned my addr 172.21.15.16:0/1904588270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:35.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5692a00640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56940cd270 0x7f56941578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:35.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693e00640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5694157e10 0x7f5694150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 -- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5694157e10 msgr2=0x7f5694150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5694157e10 0x7f5694150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 -- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56940cd270 msgr2=0x7f56941578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56940cd270 0x7f56941578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 -- 172.21.15.16:0/1904588270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569415fd20 con 0x7f56940cb0f0 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f5693400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 0x7f5694153bc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f568400c460 tx=0x7f568400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684017070 con 0x7f56940cb0f0 2024-09-17T05:07:35.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56840053c0 con 0x7f56940cb0f0 2024-09-17T05:07:35.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684003860 con 0x7f56940cb0f0 2024-09-17T05:07:35.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5694152430 con 0x7f56940cb0f0 2024-09-17T05:07:35.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941529f0 con 0x7f56940cb0f0 2024-09-17T05:07:35.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569415c7d0 con 0x7f56940cb0f0 2024-09-17T05:07:35.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.331+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f568401a070 con 0x7f56940cb0f0 2024-09-17T05:07:35.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.331+0000 7f568b400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:35.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.331+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5249+0+0 (secure 0 0 0) 0x7f56840933d0 con 0x7f56940cb0f0 2024-09-17T05:07:35.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.331+0000 7f5692a00640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:35.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.331+0000 7f5692a00640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f56800af000 tx=0x7f568001aff0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:35.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.335+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f568405dec0 con 0x7f56940cb0f0 2024-09-17T05:07:35.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:35.511+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f56940c9b60 con 0x7f56940cb0f0 2024-09-17T05:07:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.323+0000 7f568b400640 1 -- 172.21.15.16:0/1904588270 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) ==== 138+0+0 (secure 0 0 0) 0x7f56840628f0 con 0x7f56940cb0f0 2024-09-17T05:07:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:07:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 msgr2=0x7f5674075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f56800af000 tx=0x7f568001aff0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 msgr2=0x7f5694153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 0x7f5694153bc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f568400c460 tx=0x7f568400c920 comp rx=0 tx=0).stop 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 shutdown_connections 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5694157e10 0x7f5694150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56940cd270 0x7f56941578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 --2- 172.21.15.16:0/1904588270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56940cb0f0 0x7f5694153bc0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 >> 172.21.15.16:0/1904588270 conn(0x7f56940d22a0 msgr2=0x7f56941542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 shutdown_connections 2024-09-17T05:07:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.327+0000 7f569a400640 1 -- 172.21.15.16:0/1904588270 wait complete. 2024-09-17T05:07:36.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:07:36.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 -- 172.21.15.16:0/235135789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980748d0 msgr2=0x7efd98074ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 --2- 172.21.15.16:0/235135789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980748d0 0x7efd98074ce0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7efd98158880 tx=0x7efd8801c770 comp rx=0 tx=0).stop 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 -- 172.21.15.16:0/235135789 shutdown_connections 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 --2- 172.21.15.16:0/235135789 >> v2:172.21.15.104:3300/0 conn(0x7efd9815e2a0 0x7efd9815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 --2- 172.21.15.16:0/235135789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980748d0 0x7efd98074ce0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 -- 172.21.15.16:0/235135789 >> 172.21.15.16:0/235135789 conn(0x7efd9806f900 msgr2=0x7efd980b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 -- 172.21.15.16:0/235135789 shutdown_connections 2024-09-17T05:07:36.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.527+0000 7efd9ea00640 1 -- 172.21.15.16:0/235135789 wait complete. 2024-09-17T05:07:36.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 Processor -- start 2024-09-17T05:07:36.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 -- start start 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd980748d0 0x7efd98162210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 0x7efd981f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd9815e2a0 0x7efd981fa360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd9806da30 con 0x7efd980cf140 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efd9806d8b0 con 0x7efd980748d0 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efd9806dbb0 con 0x7efd9815e2a0 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 0x7efd981f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 0x7efd981f9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47968/0 (socket says 172.21.15.16:47968) 2024-09-17T05:07:36.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 -- 172.21.15.16:0/565903613 learned_addr learned my addr 172.21.15.16:0/565903613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97e00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd980748d0 0x7efd98162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd9ca00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd9815e2a0 0x7efd981fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 -- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd9815e2a0 msgr2=0x7efd981fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd9815e2a0 0x7efd981fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 -- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd980748d0 msgr2=0x7efd98162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd980748d0 0x7efd98162210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:36.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.531+0000 7efd97400640 1 -- 172.21.15.16:0/565903613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd98162750 con 0x7efd980cf140 2024-09-17T05:07:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd97400640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 0x7efd981f9e20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efd840058d0 tx=0x7efd8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8401b070 con 0x7efd980cf140 2024-09-17T05:07:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd8400ae50 con 0x7efd980cf140 2024-09-17T05:07:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8400e9c0 con 0x7efd980cf140 2024-09-17T05:07:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd981525b0 con 0x7efd980cf140 2024-09-17T05:07:36.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd98152b70 con 0x7efd980cf140 2024-09-17T05:07:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd98152320 con 0x7efd980cf140 2024-09-17T05:07:36.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd8401e070 con 0x7efd980cf140 2024-09-17T05:07:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.535+0000 7efd8fe00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd78073200 0x7efd78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.539+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5619+0+0 (secure 0 0 0) 0x7efd840944e0 con 0x7efd980cf140 2024-09-17T05:07:36.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.539+0000 7efd97e00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd78073200 0x7efd78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:36.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.539+0000 7efd97e00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd78073200 0x7efd78075710 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efd880af000 tx=0x7efd8801afb0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:36.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.543+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd8400de60 con 0x7efd980cf140 2024-09-17T05:07:36.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:36.719+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efd9815e6b0 con 0x7efd980cf140 2024-09-17T05:07:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.335+0000 7efd8fe00640 1 -- 172.21.15.16:0/565903613 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) ==== 170+0+0 (secure 0 0 0) 0x7efd8405ef00 con 0x7efd980cf140 2024-09-17T05:07:37.334 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:07:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd78073200 msgr2=0x7efd78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd78073200 0x7efd78075710 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efd880af000 tx=0x7efd8801afb0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 msgr2=0x7efd981f9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:37.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 0x7efd981f9e20 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efd840058d0 tx=0x7efd8400a620 comp rx=0 tx=0).stop 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 shutdown_connections 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efd78073200 0x7efd78075710 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efd9815e2a0 0x7efd981fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd980cf140 0x7efd981f9e20 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 --2- 172.21.15.16:0/565903613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd980748d0 0x7efd98162210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 >> 172.21.15.16:0/565903613 conn(0x7efd9806f900 msgr2=0x7efd9814f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 shutdown_connections 2024-09-17T05:07:37.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.339+0000 7efd9ea00640 1 -- 172.21.15.16:0/565903613 wait complete. 2024-09-17T05:07:37.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 -- 172.21.15.16:0/1336232082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a806f270 msgr2=0x7f91a806f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 --2- 172.21.15.16:0/1336232082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a806f270 0x7f91a806f680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009bc0 tx=0x7f91a001ca60 comp rx=0 tx=0).stop 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 -- 172.21.15.16:0/1336232082 shutdown_connections 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 --2- 172.21.15.16:0/1336232082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a806f270 0x7f91a806f680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 --2- 172.21.15.16:0/1336232082 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91a80c5400 0x7f91a80c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 -- 172.21.15.16:0/1336232082 >> 172.21.15.16:0/1336232082 conn(0x7f91a80ccfe0 msgr2=0x7f91a80cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 -- 172.21.15.16:0/1336232082 shutdown_connections 2024-09-17T05:07:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 -- 172.21.15.16:0/1336232082 wait complete. 2024-09-17T05:07:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 Processor -- start 2024-09-17T05:07:37.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.547+0000 7f91aea00640 1 -- start start 2024-09-17T05:07:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91a806ce90 0x7f91a815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 0x7f91a81515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a80c5400 0x7f91a81567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8168a50 con 0x7f91a806f270 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81688d0 con 0x7f91a80c5400 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f91a8168bd0 con 0x7f91a806ce90 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 0x7f91a81515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 0x7f91a81515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42300/0 (socket says 172.21.15.16:42300) 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 -- 172.21.15.16:0/99862629 learned_addr learned my addr 172.21.15.16:0/99862629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91a7e00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91a806ce90 0x7f91a815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aca00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a80c5400 0x7f91a81567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 -- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91a806ce90 msgr2=0x7f91a815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91a806ce90 0x7f91a815dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 -- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a80c5400 msgr2=0x7f91a81567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a80c5400 0x7f91a81567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 -- 172.21.15.16:0/99862629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a815e1a0 con 0x7f91a806f270 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aca00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a80c5400 0x7f91a81567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f919f400640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 0x7f91a81515c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f919000e770 tx=0x7f919000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000ccc0 con 0x7f91a806f270 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a8151b00 con 0x7f91a806f270 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f919000ee80 con 0x7f91a806f270 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190010670 con 0x7f91a806f270 2024-09-17T05:07:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a813dde0 con 0x7f91a806f270 2024-09-17T05:07:37.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.551+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a806f680 con 0x7f91a806f270 2024-09-17T05:07:37.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.555+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9190010810 con 0x7f91a806f270 2024-09-17T05:07:37.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.555+0000 7f91a4c00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9184073230 0x7f9184075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:37.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.555+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6001+0+0 (secure 0 0 0) 0x7f9190094d70 con 0x7f91a806f270 2024-09-17T05:07:37.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.555+0000 7f91a7e00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9184073230 0x7f9184075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:37.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.555+0000 7f91a7e00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9184073230 0x7f9184075740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9198001840 tx=0x7f9198009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:37.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.559+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f919005fc60 con 0x7f91a806f270 2024-09-17T05:07:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:37.771+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f91a813d890 con 0x7f91a806f270 2024-09-17T05:07:38.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.355+0000 7f91a4c00640 1 -- 172.21.15.16:0/99862629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v62) ==== 174+0+0 (secure 0 0 0) 0x7f919005f560 con 0x7f91a806f270 2024-09-17T05:07:38.359 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 15 and data pool 16 2024-09-17T05:07:38.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.363+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9184073230 msgr2=0x7f9184075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.363+0000 7f91aea00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9184073230 0x7f9184075740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9198001840 tx=0x7f9198009040 comp rx=0 tx=0).stop 2024-09-17T05:07:38.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.363+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 msgr2=0x7f91a81515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.363+0000 7f91aea00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 0x7f91a81515c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f919000e770 tx=0x7f919000ec30 comp rx=0 tx=0).stop 2024-09-17T05:07:38.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.367+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 shutdown_connections 2024-09-17T05:07:38.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.367+0000 7f91aea00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9184073230 0x7f9184075740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.367+0000 7f91aea00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a80c5400 0x7f91a81567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.367+0000 7f91aea00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a806f270 0x7f91a81515c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.367+0000 7f91aea00640 1 --2- 172.21.15.16:0/99862629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f91a806ce90 0x7f91a815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.367+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 >> 172.21.15.16:0/99862629 conn(0x7f91a80ccfe0 msgr2=0x7f91a80af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.371+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 shutdown_connections 2024-09-17T05:07:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.375+0000 7f91aea00640 1 -- 172.21.15.16:0/99862629 wait complete. 2024-09-17T05:07:38.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 -- 172.21.15.16:0/1221756921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 msgr2=0x7f204815c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 --2- 172.21.15.16:0/1221756921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f204815c680 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f2038009900 tx=0x7f203801c850 comp rx=0 tx=0).stop 2024-09-17T05:07:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 -- 172.21.15.16:0/1221756921 shutdown_connections 2024-09-17T05:07:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 --2- 172.21.15.16:0/1221756921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f204815c680 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 -- 172.21.15.16:0/1221756921 >> 172.21.15.16:0/1221756921 conn(0x7f204806f900 msgr2=0x7f20480b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 -- 172.21.15.16:0/1221756921 shutdown_connections 2024-09-17T05:07:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 -- 172.21.15.16:0/1221756921 wait complete. 2024-09-17T05:07:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 Processor -- start 2024-09-17T05:07:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.571+0000 7f2050800640 1 -- start start 2024-09-17T05:07:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2048073240 0x7f20481439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20480b2f50 0x7f20480b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f20481f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2048169cc0 con 0x7f204815c270 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2048169b40 con 0x7f20480b2f50 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2048169e40 con 0x7f2048073240 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f20481f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f20481f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42304/0 (socket says 172.21.15.16:42304) 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 -- 172.21.15.16:0/2487194414 learned_addr learned my addr 172.21.15.16:0/2487194414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204d400640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20480b2f50 0x7f20480b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204de00640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2048073240 0x7f20481439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 -- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2048073240 msgr2=0x7f20481439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2048073240 0x7f20481439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 -- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20480b2f50 msgr2=0x7f20480b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20480b2f50 0x7f20480b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 -- 172.21.15.16:0/2487194414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048143f30 con 0x7f204815c270 2024-09-17T05:07:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f204e800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f20481f84c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f204400c6e0 tx=0x7f204400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044003b80 con 0x7f204815c270 2024-09-17T05:07:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2044003d20 con 0x7f204815c270 2024-09-17T05:07:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20480b04f0 con 0x7f204815c270 2024-09-17T05:07:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044005330 con 0x7f204815c270 2024-09-17T05:07:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20480b0ab0 con 0x7f204815c270 2024-09-17T05:07:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.575+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20480b0260 con 0x7f204815c270 2024-09-17T05:07:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.579+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20440054d0 con 0x7f204815c270 2024-09-17T05:07:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.579+0000 7f2042000640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f202c073230 0x7f202c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.579+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f2044093f80 con 0x7f204815c270 2024-09-17T05:07:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.579+0000 7f204de00640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f202c073230 0x7f202c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.579+0000 7f204de00640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f202c073230 0x7f202c075740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f203c007fd0 tx=0x7f203c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.583+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f204405e850 con 0x7f204815c270 2024-09-17T05:07:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.755+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f204815e8c0 con 0x7f204815c270 2024-09-17T05:07:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2042000640 1 -- 172.21.15.16:0/2487194414 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16950 (secure 0 0 0) 0x7f2044063280 con 0x7f204815c270 2024-09-17T05:07:38.758 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:38.758 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:37.774103+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:07:35.523800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T05:07:36.731373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f202c073230 msgr2=0x7f202c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f202c073230 0x7f202c075740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f203c007fd0 tx=0x7f203c009040 comp rx=0 tx=0).stop 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 msgr2=0x7f20481f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f20481f84c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f204400c6e0 tx=0x7f204400cba0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 shutdown_connections 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f202c073230 0x7f202c075740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f204815c270 0x7f20481f84c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20480b2f50 0x7f20480b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 --2- 172.21.15.16:0/2487194414 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2048073240 0x7f20481439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 >> 172.21.15.16:0/2487194414 conn(0x7f204806f900 msgr2=0x7f2048073650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 shutdown_connections 2024-09-17T05:07:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.759+0000 7f2050800640 1 -- 172.21.15.16:0/2487194414 wait complete. 2024-09-17T05:07:38.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:38.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 -- 172.21.15.16:0/436363340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cecf0 msgr2=0x7f51b006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 --2- 172.21.15.16:0/436363340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cecf0 0x7f51b006dfa0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5198009900 tx=0x7f519801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 -- 172.21.15.16:0/436363340 shutdown_connections 2024-09-17T05:07:38.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 --2- 172.21.15.16:0/436363340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cecf0 0x7f51b006dfa0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 --2- 172.21.15.16:0/436363340 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51b00cfa80 0x7f51b015b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 -- 172.21.15.16:0/436363340 >> 172.21.15.16:0/436363340 conn(0x7f51b006fcf0 msgr2=0x7f51b0070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 -- 172.21.15.16:0/436363340 shutdown_connections 2024-09-17T05:07:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.939+0000 7f51b5c00640 1 -- 172.21.15.16:0/436363340 wait complete. 2024-09-17T05:07:38.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 Processor -- start 2024-09-17T05:07:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 -- start start 2024-09-17T05:07:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51b00cecf0 0x7f51b01f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 0x7f51b00b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51b00cfa80 0x7f51b00a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51b0169440 con 0x7f51b00cf100 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51b01692c0 con 0x7f51b00cfa80 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51b5c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f51b01695c0 con 0x7f51b00cecf0 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.943+0000 7f51a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 0x7f51b00b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51aea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51b00cecf0 0x7f51b01f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 0x7f51b00b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42322/0 (socket says 172.21.15.16:42322) 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 -- 172.21.15.16:0/1064960628 learned_addr learned my addr 172.21.15.16:0/1064960628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51af400640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51b00cfa80 0x7f51b00a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 -- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51b00cecf0 msgr2=0x7f51b01f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51b00cecf0 0x7f51b01f9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 -- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51b00cfa80 msgr2=0x7f51b00a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51b00cfa80 0x7f51b00a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 -- 172.21.15.16:0/1064960628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b014d450 con 0x7f51b00cf100 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51af400640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51b00cfa80 0x7f51b00a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a6000640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 0x7f51b00b1e60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5198009850 tx=0x7f5198005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51980b2040 con 0x7f51b00cf100 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5198005320 con 0x7f51b00cf100 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51980046d0 con 0x7f51b00cf100 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b01fcf80 con 0x7f51b00cf100 2024-09-17T05:07:38.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b01f9170 con 0x7f51b00cf100 2024-09-17T05:07:38.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b006dfa0 con 0x7f51b00cf100 2024-09-17T05:07:38.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.947+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f51980bd060 con 0x7f51b00cf100 2024-09-17T05:07:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.951+0000 7f51a7400640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f518c073230 0x7f518c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.951+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f519812e190 con 0x7f51b00cf100 2024-09-17T05:07:38.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.951+0000 7f51aea00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f518c073230 0x7f518c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:38.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.951+0000 7f51aea00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f518c073230 0x7f518c075740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f51b0073010 tx=0x7f51a000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:38.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:38.955+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f51980164d0 con 0x7f51b00cf100 2024-09-17T05:07:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.131+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51b0071cb0 con 0x7f51b00cf100 2024-09-17T05:07:39.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.131+0000 7f51a7400640 1 -- 172.21.15.16:0/1064960628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16950 (secure 0 0 0) 0x7f51980f8f80 con 0x7f51b00cf100 2024-09-17T05:07:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:39.130 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:37.774103+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:07:35.523800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T05:07:36.731373+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f518c073230 msgr2=0x7f518c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f518c073230 0x7f518c075740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f51b0073010 tx=0x7f51a000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 msgr2=0x7f51b00b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 0x7f51b00b1e60 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5198009850 tx=0x7f5198005760 comp rx=0 tx=0).stop 2024-09-17T05:07:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 shutdown_connections 2024-09-17T05:07:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f518c073230 0x7f518c075740 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51b00cfa80 0x7f51b00a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51b00cf100 0x7f51b00b1e60 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 --2- 172.21.15.16:0/1064960628 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51b00cecf0 0x7f51b01f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 >> 172.21.15.16:0/1064960628 conn(0x7f51b006fcf0 msgr2=0x7f51b0070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 shutdown_connections 2024-09-17T05:07:39.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.135+0000 7f51b5c00640 1 -- 172.21.15.16:0/1064960628 wait complete. 2024-09-17T05:07:39.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 -- 172.21.15.16:0/2009826694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 msgr2=0x7fdfec0cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 --2- 172.21.15.16:0/2009826694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 0x7fdfec0cf870 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdfec06ea60 tx=0x7fdfe401cb80 comp rx=0 tx=0).stop 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 -- 172.21.15.16:0/2009826694 shutdown_connections 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 --2- 172.21.15.16:0/2009826694 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 0x7fdfec0cf870 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 --2- 172.21.15.16:0/2009826694 >> v2:172.21.15.16:3300/0 conn(0x7fdfec15e990 0x7fdfec0cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 -- 172.21.15.16:0/2009826694 >> 172.21.15.16:0/2009826694 conn(0x7fdfec0cd980 msgr2=0x7fdfec0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 -- 172.21.15.16:0/2009826694 shutdown_connections 2024-09-17T05:07:39.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.315+0000 7fdff3800640 1 -- 172.21.15.16:0/2009826694 wait complete. 2024-09-17T05:07:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 Processor -- start 2024-09-17T05:07:39.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 -- start start 2024-09-17T05:07:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdfec0caf50 0x7fdfec1618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 0x7fdfec1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 0x7fdfec162c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec16c4c0 con 0x7fdfec15e990 2024-09-17T05:07:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec16c340 con 0x7fdfec0cf460 2024-09-17T05:07:39.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff3800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdfec16c640 con 0x7fdfec0caf50 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 0x7fdfec162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 0x7fdfec162c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42340/0 (socket says 172.21.15.16:42340) 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 -- 172.21.15.16:0/353986514 learned_addr learned my addr 172.21.15.16:0/353986514 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff0e00640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdfec0caf50 0x7fdfec1618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdfe3e00640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 0x7fdfec1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 -- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdfec0caf50 msgr2=0x7fdfec1618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdfec0caf50 0x7fdfec1618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 -- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 msgr2=0x7fdfec1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 0x7fdfec1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:39.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.319+0000 7fdff1800640 1 -- 172.21.15.16:0/353986514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec160840 con 0x7fdfec15e990 2024-09-17T05:07:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdff1800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 0x7fdfec162c00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401a7b0 tx=0x7fdfe4009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40bb070 con 0x7fdfec15e990 2024-09-17T05:07:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfe4004b40 con 0x7fdfec15e990 2024-09-17T05:07:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe40b2040 con 0x7fdfec15e990 2024-09-17T05:07:39.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec160b20 con 0x7fdfec15e990 2024-09-17T05:07:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1610e0 con 0x7fdfec15e990 2024-09-17T05:07:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfec0cf870 con 0x7fdfec15e990 2024-09-17T05:07:39.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdfe4004000 con 0x7fdfec15e990 2024-09-17T05:07:39.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.323+0000 7fdfe9600640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdfc8073230 0x7fdfc8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:39.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.327+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fdfe412e070 con 0x7fdfec15e990 2024-09-17T05:07:39.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.327+0000 7fdff0e00640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdfc8073230 0x7fdfc8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:39.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.327+0000 7fdff0e00640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdfc8073230 0x7fdfc8075740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009ad0 tx=0x7fdfdc009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:39.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.331+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdfe40f1db0 con 0x7fdfec15e990 2024-09-17T05:07:39.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:39.543+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdfec06a180 con 0x7fdfec15e990 2024-09-17T05:07:40.324 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-09-17T05:07:40.327+0000 7f3664000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:07:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdfe9600640 1 -- 172.21.15.16:0/353986514 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) ==== 118+0+0 (secure 0 0 0) 0x7fdfe4016770 con 0x7fdfec15e990 2024-09-17T05:07:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdfc8073230 msgr2=0x7fdfc8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdfc8073230 0x7fdfc8075740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009ad0 tx=0x7fdfdc009260 comp rx=0 tx=0).stop 2024-09-17T05:07:40.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 msgr2=0x7fdfec162c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 0x7fdfec162c00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401a7b0 tx=0x7fdfe4009280 comp rx=0 tx=0).stop 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 shutdown_connections 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdfc8073230 0x7fdfc8075740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec15e990 0x7fdfec162c00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdfec0cf460 0x7fdfec1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 --2- 172.21.15.16:0/353986514 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdfec0caf50 0x7fdfec1618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.367+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 >> 172.21.15.16:0/353986514 conn(0x7fdfec0cd980 msgr2=0x7fdfec1570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.371+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 shutdown_connections 2024-09-17T05:07:40.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.371+0000 7fdff3800640 1 -- 172.21.15.16:0/353986514 wait complete. 2024-09-17T05:07:40.392 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:07:40.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 -- 172.21.15.16:0/2408214853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc004810 msgr2=0x7f10fc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 --2- 172.21.15.16:0/2408214853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc004810 0x7f10fc004c20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f110000c2e0 tx=0x7f110001cf20 comp rx=0 tx=0).stop 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 -- 172.21.15.16:0/2408214853 shutdown_connections 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 --2- 172.21.15.16:0/2408214853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10fc004810 0x7f10fc004c20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 -- 172.21.15.16:0/2408214853 >> 172.21.15.16:0/2408214853 conn(0x7f1118073330 msgr2=0x7f11180b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 -- 172.21.15.16:0/2408214853 shutdown_connections 2024-09-17T05:07:40.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.559+0000 7f111d600640 1 -- 172.21.15.16:0/2408214853 wait complete. 2024-09-17T05:07:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 Processor -- start 2024-09-17T05:07:40.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 -- start start 2024-09-17T05:07:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10fc004810 0x7f11181ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118152460 0x7f11180aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 0x7f11180cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11180b1a40 con 0x7f11180af4a0 2024-09-17T05:07:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11180b18c0 con 0x7f1118152460 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f11180b1bc0 con 0x7f10fc004810 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 0x7f11180cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 0x7f11180cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42362/0 (socket says 172.21.15.16:42362) 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f111cc00640 1 -- 172.21.15.16:0/3921830054 learned_addr learned my addr 172.21.15.16:0/3921830054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f1117e00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10fc004810 0x7f11181ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.563+0000 7f110f400640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118152460 0x7f11180aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111cc00640 1 -- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10fc004810 msgr2=0x7f11181ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111cc00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10fc004810 0x7f11181ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111cc00640 1 -- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118152460 msgr2=0x7f11180aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111cc00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118152460 0x7f11180aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111cc00640 1 -- 172.21.15.16:0/3921830054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11180af8b0 con 0x7f11180af4a0 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f110f400640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118152460 0x7f11180aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111cc00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 0x7f11180cc9c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f11180cc990 tx=0x7f1110011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110004440 con 0x7f11180af4a0 2024-09-17T05:07:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1110016070 con 0x7f11180af4a0 2024-09-17T05:07:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11180cefe0 con 0x7f11180af4a0 2024-09-17T05:07:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111000fbe0 con 0x7f11180af4a0 2024-09-17T05:07:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11180ced00 con 0x7f11180af4a0 2024-09-17T05:07:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111806e7d0 con 0x7f11180af4a0 2024-09-17T05:07:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.567+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1110003600 con 0x7f11180af4a0 2024-09-17T05:07:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.571+0000 7f1114c00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10f4073230 0x7f10f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.571+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f1110094b30 con 0x7f11180af4a0 2024-09-17T05:07:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.571+0000 7f1117e00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10f4073230 0x7f10f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.571+0000 7f1117e00640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10f4073230 0x7f10f4075740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1118052280 tx=0x7f1108006f00 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.575+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f11180af8b0 con 0x7f11180af4a0 2024-09-17T05:07:40.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:40.783+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f111815f830 con 0x7f11180af4a0 2024-09-17T05:07:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.367+0000 7f1114c00640 1 -- 172.21.15.16:0/3921830054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7f111005fa60 con 0x7f11180af4a0 2024-09-17T05:07:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10f4073230 msgr2=0x7f10f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10f4073230 0x7f10f4075740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1118052280 tx=0x7f1108006f00 comp rx=0 tx=0).stop 2024-09-17T05:07:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 msgr2=0x7f11180cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 0x7f11180cc9c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f11180cc990 tx=0x7f1110011e10 comp rx=0 tx=0).stop 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 shutdown_connections 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10f4073230 0x7f10f4075740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11180af4a0 0x7f11180cc9c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118152460 0x7f11180aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 --2- 172.21.15.16:0/3921830054 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f10fc004810 0x7f11181ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 >> 172.21.15.16:0/3921830054 conn(0x7f1118073330 msgr2=0x7f111806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 shutdown_connections 2024-09-17T05:07:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.371+0000 7f111d600640 1 -- 172.21.15.16:0/3921830054 wait complete. 2024-09-17T05:07:41.395 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:07:41.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 -- 172.21.15.16:0/1149517473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c003c20 msgr2=0x7faf0c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 --2- 172.21.15.16:0/1149517473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf0c003c20 0x7faf0c004070 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7faf18009b30 tx=0x7faf1801d250 comp rx=0 tx=0).stop 2024-09-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf26800640 1 -- 172.21.15.16:0/1149517473 reap_dead start 2024-09-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 -- 172.21.15.16:0/1149517473 shutdown_connections 2024-09-17T05:07:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 -- 172.21.15.16:0/1149517473 >> 172.21.15.16:0/1149517473 conn(0x7faf280d2270 msgr2=0x7faf280d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 -- 172.21.15.16:0/1149517473 shutdown_connections 2024-09-17T05:07:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 -- 172.21.15.16:0/1149517473 wait complete. 2024-09-17T05:07:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 Processor -- start 2024-09-17T05:07:41.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.563+0000 7faf2cc00640 1 -- start start 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0c003c20 0x7faf2815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 0x7faf28165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf2815f150 0x7faf2815f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf28168eb0 con 0x7faf2815fd90 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faf28168d30 con 0x7faf0c003c20 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7faf28169030 con 0x7faf2815f150 2024-09-17T05:07:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 0x7faf28165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 0x7faf28165820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42386/0 (socket says 172.21.15.16:42386) 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 -- 172.21.15.16:0/3266867531 learned_addr learned my addr 172.21.15.16:0/3266867531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf25e00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0c003c20 0x7faf2815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf26800640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf2815f150 0x7faf2815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 -- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf2815f150 msgr2=0x7faf2815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf2815f150 0x7faf2815f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 -- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0c003c20 msgr2=0x7faf2815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0c003c20 0x7faf2815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 -- 172.21.15.16:0/3266867531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2815e0b0 con 0x7faf2815fd90 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf26800640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf2815f150 0x7faf2815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1fe00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 0x7faf28165820 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7faf10001250 tx=0x7faf10001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10015070 con 0x7faf2815fd90 2024-09-17T05:07:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf100109a0 con 0x7faf2815fd90 2024-09-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10010df0 con 0x7faf2815fd90 2024-09-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2815e390 con 0x7faf2815fd90 2024-09-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2815e950 con 0x7faf2815fd90 2024-09-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.567+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf280c9850 con 0x7faf2815fd90 2024-09-17T05:07:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.571+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf10018070 con 0x7faf2815fd90 2024-09-17T05:07:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.571+0000 7faf1e000640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf04073200 0x7faf04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.571+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7faf10094840 con 0x7faf2815fd90 2024-09-17T05:07:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.571+0000 7faf25e00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf04073200 0x7faf04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.571+0000 7faf25e00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf04073200 0x7faf04075710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faf180ae000 tx=0x7faf18019ff0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.575+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf1005f6c0 con 0x7faf2815fd90 2024-09-17T05:07:41.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf2815c3b0 con 0x7faf2815fd90 2024-09-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf1e000640 1 -- 172.21.15.16:0/3266867531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7faf1005efc0 con 0x7faf2815fd90 2024-09-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:41.785 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:07:37.774040+0000","modified":"2024-09-17T05:07:40.784518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2345541365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2345541365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2345541365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-17T05:07:41.786 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-09-17T05:07:41.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf04073200 msgr2=0x7faf04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf04073200 0x7faf04075710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7faf180ae000 tx=0x7faf18019ff0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 msgr2=0x7faf28165820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 0x7faf28165820 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7faf10001250 tx=0x7faf10001830 comp rx=0 tx=0).stop 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 shutdown_connections 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf04073200 0x7faf04075710 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7faf2815f150 0x7faf2815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2815fd90 0x7faf28165820 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 --2- 172.21.15.16:0/3266867531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf0c003c20 0x7faf2815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 >> 172.21.15.16:0/3266867531 conn(0x7faf280d2270 msgr2=0x7faf280cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 shutdown_connections 2024-09-17T05:07:41.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:41.787+0000 7faf2cc00640 1 -- 172.21.15.16:0/3266867531 wait complete. 2024-09-17T05:07:41.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:42.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 -- 172.21.15.16:0/982188970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 msgr2=0x7fb8f4074030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/982188970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4074030 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009bc0 tx=0x7fb8dc01cb70 comp rx=0 tx=0).stop 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 -- 172.21.15.16:0/982188970 shutdown_connections 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/982188970 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8f406d450 0x7fb8f406d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/982188970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4074030 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 -- 172.21.15.16:0/982188970 >> 172.21.15.16:0/982188970 conn(0x7fb8f40cffa0 msgr2=0x7fb8f40d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 -- 172.21.15.16:0/982188970 shutdown_connections 2024-09-17T05:07:42.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.011+0000 7fb8f8c00640 1 -- 172.21.15.16:0/982188970 wait complete. 2024-09-17T05:07:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 Processor -- start 2024-09-17T05:07:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 -- start start 2024-09-17T05:07:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f406d450 0x7fb8f4166b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8f416b2a0 0x7fb8f416ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f41789a0 con 0x7fb8f4073c20 2024-09-17T05:07:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4178820 con 0x7fb8f406d450 2024-09-17T05:07:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f4178b20 con 0x7fb8f416b2a0 2024-09-17T05:07:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42406/0 (socket says 172.21.15.16:42406) 2024-09-17T05:07:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 -- 172.21.15.16:0/2862187529 learned_addr learned my addr 172.21.15.16:0/2862187529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f1e00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f406d450 0x7fb8f4166b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f2800640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8f416b2a0 0x7fb8f416ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 -- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8f416b2a0 msgr2=0x7fb8f416ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8f416b2a0 0x7fb8f416ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 -- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f406d450 msgr2=0x7fb8f4166b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f406d450 0x7fb8f4166b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 -- 172.21.15.16:0/2862187529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f4174850 con 0x7fb8f4073c20 2024-09-17T05:07:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8e9400640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4155c70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009b10 tx=0x7fb8dc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0b2040 con 0x7fb8f4073c20 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8dc004330 con 0x7fb8f4073c20 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f41737b0 con 0x7fb8f4073c20 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc004720 con 0x7fb8f4073c20 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.015+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f4173d70 con 0x7fb8f4073c20 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.019+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f406d860 con 0x7fb8f4073c20 2024-09-17T05:07:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.019+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb8dc0bd060 con 0x7fb8f4073c20 2024-09-17T05:07:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.019+0000 7fb8eaa00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8d0073200 0x7fb8d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.019+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fb8dc12e120 con 0x7fb8f4073c20 2024-09-17T05:07:42.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.019+0000 7fb8f1e00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8d0073200 0x7fb8d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.019+0000 7fb8f1e00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8d0073200 0x7fb8d0075710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000f70 tx=0x7fb8e4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:42.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.023+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb8dc016790 con 0x7fb8f4073c20 2024-09-17T05:07:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.199+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8f4070c40 con 0x7fb8f4073c20 2024-09-17T05:07:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8eaa00640 1 -- 172.21.15.16:0/2862187529 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16939 (secure 0 0 0) 0x7fb8dc0f8f10 con 0x7fb8f4073c20 2024-09-17T05:07:42.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:42.201 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:39.352002+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:07:35.523800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T05:07:36.731373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8d0073200 msgr2=0x7fb8d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8d0073200 0x7fb8d0075710 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000f70 tx=0x7fb8e4009040 comp rx=0 tx=0).stop 2024-09-17T05:07:42.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 msgr2=0x7fb8f4155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4155c70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009b10 tx=0x7fb8dc005ae0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 shutdown_connections 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8d0073200 0x7fb8d0075710 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb8f416b2a0 0x7fb8f416ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f4073c20 0x7fb8f4155c70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 --2- 172.21.15.16:0/2862187529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8f406d450 0x7fb8f4166b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 >> 172.21.15.16:0/2862187529 conn(0x7fb8f40cffa0 msgr2=0x7fb8f41653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 shutdown_connections 2024-09-17T05:07:42.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.203+0000 7fb8f8c00640 1 -- 172.21.15.16:0/2862187529 wait complete. 2024-09-17T05:07:42.224 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:07:42.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:07:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 -- 172.21.15.16:0/2566043152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840d1c90 msgr2=0x7f098415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 --2- 172.21.15.16:0/2566043152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840d1c90 0x7f098415cab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0970009840 tx=0x7f097001c820 comp rx=0 tx=0).stop 2024-09-17T05:07:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 -- 172.21.15.16:0/2566043152 shutdown_connections 2024-09-17T05:07:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 --2- 172.21.15.16:0/2566043152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840d1c90 0x7f098415cab0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 -- 172.21.15.16:0/2566043152 >> 172.21.15.16:0/2566043152 conn(0x7f09840d2270 msgr2=0x7f09840d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 -- 172.21.15.16:0/2566043152 shutdown_connections 2024-09-17T05:07:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.383+0000 7f098c400640 1 -- 172.21.15.16:0/2566043152 wait complete. 2024-09-17T05:07:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 Processor -- start 2024-09-17T05:07:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 -- start start 2024-09-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 0x7f0984153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09840cc8c0 0x7f0984152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09841f8720 0x7f0984152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f098416b040 con 0x7f09840c9440 2024-09-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f098416aec0 con 0x7f09841f8720 2024-09-17T05:07:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098c400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f098416b1c0 con 0x7f09840cc8c0 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 0x7f0984153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 0x7f0984153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42426/0 (socket says 172.21.15.16:42426) 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 -- 172.21.15.16:0/2886392757 learned_addr learned my addr 172.21.15.16:0/2886392757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989000640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09840cc8c0 0x7f0984152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f098a400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09841f8720 0x7f0984152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 -- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09840cc8c0 msgr2=0x7f0984152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09840cc8c0 0x7f0984152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 -- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09841f8720 msgr2=0x7f0984152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09841f8720 0x7f0984152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.387+0000 7f0989a00640 1 -- 172.21.15.16:0/2886392757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09840b5430 con 0x7f09840c9440 2024-09-17T05:07:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f0989a00640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 0x7f0984153900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f097800c3f0 tx=0x7f097800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978011790 con 0x7f09840c9440 2024-09-17T05:07:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0978017070 con 0x7f09840c9440 2024-09-17T05:07:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0978011ce0 con 0x7f09840c9440 2024-09-17T05:07:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09840b5710 con 0x7f09840c9440 2024-09-17T05:07:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09840b80e0 con 0x7f09840c9440 2024-09-17T05:07:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098415cab0 con 0x7f09840c9440 2024-09-17T05:07:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.391+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f097801c460 con 0x7f09840c9440 2024-09-17T05:07:42.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.395+0000 7f0981600640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0964073230 0x7f0964075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.395+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f0978094000 con 0x7f09840c9440 2024-09-17T05:07:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.395+0000 7f0989000640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0964073230 0x7f0964075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.395+0000 7f0989000640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0964073230 0x7f0964075740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f09700af000 tx=0x7f097001b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.399+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f097805e840 con 0x7f09840c9440 2024-09-17T05:07:42.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.567+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0984076230 con 0x7f0964073230 2024-09-17T05:07:42.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.575+0000 7f0981600640 1 -- 172.21.15.16:0/2886392757 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247220 (secure 0 0 0) 0x7f0984076230 con 0x7f0964073230 2024-09-17T05:07:42.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:42.578 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:07:42.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0964073230 msgr2=0x7f0964075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0964073230 0x7f0964075740 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f09700af000 tx=0x7f097001b060 comp rx=0 tx=0).stop 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 msgr2=0x7f0984153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 0x7f0984153900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f097800c3f0 tx=0x7f097800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 shutdown_connections 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0964073230 0x7f0964075740 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09841f8720 0x7f0984152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f09840cc8c0 0x7f0984152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 --2- 172.21.15.16:0/2886392757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09840c9440 0x7f0984153900 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 >> 172.21.15.16:0/2886392757 conn(0x7f09840d2270 msgr2=0x7f09840d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 shutdown_connections 2024-09-17T05:07:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.579+0000 7f098c400640 1 -- 172.21.15.16:0/2886392757 wait complete. 2024-09-17T05:07:42.598 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-17T05:07:41.570418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268440,"kb_used_data":10584,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706280,"statfs":{"total":773094113280,"available":772819230720,"internally_reserved":0,"allocated":10838016,"data_stored":5578435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.244815"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354142+0000","last_change":"2024-09-17T05:07:37.354142+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354142+0000","last_undegraded":"2024-09-17T05:07:37.354142+0000","last_fullsized":"2024-09-17T05:07:37.354142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344873+0000","last_change":"2024-09-17T05:07:37.344873+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344873+0000","last_undegraded":"2024-09-17T05:07:37.344873+0000","last_fullsized":"2024-09-17T05:07:37.344873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365386+0000","last_change":"2024-09-17T05:07:38.373066+0000","last_active":"2024-09-17T05:07:39.365386+0000","last_peered":"2024-09-17T05:07:39.365386+0000","last_clean":"2024-09-17T05:07:39.365386+0000","last_became_active":"2024-09-17T05:07:38.372726+0000","last_became_peered":"2024-09-17T05:07:38.372726+0000","last_unstale":"2024-09-17T05:07:39.365386+0000","last_undegraded":"2024-09-17T05:07:39.365386+0000","last_fullsized":"2024-09-17T05:07:39.365386+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:40.211315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353488+0000","last_change":"2024-09-17T05:07:37.353488+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353488+0000","last_undegraded":"2024-09-17T05:07:37.353488+0000","last_fullsized":"2024-09-17T05:07:37.353488+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353790+0000","last_change":"2024-09-17T05:07:37.353790+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353790+0000","last_undegraded":"2024-09-17T05:07:37.353790+0000","last_fullsized":"2024-09-17T05:07:37.353790+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352451+0000","last_change":"2024-09-17T05:07:37.352451+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352451+0000","last_undegraded":"2024-09-17T05:07:37.352451+0000","last_fullsized":"2024-09-17T05:07:37.352451+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352553+0000","last_change":"2024-09-17T05:07:37.352553+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352553+0000","last_undegraded":"2024-09-17T05:07:37.352553+0000","last_fullsized":"2024-09-17T05:07:37.352553+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347288+0000","last_change":"2024-09-17T05:07:37.347288+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347288+0000","last_undegraded":"2024-09-17T05:07:37.347288+0000","last_fullsized":"2024-09-17T05:07:37.347288+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344945+0000","last_change":"2024-09-17T05:07:37.344945+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344945+0000","last_undegraded":"2024-09-17T05:07:37.344945+0000","last_fullsized":"2024-09-17T05:07:37.344945+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.349207+0000","last_change":"2024-09-17T05:07:37.349207+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.349207+0000","last_undegraded":"2024-09-17T05:07:37.349207+0000","last_fullsized":"2024-09-17T05:07:37.349207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348392+0000","last_change":"2024-09-17T05:07:37.348392+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348392+0000","last_undegraded":"2024-09-17T05:07:37.348392+0000","last_fullsized":"2024-09-17T05:07:37.348392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.351164+0000","last_change":"2024-09-17T05:07:37.351164+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.351164+0000","last_undegraded":"2024-09-17T05:07:37.351164+0000","last_fullsized":"2024-09-17T05:07:37.351164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354316+0000","last_change":"2024-09-17T05:07:37.354316+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354316+0000","last_undegraded":"2024-09-17T05:07:37.354316+0000","last_fullsized":"2024-09-17T05:07:37.354316+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350574+0000","last_change":"2024-09-17T05:07:37.350574+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350574+0000","last_undegraded":"2024-09-17T05:07:37.350574+0000","last_fullsized":"2024-09-17T05:07:37.350574+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343241+0000","last_change":"2024-09-17T05:07:37.343241+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343241+0000","last_undegraded":"2024-09-17T05:07:37.343241+0000","last_fullsized":"2024-09-17T05:07:37.343241+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.359263+0000","last_change":"2024-09-17T05:07:37.359263+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.359263+0000","last_undegraded":"2024-09-17T05:07:37.359263+0000","last_fullsized":"2024-09-17T05:07:37.359263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365037+0000","last_change":"2024-09-17T05:07:38.369916+0000","last_active":"2024-09-17T05:07:39.365037+0000","last_peered":"2024-09-17T05:07:39.365037+0000","last_clean":"2024-09-17T05:07:39.365037+0000","last_became_active":"2024-09-17T05:07:38.369536+0000","last_became_peered":"2024-09-17T05:07:38.369536+0000","last_unstale":"2024-09-17T05:07:39.365037+0000","last_undegraded":"2024-09-17T05:07:39.365037+0000","last_fullsized":"2024-09-17T05:07:39.365037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:07.275670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344273+0000","last_change":"2024-09-17T05:07:37.344273+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344273+0000","last_undegraded":"2024-09-17T05:07:37.344273+0000","last_fullsized":"2024-09-17T05:07:37.344273+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358759+0000","last_change":"2024-09-17T05:07:37.358759+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358759+0000","last_undegraded":"2024-09-17T05:07:37.358759+0000","last_fullsized":"2024-09-17T05:07:37.358759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358844+0000","last_change":"2024-09-17T05:07:37.358844+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358844+0000","last_undegraded":"2024-09-17T05:07:37.358844+0000","last_fullsized":"2024-09-17T05:07:37.358844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364572+0000","last_change":"2024-09-17T05:07:38.365788+0000","last_active":"2024-09-17T05:07:39.364572+0000","last_peered":"2024-09-17T05:07:39.364572+0000","last_clean":"2024-09-17T05:07:39.364572+0000","last_became_active":"2024-09-17T05:07:38.364783+0000","last_became_peered":"2024-09-17T05:07:38.364783+0000","last_unstale":"2024-09-17T05:07:39.364572+0000","last_undegraded":"2024-09-17T05:07:39.364572+0000","last_fullsized":"2024-09-17T05:07:39.364572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:32.081371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352974+0000","last_change":"2024-09-17T05:07:37.352974+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352974+0000","last_undegraded":"2024-09-17T05:07:37.352974+0000","last_fullsized":"2024-09-17T05:07:37.352974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347298+0000","last_change":"2024-09-17T05:07:37.347298+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347298+0000","last_undegraded":"2024-09-17T05:07:37.347298+0000","last_fullsized":"2024-09-17T05:07:37.347298+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346741+0000","last_change":"2024-09-17T05:07:37.346741+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346741+0000","last_undegraded":"2024-09-17T05:07:37.346741+0000","last_fullsized":"2024-09-17T05:07:37.346741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.355031+0000","last_change":"2024-09-17T05:07:37.355031+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.355031+0000","last_undegraded":"2024-09-17T05:07:37.355031+0000","last_fullsized":"2024-09-17T05:07:37.355031+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354956+0000","last_change":"2024-09-17T05:07:37.354956+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354956+0000","last_undegraded":"2024-09-17T05:07:37.354956+0000","last_fullsized":"2024-09-17T05:07:37.354956+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354198+0000","last_change":"2024-09-17T05:07:37.354198+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354198+0000","last_undegraded":"2024-09-17T05:07:37.354198+0000","last_fullsized":"2024-09-17T05:07:37.354198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350568+0000","last_change":"2024-09-17T05:07:37.350568+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350568+0000","last_undegraded":"2024-09-17T05:07:37.350568+0000","last_fullsized":"2024-09-17T05:07:37.350568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347094+0000","last_change":"2024-09-17T05:07:37.347094+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347094+0000","last_undegraded":"2024-09-17T05:07:37.347094+0000","last_fullsized":"2024-09-17T05:07:37.347094+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347191+0000","last_change":"2024-09-17T05:07:37.347191+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347191+0000","last_undegraded":"2024-09-17T05:07:37.347191+0000","last_fullsized":"2024-09-17T05:07:37.347191+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350459+0000","last_change":"2024-09-17T05:07:37.350459+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350459+0000","last_undegraded":"2024-09-17T05:07:37.350459+0000","last_fullsized":"2024-09-17T05:07:37.350459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348834+0000","last_change":"2024-09-17T05:07:37.348834+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348834+0000","last_undegraded":"2024-09-17T05:07:37.348834+0000","last_fullsized":"2024-09-17T05:07:37.348834+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363817+0000","last_change":"2024-09-17T05:07:38.351295+0000","last_active":"2024-09-17T05:07:39.363817+0000","last_peered":"2024-09-17T05:07:39.363817+0000","last_clean":"2024-09-17T05:07:39.363817+0000","last_became_active":"2024-09-17T05:07:38.348971+0000","last_became_peered":"2024-09-17T05:07:38.348971+0000","last_unstale":"2024-09-17T05:07:39.363817+0000","last_undegraded":"2024-09-17T05:07:39.363817+0000","last_fullsized":"2024-09-17T05:07:39.363817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:12.675618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365425+0000","last_change":"2024-09-17T05:07:37.361640+0000","last_active":"2024-09-17T05:07:37.365425+0000","last_peered":"2024-09-17T05:07:37.365425+0000","last_clean":"2024-09-17T05:07:37.365425+0000","last_became_active":"2024-09-17T05:07:37.360159+0000","last_became_peered":"2024-09-17T05:07:37.360159+0000","last_unstale":"2024-09-17T05:07:37.365425+0000","last_undegraded":"2024-09-17T05:07:37.365425+0000","last_fullsized":"2024-09-17T05:07:37.365425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:22.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346712+0000","last_change":"2024-09-17T05:07:37.346712+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346712+0000","last_undegraded":"2024-09-17T05:07:37.346712+0000","last_fullsized":"2024-09-17T05:07:37.346712+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364559+0000","last_change":"2024-09-17T05:07:37.367481+0000","last_active":"2024-09-17T05:07:39.364559+0000","last_peered":"2024-09-17T05:07:39.364559+0000","last_clean":"2024-09-17T05:07:39.364559+0000","last_became_active":"2024-09-17T05:07:37.367109+0000","last_became_peered":"2024-09-17T05:07:37.367109+0000","last_unstale":"2024-09-17T05:07:39.364559+0000","last_undegraded":"2024-09-17T05:07:39.364559+0000","last_fullsized":"2024-09-17T05:07:39.364559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:25.251480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344456+0000","last_change":"2024-09-17T05:07:37.344456+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344456+0000","last_undegraded":"2024-09-17T05:07:37.344456+0000","last_fullsized":"2024-09-17T05:07:37.344456+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.363644+0000","last_change":"2024-09-17T05:07:37.360648+0000","last_active":"2024-09-17T05:07:37.363644+0000","last_peered":"2024-09-17T05:07:37.363644+0000","last_clean":"2024-09-17T05:07:37.363644+0000","last_became_active":"2024-09-17T05:07:37.356573+0000","last_became_peered":"2024-09-17T05:07:37.356573+0000","last_unstale":"2024-09-17T05:07:37.363644+0000","last_undegraded":"2024-09-17T05:07:37.363644+0000","last_fullsized":"2024-09-17T05:07:37.363644+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:23.902999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346600+0000","last_change":"2024-09-17T05:07:37.346600+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346600+0000","last_undegraded":"2024-09-17T05:07:37.346600+0000","last_fullsized":"2024-09-17T05:07:37.346600+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367346+0000","last_change":"2024-09-17T05:07:37.364681+0000","last_active":"2024-09-17T05:07:37.367346+0000","last_peered":"2024-09-17T05:07:37.367346+0000","last_clean":"2024-09-17T05:07:37.367346+0000","last_became_active":"2024-09-17T05:07:37.364348+0000","last_became_peered":"2024-09-17T05:07:37.364348+0000","last_unstale":"2024-09-17T05:07:37.367346+0000","last_undegraded":"2024-09-17T05:07:37.367346+0000","last_fullsized":"2024-09-17T05:07:37.367346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:09.224428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.349759+0000","last_change":"2024-09-17T05:07:37.349759+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.349759+0000","last_undegraded":"2024-09-17T05:07:37.349759+0000","last_fullsized":"2024-09-17T05:07:37.349759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371989+0000","last_change":"2024-09-17T05:07:37.368233+0000","last_active":"2024-09-17T05:07:37.371989+0000","last_peered":"2024-09-17T05:07:37.371989+0000","last_clean":"2024-09-17T05:07:37.371989+0000","last_became_active":"2024-09-17T05:07:37.367904+0000","last_became_peered":"2024-09-17T05:07:37.367904+0000","last_unstale":"2024-09-17T05:07:37.371989+0000","last_undegraded":"2024-09-17T05:07:37.371989+0000","last_fullsized":"2024-09-17T05:07:37.371989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:38.417939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354832+0000","last_change":"2024-09-17T05:07:37.354832+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354832+0000","last_undegraded":"2024-09-17T05:07:37.354832+0000","last_fullsized":"2024-09-17T05:07:37.354832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.372345+0000","last_change":"2024-09-17T05:07:37.368607+0000","last_active":"2024-09-17T05:07:37.372345+0000","last_peered":"2024-09-17T05:07:37.372345+0000","last_clean":"2024-09-17T05:07:37.372345+0000","last_became_active":"2024-09-17T05:07:37.368307+0000","last_became_peered":"2024-09-17T05:07:37.368307+0000","last_unstale":"2024-09-17T05:07:37.372345+0000","last_undegraded":"2024-09-17T05:07:37.372345+0000","last_fullsized":"2024-09-17T05:07:37.372345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346749+0000","last_change":"2024-09-17T05:07:37.346749+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346749+0000","last_undegraded":"2024-09-17T05:07:37.346749+0000","last_fullsized":"2024-09-17T05:07:37.346749+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371473+0000","last_change":"2024-09-17T05:07:37.367847+0000","last_active":"2024-09-17T05:07:37.371473+0000","last_peered":"2024-09-17T05:07:37.371473+0000","last_clean":"2024-09-17T05:07:37.371473+0000","last_became_active":"2024-09-17T05:07:37.367465+0000","last_became_peered":"2024-09-17T05:07:37.367465+0000","last_unstale":"2024-09-17T05:07:37.371473+0000","last_undegraded":"2024-09-17T05:07:37.371473+0000","last_fullsized":"2024-09-17T05:07:37.371473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:41.729615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353184+0000","last_change":"2024-09-17T05:07:37.353184+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353184+0000","last_undegraded":"2024-09-17T05:07:37.353184+0000","last_fullsized":"2024-09-17T05:07:37.353184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367733+0000","last_change":"2024-09-17T05:07:37.364825+0000","last_active":"2024-09-17T05:07:37.367733+0000","last_peered":"2024-09-17T05:07:37.367733+0000","last_clean":"2024-09-17T05:07:37.367733+0000","last_became_active":"2024-09-17T05:07:37.364481+0000","last_became_peered":"2024-09-17T05:07:37.364481+0000","last_unstale":"2024-09-17T05:07:37.367733+0000","last_undegraded":"2024-09-17T05:07:37.367733+0000","last_fullsized":"2024-09-17T05:07:37.367733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:05.942761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348642+0000","last_change":"2024-09-17T05:07:37.348642+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348642+0000","last_undegraded":"2024-09-17T05:07:37.348642+0000","last_fullsized":"2024-09-17T05:07:37.348642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366699+0000","last_change":"2024-09-17T05:07:37.362446+0000","last_active":"2024-09-17T05:07:37.366699+0000","last_peered":"2024-09-17T05:07:37.366699+0000","last_clean":"2024-09-17T05:07:37.366699+0000","last_became_active":"2024-09-17T05:07:37.362104+0000","last_became_peered":"2024-09-17T05:07:37.362104+0000","last_unstale":"2024-09-17T05:07:37.366699+0000","last_undegraded":"2024-09-17T05:07:37.366699+0000","last_fullsized":"2024-09-17T05:07:37.366699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:41.729615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348460+0000","last_change":"2024-09-17T05:07:37.348460+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348460+0000","last_undegraded":"2024-09-17T05:07:37.348460+0000","last_fullsized":"2024-09-17T05:07:37.348460+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.361947+0000","last_change":"2024-09-17T05:07:37.359338+0000","last_active":"2024-09-17T05:07:37.361947+0000","last_peered":"2024-09-17T05:07:37.361947+0000","last_clean":"2024-09-17T05:07:37.361947+0000","last_became_active":"2024-09-17T05:07:37.359008+0000","last_became_peered":"2024-09-17T05:07:37.359008+0000","last_unstale":"2024-09-17T05:07:37.361947+0000","last_undegraded":"2024-09-17T05:07:37.361947+0000","last_fullsized":"2024-09-17T05:07:37.361947+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348027+0000","last_change":"2024-09-17T05:07:37.348027+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348027+0000","last_undegraded":"2024-09-17T05:07:37.348027+0000","last_fullsized":"2024-09-17T05:07:37.348027+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365107+0000","last_change":"2024-09-17T05:07:37.362270+0000","last_active":"2024-09-17T05:07:37.365107+0000","last_peered":"2024-09-17T05:07:37.365107+0000","last_clean":"2024-09-17T05:07:37.365107+0000","last_became_active":"2024-09-17T05:07:37.361924+0000","last_became_peered":"2024-09-17T05:07:37.361924+0000","last_unstale":"2024-09-17T05:07:37.365107+0000","last_undegraded":"2024-09-17T05:07:37.365107+0000","last_fullsized":"2024-09-17T05:07:37.365107+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:26.077987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347252+0000","last_change":"2024-09-17T05:07:37.347252+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347252+0000","last_undegraded":"2024-09-17T05:07:37.347252+0000","last_fullsized":"2024-09-17T05:07:37.347252+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371818+0000","last_change":"2024-09-17T05:07:37.365192+0000","last_active":"2024-09-17T05:07:37.371818+0000","last_peered":"2024-09-17T05:07:37.371818+0000","last_clean":"2024-09-17T05:07:37.371818+0000","last_became_active":"2024-09-17T05:07:37.364884+0000","last_became_peered":"2024-09-17T05:07:37.364884+0000","last_unstale":"2024-09-17T05:07:37.371818+0000","last_undegraded":"2024-09-17T05:07:37.371818+0000","last_fullsized":"2024-09-17T05:07:37.371818+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:45.779925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343962+0000","last_change":"2024-09-17T05:07:37.343962+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343962+0000","last_undegraded":"2024-09-17T05:07:37.343962+0000","last_fullsized":"2024-09-17T05:07:37.343962+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.364152+0000","last_change":"2024-09-17T05:07:37.358555+0000","last_active":"2024-09-17T05:07:37.364152+0000","last_peered":"2024-09-17T05:07:37.364152+0000","last_clean":"2024-09-17T05:07:37.364152+0000","last_became_active":"2024-09-17T05:07:37.358188+0000","last_became_peered":"2024-09-17T05:07:37.358188+0000","last_unstale":"2024-09-17T05:07:37.364152+0000","last_undegraded":"2024-09-17T05:07:37.364152+0000","last_fullsized":"2024-09-17T05:07:37.364152+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:55.685542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.345638+0000","last_change":"2024-09-17T05:07:37.345638+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.345638+0000","last_undegraded":"2024-09-17T05:07:37.345638+0000","last_fullsized":"2024-09-17T05:07:37.345638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366332+0000","last_change":"2024-09-17T05:07:37.363006+0000","last_active":"2024-09-17T05:07:37.366332+0000","last_peered":"2024-09-17T05:07:37.366332+0000","last_clean":"2024-09-17T05:07:37.366332+0000","last_became_active":"2024-09-17T05:07:37.362664+0000","last_became_peered":"2024-09-17T05:07:37.362664+0000","last_unstale":"2024-09-17T05:07:37.366332+0000","last_undegraded":"2024-09-17T05:07:37.366332+0000","last_fullsized":"2024-09-17T05:07:37.366332+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:34.154990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353663+0000","last_change":"2024-09-17T05:07:37.353663+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353663+0000","last_undegraded":"2024-09-17T05:07:37.353663+0000","last_fullsized":"2024-09-17T05:07:37.353663+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.368799+0000","last_change":"2024-09-17T05:07:37.365145+0000","last_active":"2024-09-17T05:07:37.368799+0000","last_peered":"2024-09-17T05:07:37.368799+0000","last_clean":"2024-09-17T05:07:37.368799+0000","last_became_active":"2024-09-17T05:07:37.364775+0000","last_became_peered":"2024-09-17T05:07:37.364775+0000","last_unstale":"2024-09-17T05:07:37.368799+0000","last_undegraded":"2024-09-17T05:07:37.368799+0000","last_fullsized":"2024-09-17T05:07:37.368799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.877036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365284+0000","last_change":"2024-09-17T05:07:38.369937+0000","last_active":"2024-09-17T05:07:39.365284+0000","last_peered":"2024-09-17T05:07:39.365284+0000","last_clean":"2024-09-17T05:07:39.365284+0000","last_became_active":"2024-09-17T05:07:38.369450+0000","last_became_peered":"2024-09-17T05:07:38.369450+0000","last_unstale":"2024-09-17T05:07:39.365284+0000","last_undegraded":"2024-09-17T05:07:39.365284+0000","last_fullsized":"2024-09-17T05:07:39.365284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:59.251209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.372162+0000","last_change":"2024-09-17T05:07:37.368862+0000","last_active":"2024-09-17T05:07:37.372162+0000","last_peered":"2024-09-17T05:07:37.372162+0000","last_clean":"2024-09-17T05:07:37.372162+0000","last_became_active":"2024-09-17T05:07:37.368509+0000","last_became_peered":"2024-09-17T05:07:37.368509+0000","last_unstale":"2024-09-17T05:07:37.372162+0000","last_undegraded":"2024-09-17T05:07:37.372162+0000","last_fullsized":"2024-09-17T05:07:37.372162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:56.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346070+0000","last_change":"2024-09-17T05:07:37.346070+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346070+0000","last_undegraded":"2024-09-17T05:07:37.346070+0000","last_fullsized":"2024-09-17T05:07:37.346070+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367208+0000","last_change":"2024-09-17T05:07:37.364495+0000","last_active":"2024-09-17T05:07:37.367208+0000","last_peered":"2024-09-17T05:07:37.367208+0000","last_clean":"2024-09-17T05:07:37.367208+0000","last_became_active":"2024-09-17T05:07:37.364156+0000","last_became_peered":"2024-09-17T05:07:37.364156+0000","last_unstale":"2024-09-17T05:07:37.367208+0000","last_undegraded":"2024-09-17T05:07:37.367208+0000","last_fullsized":"2024-09-17T05:07:37.367208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347829+0000","last_change":"2024-09-17T05:07:37.347829+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347829+0000","last_undegraded":"2024-09-17T05:07:37.347829+0000","last_fullsized":"2024-09-17T05:07:37.347829+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371399+0000","last_change":"2024-09-17T05:07:37.367221+0000","last_active":"2024-09-17T05:07:37.371399+0000","last_peered":"2024-09-17T05:07:37.371399+0000","last_clean":"2024-09-17T05:07:37.371399+0000","last_became_active":"2024-09-17T05:07:37.365939+0000","last_became_peered":"2024-09-17T05:07:37.365939+0000","last_unstale":"2024-09-17T05:07:37.371399+0000","last_undegraded":"2024-09-17T05:07:37.371399+0000","last_fullsized":"2024-09-17T05:07:37.371399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:26.077987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347804+0000","last_change":"2024-09-17T05:07:37.347804+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347804+0000","last_undegraded":"2024-09-17T05:07:37.347804+0000","last_fullsized":"2024-09-17T05:07:37.347804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369481+0000","last_change":"2024-09-17T05:07:37.367645+0000","last_active":"2024-09-17T05:07:37.369481+0000","last_peered":"2024-09-17T05:07:37.369481+0000","last_clean":"2024-09-17T05:07:37.369481+0000","last_became_active":"2024-09-17T05:07:37.367258+0000","last_became_peered":"2024-09-17T05:07:37.367258+0000","last_unstale":"2024-09-17T05:07:37.369481+0000","last_undegraded":"2024-09-17T05:07:37.369481+0000","last_fullsized":"2024-09-17T05:07:37.369481+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352275+0000","last_change":"2024-09-17T05:07:37.352275+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352275+0000","last_undegraded":"2024-09-17T05:07:37.352275+0000","last_fullsized":"2024-09-17T05:07:37.352275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364438+0000","last_change":"2024-09-17T05:07:37.364934+0000","last_active":"2024-09-17T05:07:39.364438+0000","last_peered":"2024-09-17T05:07:39.364438+0000","last_clean":"2024-09-17T05:07:39.364438+0000","last_became_active":"2024-09-17T05:07:37.364564+0000","last_became_peered":"2024-09-17T05:07:37.364564+0000","last_unstale":"2024-09-17T05:07:39.364438+0000","last_undegraded":"2024-09-17T05:07:39.364438+0000","last_fullsized":"2024-09-17T05:07:39.364438+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:25.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346823+0000","last_change":"2024-09-17T05:07:37.346823+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346823+0000","last_undegraded":"2024-09-17T05:07:37.346823+0000","last_fullsized":"2024-09-17T05:07:37.346823+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363798+0000","last_change":"2024-09-17T05:07:37.359369+0000","last_active":"2024-09-17T05:07:39.363798+0000","last_peered":"2024-09-17T05:07:39.363798+0000","last_clean":"2024-09-17T05:07:39.363798+0000","last_became_active":"2024-09-17T05:07:37.359073+0000","last_became_peered":"2024-09-17T05:07:37.359073+0000","last_unstale":"2024-09-17T05:07:39.363798+0000","last_undegraded":"2024-09-17T05:07:39.363798+0000","last_fullsized":"2024-09-17T05:07:39.363798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:59.202139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354263+0000","last_change":"2024-09-17T05:07:37.354263+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354263+0000","last_undegraded":"2024-09-17T05:07:37.354263+0000","last_fullsized":"2024-09-17T05:07:37.354263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365151+0000","last_change":"2024-09-17T05:07:37.361606+0000","last_active":"2024-09-17T05:07:37.365151+0000","last_peered":"2024-09-17T05:07:37.365151+0000","last_clean":"2024-09-17T05:07:37.365151+0000","last_became_active":"2024-09-17T05:07:37.361002+0000","last_became_peered":"2024-09-17T05:07:37.361002+0000","last_unstale":"2024-09-17T05:07:37.365151+0000","last_undegraded":"2024-09-17T05:07:37.365151+0000","last_fullsized":"2024-09-17T05:07:37.365151+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:31.412375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353665+0000","last_change":"2024-09-17T05:07:37.353665+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353665+0000","last_undegraded":"2024-09-17T05:07:37.353665+0000","last_fullsized":"2024-09-17T05:07:37.353665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366271+0000","last_change":"2024-09-17T05:07:37.362489+0000","last_active":"2024-09-17T05:07:37.366271+0000","last_peered":"2024-09-17T05:07:37.366271+0000","last_clean":"2024-09-17T05:07:37.366271+0000","last_became_active":"2024-09-17T05:07:37.361729+0000","last_became_peered":"2024-09-17T05:07:37.361729+0000","last_unstale":"2024-09-17T05:07:37.366271+0000","last_undegraded":"2024-09-17T05:07:37.366271+0000","last_fullsized":"2024-09-17T05:07:37.366271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348338+0000","last_change":"2024-09-17T05:07:37.348338+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348338+0000","last_undegraded":"2024-09-17T05:07:37.348338+0000","last_fullsized":"2024-09-17T05:07:37.348338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365507+0000","last_change":"2024-09-17T05:07:37.360081+0000","last_active":"2024-09-17T05:07:39.365507+0000","last_peered":"2024-09-17T05:07:39.365507+0000","last_clean":"2024-09-17T05:07:39.365507+0000","last_became_active":"2024-09-17T05:07:37.359720+0000","last_became_peered":"2024-09-17T05:07:37.359720+0000","last_unstale":"2024-09-17T05:07:39.365507+0000","last_undegraded":"2024-09-17T05:07:39.365507+0000","last_fullsized":"2024-09-17T05:07:39.365507+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:16.580036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347399+0000","last_change":"2024-09-17T05:07:37.347399+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347399+0000","last_undegraded":"2024-09-17T05:07:37.347399+0000","last_fullsized":"2024-09-17T05:07:37.347399+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.370455+0000","last_change":"2024-09-17T05:07:37.358084+0000","last_active":"2024-09-17T05:07:39.370455+0000","last_peered":"2024-09-17T05:07:39.370455+0000","last_clean":"2024-09-17T05:07:39.370455+0000","last_became_active":"2024-09-17T05:07:37.357773+0000","last_became_peered":"2024-09-17T05:07:37.357773+0000","last_unstale":"2024-09-17T05:07:39.370455+0000","last_undegraded":"2024-09-17T05:07:39.370455+0000","last_fullsized":"2024-09-17T05:07:39.370455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:02.583665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348583+0000","last_change":"2024-09-17T05:07:37.348583+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348583+0000","last_undegraded":"2024-09-17T05:07:37.348583+0000","last_fullsized":"2024-09-17T05:07:37.348583+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.354323+0000","last_change":"2024-09-17T05:07:37.350958+0000","last_active":"2024-09-17T05:07:37.354323+0000","last_peered":"2024-09-17T05:07:37.354323+0000","last_clean":"2024-09-17T05:07:37.354323+0000","last_became_active":"2024-09-17T05:07:37.349820+0000","last_became_peered":"2024-09-17T05:07:37.349820+0000","last_unstale":"2024-09-17T05:07:37.354323+0000","last_undegraded":"2024-09-17T05:07:37.354323+0000","last_fullsized":"2024-09-17T05:07:37.354323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:25.046789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348957+0000","last_change":"2024-09-17T05:07:37.348957+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348957+0000","last_undegraded":"2024-09-17T05:07:37.348957+0000","last_fullsized":"2024-09-17T05:07:37.348957+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369179+0000","last_change":"2024-09-17T05:07:37.368314+0000","last_active":"2024-09-17T05:07:37.369179+0000","last_peered":"2024-09-17T05:07:37.369179+0000","last_clean":"2024-09-17T05:07:37.369179+0000","last_became_active":"2024-09-17T05:07:37.367953+0000","last_became_peered":"2024-09-17T05:07:37.367953+0000","last_unstale":"2024-09-17T05:07:37.369179+0000","last_undegraded":"2024-09-17T05:07:37.369179+0000","last_fullsized":"2024-09-17T05:07:37.369179+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:21.094340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352857+0000","last_change":"2024-09-17T05:07:37.352857+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352857+0000","last_undegraded":"2024-09-17T05:07:37.352857+0000","last_fullsized":"2024-09-17T05:07:37.352857+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371407+0000","last_change":"2024-09-17T05:07:37.364116+0000","last_active":"2024-09-17T05:07:37.371407+0000","last_peered":"2024-09-17T05:07:37.371407+0000","last_clean":"2024-09-17T05:07:37.371407+0000","last_became_active":"2024-09-17T05:07:37.363791+0000","last_became_peered":"2024-09-17T05:07:37.363791+0000","last_unstale":"2024-09-17T05:07:37.371407+0000","last_undegraded":"2024-09-17T05:07:37.371407+0000","last_fullsized":"2024-09-17T05:07:37.371407+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:14.819276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358208+0000","last_change":"2024-09-17T05:07:37.358208+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358208+0000","last_undegraded":"2024-09-17T05:07:37.358208+0000","last_fullsized":"2024-09-17T05:07:37.358208+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364119+0000","last_change":"2024-09-17T05:07:37.357614+0000","last_active":"2024-09-17T05:07:39.364119+0000","last_peered":"2024-09-17T05:07:39.364119+0000","last_clean":"2024-09-17T05:07:39.364119+0000","last_became_active":"2024-09-17T05:07:37.357276+0000","last_became_peered":"2024-09-17T05:07:37.357276+0000","last_unstale":"2024-09-17T05:07:39.364119+0000","last_undegraded":"2024-09-17T05:07:39.364119+0000","last_fullsized":"2024-09-17T05:07:39.364119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:39.232366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347380+0000","last_change":"2024-09-17T05:07:37.347380+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347380+0000","last_undegraded":"2024-09-17T05:07:37.347380+0000","last_fullsized":"2024-09-17T05:07:37.347380+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369451+0000","last_change":"2024-09-17T05:07:37.364772+0000","last_active":"2024-09-17T05:07:37.369451+0000","last_peered":"2024-09-17T05:07:37.369451+0000","last_clean":"2024-09-17T05:07:37.369451+0000","last_became_active":"2024-09-17T05:07:37.364468+0000","last_became_peered":"2024-09-17T05:07:37.364468+0000","last_unstale":"2024-09-17T05:07:37.369451+0000","last_undegraded":"2024-09-17T05:07:37.369451+0000","last_fullsized":"2024-09-17T05:07:37.369451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:31.412375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343458+0000","last_change":"2024-09-17T05:07:37.343458+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343458+0000","last_undegraded":"2024-09-17T05:07:37.343458+0000","last_fullsized":"2024-09-17T05:07:37.343458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363935+0000","last_change":"2024-09-17T05:07:37.355234+0000","last_active":"2024-09-17T05:07:39.363935+0000","last_peered":"2024-09-17T05:07:39.363935+0000","last_clean":"2024-09-17T05:07:39.363935+0000","last_became_active":"2024-09-17T05:07:37.354718+0000","last_became_peered":"2024-09-17T05:07:37.354718+0000","last_unstale":"2024-09-17T05:07:39.363935+0000","last_undegraded":"2024-09-17T05:07:39.363935+0000","last_fullsized":"2024-09-17T05:07:39.363935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.673587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347653+0000","last_change":"2024-09-17T05:07:37.347653+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347653+0000","last_undegraded":"2024-09-17T05:07:37.347653+0000","last_fullsized":"2024-09-17T05:07:37.347653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367557+0000","last_change":"2024-09-17T05:07:37.363388+0000","last_active":"2024-09-17T05:07:37.367557+0000","last_peered":"2024-09-17T05:07:37.367557+0000","last_clean":"2024-09-17T05:07:37.367557+0000","last_became_active":"2024-09-17T05:07:37.362169+0000","last_became_peered":"2024-09-17T05:07:37.362169+0000","last_unstale":"2024-09-17T05:07:37.367557+0000","last_undegraded":"2024-09-17T05:07:37.367557+0000","last_fullsized":"2024-09-17T05:07:37.367557+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:17.013225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.359581+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:07:37.359581+0000","last_peered":"2024-09-17T05:07:37.359581+0000","last_clean":"2024-09-17T05:07:37.359581+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:07:37.359581+0000","last_undegraded":"2024-09-17T05:07:37.359581+0000","last_fullsized":"2024-09-17T05:07:37.359581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36528,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335312,"statfs":{"total":96636764160,"available":96599359488,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1789952,"data_stored":1134999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1835008,"data_stored":1155649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:07:42.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1036232858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 msgr2=0x7fe6dc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1036232858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc15bb60 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009900 tx=0x7fe6cc01c850 comp rx=0 tx=0).stop 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1036232858 shutdown_connections 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1036232858 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6dc15c0a0 0x7fe6dc15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1036232858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc15bb60 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1036232858 >> 172.21.15.16:0/1036232858 conn(0x7fe6dc06f900 msgr2=0x7fe6dc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1036232858 shutdown_connections 2024-09-17T05:07:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1036232858 wait complete. 2024-09-17T05:07:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 Processor -- start 2024-09-17T05:07:42.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.803+0000 7fe6e1e00640 1 -- start start 2024-09-17T05:07:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc1603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6dc15c0a0 0x7fe6dc0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6dc15f150 0x7fe6dc0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc169120 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc168fa0 con 0x7fe6dc15c0a0 2024-09-17T05:07:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc1692a0 con 0x7fe6dc15f150 2024-09-17T05:07:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc1603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc1603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42448/0 (socket says 172.21.15.16:42448) 2024-09-17T05:07:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 -- 172.21.15.16:0/1680545104 learned_addr learned my addr 172.21.15.16:0/1680545104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 -- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6dc15f150 msgr2=0x7fe6dc0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6dc15f150 0x7fe6dc0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 -- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6dc15c0a0 msgr2=0x7fe6dc0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6dc15c0a0 0x7fe6dc0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 -- 172.21.15.16:0/1680545104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc161140 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6daa00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc1603d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009850 tx=0x7fe6cc0a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0b2040 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6cc004230 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc161420 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc003600 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1619e0 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.807+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc15bb60 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.811+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6cc0bf020 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.811+0000 7fe6d3400640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe6b8073230 0x7fe6b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.811+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fe6cc12ec40 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.811+0000 7fe6d3e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe6b8073230 0x7fe6b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:42.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.815+0000 7fe6d3e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe6b8073230 0x7fe6b8075740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40058d0 tx=0x7fe6c4005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:42.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.815+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe6cc0c6020 con 0x7fe6dc0b2f50 2024-09-17T05:07:42.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.987+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6dc15d040 con 0x7fe6b8073230 2024-09-17T05:07:42.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.991+0000 7fe6d3400640 1 -- 172.21.15.16:0/1680545104 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247220 (secure 0 0 0) 0x7fe6dc15d040 con 0x7fe6b8073230 2024-09-17T05:07:42.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:07:42.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe6b8073230 msgr2=0x7fe6b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe6b8073230 0x7fe6b8075740 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40058d0 tx=0x7fe6c4005860 comp rx=0 tx=0).stop 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 msgr2=0x7fe6dc1603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc1603d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009850 tx=0x7fe6cc0a5840 comp rx=0 tx=0).stop 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 shutdown_connections 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe6b8073230 0x7fe6b8075740 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe6dc15f150 0x7fe6dc0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6dc15c0a0 0x7fe6dc0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 --2- 172.21.15.16:0/1680545104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc0b2f50 0x7fe6dc1603d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 >> 172.21.15.16:0/1680545104 conn(0x7fe6dc06f900 msgr2=0x7fe6dc165ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 shutdown_connections 2024-09-17T05:07:42.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:42.995+0000 7fe6e1e00640 1 -- 172.21.15.16:0/1680545104 wait complete. 2024-09-17T05:07:43.015 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-17T05:07:41.570418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268440,"kb_used_data":10584,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706280,"statfs":{"total":773094113280,"available":772819230720,"internally_reserved":0,"allocated":10838016,"data_stored":5578435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.244815"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354142+0000","last_change":"2024-09-17T05:07:37.354142+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354142+0000","last_undegraded":"2024-09-17T05:07:37.354142+0000","last_fullsized":"2024-09-17T05:07:37.354142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344873+0000","last_change":"2024-09-17T05:07:37.344873+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344873+0000","last_undegraded":"2024-09-17T05:07:37.344873+0000","last_fullsized":"2024-09-17T05:07:37.344873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365386+0000","last_change":"2024-09-17T05:07:38.373066+0000","last_active":"2024-09-17T05:07:39.365386+0000","last_peered":"2024-09-17T05:07:39.365386+0000","last_clean":"2024-09-17T05:07:39.365386+0000","last_became_active":"2024-09-17T05:07:38.372726+0000","last_became_peered":"2024-09-17T05:07:38.372726+0000","last_unstale":"2024-09-17T05:07:39.365386+0000","last_undegraded":"2024-09-17T05:07:39.365386+0000","last_fullsized":"2024-09-17T05:07:39.365386+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:40.211315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353488+0000","last_change":"2024-09-17T05:07:37.353488+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353488+0000","last_undegraded":"2024-09-17T05:07:37.353488+0000","last_fullsized":"2024-09-17T05:07:37.353488+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353790+0000","last_change":"2024-09-17T05:07:37.353790+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353790+0000","last_undegraded":"2024-09-17T05:07:37.353790+0000","last_fullsized":"2024-09-17T05:07:37.353790+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352451+0000","last_change":"2024-09-17T05:07:37.352451+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352451+0000","last_undegraded":"2024-09-17T05:07:37.352451+0000","last_fullsized":"2024-09-17T05:07:37.352451+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352553+0000","last_change":"2024-09-17T05:07:37.352553+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352553+0000","last_undegraded":"2024-09-17T05:07:37.352553+0000","last_fullsized":"2024-09-17T05:07:37.352553+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347288+0000","last_change":"2024-09-17T05:07:37.347288+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347288+0000","last_undegraded":"2024-09-17T05:07:37.347288+0000","last_fullsized":"2024-09-17T05:07:37.347288+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344945+0000","last_change":"2024-09-17T05:07:37.344945+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344945+0000","last_undegraded":"2024-09-17T05:07:37.344945+0000","last_fullsized":"2024-09-17T05:07:37.344945+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.349207+0000","last_change":"2024-09-17T05:07:37.349207+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.349207+0000","last_undegraded":"2024-09-17T05:07:37.349207+0000","last_fullsized":"2024-09-17T05:07:37.349207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348392+0000","last_change":"2024-09-17T05:07:37.348392+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348392+0000","last_undegraded":"2024-09-17T05:07:37.348392+0000","last_fullsized":"2024-09-17T05:07:37.348392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.351164+0000","last_change":"2024-09-17T05:07:37.351164+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.351164+0000","last_undegraded":"2024-09-17T05:07:37.351164+0000","last_fullsized":"2024-09-17T05:07:37.351164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354316+0000","last_change":"2024-09-17T05:07:37.354316+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354316+0000","last_undegraded":"2024-09-17T05:07:37.354316+0000","last_fullsized":"2024-09-17T05:07:37.354316+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350574+0000","last_change":"2024-09-17T05:07:37.350574+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350574+0000","last_undegraded":"2024-09-17T05:07:37.350574+0000","last_fullsized":"2024-09-17T05:07:37.350574+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343241+0000","last_change":"2024-09-17T05:07:37.343241+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343241+0000","last_undegraded":"2024-09-17T05:07:37.343241+0000","last_fullsized":"2024-09-17T05:07:37.343241+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.359263+0000","last_change":"2024-09-17T05:07:37.359263+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.359263+0000","last_undegraded":"2024-09-17T05:07:37.359263+0000","last_fullsized":"2024-09-17T05:07:37.359263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365037+0000","last_change":"2024-09-17T05:07:38.369916+0000","last_active":"2024-09-17T05:07:39.365037+0000","last_peered":"2024-09-17T05:07:39.365037+0000","last_clean":"2024-09-17T05:07:39.365037+0000","last_became_active":"2024-09-17T05:07:38.369536+0000","last_became_peered":"2024-09-17T05:07:38.369536+0000","last_unstale":"2024-09-17T05:07:39.365037+0000","last_undegraded":"2024-09-17T05:07:39.365037+0000","last_fullsized":"2024-09-17T05:07:39.365037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:07.275670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344273+0000","last_change":"2024-09-17T05:07:37.344273+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344273+0000","last_undegraded":"2024-09-17T05:07:37.344273+0000","last_fullsized":"2024-09-17T05:07:37.344273+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358759+0000","last_change":"2024-09-17T05:07:37.358759+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358759+0000","last_undegraded":"2024-09-17T05:07:37.358759+0000","last_fullsized":"2024-09-17T05:07:37.358759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358844+0000","last_change":"2024-09-17T05:07:37.358844+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358844+0000","last_undegraded":"2024-09-17T05:07:37.358844+0000","last_fullsized":"2024-09-17T05:07:37.358844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364572+0000","last_change":"2024-09-17T05:07:38.365788+0000","last_active":"2024-09-17T05:07:39.364572+0000","last_peered":"2024-09-17T05:07:39.364572+0000","last_clean":"2024-09-17T05:07:39.364572+0000","last_became_active":"2024-09-17T05:07:38.364783+0000","last_became_peered":"2024-09-17T05:07:38.364783+0000","last_unstale":"2024-09-17T05:07:39.364572+0000","last_undegraded":"2024-09-17T05:07:39.364572+0000","last_fullsized":"2024-09-17T05:07:39.364572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:32.081371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352974+0000","last_change":"2024-09-17T05:07:37.352974+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352974+0000","last_undegraded":"2024-09-17T05:07:37.352974+0000","last_fullsized":"2024-09-17T05:07:37.352974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347298+0000","last_change":"2024-09-17T05:07:37.347298+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347298+0000","last_undegraded":"2024-09-17T05:07:37.347298+0000","last_fullsized":"2024-09-17T05:07:37.347298+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346741+0000","last_change":"2024-09-17T05:07:37.346741+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346741+0000","last_undegraded":"2024-09-17T05:07:37.346741+0000","last_fullsized":"2024-09-17T05:07:37.346741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.355031+0000","last_change":"2024-09-17T05:07:37.355031+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.355031+0000","last_undegraded":"2024-09-17T05:07:37.355031+0000","last_fullsized":"2024-09-17T05:07:37.355031+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354956+0000","last_change":"2024-09-17T05:07:37.354956+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354956+0000","last_undegraded":"2024-09-17T05:07:37.354956+0000","last_fullsized":"2024-09-17T05:07:37.354956+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354198+0000","last_change":"2024-09-17T05:07:37.354198+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354198+0000","last_undegraded":"2024-09-17T05:07:37.354198+0000","last_fullsized":"2024-09-17T05:07:37.354198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350568+0000","last_change":"2024-09-17T05:07:37.350568+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350568+0000","last_undegraded":"2024-09-17T05:07:37.350568+0000","last_fullsized":"2024-09-17T05:07:37.350568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347094+0000","last_change":"2024-09-17T05:07:37.347094+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347094+0000","last_undegraded":"2024-09-17T05:07:37.347094+0000","last_fullsized":"2024-09-17T05:07:37.347094+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347191+0000","last_change":"2024-09-17T05:07:37.347191+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347191+0000","last_undegraded":"2024-09-17T05:07:37.347191+0000","last_fullsized":"2024-09-17T05:07:37.347191+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350459+0000","last_change":"2024-09-17T05:07:37.350459+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350459+0000","last_undegraded":"2024-09-17T05:07:37.350459+0000","last_fullsized":"2024-09-17T05:07:37.350459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348834+0000","last_change":"2024-09-17T05:07:37.348834+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348834+0000","last_undegraded":"2024-09-17T05:07:37.348834+0000","last_fullsized":"2024-09-17T05:07:37.348834+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363817+0000","last_change":"2024-09-17T05:07:38.351295+0000","last_active":"2024-09-17T05:07:39.363817+0000","last_peered":"2024-09-17T05:07:39.363817+0000","last_clean":"2024-09-17T05:07:39.363817+0000","last_became_active":"2024-09-17T05:07:38.348971+0000","last_became_peered":"2024-09-17T05:07:38.348971+0000","last_unstale":"2024-09-17T05:07:39.363817+0000","last_undegraded":"2024-09-17T05:07:39.363817+0000","last_fullsized":"2024-09-17T05:07:39.363817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:12.675618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365425+0000","last_change":"2024-09-17T05:07:37.361640+0000","last_active":"2024-09-17T05:07:37.365425+0000","last_peered":"2024-09-17T05:07:37.365425+0000","last_clean":"2024-09-17T05:07:37.365425+0000","last_became_active":"2024-09-17T05:07:37.360159+0000","last_became_peered":"2024-09-17T05:07:37.360159+0000","last_unstale":"2024-09-17T05:07:37.365425+0000","last_undegraded":"2024-09-17T05:07:37.365425+0000","last_fullsized":"2024-09-17T05:07:37.365425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:22.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346712+0000","last_change":"2024-09-17T05:07:37.346712+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346712+0000","last_undegraded":"2024-09-17T05:07:37.346712+0000","last_fullsized":"2024-09-17T05:07:37.346712+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364559+0000","last_change":"2024-09-17T05:07:37.367481+0000","last_active":"2024-09-17T05:07:39.364559+0000","last_peered":"2024-09-17T05:07:39.364559+0000","last_clean":"2024-09-17T05:07:39.364559+0000","last_became_active":"2024-09-17T05:07:37.367109+0000","last_became_peered":"2024-09-17T05:07:37.367109+0000","last_unstale":"2024-09-17T05:07:39.364559+0000","last_undegraded":"2024-09-17T05:07:39.364559+0000","last_fullsized":"2024-09-17T05:07:39.364559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:25.251480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344456+0000","last_change":"2024-09-17T05:07:37.344456+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344456+0000","last_undegraded":"2024-09-17T05:07:37.344456+0000","last_fullsized":"2024-09-17T05:07:37.344456+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.363644+0000","last_change":"2024-09-17T05:07:37.360648+0000","last_active":"2024-09-17T05:07:37.363644+0000","last_peered":"2024-09-17T05:07:37.363644+0000","last_clean":"2024-09-17T05:07:37.363644+0000","last_became_active":"2024-09-17T05:07:37.356573+0000","last_became_peered":"2024-09-17T05:07:37.356573+0000","last_unstale":"2024-09-17T05:07:37.363644+0000","last_undegraded":"2024-09-17T05:07:37.363644+0000","last_fullsized":"2024-09-17T05:07:37.363644+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:23.902999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346600+0000","last_change":"2024-09-17T05:07:37.346600+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346600+0000","last_undegraded":"2024-09-17T05:07:37.346600+0000","last_fullsized":"2024-09-17T05:07:37.346600+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367346+0000","last_change":"2024-09-17T05:07:37.364681+0000","last_active":"2024-09-17T05:07:37.367346+0000","last_peered":"2024-09-17T05:07:37.367346+0000","last_clean":"2024-09-17T05:07:37.367346+0000","last_became_active":"2024-09-17T05:07:37.364348+0000","last_became_peered":"2024-09-17T05:07:37.364348+0000","last_unstale":"2024-09-17T05:07:37.367346+0000","last_undegraded":"2024-09-17T05:07:37.367346+0000","last_fullsized":"2024-09-17T05:07:37.367346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:09.224428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.349759+0000","last_change":"2024-09-17T05:07:37.349759+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.349759+0000","last_undegraded":"2024-09-17T05:07:37.349759+0000","last_fullsized":"2024-09-17T05:07:37.349759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371989+0000","last_change":"2024-09-17T05:07:37.368233+0000","last_active":"2024-09-17T05:07:37.371989+0000","last_peered":"2024-09-17T05:07:37.371989+0000","last_clean":"2024-09-17T05:07:37.371989+0000","last_became_active":"2024-09-17T05:07:37.367904+0000","last_became_peered":"2024-09-17T05:07:37.367904+0000","last_unstale":"2024-09-17T05:07:37.371989+0000","last_undegraded":"2024-09-17T05:07:37.371989+0000","last_fullsized":"2024-09-17T05:07:37.371989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:38.417939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354832+0000","last_change":"2024-09-17T05:07:37.354832+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354832+0000","last_undegraded":"2024-09-17T05:07:37.354832+0000","last_fullsized":"2024-09-17T05:07:37.354832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.372345+0000","last_change":"2024-09-17T05:07:37.368607+0000","last_active":"2024-09-17T05:07:37.372345+0000","last_peered":"2024-09-17T05:07:37.372345+0000","last_clean":"2024-09-17T05:07:37.372345+0000","last_became_active":"2024-09-17T05:07:37.368307+0000","last_became_peered":"2024-09-17T05:07:37.368307+0000","last_unstale":"2024-09-17T05:07:37.372345+0000","last_undegraded":"2024-09-17T05:07:37.372345+0000","last_fullsized":"2024-09-17T05:07:37.372345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346749+0000","last_change":"2024-09-17T05:07:37.346749+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346749+0000","last_undegraded":"2024-09-17T05:07:37.346749+0000","last_fullsized":"2024-09-17T05:07:37.346749+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371473+0000","last_change":"2024-09-17T05:07:37.367847+0000","last_active":"2024-09-17T05:07:37.371473+0000","last_peered":"2024-09-17T05:07:37.371473+0000","last_clean":"2024-09-17T05:07:37.371473+0000","last_became_active":"2024-09-17T05:07:37.367465+0000","last_became_peered":"2024-09-17T05:07:37.367465+0000","last_unstale":"2024-09-17T05:07:37.371473+0000","last_undegraded":"2024-09-17T05:07:37.371473+0000","last_fullsized":"2024-09-17T05:07:37.371473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:41.729615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353184+0000","last_change":"2024-09-17T05:07:37.353184+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353184+0000","last_undegraded":"2024-09-17T05:07:37.353184+0000","last_fullsized":"2024-09-17T05:07:37.353184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367733+0000","last_change":"2024-09-17T05:07:37.364825+0000","last_active":"2024-09-17T05:07:37.367733+0000","last_peered":"2024-09-17T05:07:37.367733+0000","last_clean":"2024-09-17T05:07:37.367733+0000","last_became_active":"2024-09-17T05:07:37.364481+0000","last_became_peered":"2024-09-17T05:07:37.364481+0000","last_unstale":"2024-09-17T05:07:37.367733+0000","last_undegraded":"2024-09-17T05:07:37.367733+0000","last_fullsized":"2024-09-17T05:07:37.367733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:05.942761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348642+0000","last_change":"2024-09-17T05:07:37.348642+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348642+0000","last_undegraded":"2024-09-17T05:07:37.348642+0000","last_fullsized":"2024-09-17T05:07:37.348642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366699+0000","last_change":"2024-09-17T05:07:37.362446+0000","last_active":"2024-09-17T05:07:37.366699+0000","last_peered":"2024-09-17T05:07:37.366699+0000","last_clean":"2024-09-17T05:07:37.366699+0000","last_became_active":"2024-09-17T05:07:37.362104+0000","last_became_peered":"2024-09-17T05:07:37.362104+0000","last_unstale":"2024-09-17T05:07:37.366699+0000","last_undegraded":"2024-09-17T05:07:37.366699+0000","last_fullsized":"2024-09-17T05:07:37.366699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:41.729615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348460+0000","last_change":"2024-09-17T05:07:37.348460+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348460+0000","last_undegraded":"2024-09-17T05:07:37.348460+0000","last_fullsized":"2024-09-17T05:07:37.348460+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.361947+0000","last_change":"2024-09-17T05:07:37.359338+0000","last_active":"2024-09-17T05:07:37.361947+0000","last_peered":"2024-09-17T05:07:37.361947+0000","last_clean":"2024-09-17T05:07:37.361947+0000","last_became_active":"2024-09-17T05:07:37.359008+0000","last_became_peered":"2024-09-17T05:07:37.359008+0000","last_unstale":"2024-09-17T05:07:37.361947+0000","last_undegraded":"2024-09-17T05:07:37.361947+0000","last_fullsized":"2024-09-17T05:07:37.361947+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348027+0000","last_change":"2024-09-17T05:07:37.348027+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348027+0000","last_undegraded":"2024-09-17T05:07:37.348027+0000","last_fullsized":"2024-09-17T05:07:37.348027+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365107+0000","last_change":"2024-09-17T05:07:37.362270+0000","last_active":"2024-09-17T05:07:37.365107+0000","last_peered":"2024-09-17T05:07:37.365107+0000","last_clean":"2024-09-17T05:07:37.365107+0000","last_became_active":"2024-09-17T05:07:37.361924+0000","last_became_peered":"2024-09-17T05:07:37.361924+0000","last_unstale":"2024-09-17T05:07:37.365107+0000","last_undegraded":"2024-09-17T05:07:37.365107+0000","last_fullsized":"2024-09-17T05:07:37.365107+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:26.077987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347252+0000","last_change":"2024-09-17T05:07:37.347252+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347252+0000","last_undegraded":"2024-09-17T05:07:37.347252+0000","last_fullsized":"2024-09-17T05:07:37.347252+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371818+0000","last_change":"2024-09-17T05:07:37.365192+0000","last_active":"2024-09-17T05:07:37.371818+0000","last_peered":"2024-09-17T05:07:37.371818+0000","last_clean":"2024-09-17T05:07:37.371818+0000","last_became_active":"2024-09-17T05:07:37.364884+0000","last_became_peered":"2024-09-17T05:07:37.364884+0000","last_unstale":"2024-09-17T05:07:37.371818+0000","last_undegraded":"2024-09-17T05:07:37.371818+0000","last_fullsized":"2024-09-17T05:07:37.371818+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:45.779925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343962+0000","last_change":"2024-09-17T05:07:37.343962+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343962+0000","last_undegraded":"2024-09-17T05:07:37.343962+0000","last_fullsized":"2024-09-17T05:07:37.343962+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.364152+0000","last_change":"2024-09-17T05:07:37.358555+0000","last_active":"2024-09-17T05:07:37.364152+0000","last_peered":"2024-09-17T05:07:37.364152+0000","last_clean":"2024-09-17T05:07:37.364152+0000","last_became_active":"2024-09-17T05:07:37.358188+0000","last_became_peered":"2024-09-17T05:07:37.358188+0000","last_unstale":"2024-09-17T05:07:37.364152+0000","last_undegraded":"2024-09-17T05:07:37.364152+0000","last_fullsized":"2024-09-17T05:07:37.364152+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:55.685542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.345638+0000","last_change":"2024-09-17T05:07:37.345638+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.345638+0000","last_undegraded":"2024-09-17T05:07:37.345638+0000","last_fullsized":"2024-09-17T05:07:37.345638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366332+0000","last_change":"2024-09-17T05:07:37.363006+0000","last_active":"2024-09-17T05:07:37.366332+0000","last_peered":"2024-09-17T05:07:37.366332+0000","last_clean":"2024-09-17T05:07:37.366332+0000","last_became_active":"2024-09-17T05:07:37.362664+0000","last_became_peered":"2024-09-17T05:07:37.362664+0000","last_unstale":"2024-09-17T05:07:37.366332+0000","last_undegraded":"2024-09-17T05:07:37.366332+0000","last_fullsized":"2024-09-17T05:07:37.366332+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:34.154990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353663+0000","last_change":"2024-09-17T05:07:37.353663+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353663+0000","last_undegraded":"2024-09-17T05:07:37.353663+0000","last_fullsized":"2024-09-17T05:07:37.353663+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.368799+0000","last_change":"2024-09-17T05:07:37.365145+0000","last_active":"2024-09-17T05:07:37.368799+0000","last_peered":"2024-09-17T05:07:37.368799+0000","last_clean":"2024-09-17T05:07:37.368799+0000","last_became_active":"2024-09-17T05:07:37.364775+0000","last_became_peered":"2024-09-17T05:07:37.364775+0000","last_unstale":"2024-09-17T05:07:37.368799+0000","last_undegraded":"2024-09-17T05:07:37.368799+0000","last_fullsized":"2024-09-17T05:07:37.368799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.877036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365284+0000","last_change":"2024-09-17T05:07:38.369937+0000","last_active":"2024-09-17T05:07:39.365284+0000","last_peered":"2024-09-17T05:07:39.365284+0000","last_clean":"2024-09-17T05:07:39.365284+0000","last_became_active":"2024-09-17T05:07:38.369450+0000","last_became_peered":"2024-09-17T05:07:38.369450+0000","last_unstale":"2024-09-17T05:07:39.365284+0000","last_undegraded":"2024-09-17T05:07:39.365284+0000","last_fullsized":"2024-09-17T05:07:39.365284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:59.251209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.372162+0000","last_change":"2024-09-17T05:07:37.368862+0000","last_active":"2024-09-17T05:07:37.372162+0000","last_peered":"2024-09-17T05:07:37.372162+0000","last_clean":"2024-09-17T05:07:37.372162+0000","last_became_active":"2024-09-17T05:07:37.368509+0000","last_became_peered":"2024-09-17T05:07:37.368509+0000","last_unstale":"2024-09-17T05:07:37.372162+0000","last_undegraded":"2024-09-17T05:07:37.372162+0000","last_fullsized":"2024-09-17T05:07:37.372162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:56.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346070+0000","last_change":"2024-09-17T05:07:37.346070+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346070+0000","last_undegraded":"2024-09-17T05:07:37.346070+0000","last_fullsized":"2024-09-17T05:07:37.346070+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367208+0000","last_change":"2024-09-17T05:07:37.364495+0000","last_active":"2024-09-17T05:07:37.367208+0000","last_peered":"2024-09-17T05:07:37.367208+0000","last_clean":"2024-09-17T05:07:37.367208+0000","last_became_active":"2024-09-17T05:07:37.364156+0000","last_became_peered":"2024-09-17T05:07:37.364156+0000","last_unstale":"2024-09-17T05:07:37.367208+0000","last_undegraded":"2024-09-17T05:07:37.367208+0000","last_fullsized":"2024-09-17T05:07:37.367208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347829+0000","last_change":"2024-09-17T05:07:37.347829+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347829+0000","last_undegraded":"2024-09-17T05:07:37.347829+0000","last_fullsized":"2024-09-17T05:07:37.347829+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371399+0000","last_change":"2024-09-17T05:07:37.367221+0000","last_active":"2024-09-17T05:07:37.371399+0000","last_peered":"2024-09-17T05:07:37.371399+0000","last_clean":"2024-09-17T05:07:37.371399+0000","last_became_active":"2024-09-17T05:07:37.365939+0000","last_became_peered":"2024-09-17T05:07:37.365939+0000","last_unstale":"2024-09-17T05:07:37.371399+0000","last_undegraded":"2024-09-17T05:07:37.371399+0000","last_fullsized":"2024-09-17T05:07:37.371399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:26.077987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347804+0000","last_change":"2024-09-17T05:07:37.347804+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347804+0000","last_undegraded":"2024-09-17T05:07:37.347804+0000","last_fullsized":"2024-09-17T05:07:37.347804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369481+0000","last_change":"2024-09-17T05:07:37.367645+0000","last_active":"2024-09-17T05:07:37.369481+0000","last_peered":"2024-09-17T05:07:37.369481+0000","last_clean":"2024-09-17T05:07:37.369481+0000","last_became_active":"2024-09-17T05:07:37.367258+0000","last_became_peered":"2024-09-17T05:07:37.367258+0000","last_unstale":"2024-09-17T05:07:37.369481+0000","last_undegraded":"2024-09-17T05:07:37.369481+0000","last_fullsized":"2024-09-17T05:07:37.369481+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352275+0000","last_change":"2024-09-17T05:07:37.352275+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352275+0000","last_undegraded":"2024-09-17T05:07:37.352275+0000","last_fullsized":"2024-09-17T05:07:37.352275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364438+0000","last_change":"2024-09-17T05:07:37.364934+0000","last_active":"2024-09-17T05:07:39.364438+0000","last_peered":"2024-09-17T05:07:39.364438+0000","last_clean":"2024-09-17T05:07:39.364438+0000","last_became_active":"2024-09-17T05:07:37.364564+0000","last_became_peered":"2024-09-17T05:07:37.364564+0000","last_unstale":"2024-09-17T05:07:39.364438+0000","last_undegraded":"2024-09-17T05:07:39.364438+0000","last_fullsized":"2024-09-17T05:07:39.364438+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:25.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346823+0000","last_change":"2024-09-17T05:07:37.346823+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346823+0000","last_undegraded":"2024-09-17T05:07:37.346823+0000","last_fullsized":"2024-09-17T05:07:37.346823+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363798+0000","last_change":"2024-09-17T05:07:37.359369+0000","last_active":"2024-09-17T05:07:39.363798+0000","last_peered":"2024-09-17T05:07:39.363798+0000","last_clean":"2024-09-17T05:07:39.363798+0000","last_became_active":"2024-09-17T05:07:37.359073+0000","last_became_peered":"2024-09-17T05:07:37.359073+0000","last_unstale":"2024-09-17T05:07:39.363798+0000","last_undegraded":"2024-09-17T05:07:39.363798+0000","last_fullsized":"2024-09-17T05:07:39.363798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:59.202139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354263+0000","last_change":"2024-09-17T05:07:37.354263+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354263+0000","last_undegraded":"2024-09-17T05:07:37.354263+0000","last_fullsized":"2024-09-17T05:07:37.354263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365151+0000","last_change":"2024-09-17T05:07:37.361606+0000","last_active":"2024-09-17T05:07:37.365151+0000","last_peered":"2024-09-17T05:07:37.365151+0000","last_clean":"2024-09-17T05:07:37.365151+0000","last_became_active":"2024-09-17T05:07:37.361002+0000","last_became_peered":"2024-09-17T05:07:37.361002+0000","last_unstale":"2024-09-17T05:07:37.365151+0000","last_undegraded":"2024-09-17T05:07:37.365151+0000","last_fullsized":"2024-09-17T05:07:37.365151+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:31.412375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353665+0000","last_change":"2024-09-17T05:07:37.353665+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353665+0000","last_undegraded":"2024-09-17T05:07:37.353665+0000","last_fullsized":"2024-09-17T05:07:37.353665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366271+0000","last_change":"2024-09-17T05:07:37.362489+0000","last_active":"2024-09-17T05:07:37.366271+0000","last_peered":"2024-09-17T05:07:37.366271+0000","last_clean":"2024-09-17T05:07:37.366271+0000","last_became_active":"2024-09-17T05:07:37.361729+0000","last_became_peered":"2024-09-17T05:07:37.361729+0000","last_unstale":"2024-09-17T05:07:37.366271+0000","last_undegraded":"2024-09-17T05:07:37.366271+0000","last_fullsized":"2024-09-17T05:07:37.366271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348338+0000","last_change":"2024-09-17T05:07:37.348338+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348338+0000","last_undegraded":"2024-09-17T05:07:37.348338+0000","last_fullsized":"2024-09-17T05:07:37.348338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365507+0000","last_change":"2024-09-17T05:07:37.360081+0000","last_active":"2024-09-17T05:07:39.365507+0000","last_peered":"2024-09-17T05:07:39.365507+0000","last_clean":"2024-09-17T05:07:39.365507+0000","last_became_active":"2024-09-17T05:07:37.359720+0000","last_became_peered":"2024-09-17T05:07:37.359720+0000","last_unstale":"2024-09-17T05:07:39.365507+0000","last_undegraded":"2024-09-17T05:07:39.365507+0000","last_fullsized":"2024-09-17T05:07:39.365507+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:16.580036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347399+0000","last_change":"2024-09-17T05:07:37.347399+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347399+0000","last_undegraded":"2024-09-17T05:07:37.347399+0000","last_fullsized":"2024-09-17T05:07:37.347399+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.370455+0000","last_change":"2024-09-17T05:07:37.358084+0000","last_active":"2024-09-17T05:07:39.370455+0000","last_peered":"2024-09-17T05:07:39.370455+0000","last_clean":"2024-09-17T05:07:39.370455+0000","last_became_active":"2024-09-17T05:07:37.357773+0000","last_became_peered":"2024-09-17T05:07:37.357773+0000","last_unstale":"2024-09-17T05:07:39.370455+0000","last_undegraded":"2024-09-17T05:07:39.370455+0000","last_fullsized":"2024-09-17T05:07:39.370455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:02.583665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348583+0000","last_change":"2024-09-17T05:07:37.348583+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348583+0000","last_undegraded":"2024-09-17T05:07:37.348583+0000","last_fullsized":"2024-09-17T05:07:37.348583+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.354323+0000","last_change":"2024-09-17T05:07:37.350958+0000","last_active":"2024-09-17T05:07:37.354323+0000","last_peered":"2024-09-17T05:07:37.354323+0000","last_clean":"2024-09-17T05:07:37.354323+0000","last_became_active":"2024-09-17T05:07:37.349820+0000","last_became_peered":"2024-09-17T05:07:37.349820+0000","last_unstale":"2024-09-17T05:07:37.354323+0000","last_undegraded":"2024-09-17T05:07:37.354323+0000","last_fullsized":"2024-09-17T05:07:37.354323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:25.046789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348957+0000","last_change":"2024-09-17T05:07:37.348957+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348957+0000","last_undegraded":"2024-09-17T05:07:37.348957+0000","last_fullsized":"2024-09-17T05:07:37.348957+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369179+0000","last_change":"2024-09-17T05:07:37.368314+0000","last_active":"2024-09-17T05:07:37.369179+0000","last_peered":"2024-09-17T05:07:37.369179+0000","last_clean":"2024-09-17T05:07:37.369179+0000","last_became_active":"2024-09-17T05:07:37.367953+0000","last_became_peered":"2024-09-17T05:07:37.367953+0000","last_unstale":"2024-09-17T05:07:37.369179+0000","last_undegraded":"2024-09-17T05:07:37.369179+0000","last_fullsized":"2024-09-17T05:07:37.369179+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:21.094340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352857+0000","last_change":"2024-09-17T05:07:37.352857+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352857+0000","last_undegraded":"2024-09-17T05:07:37.352857+0000","last_fullsized":"2024-09-17T05:07:37.352857+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371407+0000","last_change":"2024-09-17T05:07:37.364116+0000","last_active":"2024-09-17T05:07:37.371407+0000","last_peered":"2024-09-17T05:07:37.371407+0000","last_clean":"2024-09-17T05:07:37.371407+0000","last_became_active":"2024-09-17T05:07:37.363791+0000","last_became_peered":"2024-09-17T05:07:37.363791+0000","last_unstale":"2024-09-17T05:07:37.371407+0000","last_undegraded":"2024-09-17T05:07:37.371407+0000","last_fullsized":"2024-09-17T05:07:37.371407+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:14.819276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358208+0000","last_change":"2024-09-17T05:07:37.358208+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358208+0000","last_undegraded":"2024-09-17T05:07:37.358208+0000","last_fullsized":"2024-09-17T05:07:37.358208+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364119+0000","last_change":"2024-09-17T05:07:37.357614+0000","last_active":"2024-09-17T05:07:39.364119+0000","last_peered":"2024-09-17T05:07:39.364119+0000","last_clean":"2024-09-17T05:07:39.364119+0000","last_became_active":"2024-09-17T05:07:37.357276+0000","last_became_peered":"2024-09-17T05:07:37.357276+0000","last_unstale":"2024-09-17T05:07:39.364119+0000","last_undegraded":"2024-09-17T05:07:39.364119+0000","last_fullsized":"2024-09-17T05:07:39.364119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:39.232366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347380+0000","last_change":"2024-09-17T05:07:37.347380+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347380+0000","last_undegraded":"2024-09-17T05:07:37.347380+0000","last_fullsized":"2024-09-17T05:07:37.347380+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369451+0000","last_change":"2024-09-17T05:07:37.364772+0000","last_active":"2024-09-17T05:07:37.369451+0000","last_peered":"2024-09-17T05:07:37.369451+0000","last_clean":"2024-09-17T05:07:37.369451+0000","last_became_active":"2024-09-17T05:07:37.364468+0000","last_became_peered":"2024-09-17T05:07:37.364468+0000","last_unstale":"2024-09-17T05:07:37.369451+0000","last_undegraded":"2024-09-17T05:07:37.369451+0000","last_fullsized":"2024-09-17T05:07:37.369451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:31.412375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343458+0000","last_change":"2024-09-17T05:07:37.343458+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343458+0000","last_undegraded":"2024-09-17T05:07:37.343458+0000","last_fullsized":"2024-09-17T05:07:37.343458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363935+0000","last_change":"2024-09-17T05:07:37.355234+0000","last_active":"2024-09-17T05:07:39.363935+0000","last_peered":"2024-09-17T05:07:39.363935+0000","last_clean":"2024-09-17T05:07:39.363935+0000","last_became_active":"2024-09-17T05:07:37.354718+0000","last_became_peered":"2024-09-17T05:07:37.354718+0000","last_unstale":"2024-09-17T05:07:39.363935+0000","last_undegraded":"2024-09-17T05:07:39.363935+0000","last_fullsized":"2024-09-17T05:07:39.363935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.673587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347653+0000","last_change":"2024-09-17T05:07:37.347653+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347653+0000","last_undegraded":"2024-09-17T05:07:37.347653+0000","last_fullsized":"2024-09-17T05:07:37.347653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367557+0000","last_change":"2024-09-17T05:07:37.363388+0000","last_active":"2024-09-17T05:07:37.367557+0000","last_peered":"2024-09-17T05:07:37.367557+0000","last_clean":"2024-09-17T05:07:37.367557+0000","last_became_active":"2024-09-17T05:07:37.362169+0000","last_became_peered":"2024-09-17T05:07:37.362169+0000","last_unstale":"2024-09-17T05:07:37.367557+0000","last_undegraded":"2024-09-17T05:07:37.367557+0000","last_fullsized":"2024-09-17T05:07:37.367557+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:17.013225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.359581+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:07:37.359581+0000","last_peered":"2024-09-17T05:07:37.359581+0000","last_clean":"2024-09-17T05:07:37.359581+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:07:37.359581+0000","last_undegraded":"2024-09-17T05:07:37.359581+0000","last_fullsized":"2024-09-17T05:07:37.359581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36528,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335312,"statfs":{"total":96636764160,"available":96599359488,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1789952,"data_stored":1134999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1835008,"data_stored":1155649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:07:43.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:07:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60c6000640 1 -- 172.21.15.16:0/2362881723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4004d50 msgr2=0x7f60a40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60c6000640 1 --2- 172.21.15.16:0/2362881723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a4004d50 0x7f60a40051a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f60c015f0b0 tx=0x7f60b801d260 comp rx=0 tx=0).stop 2024-09-17T05:07:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60bfe00640 1 -- 172.21.15.16:0/2362881723 reap_dead start 2024-09-17T05:07:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60c6000640 1 -- 172.21.15.16:0/2362881723 shutdown_connections 2024-09-17T05:07:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60c6000640 1 -- 172.21.15.16:0/2362881723 >> 172.21.15.16:0/2362881723 conn(0x7f60c006f900 msgr2=0x7f60c00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60c6000640 1 -- 172.21.15.16:0/2362881723 shutdown_connections 2024-09-17T05:07:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.219+0000 7f60c6000640 1 -- 172.21.15.16:0/2362881723 wait complete. 2024-09-17T05:07:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 Processor -- start 2024-09-17T05:07:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 -- start start 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a4004d50 0x7f60c0146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60c0147700 0x7f60c006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 0x7f60c01541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60c0167c30 con 0x7f60c006dc60 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60c0167ab0 con 0x7f60a4004d50 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60c6000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f60c0167db0 con 0x7f60c0147700 2024-09-17T05:07:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 0x7f60c01541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 0x7f60c01541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42478/0 (socket says 172.21.15.16:42478) 2024-09-17T05:07:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 -- 172.21.15.16:0/605784576 learned_addr learned my addr 172.21.15.16:0/605784576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bf400640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a4004d50 0x7f60c0146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60b7e00640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60c0147700 0x7f60c006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 -- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60c0147700 msgr2=0x7f60c006d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60c0147700 0x7f60c006d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 -- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a4004d50 msgr2=0x7f60c0146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a4004d50 0x7f60c0146e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 -- 172.21.15.16:0/605784576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c006e070 con 0x7f60c006dc60 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bf400640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a4004d50 0x7f60c0146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:07:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.223+0000 7f60bfe00640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 0x7f60c01541a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f60b8000f90 tx=0x7f60b8004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b80011d0 con 0x7f60c006dc60 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60b8003980 con 0x7f60c006dc60 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8002020 con 0x7f60c006dc60 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c01f66e0 con 0x7f60c006dc60 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c01f6ca0 con 0x7f60c006dc60 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c01fa820 con 0x7f60c006dc60 2024-09-17T05:07:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f60b80c1060 con 0x7f60c006dc60 2024-09-17T05:07:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60b7400640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f609c073200 0x7f609c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f60b812e300 con 0x7f60c006dc60 2024-09-17T05:07:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.227+0000 7f60bf400640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f609c073200 0x7f609c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.231+0000 7f60bf400640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f609c073200 0x7f609c075710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006660 tx=0x7f60b0006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:43.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.231+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f60c006e070 con 0x7f60c006dc60 2024-09-17T05:07:43.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.403+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60c0073650 con 0x7f609c073200 2024-09-17T05:07:43.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.411+0000 7f60b7400640 1 -- 172.21.15.16:0/605784576 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247220 (secure 0 0 0) 0x7f60c0073650 con 0x7f609c073200 2024-09-17T05:07:43.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:43.412 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f609c073200 msgr2=0x7f609c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f609c073200 0x7f609c075710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f60b0006660 tx=0x7f60b0006b20 comp rx=0 tx=0).stop 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 msgr2=0x7f60c01541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 0x7f60c01541a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f60b8000f90 tx=0x7f60b8004e60 comp rx=0 tx=0).stop 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 shutdown_connections 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f609c073200 0x7f609c075710 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60c006dc60 0x7f60c01541a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f60c0147700 0x7f60c006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 --2- 172.21.15.16:0/605784576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60a4004d50 0x7f60c0146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 >> 172.21.15.16:0/605784576 conn(0x7f60c006f900 msgr2=0x7f60c015d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:43.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 shutdown_connections 2024-09-17T05:07:43.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.415+0000 7f60c6000640 1 -- 172.21.15.16:0/605784576 wait complete. 2024-09-17T05:07:43.433 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-17T05:07:41.570418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268440,"kb_used_data":10584,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706280,"statfs":{"total":773094113280,"available":772819230720,"internally_reserved":0,"allocated":10838016,"data_stored":5578435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.244815"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354142+0000","last_change":"2024-09-17T05:07:37.354142+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354142+0000","last_undegraded":"2024-09-17T05:07:37.354142+0000","last_fullsized":"2024-09-17T05:07:37.354142+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344873+0000","last_change":"2024-09-17T05:07:37.344873+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344873+0000","last_undegraded":"2024-09-17T05:07:37.344873+0000","last_fullsized":"2024-09-17T05:07:37.344873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365386+0000","last_change":"2024-09-17T05:07:38.373066+0000","last_active":"2024-09-17T05:07:39.365386+0000","last_peered":"2024-09-17T05:07:39.365386+0000","last_clean":"2024-09-17T05:07:39.365386+0000","last_became_active":"2024-09-17T05:07:38.372726+0000","last_became_peered":"2024-09-17T05:07:38.372726+0000","last_unstale":"2024-09-17T05:07:39.365386+0000","last_undegraded":"2024-09-17T05:07:39.365386+0000","last_fullsized":"2024-09-17T05:07:39.365386+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:40.211315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353488+0000","last_change":"2024-09-17T05:07:37.353488+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353488+0000","last_undegraded":"2024-09-17T05:07:37.353488+0000","last_fullsized":"2024-09-17T05:07:37.353488+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353790+0000","last_change":"2024-09-17T05:07:37.353790+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353790+0000","last_undegraded":"2024-09-17T05:07:37.353790+0000","last_fullsized":"2024-09-17T05:07:37.353790+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352451+0000","last_change":"2024-09-17T05:07:37.352451+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352451+0000","last_undegraded":"2024-09-17T05:07:37.352451+0000","last_fullsized":"2024-09-17T05:07:37.352451+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352553+0000","last_change":"2024-09-17T05:07:37.352553+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352553+0000","last_undegraded":"2024-09-17T05:07:37.352553+0000","last_fullsized":"2024-09-17T05:07:37.352553+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347288+0000","last_change":"2024-09-17T05:07:37.347288+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347288+0000","last_undegraded":"2024-09-17T05:07:37.347288+0000","last_fullsized":"2024-09-17T05:07:37.347288+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344945+0000","last_change":"2024-09-17T05:07:37.344945+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344945+0000","last_undegraded":"2024-09-17T05:07:37.344945+0000","last_fullsized":"2024-09-17T05:07:37.344945+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.349207+0000","last_change":"2024-09-17T05:07:37.349207+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.349207+0000","last_undegraded":"2024-09-17T05:07:37.349207+0000","last_fullsized":"2024-09-17T05:07:37.349207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348392+0000","last_change":"2024-09-17T05:07:37.348392+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348392+0000","last_undegraded":"2024-09-17T05:07:37.348392+0000","last_fullsized":"2024-09-17T05:07:37.348392+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.351164+0000","last_change":"2024-09-17T05:07:37.351164+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.351164+0000","last_undegraded":"2024-09-17T05:07:37.351164+0000","last_fullsized":"2024-09-17T05:07:37.351164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354316+0000","last_change":"2024-09-17T05:07:37.354316+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354316+0000","last_undegraded":"2024-09-17T05:07:37.354316+0000","last_fullsized":"2024-09-17T05:07:37.354316+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350574+0000","last_change":"2024-09-17T05:07:37.350574+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350574+0000","last_undegraded":"2024-09-17T05:07:37.350574+0000","last_fullsized":"2024-09-17T05:07:37.350574+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343241+0000","last_change":"2024-09-17T05:07:37.343241+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343241+0000","last_undegraded":"2024-09-17T05:07:37.343241+0000","last_fullsized":"2024-09-17T05:07:37.343241+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.359263+0000","last_change":"2024-09-17T05:07:37.359263+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.359263+0000","last_undegraded":"2024-09-17T05:07:37.359263+0000","last_fullsized":"2024-09-17T05:07:37.359263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365037+0000","last_change":"2024-09-17T05:07:38.369916+0000","last_active":"2024-09-17T05:07:39.365037+0000","last_peered":"2024-09-17T05:07:39.365037+0000","last_clean":"2024-09-17T05:07:39.365037+0000","last_became_active":"2024-09-17T05:07:38.369536+0000","last_became_peered":"2024-09-17T05:07:38.369536+0000","last_unstale":"2024-09-17T05:07:39.365037+0000","last_undegraded":"2024-09-17T05:07:39.365037+0000","last_fullsized":"2024-09-17T05:07:39.365037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:07.275670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344273+0000","last_change":"2024-09-17T05:07:37.344273+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344273+0000","last_undegraded":"2024-09-17T05:07:37.344273+0000","last_fullsized":"2024-09-17T05:07:37.344273+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358759+0000","last_change":"2024-09-17T05:07:37.358759+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358759+0000","last_undegraded":"2024-09-17T05:07:37.358759+0000","last_fullsized":"2024-09-17T05:07:37.358759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358844+0000","last_change":"2024-09-17T05:07:37.358844+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358844+0000","last_undegraded":"2024-09-17T05:07:37.358844+0000","last_fullsized":"2024-09-17T05:07:37.358844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364572+0000","last_change":"2024-09-17T05:07:38.365788+0000","last_active":"2024-09-17T05:07:39.364572+0000","last_peered":"2024-09-17T05:07:39.364572+0000","last_clean":"2024-09-17T05:07:39.364572+0000","last_became_active":"2024-09-17T05:07:38.364783+0000","last_became_peered":"2024-09-17T05:07:38.364783+0000","last_unstale":"2024-09-17T05:07:39.364572+0000","last_undegraded":"2024-09-17T05:07:39.364572+0000","last_fullsized":"2024-09-17T05:07:39.364572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:32.081371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352974+0000","last_change":"2024-09-17T05:07:37.352974+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352974+0000","last_undegraded":"2024-09-17T05:07:37.352974+0000","last_fullsized":"2024-09-17T05:07:37.352974+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347298+0000","last_change":"2024-09-17T05:07:37.347298+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347298+0000","last_undegraded":"2024-09-17T05:07:37.347298+0000","last_fullsized":"2024-09-17T05:07:37.347298+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346741+0000","last_change":"2024-09-17T05:07:37.346741+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346741+0000","last_undegraded":"2024-09-17T05:07:37.346741+0000","last_fullsized":"2024-09-17T05:07:37.346741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.355031+0000","last_change":"2024-09-17T05:07:37.355031+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.355031+0000","last_undegraded":"2024-09-17T05:07:37.355031+0000","last_fullsized":"2024-09-17T05:07:37.355031+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354956+0000","last_change":"2024-09-17T05:07:37.354956+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354956+0000","last_undegraded":"2024-09-17T05:07:37.354956+0000","last_fullsized":"2024-09-17T05:07:37.354956+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354198+0000","last_change":"2024-09-17T05:07:37.354198+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354198+0000","last_undegraded":"2024-09-17T05:07:37.354198+0000","last_fullsized":"2024-09-17T05:07:37.354198+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350568+0000","last_change":"2024-09-17T05:07:37.350568+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350568+0000","last_undegraded":"2024-09-17T05:07:37.350568+0000","last_fullsized":"2024-09-17T05:07:37.350568+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347094+0000","last_change":"2024-09-17T05:07:37.347094+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347094+0000","last_undegraded":"2024-09-17T05:07:37.347094+0000","last_fullsized":"2024-09-17T05:07:37.347094+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347191+0000","last_change":"2024-09-17T05:07:37.347191+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347191+0000","last_undegraded":"2024-09-17T05:07:37.347191+0000","last_fullsized":"2024-09-17T05:07:37.347191+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.350459+0000","last_change":"2024-09-17T05:07:37.350459+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.350459+0000","last_undegraded":"2024-09-17T05:07:37.350459+0000","last_fullsized":"2024-09-17T05:07:37.350459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348834+0000","last_change":"2024-09-17T05:07:37.348834+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348834+0000","last_undegraded":"2024-09-17T05:07:37.348834+0000","last_fullsized":"2024-09-17T05:07:37.348834+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363817+0000","last_change":"2024-09-17T05:07:38.351295+0000","last_active":"2024-09-17T05:07:39.363817+0000","last_peered":"2024-09-17T05:07:39.363817+0000","last_clean":"2024-09-17T05:07:39.363817+0000","last_became_active":"2024-09-17T05:07:38.348971+0000","last_became_peered":"2024-09-17T05:07:38.348971+0000","last_unstale":"2024-09-17T05:07:39.363817+0000","last_undegraded":"2024-09-17T05:07:39.363817+0000","last_fullsized":"2024-09-17T05:07:39.363817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:12.675618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365425+0000","last_change":"2024-09-17T05:07:37.361640+0000","last_active":"2024-09-17T05:07:37.365425+0000","last_peered":"2024-09-17T05:07:37.365425+0000","last_clean":"2024-09-17T05:07:37.365425+0000","last_became_active":"2024-09-17T05:07:37.360159+0000","last_became_peered":"2024-09-17T05:07:37.360159+0000","last_unstale":"2024-09-17T05:07:37.365425+0000","last_undegraded":"2024-09-17T05:07:37.365425+0000","last_fullsized":"2024-09-17T05:07:37.365425+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:22.738619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346712+0000","last_change":"2024-09-17T05:07:37.346712+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346712+0000","last_undegraded":"2024-09-17T05:07:37.346712+0000","last_fullsized":"2024-09-17T05:07:37.346712+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364559+0000","last_change":"2024-09-17T05:07:37.367481+0000","last_active":"2024-09-17T05:07:39.364559+0000","last_peered":"2024-09-17T05:07:39.364559+0000","last_clean":"2024-09-17T05:07:39.364559+0000","last_became_active":"2024-09-17T05:07:37.367109+0000","last_became_peered":"2024-09-17T05:07:37.367109+0000","last_unstale":"2024-09-17T05:07:39.364559+0000","last_undegraded":"2024-09-17T05:07:39.364559+0000","last_fullsized":"2024-09-17T05:07:39.364559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:25.251480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.344456+0000","last_change":"2024-09-17T05:07:37.344456+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.344456+0000","last_undegraded":"2024-09-17T05:07:37.344456+0000","last_fullsized":"2024-09-17T05:07:37.344456+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.363644+0000","last_change":"2024-09-17T05:07:37.360648+0000","last_active":"2024-09-17T05:07:37.363644+0000","last_peered":"2024-09-17T05:07:37.363644+0000","last_clean":"2024-09-17T05:07:37.363644+0000","last_became_active":"2024-09-17T05:07:37.356573+0000","last_became_peered":"2024-09-17T05:07:37.356573+0000","last_unstale":"2024-09-17T05:07:37.363644+0000","last_undegraded":"2024-09-17T05:07:37.363644+0000","last_fullsized":"2024-09-17T05:07:37.363644+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:23.902999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346600+0000","last_change":"2024-09-17T05:07:37.346600+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346600+0000","last_undegraded":"2024-09-17T05:07:37.346600+0000","last_fullsized":"2024-09-17T05:07:37.346600+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367346+0000","last_change":"2024-09-17T05:07:37.364681+0000","last_active":"2024-09-17T05:07:37.367346+0000","last_peered":"2024-09-17T05:07:37.367346+0000","last_clean":"2024-09-17T05:07:37.367346+0000","last_became_active":"2024-09-17T05:07:37.364348+0000","last_became_peered":"2024-09-17T05:07:37.364348+0000","last_unstale":"2024-09-17T05:07:37.367346+0000","last_undegraded":"2024-09-17T05:07:37.367346+0000","last_fullsized":"2024-09-17T05:07:37.367346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:09.224428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.349759+0000","last_change":"2024-09-17T05:07:37.349759+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.349759+0000","last_undegraded":"2024-09-17T05:07:37.349759+0000","last_fullsized":"2024-09-17T05:07:37.349759+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371989+0000","last_change":"2024-09-17T05:07:37.368233+0000","last_active":"2024-09-17T05:07:37.371989+0000","last_peered":"2024-09-17T05:07:37.371989+0000","last_clean":"2024-09-17T05:07:37.371989+0000","last_became_active":"2024-09-17T05:07:37.367904+0000","last_became_peered":"2024-09-17T05:07:37.367904+0000","last_unstale":"2024-09-17T05:07:37.371989+0000","last_undegraded":"2024-09-17T05:07:37.371989+0000","last_fullsized":"2024-09-17T05:07:37.371989+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:38.417939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354832+0000","last_change":"2024-09-17T05:07:37.354832+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354832+0000","last_undegraded":"2024-09-17T05:07:37.354832+0000","last_fullsized":"2024-09-17T05:07:37.354832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.372345+0000","last_change":"2024-09-17T05:07:37.368607+0000","last_active":"2024-09-17T05:07:37.372345+0000","last_peered":"2024-09-17T05:07:37.372345+0000","last_clean":"2024-09-17T05:07:37.372345+0000","last_became_active":"2024-09-17T05:07:37.368307+0000","last_became_peered":"2024-09-17T05:07:37.368307+0000","last_unstale":"2024-09-17T05:07:37.372345+0000","last_undegraded":"2024-09-17T05:07:37.372345+0000","last_fullsized":"2024-09-17T05:07:37.372345+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346749+0000","last_change":"2024-09-17T05:07:37.346749+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346749+0000","last_undegraded":"2024-09-17T05:07:37.346749+0000","last_fullsized":"2024-09-17T05:07:37.346749+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371473+0000","last_change":"2024-09-17T05:07:37.367847+0000","last_active":"2024-09-17T05:07:37.371473+0000","last_peered":"2024-09-17T05:07:37.371473+0000","last_clean":"2024-09-17T05:07:37.371473+0000","last_became_active":"2024-09-17T05:07:37.367465+0000","last_became_peered":"2024-09-17T05:07:37.367465+0000","last_unstale":"2024-09-17T05:07:37.371473+0000","last_undegraded":"2024-09-17T05:07:37.371473+0000","last_fullsized":"2024-09-17T05:07:37.371473+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:41.729615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353184+0000","last_change":"2024-09-17T05:07:37.353184+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353184+0000","last_undegraded":"2024-09-17T05:07:37.353184+0000","last_fullsized":"2024-09-17T05:07:37.353184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367733+0000","last_change":"2024-09-17T05:07:37.364825+0000","last_active":"2024-09-17T05:07:37.367733+0000","last_peered":"2024-09-17T05:07:37.367733+0000","last_clean":"2024-09-17T05:07:37.367733+0000","last_became_active":"2024-09-17T05:07:37.364481+0000","last_became_peered":"2024-09-17T05:07:37.364481+0000","last_unstale":"2024-09-17T05:07:37.367733+0000","last_undegraded":"2024-09-17T05:07:37.367733+0000","last_fullsized":"2024-09-17T05:07:37.367733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:00:05.942761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348642+0000","last_change":"2024-09-17T05:07:37.348642+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348642+0000","last_undegraded":"2024-09-17T05:07:37.348642+0000","last_fullsized":"2024-09-17T05:07:37.348642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366699+0000","last_change":"2024-09-17T05:07:37.362446+0000","last_active":"2024-09-17T05:07:37.366699+0000","last_peered":"2024-09-17T05:07:37.366699+0000","last_clean":"2024-09-17T05:07:37.366699+0000","last_became_active":"2024-09-17T05:07:37.362104+0000","last_became_peered":"2024-09-17T05:07:37.362104+0000","last_unstale":"2024-09-17T05:07:37.366699+0000","last_undegraded":"2024-09-17T05:07:37.366699+0000","last_fullsized":"2024-09-17T05:07:37.366699+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:41.729615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348460+0000","last_change":"2024-09-17T05:07:37.348460+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348460+0000","last_undegraded":"2024-09-17T05:07:37.348460+0000","last_fullsized":"2024-09-17T05:07:37.348460+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.361947+0000","last_change":"2024-09-17T05:07:37.359338+0000","last_active":"2024-09-17T05:07:37.361947+0000","last_peered":"2024-09-17T05:07:37.361947+0000","last_clean":"2024-09-17T05:07:37.361947+0000","last_became_active":"2024-09-17T05:07:37.359008+0000","last_became_peered":"2024-09-17T05:07:37.359008+0000","last_unstale":"2024-09-17T05:07:37.361947+0000","last_undegraded":"2024-09-17T05:07:37.361947+0000","last_fullsized":"2024-09-17T05:07:37.361947+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348027+0000","last_change":"2024-09-17T05:07:37.348027+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348027+0000","last_undegraded":"2024-09-17T05:07:37.348027+0000","last_fullsized":"2024-09-17T05:07:37.348027+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365107+0000","last_change":"2024-09-17T05:07:37.362270+0000","last_active":"2024-09-17T05:07:37.365107+0000","last_peered":"2024-09-17T05:07:37.365107+0000","last_clean":"2024-09-17T05:07:37.365107+0000","last_became_active":"2024-09-17T05:07:37.361924+0000","last_became_peered":"2024-09-17T05:07:37.361924+0000","last_unstale":"2024-09-17T05:07:37.365107+0000","last_undegraded":"2024-09-17T05:07:37.365107+0000","last_fullsized":"2024-09-17T05:07:37.365107+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:26.077987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347252+0000","last_change":"2024-09-17T05:07:37.347252+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347252+0000","last_undegraded":"2024-09-17T05:07:37.347252+0000","last_fullsized":"2024-09-17T05:07:37.347252+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371818+0000","last_change":"2024-09-17T05:07:37.365192+0000","last_active":"2024-09-17T05:07:37.371818+0000","last_peered":"2024-09-17T05:07:37.371818+0000","last_clean":"2024-09-17T05:07:37.371818+0000","last_became_active":"2024-09-17T05:07:37.364884+0000","last_became_peered":"2024-09-17T05:07:37.364884+0000","last_unstale":"2024-09-17T05:07:37.371818+0000","last_undegraded":"2024-09-17T05:07:37.371818+0000","last_fullsized":"2024-09-17T05:07:37.371818+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:45.779925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343962+0000","last_change":"2024-09-17T05:07:37.343962+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343962+0000","last_undegraded":"2024-09-17T05:07:37.343962+0000","last_fullsized":"2024-09-17T05:07:37.343962+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.364152+0000","last_change":"2024-09-17T05:07:37.358555+0000","last_active":"2024-09-17T05:07:37.364152+0000","last_peered":"2024-09-17T05:07:37.364152+0000","last_clean":"2024-09-17T05:07:37.364152+0000","last_became_active":"2024-09-17T05:07:37.358188+0000","last_became_peered":"2024-09-17T05:07:37.358188+0000","last_unstale":"2024-09-17T05:07:37.364152+0000","last_undegraded":"2024-09-17T05:07:37.364152+0000","last_fullsized":"2024-09-17T05:07:37.364152+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:55.685542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.345638+0000","last_change":"2024-09-17T05:07:37.345638+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.345638+0000","last_undegraded":"2024-09-17T05:07:37.345638+0000","last_fullsized":"2024-09-17T05:07:37.345638+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366332+0000","last_change":"2024-09-17T05:07:37.363006+0000","last_active":"2024-09-17T05:07:37.366332+0000","last_peered":"2024-09-17T05:07:37.366332+0000","last_clean":"2024-09-17T05:07:37.366332+0000","last_became_active":"2024-09-17T05:07:37.362664+0000","last_became_peered":"2024-09-17T05:07:37.362664+0000","last_unstale":"2024-09-17T05:07:37.366332+0000","last_undegraded":"2024-09-17T05:07:37.366332+0000","last_fullsized":"2024-09-17T05:07:37.366332+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:34.154990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353663+0000","last_change":"2024-09-17T05:07:37.353663+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353663+0000","last_undegraded":"2024-09-17T05:07:37.353663+0000","last_fullsized":"2024-09-17T05:07:37.353663+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.368799+0000","last_change":"2024-09-17T05:07:37.365145+0000","last_active":"2024-09-17T05:07:37.368799+0000","last_peered":"2024-09-17T05:07:37.368799+0000","last_clean":"2024-09-17T05:07:37.368799+0000","last_became_active":"2024-09-17T05:07:37.364775+0000","last_became_peered":"2024-09-17T05:07:37.364775+0000","last_unstale":"2024-09-17T05:07:37.368799+0000","last_undegraded":"2024-09-17T05:07:37.368799+0000","last_fullsized":"2024-09-17T05:07:37.368799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:36.877036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365284+0000","last_change":"2024-09-17T05:07:38.369937+0000","last_active":"2024-09-17T05:07:39.365284+0000","last_peered":"2024-09-17T05:07:39.365284+0000","last_clean":"2024-09-17T05:07:39.365284+0000","last_became_active":"2024-09-17T05:07:38.369450+0000","last_became_peered":"2024-09-17T05:07:38.369450+0000","last_unstale":"2024-09-17T05:07:39.365284+0000","last_undegraded":"2024-09-17T05:07:39.365284+0000","last_fullsized":"2024-09-17T05:07:39.365284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:59.251209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.372162+0000","last_change":"2024-09-17T05:07:37.368862+0000","last_active":"2024-09-17T05:07:37.372162+0000","last_peered":"2024-09-17T05:07:37.372162+0000","last_clean":"2024-09-17T05:07:37.372162+0000","last_became_active":"2024-09-17T05:07:37.368509+0000","last_became_peered":"2024-09-17T05:07:37.368509+0000","last_unstale":"2024-09-17T05:07:37.372162+0000","last_undegraded":"2024-09-17T05:07:37.372162+0000","last_fullsized":"2024-09-17T05:07:37.372162+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:56.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346070+0000","last_change":"2024-09-17T05:07:37.346070+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346070+0000","last_undegraded":"2024-09-17T05:07:37.346070+0000","last_fullsized":"2024-09-17T05:07:37.346070+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367208+0000","last_change":"2024-09-17T05:07:37.364495+0000","last_active":"2024-09-17T05:07:37.367208+0000","last_peered":"2024-09-17T05:07:37.367208+0000","last_clean":"2024-09-17T05:07:37.367208+0000","last_became_active":"2024-09-17T05:07:37.364156+0000","last_became_peered":"2024-09-17T05:07:37.364156+0000","last_unstale":"2024-09-17T05:07:37.367208+0000","last_undegraded":"2024-09-17T05:07:37.367208+0000","last_fullsized":"2024-09-17T05:07:37.367208+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:47.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347829+0000","last_change":"2024-09-17T05:07:37.347829+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347829+0000","last_undegraded":"2024-09-17T05:07:37.347829+0000","last_fullsized":"2024-09-17T05:07:37.347829+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371399+0000","last_change":"2024-09-17T05:07:37.367221+0000","last_active":"2024-09-17T05:07:37.371399+0000","last_peered":"2024-09-17T05:07:37.371399+0000","last_clean":"2024-09-17T05:07:37.371399+0000","last_became_active":"2024-09-17T05:07:37.365939+0000","last_became_peered":"2024-09-17T05:07:37.365939+0000","last_unstale":"2024-09-17T05:07:37.371399+0000","last_undegraded":"2024-09-17T05:07:37.371399+0000","last_fullsized":"2024-09-17T05:07:37.371399+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:26.077987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347804+0000","last_change":"2024-09-17T05:07:37.347804+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347804+0000","last_undegraded":"2024-09-17T05:07:37.347804+0000","last_fullsized":"2024-09-17T05:07:37.347804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369481+0000","last_change":"2024-09-17T05:07:37.367645+0000","last_active":"2024-09-17T05:07:37.369481+0000","last_peered":"2024-09-17T05:07:37.369481+0000","last_clean":"2024-09-17T05:07:37.369481+0000","last_became_active":"2024-09-17T05:07:37.367258+0000","last_became_peered":"2024-09-17T05:07:37.367258+0000","last_unstale":"2024-09-17T05:07:37.369481+0000","last_undegraded":"2024-09-17T05:07:37.369481+0000","last_fullsized":"2024-09-17T05:07:37.369481+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352275+0000","last_change":"2024-09-17T05:07:37.352275+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352275+0000","last_undegraded":"2024-09-17T05:07:37.352275+0000","last_fullsized":"2024-09-17T05:07:37.352275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364438+0000","last_change":"2024-09-17T05:07:37.364934+0000","last_active":"2024-09-17T05:07:39.364438+0000","last_peered":"2024-09-17T05:07:39.364438+0000","last_clean":"2024-09-17T05:07:39.364438+0000","last_became_active":"2024-09-17T05:07:37.364564+0000","last_became_peered":"2024-09-17T05:07:37.364564+0000","last_unstale":"2024-09-17T05:07:39.364438+0000","last_undegraded":"2024-09-17T05:07:39.364438+0000","last_fullsized":"2024-09-17T05:07:39.364438+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:25.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.346823+0000","last_change":"2024-09-17T05:07:37.346823+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.346823+0000","last_undegraded":"2024-09-17T05:07:37.346823+0000","last_fullsized":"2024-09-17T05:07:37.346823+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363798+0000","last_change":"2024-09-17T05:07:37.359369+0000","last_active":"2024-09-17T05:07:39.363798+0000","last_peered":"2024-09-17T05:07:39.363798+0000","last_clean":"2024-09-17T05:07:39.363798+0000","last_became_active":"2024-09-17T05:07:37.359073+0000","last_became_peered":"2024-09-17T05:07:37.359073+0000","last_unstale":"2024-09-17T05:07:39.363798+0000","last_undegraded":"2024-09-17T05:07:39.363798+0000","last_fullsized":"2024-09-17T05:07:39.363798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:59.202139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.354263+0000","last_change":"2024-09-17T05:07:37.354263+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.354263+0000","last_undegraded":"2024-09-17T05:07:37.354263+0000","last_fullsized":"2024-09-17T05:07:37.354263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.365151+0000","last_change":"2024-09-17T05:07:37.361606+0000","last_active":"2024-09-17T05:07:37.365151+0000","last_peered":"2024-09-17T05:07:37.365151+0000","last_clean":"2024-09-17T05:07:37.365151+0000","last_became_active":"2024-09-17T05:07:37.361002+0000","last_became_peered":"2024-09-17T05:07:37.361002+0000","last_unstale":"2024-09-17T05:07:37.365151+0000","last_undegraded":"2024-09-17T05:07:37.365151+0000","last_fullsized":"2024-09-17T05:07:37.365151+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:31.412375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.353665+0000","last_change":"2024-09-17T05:07:37.353665+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.353665+0000","last_undegraded":"2024-09-17T05:07:37.353665+0000","last_fullsized":"2024-09-17T05:07:37.353665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.366271+0000","last_change":"2024-09-17T05:07:37.362489+0000","last_active":"2024-09-17T05:07:37.366271+0000","last_peered":"2024-09-17T05:07:37.366271+0000","last_clean":"2024-09-17T05:07:37.366271+0000","last_became_active":"2024-09-17T05:07:37.361729+0000","last_became_peered":"2024-09-17T05:07:37.361729+0000","last_unstale":"2024-09-17T05:07:37.366271+0000","last_undegraded":"2024-09-17T05:07:37.366271+0000","last_fullsized":"2024-09-17T05:07:37.366271+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348338+0000","last_change":"2024-09-17T05:07:37.348338+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348338+0000","last_undegraded":"2024-09-17T05:07:37.348338+0000","last_fullsized":"2024-09-17T05:07:37.348338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365507+0000","last_change":"2024-09-17T05:07:37.360081+0000","last_active":"2024-09-17T05:07:39.365507+0000","last_peered":"2024-09-17T05:07:39.365507+0000","last_clean":"2024-09-17T05:07:39.365507+0000","last_became_active":"2024-09-17T05:07:37.359720+0000","last_became_peered":"2024-09-17T05:07:37.359720+0000","last_unstale":"2024-09-17T05:07:39.365507+0000","last_undegraded":"2024-09-17T05:07:39.365507+0000","last_fullsized":"2024-09-17T05:07:39.365507+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:16.580036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347399+0000","last_change":"2024-09-17T05:07:37.347399+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347399+0000","last_undegraded":"2024-09-17T05:07:37.347399+0000","last_fullsized":"2024-09-17T05:07:37.347399+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.370455+0000","last_change":"2024-09-17T05:07:37.358084+0000","last_active":"2024-09-17T05:07:39.370455+0000","last_peered":"2024-09-17T05:07:39.370455+0000","last_clean":"2024-09-17T05:07:39.370455+0000","last_became_active":"2024-09-17T05:07:37.357773+0000","last_became_peered":"2024-09-17T05:07:37.357773+0000","last_unstale":"2024-09-17T05:07:39.370455+0000","last_undegraded":"2024-09-17T05:07:39.370455+0000","last_fullsized":"2024-09-17T05:07:39.370455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:02.583665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348583+0000","last_change":"2024-09-17T05:07:37.348583+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348583+0000","last_undegraded":"2024-09-17T05:07:37.348583+0000","last_fullsized":"2024-09-17T05:07:37.348583+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.354323+0000","last_change":"2024-09-17T05:07:37.350958+0000","last_active":"2024-09-17T05:07:37.354323+0000","last_peered":"2024-09-17T05:07:37.354323+0000","last_clean":"2024-09-17T05:07:37.354323+0000","last_became_active":"2024-09-17T05:07:37.349820+0000","last_became_peered":"2024-09-17T05:07:37.349820+0000","last_unstale":"2024-09-17T05:07:37.354323+0000","last_undegraded":"2024-09-17T05:07:37.354323+0000","last_fullsized":"2024-09-17T05:07:37.354323+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:25.046789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.348957+0000","last_change":"2024-09-17T05:07:37.348957+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.348957+0000","last_undegraded":"2024-09-17T05:07:37.348957+0000","last_fullsized":"2024-09-17T05:07:37.348957+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369179+0000","last_change":"2024-09-17T05:07:37.368314+0000","last_active":"2024-09-17T05:07:37.369179+0000","last_peered":"2024-09-17T05:07:37.369179+0000","last_clean":"2024-09-17T05:07:37.369179+0000","last_became_active":"2024-09-17T05:07:37.367953+0000","last_became_peered":"2024-09-17T05:07:37.367953+0000","last_unstale":"2024-09-17T05:07:37.369179+0000","last_undegraded":"2024-09-17T05:07:37.369179+0000","last_fullsized":"2024-09-17T05:07:37.369179+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:21.094340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.352857+0000","last_change":"2024-09-17T05:07:37.352857+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.352857+0000","last_undegraded":"2024-09-17T05:07:37.352857+0000","last_fullsized":"2024-09-17T05:07:37.352857+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.371407+0000","last_change":"2024-09-17T05:07:37.364116+0000","last_active":"2024-09-17T05:07:37.371407+0000","last_peered":"2024-09-17T05:07:37.371407+0000","last_clean":"2024-09-17T05:07:37.371407+0000","last_became_active":"2024-09-17T05:07:37.363791+0000","last_became_peered":"2024-09-17T05:07:37.363791+0000","last_unstale":"2024-09-17T05:07:37.371407+0000","last_undegraded":"2024-09-17T05:07:37.371407+0000","last_fullsized":"2024-09-17T05:07:37.371407+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:14.819276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.358208+0000","last_change":"2024-09-17T05:07:37.358208+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.358208+0000","last_undegraded":"2024-09-17T05:07:37.358208+0000","last_fullsized":"2024-09-17T05:07:37.358208+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364119+0000","last_change":"2024-09-17T05:07:37.357614+0000","last_active":"2024-09-17T05:07:39.364119+0000","last_peered":"2024-09-17T05:07:39.364119+0000","last_clean":"2024-09-17T05:07:39.364119+0000","last_became_active":"2024-09-17T05:07:37.357276+0000","last_became_peered":"2024-09-17T05:07:37.357276+0000","last_unstale":"2024-09-17T05:07:39.364119+0000","last_undegraded":"2024-09-17T05:07:39.364119+0000","last_fullsized":"2024-09-17T05:07:39.364119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:39.232366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347380+0000","last_change":"2024-09-17T05:07:37.347380+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347380+0000","last_undegraded":"2024-09-17T05:07:37.347380+0000","last_fullsized":"2024-09-17T05:07:37.347380+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.369451+0000","last_change":"2024-09-17T05:07:37.364772+0000","last_active":"2024-09-17T05:07:37.369451+0000","last_peered":"2024-09-17T05:07:37.369451+0000","last_clean":"2024-09-17T05:07:37.369451+0000","last_became_active":"2024-09-17T05:07:37.364468+0000","last_became_peered":"2024-09-17T05:07:37.364468+0000","last_unstale":"2024-09-17T05:07:37.369451+0000","last_undegraded":"2024-09-17T05:07:37.369451+0000","last_fullsized":"2024-09-17T05:07:37.369451+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:31.412375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.343458+0000","last_change":"2024-09-17T05:07:37.343458+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.343458+0000","last_undegraded":"2024-09-17T05:07:37.343458+0000","last_fullsized":"2024-09-17T05:07:37.343458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363935+0000","last_change":"2024-09-17T05:07:37.355234+0000","last_active":"2024-09-17T05:07:39.363935+0000","last_peered":"2024-09-17T05:07:39.363935+0000","last_clean":"2024-09-17T05:07:39.363935+0000","last_became_active":"2024-09-17T05:07:37.354718+0000","last_became_peered":"2024-09-17T05:07:37.354718+0000","last_unstale":"2024-09-17T05:07:39.363935+0000","last_undegraded":"2024-09-17T05:07:39.363935+0000","last_fullsized":"2024-09-17T05:07:39.363935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.673587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-17T05:07:37.347653+0000","last_change":"2024-09-17T05:07:37.347653+0000","last_active":"2024-09-17T05:07:37.326472+0000","last_peered":"2024-09-17T05:07:37.326472+0000","last_clean":"2024-09-17T05:07:37.326472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:07:37.347653+0000","last_undegraded":"2024-09-17T05:07:37.347653+0000","last_fullsized":"2024-09-17T05:07:37.347653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.367557+0000","last_change":"2024-09-17T05:07:37.363388+0000","last_active":"2024-09-17T05:07:37.367557+0000","last_peered":"2024-09-17T05:07:37.367557+0000","last_clean":"2024-09-17T05:07:37.367557+0000","last_became_active":"2024-09-17T05:07:37.362169+0000","last_became_peered":"2024-09-17T05:07:37.362169+0000","last_unstale":"2024-09-17T05:07:37.367557+0000","last_undegraded":"2024-09-17T05:07:37.367557+0000","last_fullsized":"2024-09-17T05:07:37.367557+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:17.013225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":260,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-17T05:07:37.359581+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:07:37.359581+0000","last_peered":"2024-09-17T05:07:37.359581+0000","last_clean":"2024-09-17T05:07:37.359581+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:07:37.359581+0000","last_undegraded":"2024-09-17T05:07:37.359581+0000","last_fullsized":"2024-09-17T05:07:37.359581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509523,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36528,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335312,"statfs":{"total":96636764160,"available":96599359488,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509522,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1208320,"data_stored":551298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1789952,"data_stored":1134999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":544631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1835008,"data_stored":1155649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:07:43.438 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3f is not active+clean 2024-09-17T05:07:43.438 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.354142+0000', 'last_change': '2024-09-17T05:07:37.354142+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.354142+0000', 'last_undegraded': '2024-09-17T05:07:37.354142+0000', 'last_fullsized': '2024-09-17T05:07:37.354142+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.438 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3e is not active+clean 2024-09-17T05:07:43.438 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.344873+0000', 'last_change': '2024-09-17T05:07:37.344873+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.344873+0000', 'last_undegraded': '2024-09-17T05:07:37.344873+0000', 'last_fullsized': '2024-09-17T05:07:37.344873+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.438 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3c is not active+clean 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.353488+0000', 'last_change': '2024-09-17T05:07:37.353488+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.353488+0000', 'last_undegraded': '2024-09-17T05:07:37.353488+0000', 'last_fullsized': '2024-09-17T05:07:37.353488+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3b is not active+clean 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.353790+0000', 'last_change': '2024-09-17T05:07:37.353790+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.353790+0000', 'last_undegraded': '2024-09-17T05:07:37.353790+0000', 'last_fullsized': '2024-09-17T05:07:37.353790+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3a is not active+clean 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.352451+0000', 'last_change': '2024-09-17T05:07:37.352451+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.352451+0000', 'last_undegraded': '2024-09-17T05:07:37.352451+0000', 'last_fullsized': '2024-09-17T05:07:37.352451+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:PG 16.39 is not active+clean 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.352553+0000', 'last_change': '2024-09-17T05:07:37.352553+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.352553+0000', 'last_undegraded': '2024-09-17T05:07:37.352553+0000', 'last_fullsized': '2024-09-17T05:07:37.352553+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.439 INFO:tasks.ceph.ceph_manager.ceph:PG 16.38 is not active+clean 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347288+0000', 'last_change': '2024-09-17T05:07:37.347288+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347288+0000', 'last_undegraded': '2024-09-17T05:07:37.347288+0000', 'last_fullsized': '2024-09-17T05:07:37.347288+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:PG 16.37 is not active+clean 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.344945+0000', 'last_change': '2024-09-17T05:07:37.344945+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.344945+0000', 'last_undegraded': '2024-09-17T05:07:37.344945+0000', 'last_fullsized': '2024-09-17T05:07:37.344945+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:PG 16.36 is not active+clean 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.349207+0000', 'last_change': '2024-09-17T05:07:37.349207+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.349207+0000', 'last_undegraded': '2024-09-17T05:07:37.349207+0000', 'last_fullsized': '2024-09-17T05:07:37.349207+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:PG 16.35 is not active+clean 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348392+0000', 'last_change': '2024-09-17T05:07:37.348392+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348392+0000', 'last_undegraded': '2024-09-17T05:07:37.348392+0000', 'last_fullsized': '2024-09-17T05:07:37.348392+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:PG 16.34 is not active+clean 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.351164+0000', 'last_change': '2024-09-17T05:07:37.351164+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.351164+0000', 'last_undegraded': '2024-09-17T05:07:37.351164+0000', 'last_fullsized': '2024-09-17T05:07:37.351164+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:PG 16.33 is not active+clean 2024-09-17T05:07:43.440 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.354316+0000', 'last_change': '2024-09-17T05:07:37.354316+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.354316+0000', 'last_undegraded': '2024-09-17T05:07:37.354316+0000', 'last_fullsized': '2024-09-17T05:07:37.354316+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:PG 16.32 is not active+clean 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.350574+0000', 'last_change': '2024-09-17T05:07:37.350574+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.350574+0000', 'last_undegraded': '2024-09-17T05:07:37.350574+0000', 'last_fullsized': '2024-09-17T05:07:37.350574+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:PG 16.31 is not active+clean 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.343241+0000', 'last_change': '2024-09-17T05:07:37.343241+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.343241+0000', 'last_undegraded': '2024-09-17T05:07:37.343241+0000', 'last_fullsized': '2024-09-17T05:07:37.343241+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:PG 16.30 is not active+clean 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.359263+0000', 'last_change': '2024-09-17T05:07:37.359263+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.359263+0000', 'last_undegraded': '2024-09-17T05:07:37.359263+0000', 'last_fullsized': '2024-09-17T05:07:37.359263+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.441 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2e is not active+clean 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.344273+0000', 'last_change': '2024-09-17T05:07:37.344273+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.344273+0000', 'last_undegraded': '2024-09-17T05:07:37.344273+0000', 'last_fullsized': '2024-09-17T05:07:37.344273+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2d is not active+clean 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.358759+0000', 'last_change': '2024-09-17T05:07:37.358759+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.358759+0000', 'last_undegraded': '2024-09-17T05:07:37.358759+0000', 'last_fullsized': '2024-09-17T05:07:37.358759+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2c is not active+clean 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.358844+0000', 'last_change': '2024-09-17T05:07:37.358844+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.358844+0000', 'last_undegraded': '2024-09-17T05:07:37.358844+0000', 'last_fullsized': '2024-09-17T05:07:37.358844+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2a is not active+clean 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.352974+0000', 'last_change': '2024-09-17T05:07:37.352974+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.352974+0000', 'last_undegraded': '2024-09-17T05:07:37.352974+0000', 'last_fullsized': '2024-09-17T05:07:37.352974+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:PG 16.29 is not active+clean 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347298+0000', 'last_change': '2024-09-17T05:07:37.347298+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347298+0000', 'last_undegraded': '2024-09-17T05:07:37.347298+0000', 'last_fullsized': '2024-09-17T05:07:37.347298+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:PG 16.28 is not active+clean 2024-09-17T05:07:43.442 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.346741+0000', 'last_change': '2024-09-17T05:07:37.346741+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.346741+0000', 'last_undegraded': '2024-09-17T05:07:37.346741+0000', 'last_fullsized': '2024-09-17T05:07:37.346741+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:PG 16.27 is not active+clean 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.355031+0000', 'last_change': '2024-09-17T05:07:37.355031+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.355031+0000', 'last_undegraded': '2024-09-17T05:07:37.355031+0000', 'last_fullsized': '2024-09-17T05:07:37.355031+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:PG 16.26 is not active+clean 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.354956+0000', 'last_change': '2024-09-17T05:07:37.354956+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.354956+0000', 'last_undegraded': '2024-09-17T05:07:37.354956+0000', 'last_fullsized': '2024-09-17T05:07:37.354956+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:PG 16.25 is not active+clean 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.354198+0000', 'last_change': '2024-09-17T05:07:37.354198+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.354198+0000', 'last_undegraded': '2024-09-17T05:07:37.354198+0000', 'last_fullsized': '2024-09-17T05:07:37.354198+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:PG 16.24 is not active+clean 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.350568+0000', 'last_change': '2024-09-17T05:07:37.350568+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.350568+0000', 'last_undegraded': '2024-09-17T05:07:37.350568+0000', 'last_fullsized': '2024-09-17T05:07:37.350568+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:PG 16.23 is not active+clean 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347094+0000', 'last_change': '2024-09-17T05:07:37.347094+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347094+0000', 'last_undegraded': '2024-09-17T05:07:37.347094+0000', 'last_fullsized': '2024-09-17T05:07:37.347094+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:PG 16.22 is not active+clean 2024-09-17T05:07:43.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347191+0000', 'last_change': '2024-09-17T05:07:37.347191+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347191+0000', 'last_undegraded': '2024-09-17T05:07:37.347191+0000', 'last_fullsized': '2024-09-17T05:07:37.347191+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:PG 16.21 is not active+clean 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.350459+0000', 'last_change': '2024-09-17T05:07:37.350459+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.350459+0000', 'last_undegraded': '2024-09-17T05:07:37.350459+0000', 'last_fullsized': '2024-09-17T05:07:37.350459+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:PG 16.20 is not active+clean 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348834+0000', 'last_change': '2024-09-17T05:07:37.348834+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348834+0000', 'last_undegraded': '2024-09-17T05:07:37.348834+0000', 'last_fullsized': '2024-09-17T05:07:37.348834+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1 is not active+clean 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.346712+0000', 'last_change': '2024-09-17T05:07:37.346712+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.346712+0000', 'last_undegraded': '2024-09-17T05:07:37.346712+0000', 'last_fullsized': '2024-09-17T05:07:37.346712+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2 is not active+clean 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.344456+0000', 'last_change': '2024-09-17T05:07:37.344456+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.344456+0000', 'last_undegraded': '2024-09-17T05:07:37.344456+0000', 'last_fullsized': '2024-09-17T05:07:37.344456+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3 is not active+clean 2024-09-17T05:07:43.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.346600+0000', 'last_change': '2024-09-17T05:07:37.346600+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.346600+0000', 'last_undegraded': '2024-09-17T05:07:37.346600+0000', 'last_fullsized': '2024-09-17T05:07:37.346600+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:PG 16.4 is not active+clean 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.349759+0000', 'last_change': '2024-09-17T05:07:37.349759+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.349759+0000', 'last_undegraded': '2024-09-17T05:07:37.349759+0000', 'last_fullsized': '2024-09-17T05:07:37.349759+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:PG 16.5 is not active+clean 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.354832+0000', 'last_change': '2024-09-17T05:07:37.354832+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.354832+0000', 'last_undegraded': '2024-09-17T05:07:37.354832+0000', 'last_fullsized': '2024-09-17T05:07:37.354832+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:PG 16.6 is not active+clean 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.346749+0000', 'last_change': '2024-09-17T05:07:37.346749+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.346749+0000', 'last_undegraded': '2024-09-17T05:07:37.346749+0000', 'last_fullsized': '2024-09-17T05:07:37.346749+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:PG 16.7 is not active+clean 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.353184+0000', 'last_change': '2024-09-17T05:07:37.353184+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.353184+0000', 'last_undegraded': '2024-09-17T05:07:37.353184+0000', 'last_fullsized': '2024-09-17T05:07:37.353184+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:PG 16.8 is not active+clean 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348642+0000', 'last_change': '2024-09-17T05:07:37.348642+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348642+0000', 'last_undegraded': '2024-09-17T05:07:37.348642+0000', 'last_fullsized': '2024-09-17T05:07:37.348642+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.445 INFO:tasks.ceph.ceph_manager.ceph:PG 16.9 is not active+clean 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348460+0000', 'last_change': '2024-09-17T05:07:37.348460+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348460+0000', 'last_undegraded': '2024-09-17T05:07:37.348460+0000', 'last_fullsized': '2024-09-17T05:07:37.348460+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:PG 16.a is not active+clean 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348027+0000', 'last_change': '2024-09-17T05:07:37.348027+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348027+0000', 'last_undegraded': '2024-09-17T05:07:37.348027+0000', 'last_fullsized': '2024-09-17T05:07:37.348027+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:PG 16.b is not active+clean 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347252+0000', 'last_change': '2024-09-17T05:07:37.347252+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347252+0000', 'last_undegraded': '2024-09-17T05:07:37.347252+0000', 'last_fullsized': '2024-09-17T05:07:37.347252+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:PG 16.c is not active+clean 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.343962+0000', 'last_change': '2024-09-17T05:07:37.343962+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.343962+0000', 'last_undegraded': '2024-09-17T05:07:37.343962+0000', 'last_fullsized': '2024-09-17T05:07:37.343962+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:PG 16.d is not active+clean 2024-09-17T05:07:43.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.345638+0000', 'last_change': '2024-09-17T05:07:37.345638+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.345638+0000', 'last_undegraded': '2024-09-17T05:07:37.345638+0000', 'last_fullsized': '2024-09-17T05:07:37.345638+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:PG 16.e is not active+clean 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.353663+0000', 'last_change': '2024-09-17T05:07:37.353663+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.353663+0000', 'last_undegraded': '2024-09-17T05:07:37.353663+0000', 'last_fullsized': '2024-09-17T05:07:37.353663+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:PG 16.10 is not active+clean 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.346070+0000', 'last_change': '2024-09-17T05:07:37.346070+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.346070+0000', 'last_undegraded': '2024-09-17T05:07:37.346070+0000', 'last_fullsized': '2024-09-17T05:07:37.346070+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:PG 16.12 is not active+clean 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347829+0000', 'last_change': '2024-09-17T05:07:37.347829+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347829+0000', 'last_undegraded': '2024-09-17T05:07:37.347829+0000', 'last_fullsized': '2024-09-17T05:07:37.347829+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.447 INFO:tasks.ceph.ceph_manager.ceph:PG 16.13 is not active+clean 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347804+0000', 'last_change': '2024-09-17T05:07:37.347804+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347804+0000', 'last_undegraded': '2024-09-17T05:07:37.347804+0000', 'last_fullsized': '2024-09-17T05:07:37.347804+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 16.14 is not active+clean 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.352275+0000', 'last_change': '2024-09-17T05:07:37.352275+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.352275+0000', 'last_undegraded': '2024-09-17T05:07:37.352275+0000', 'last_fullsized': '2024-09-17T05:07:37.352275+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 16.15 is not active+clean 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.346823+0000', 'last_change': '2024-09-17T05:07:37.346823+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.346823+0000', 'last_undegraded': '2024-09-17T05:07:37.346823+0000', 'last_fullsized': '2024-09-17T05:07:37.346823+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:PG 16.16 is not active+clean 2024-09-17T05:07:43.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.354263+0000', 'last_change': '2024-09-17T05:07:37.354263+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.354263+0000', 'last_undegraded': '2024-09-17T05:07:37.354263+0000', 'last_fullsized': '2024-09-17T05:07:37.354263+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 16.17 is not active+clean 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.353665+0000', 'last_change': '2024-09-17T05:07:37.353665+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.353665+0000', 'last_undegraded': '2024-09-17T05:07:37.353665+0000', 'last_fullsized': '2024-09-17T05:07:37.353665+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 16.18 is not active+clean 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348338+0000', 'last_change': '2024-09-17T05:07:37.348338+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348338+0000', 'last_undegraded': '2024-09-17T05:07:37.348338+0000', 'last_fullsized': '2024-09-17T05:07:37.348338+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 16.19 is not active+clean 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347399+0000', 'last_change': '2024-09-17T05:07:37.347399+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347399+0000', 'last_undegraded': '2024-09-17T05:07:37.347399+0000', 'last_fullsized': '2024-09-17T05:07:37.347399+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1a is not active+clean 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348583+0000', 'last_change': '2024-09-17T05:07:37.348583+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348583+0000', 'last_undegraded': '2024-09-17T05:07:37.348583+0000', 'last_fullsized': '2024-09-17T05:07:37.348583+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1b is not active+clean 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.348957+0000', 'last_change': '2024-09-17T05:07:37.348957+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.348957+0000', 'last_undegraded': '2024-09-17T05:07:37.348957+0000', 'last_fullsized': '2024-09-17T05:07:37.348957+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:07:43.449 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1c is not active+clean 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.352857+0000', 'last_change': '2024-09-17T05:07:37.352857+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.352857+0000', 'last_undegraded': '2024-09-17T05:07:37.352857+0000', 'last_fullsized': '2024-09-17T05:07:37.352857+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1d is not active+clean 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.358208+0000', 'last_change': '2024-09-17T05:07:37.358208+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.358208+0000', 'last_undegraded': '2024-09-17T05:07:37.358208+0000', 'last_fullsized': '2024-09-17T05:07:37.358208+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1e is not active+clean 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347380+0000', 'last_change': '2024-09-17T05:07:37.347380+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347380+0000', 'last_undegraded': '2024-09-17T05:07:37.347380+0000', 'last_fullsized': '2024-09-17T05:07:37.347380+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 16.1f is not active+clean 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.343458+0000', 'last_change': '2024-09-17T05:07:37.343458+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.343458+0000', 'last_undegraded': '2024-09-17T05:07:37.343458+0000', 'last_fullsized': '2024-09-17T05:07:37.343458+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:PG 16.11 is not active+clean 2024-09-17T05:07:43.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 98, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:07:37.347653+0000', 'last_change': '2024-09-17T05:07:37.347653+0000', 'last_active': '2024-09-17T05:07:37.326472+0000', 'last_peered': '2024-09-17T05:07:37.326472+0000', 'last_clean': '2024-09-17T05:07:37.326472+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:07:37.347653+0000', 'last_undegraded': '2024-09-17T05:07:37.347653+0000', 'last_fullsized': '2024-09-17T05:07:37.347653+0000', 'mapping_epoch': 98, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 98, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'last_clean_scrub_stamp': '2024-09-17T05:07:37.326472+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:07:43.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:07:43.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f539de00640 1 -- 172.21.15.16:0/1748511416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 msgr2=0x7f537c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f539de00640 1 --2- 172.21.15.16:0/1748511416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f537c00b9d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5398074e80 tx=0x7f539001fa20 comp rx=0 tx=0).stop 2024-09-17T05:07:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f5397400640 1 -- 172.21.15.16:0/1748511416 reap_dead start 2024-09-17T05:07:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f539de00640 1 -- 172.21.15.16:0/1748511416 shutdown_connections 2024-09-17T05:07:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f539de00640 1 -- 172.21.15.16:0/1748511416 >> 172.21.15.16:0/1748511416 conn(0x7f53980cecc0 msgr2=0x7f53980cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f539de00640 1 -- 172.21.15.16:0/1748511416 shutdown_connections 2024-09-17T05:07:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.639+0000 7f539de00640 1 -- 172.21.15.16:0/1748511416 wait complete. 2024-09-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 Processor -- start 2024-09-17T05:07:43.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 -- start start 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f5398155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53980b5b90 0x7f53980b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f539806e600 0x7f539806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53980d1e60 con 0x7f537c005160 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53980d1ce0 con 0x7f53980b5b90 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f539de00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f539815d610 con 0x7f539806e600 2024-09-17T05:07:43.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f5398155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f5398155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42488/0 (socket says 172.21.15.16:42488) 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 -- 172.21.15.16:0/151069396 learned_addr learned my addr 172.21.15.16:0/151069396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f538fe00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53980b5b90 0x7f53980b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5397400640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f539806e600 0x7f539806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 -- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f539806e600 msgr2=0x7f539806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f539806e600 0x7f539806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 -- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53980b5b90 msgr2=0x7f53980b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53980b5b90 0x7f53980b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 -- 172.21.15.16:0/151069396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539806ef50 con 0x7f537c005160 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.643+0000 7f5396a00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f5398155870 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5388011b10 tx=0x7f5388011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538801e070 con 0x7f537c005160 2024-09-17T05:07:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f538800fc50 con 0x7f537c005160 2024-09-17T05:07:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53980b6610 con 0x7f537c005160 2024-09-17T05:07:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388019410 con 0x7f537c005160 2024-09-17T05:07:43.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53980cb800 con 0x7f537c005160 2024-09-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539815cf00 con 0x7f537c005160 2024-09-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5388021070 con 0x7f537c005160 2024-09-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f538f400640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5374073230 0x7f5374075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.647+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f5388094ce0 con 0x7f537c005160 2024-09-17T05:07:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.651+0000 7f538fe00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5374073230 0x7f5374075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:43.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.651+0000 7f538fe00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5374073230 0x7f5374075740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f53980c5d00 tx=0x7f53800091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:43.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.655+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f539806ef50 con 0x7f537c005160 2024-09-17T05:07:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f539815e5f0 con 0x7f537c005160 2024-09-17T05:07:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f538f400640 1 -- 172.21.15.16:0/151069396 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1139 (secure 0 0 0) 0x7f538805fb60 con 0x7f537c005160 2024-09-17T05:07:43.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:43.898 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":421,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1726549257,"num_in_osds":8,"osd_in_since":1726549246,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":59},{"state_name":"active+clean","count":38}],"num_pgs":97,"num_pools":3,"num_objects":5,"data_bytes":590368,"bytes_used":274882560,"bytes_avail":772819230720,"bytes_total":773094113280,"inactive_pgs_ratio":0.60824739933013916,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":6993}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-17T05:05:59.539048+0000","services":{}},"progress_events":{}} 2024-09-17T05:07:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5374073230 msgr2=0x7f5374075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5374073230 0x7f5374075740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f53980c5d00 tx=0x7f53800091e0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 msgr2=0x7f5398155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f5398155870 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f5388011b10 tx=0x7f5388011f10 comp rx=0 tx=0).stop 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 shutdown_connections 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5374073230 0x7f5374075740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f539806e600 0x7f539806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53980b5b90 0x7f53980b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 --2- 172.21.15.16:0/151069396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f537c005160 0x7f5398155870 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 >> 172.21.15.16:0/151069396 conn(0x7f53980cecc0 msgr2=0x7f5398159900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 shutdown_connections 2024-09-17T05:07:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:43.899+0000 7f539de00640 1 -- 172.21.15.16:0/151069396 wait complete. 2024-09-17T05:07:43.920 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:07:43.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:07:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65c6400640 1 -- 172.21.15.16:0/1007018867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a4003c20 msgr2=0x7f65a4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65c6400640 1 --2- 172.21.15.16:0/1007018867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65a4003c20 0x7f65a4004070 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009b40 tx=0x7f65b001d270 comp rx=0 tx=0).stop 2024-09-17T05:07:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65bfe00640 1 -- 172.21.15.16:0/1007018867 reap_dead start 2024-09-17T05:07:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65c6400640 1 -- 172.21.15.16:0/1007018867 shutdown_connections 2024-09-17T05:07:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65c6400640 1 -- 172.21.15.16:0/1007018867 >> 172.21.15.16:0/1007018867 conn(0x7f65c006f900 msgr2=0x7f65c00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65c6400640 1 -- 172.21.15.16:0/1007018867 shutdown_connections 2024-09-17T05:07:44.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.079+0000 7f65c6400640 1 -- 172.21.15.16:0/1007018867 wait complete. 2024-09-17T05:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 Processor -- start 2024-09-17T05:07:44.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 -- start start 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f65a4003c20 0x7f65c0149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 0x7f65c00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65c01f88c0 0x7f65c01f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0167c30 con 0x7f65c00cee50 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65c0167ab0 con 0x7f65c01f88c0 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65c6400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f65c0167db0 con 0x7f65a4003c20 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 0x7f65c00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 0x7f65c00cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42510/0 (socket says 172.21.15.16:42510) 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 -- 172.21.15.16:0/3507148158 learned_addr learned my addr 172.21.15.16:0/3507148158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:44.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65bf400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f65a4003c20 0x7f65c0149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65bfe00640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65c01f88c0 0x7f65c01f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 -- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f65a4003c20 msgr2=0x7f65c0149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f65a4003c20 0x7f65c0149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 -- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65c01f88c0 msgr2=0x7f65c01f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65c01f88c0 0x7f65c01f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.083+0000 7f65b7e00640 1 -- 172.21.15.16:0/3507148158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c00ce090 con 0x7f65c00cee50 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65b7e00640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 0x7f65c00cf260 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f65a8004cb0 tx=0x7f65a8001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8015020 con 0x7f65c00cee50 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65a8011980 con 0x7f65c00cee50 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8011dd0 con 0x7f65c00cee50 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c00ce370 con 0x7f65c00cee50 2024-09-17T05:07:44.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c0149ab0 con 0x7f65c00cee50 2024-09-17T05:07:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c0073650 con 0x7f65c00cee50 2024-09-17T05:07:44.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f65a8018020 con 0x7f65c00cee50 2024-09-17T05:07:44.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.087+0000 7f65b7400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f659c073200 0x7f659c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:44.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.091+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f65a8094e40 con 0x7f65c00cee50 2024-09-17T05:07:44.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.091+0000 7f65bf400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f659c073200 0x7f659c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.091+0000 7f65bf400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f659c073200 0x7f659c075710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65b00ad000 tx=0x7f65b001a010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:44.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.095+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f65a801f020 con 0x7f65c00cee50 2024-09-17T05:07:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.267+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65c00cde40 con 0x7f659c073200 2024-09-17T05:07:44.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.271+0000 7f65b7400640 1 -- 172.21.15.16:0/3507148158 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252177 (secure 0 0 0) 0x7f65c00cde40 con 0x7f659c073200 2024-09-17T05:07:44.273 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f659c073200 msgr2=0x7f659c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f659c073200 0x7f659c075710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f65b00ad000 tx=0x7f65b001a010 comp rx=0 tx=0).stop 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 msgr2=0x7f65c00cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 0x7f65c00cf260 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f65a8004cb0 tx=0x7f65a8001810 comp rx=0 tx=0).stop 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 shutdown_connections 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f659c073200 0x7f659c075710 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65c01f88c0 0x7f65c01f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65c00cee50 0x7f65c00cf260 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 --2- 172.21.15.16:0/3507148158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f65a4003c20 0x7f65c0149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:44.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 >> 172.21.15.16:0/3507148158 conn(0x7f65c006f900 msgr2=0x7f65c015d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 shutdown_connections 2024-09-17T05:07:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:44.275+0000 7f65c6400640 1 -- 172.21.15.16:0/3507148158 wait complete. 2024-09-17T05:07:44.294 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-17T05:07:43.570948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268712,"kb_used_data":10856,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706008,"statfs":{"total":773094113280,"available":772818952192,"internally_reserved":0,"allocated":11116544,"data_stored":5706812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.234815"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365181+0000","last_change":"2024-09-17T05:07:38.349474+0000","last_active":"2024-09-17T05:07:39.365181+0000","last_peered":"2024-09-17T05:07:39.365181+0000","last_clean":"2024-09-17T05:07:39.365181+0000","last_became_active":"2024-09-17T05:07:38.349166+0000","last_became_peered":"2024-09-17T05:07:38.349166+0000","last_unstale":"2024-09-17T05:07:39.365181+0000","last_undegraded":"2024-09-17T05:07:39.365181+0000","last_fullsized":"2024-09-17T05:07:39.365181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:03.183330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415351+0000","last_change":"2024-09-17T05:07:38.378880+0000","last_active":"2024-09-17T05:07:39.415351+0000","last_peered":"2024-09-17T05:07:39.415351+0000","last_clean":"2024-09-17T05:07:39.415351+0000","last_became_active":"2024-09-17T05:07:38.378565+0000","last_became_peered":"2024-09-17T05:07:38.378565+0000","last_unstale":"2024-09-17T05:07:39.415351+0000","last_undegraded":"2024-09-17T05:07:39.415351+0000","last_fullsized":"2024-09-17T05:07:39.415351+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:45.576630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365386+0000","last_change":"2024-09-17T05:07:38.373066+0000","last_active":"2024-09-17T05:07:39.365386+0000","last_peered":"2024-09-17T05:07:39.365386+0000","last_clean":"2024-09-17T05:07:39.365386+0000","last_became_active":"2024-09-17T05:07:38.372726+0000","last_became_peered":"2024-09-17T05:07:38.372726+0000","last_unstale":"2024-09-17T05:07:39.365386+0000","last_undegraded":"2024-09-17T05:07:39.365386+0000","last_fullsized":"2024-09-17T05:07:39.365386+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:40.211315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363993+0000","last_change":"2024-09-17T05:07:38.371262+0000","last_active":"2024-09-17T05:07:39.363993+0000","last_peered":"2024-09-17T05:07:39.363993+0000","last_clean":"2024-09-17T05:07:39.363993+0000","last_became_active":"2024-09-17T05:07:38.370812+0000","last_became_peered":"2024-09-17T05:07:38.370812+0000","last_unstale":"2024-09-17T05:07:39.363993+0000","last_undegraded":"2024-09-17T05:07:39.363993+0000","last_fullsized":"2024-09-17T05:07:39.363993+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:40.753264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364724+0000","last_change":"2024-09-17T05:07:38.364588+0000","last_active":"2024-09-17T05:07:39.364724+0000","last_peered":"2024-09-17T05:07:39.364724+0000","last_clean":"2024-09-17T05:07:39.364724+0000","last_became_active":"2024-09-17T05:07:38.364271+0000","last_became_peered":"2024-09-17T05:07:38.364271+0000","last_unstale":"2024-09-17T05:07:39.364724+0000","last_undegraded":"2024-09-17T05:07:39.364724+0000","last_fullsized":"2024-09-17T05:07:39.364724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:36.712050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362349+0000","last_change":"2024-09-17T05:07:38.354213+0000","last_active":"2024-09-17T05:07:39.362349+0000","last_peered":"2024-09-17T05:07:39.362349+0000","last_clean":"2024-09-17T05:07:39.362349+0000","last_became_active":"2024-09-17T05:07:38.353898+0000","last_became_peered":"2024-09-17T05:07:38.353898+0000","last_unstale":"2024-09-17T05:07:39.362349+0000","last_undegraded":"2024-09-17T05:07:39.362349+0000","last_fullsized":"2024-09-17T05:07:39.362349+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:15.826789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362073+0000","last_change":"2024-09-17T05:07:38.352925+0000","last_active":"2024-09-17T05:07:39.362073+0000","last_peered":"2024-09-17T05:07:39.362073+0000","last_clean":"2024-09-17T05:07:39.362073+0000","last_became_active":"2024-09-17T05:07:38.352587+0000","last_became_peered":"2024-09-17T05:07:38.352587+0000","last_unstale":"2024-09-17T05:07:39.362073+0000","last_undegraded":"2024-09-17T05:07:39.362073+0000","last_fullsized":"2024-09-17T05:07:39.362073+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:05.811532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414053+0000","last_change":"2024-09-17T05:07:38.374955+0000","last_active":"2024-09-17T05:07:39.414053+0000","last_peered":"2024-09-17T05:07:39.414053+0000","last_clean":"2024-09-17T05:07:39.414053+0000","last_became_active":"2024-09-17T05:07:38.374642+0000","last_became_peered":"2024-09-17T05:07:38.374642+0000","last_unstale":"2024-09-17T05:07:39.414053+0000","last_undegraded":"2024-09-17T05:07:39.414053+0000","last_fullsized":"2024-09-17T05:07:39.414053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:03.975633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416424+0000","last_change":"2024-09-17T05:07:38.371399+0000","last_active":"2024-09-17T05:07:39.416424+0000","last_peered":"2024-09-17T05:07:39.416424+0000","last_clean":"2024-09-17T05:07:39.416424+0000","last_became_active":"2024-09-17T05:07:38.371082+0000","last_became_peered":"2024-09-17T05:07:38.371082+0000","last_unstale":"2024-09-17T05:07:39.416424+0000","last_undegraded":"2024-09-17T05:07:39.416424+0000","last_fullsized":"2024-09-17T05:07:39.416424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:10.098240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414665+0000","last_change":"2024-09-17T05:07:38.377561+0000","last_active":"2024-09-17T05:07:39.414665+0000","last_peered":"2024-09-17T05:07:39.414665+0000","last_clean":"2024-09-17T05:07:39.414665+0000","last_became_active":"2024-09-17T05:07:38.377253+0000","last_became_peered":"2024-09-17T05:07:38.377253+0000","last_unstale":"2024-09-17T05:07:39.414665+0000","last_undegraded":"2024-09-17T05:07:39.414665+0000","last_fullsized":"2024-09-17T05:07:39.414665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:52.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416846+0000","last_change":"2024-09-17T05:07:38.367526+0000","last_active":"2024-09-17T05:07:39.416846+0000","last_peered":"2024-09-17T05:07:39.416846+0000","last_clean":"2024-09-17T05:07:39.416846+0000","last_became_active":"2024-09-17T05:07:38.367209+0000","last_became_peered":"2024-09-17T05:07:38.367209+0000","last_unstale":"2024-09-17T05:07:39.416846+0000","last_undegraded":"2024-09-17T05:07:39.416846+0000","last_fullsized":"2024-09-17T05:07:39.416846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:26.936939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362606+0000","last_change":"2024-09-17T05:07:38.375041+0000","last_active":"2024-09-17T05:07:39.362606+0000","last_peered":"2024-09-17T05:07:39.362606+0000","last_clean":"2024-09-17T05:07:39.362606+0000","last_became_active":"2024-09-17T05:07:38.372665+0000","last_became_peered":"2024-09-17T05:07:38.372665+0000","last_unstale":"2024-09-17T05:07:39.362606+0000","last_undegraded":"2024-09-17T05:07:39.362606+0000","last_fullsized":"2024-09-17T05:07:39.362606+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:39.425452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364112+0000","last_change":"2024-09-17T05:07:38.351889+0000","last_active":"2024-09-17T05:07:39.364112+0000","last_peered":"2024-09-17T05:07:39.364112+0000","last_clean":"2024-09-17T05:07:39.364112+0000","last_became_active":"2024-09-17T05:07:38.351576+0000","last_became_peered":"2024-09-17T05:07:38.351576+0000","last_unstale":"2024-09-17T05:07:39.364112+0000","last_undegraded":"2024-09-17T05:07:39.364112+0000","last_fullsized":"2024-09-17T05:07:39.364112+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:07.224493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362408+0000","last_change":"2024-09-17T05:07:38.371821+0000","last_active":"2024-09-17T05:07:39.362408+0000","last_peered":"2024-09-17T05:07:39.362408+0000","last_clean":"2024-09-17T05:07:39.362408+0000","last_became_active":"2024-09-17T05:07:38.371476+0000","last_became_peered":"2024-09-17T05:07:38.371476+0000","last_unstale":"2024-09-17T05:07:39.362408+0000","last_undegraded":"2024-09-17T05:07:39.362408+0000","last_fullsized":"2024-09-17T05:07:39.362408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:46.787438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415065+0000","last_change":"2024-09-17T05:07:38.368316+0000","last_active":"2024-09-17T05:07:39.415065+0000","last_peered":"2024-09-17T05:07:39.415065+0000","last_clean":"2024-09-17T05:07:39.415065+0000","last_became_active":"2024-09-17T05:07:38.367970+0000","last_became_peered":"2024-09-17T05:07:38.367970+0000","last_unstale":"2024-09-17T05:07:39.415065+0000","last_undegraded":"2024-09-17T05:07:39.415065+0000","last_fullsized":"2024-09-17T05:07:39.415065+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:13.001958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415658+0000","last_change":"2024-09-17T05:07:38.353082+0000","last_active":"2024-09-17T05:07:39.415658+0000","last_peered":"2024-09-17T05:07:39.415658+0000","last_clean":"2024-09-17T05:07:39.415658+0000","last_became_active":"2024-09-17T05:07:38.352602+0000","last_became_peered":"2024-09-17T05:07:38.352602+0000","last_unstale":"2024-09-17T05:07:39.415658+0000","last_undegraded":"2024-09-17T05:07:39.415658+0000","last_fullsized":"2024-09-17T05:07:39.415658+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:22.101853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365037+0000","last_change":"2024-09-17T05:07:38.369916+0000","last_active":"2024-09-17T05:07:39.365037+0000","last_peered":"2024-09-17T05:07:39.365037+0000","last_clean":"2024-09-17T05:07:39.365037+0000","last_became_active":"2024-09-17T05:07:38.369536+0000","last_became_peered":"2024-09-17T05:07:38.369536+0000","last_unstale":"2024-09-17T05:07:39.365037+0000","last_undegraded":"2024-09-17T05:07:39.365037+0000","last_fullsized":"2024-09-17T05:07:39.365037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:07.275670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416012+0000","last_change":"2024-09-17T05:07:38.365371+0000","last_active":"2024-09-17T05:07:39.416012+0000","last_peered":"2024-09-17T05:07:39.416012+0000","last_clean":"2024-09-17T05:07:39.416012+0000","last_became_active":"2024-09-17T05:07:38.365027+0000","last_became_peered":"2024-09-17T05:07:38.365027+0000","last_unstale":"2024-09-17T05:07:39.416012+0000","last_undegraded":"2024-09-17T05:07:39.416012+0000","last_fullsized":"2024-09-17T05:07:39.416012+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:13.836046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364710+0000","last_change":"2024-09-17T05:07:38.372948+0000","last_active":"2024-09-17T05:07:39.364710+0000","last_peered":"2024-09-17T05:07:39.364710+0000","last_clean":"2024-09-17T05:07:39.364710+0000","last_became_active":"2024-09-17T05:07:38.372526+0000","last_became_peered":"2024-09-17T05:07:38.372526+0000","last_unstale":"2024-09-17T05:07:39.364710+0000","last_undegraded":"2024-09-17T05:07:39.364710+0000","last_fullsized":"2024-09-17T05:07:39.364710+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:50.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417054+0000","last_change":"2024-09-17T05:07:38.366601+0000","last_active":"2024-09-17T05:07:39.417054+0000","last_peered":"2024-09-17T05:07:39.417054+0000","last_clean":"2024-09-17T05:07:39.417054+0000","last_became_active":"2024-09-17T05:07:38.366223+0000","last_became_peered":"2024-09-17T05:07:38.366223+0000","last_unstale":"2024-09-17T05:07:39.417054+0000","last_undegraded":"2024-09-17T05:07:39.417054+0000","last_fullsized":"2024-09-17T05:07:39.417054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:05.811532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364572+0000","last_change":"2024-09-17T05:07:38.365788+0000","last_active":"2024-09-17T05:07:39.364572+0000","last_peered":"2024-09-17T05:07:39.364572+0000","last_clean":"2024-09-17T05:07:39.364572+0000","last_became_active":"2024-09-17T05:07:38.364783+0000","last_became_peered":"2024-09-17T05:07:38.364783+0000","last_unstale":"2024-09-17T05:07:39.364572+0000","last_undegraded":"2024-09-17T05:07:39.364572+0000","last_fullsized":"2024-09-17T05:07:39.364572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:32.081371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414574+0000","last_change":"2024-09-17T05:07:38.376620+0000","last_active":"2024-09-17T05:07:39.414574+0000","last_peered":"2024-09-17T05:07:39.414574+0000","last_clean":"2024-09-17T05:07:39.414574+0000","last_became_active":"2024-09-17T05:07:38.376316+0000","last_became_peered":"2024-09-17T05:07:38.376316+0000","last_unstale":"2024-09-17T05:07:39.414574+0000","last_undegraded":"2024-09-17T05:07:39.414574+0000","last_fullsized":"2024-09-17T05:07:39.414574+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:15.492636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415287+0000","last_change":"2024-09-17T05:07:38.384020+0000","last_active":"2024-09-17T05:07:39.415287+0000","last_peered":"2024-09-17T05:07:39.415287+0000","last_clean":"2024-09-17T05:07:39.415287+0000","last_became_active":"2024-09-17T05:07:38.383707+0000","last_became_peered":"2024-09-17T05:07:38.383707+0000","last_unstale":"2024-09-17T05:07:39.415287+0000","last_undegraded":"2024-09-17T05:07:39.415287+0000","last_fullsized":"2024-09-17T05:07:39.415287+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:25.182650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414053+0000","last_change":"2024-09-17T05:07:38.361287+0000","last_active":"2024-09-17T05:07:39.414053+0000","last_peered":"2024-09-17T05:07:39.414053+0000","last_clean":"2024-09-17T05:07:39.414053+0000","last_became_active":"2024-09-17T05:07:38.360906+0000","last_became_peered":"2024-09-17T05:07:38.360906+0000","last_unstale":"2024-09-17T05:07:39.414053+0000","last_undegraded":"2024-09-17T05:07:39.414053+0000","last_fullsized":"2024-09-17T05:07:39.414053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:25.433058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417259+0000","last_change":"2024-09-17T05:07:38.360975+0000","last_active":"2024-09-17T05:07:39.417259+0000","last_peered":"2024-09-17T05:07:39.417259+0000","last_clean":"2024-09-17T05:07:39.417259+0000","last_became_active":"2024-09-17T05:07:38.360604+0000","last_became_peered":"2024-09-17T05:07:38.360604+0000","last_unstale":"2024-09-17T05:07:39.417259+0000","last_undegraded":"2024-09-17T05:07:39.417259+0000","last_fullsized":"2024-09-17T05:07:39.417259+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:26.054302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417368+0000","last_change":"2024-09-17T05:07:38.366400+0000","last_active":"2024-09-17T05:07:39.417368+0000","last_peered":"2024-09-17T05:07:39.417368+0000","last_clean":"2024-09-17T05:07:39.417368+0000","last_became_active":"2024-09-17T05:07:38.360850+0000","last_became_peered":"2024-09-17T05:07:38.360850+0000","last_unstale":"2024-09-17T05:07:39.417368+0000","last_undegraded":"2024-09-17T05:07:39.417368+0000","last_fullsized":"2024-09-17T05:07:39.417368+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:15.826789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417201+0000","last_change":"2024-09-17T05:07:38.365699+0000","last_active":"2024-09-17T05:07:39.417201+0000","last_peered":"2024-09-17T05:07:39.417201+0000","last_clean":"2024-09-17T05:07:39.417201+0000","last_became_active":"2024-09-17T05:07:38.364854+0000","last_became_peered":"2024-09-17T05:07:38.364854+0000","last_unstale":"2024-09-17T05:07:39.417201+0000","last_undegraded":"2024-09-17T05:07:39.417201+0000","last_fullsized":"2024-09-17T05:07:39.417201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:53.530325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362699+0000","last_change":"2024-09-17T05:07:38.358348+0000","last_active":"2024-09-17T05:07:39.362699+0000","last_peered":"2024-09-17T05:07:39.362699+0000","last_clean":"2024-09-17T05:07:39.362699+0000","last_became_active":"2024-09-17T05:07:38.358031+0000","last_became_peered":"2024-09-17T05:07:38.358031+0000","last_unstale":"2024-09-17T05:07:39.362699+0000","last_undegraded":"2024-09-17T05:07:39.362699+0000","last_fullsized":"2024-09-17T05:07:39.362699+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:57.952363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414835+0000","last_change":"2024-09-17T05:07:38.375974+0000","last_active":"2024-09-17T05:07:39.414835+0000","last_peered":"2024-09-17T05:07:39.414835+0000","last_clean":"2024-09-17T05:07:39.414835+0000","last_became_active":"2024-09-17T05:07:38.375278+0000","last_became_peered":"2024-09-17T05:07:38.375278+0000","last_unstale":"2024-09-17T05:07:39.414835+0000","last_undegraded":"2024-09-17T05:07:39.414835+0000","last_fullsized":"2024-09-17T05:07:39.414835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:44.488365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416613+0000","last_change":"2024-09-17T05:07:38.366829+0000","last_active":"2024-09-17T05:07:39.416613+0000","last_peered":"2024-09-17T05:07:39.416613+0000","last_clean":"2024-09-17T05:07:39.416613+0000","last_became_active":"2024-09-17T05:07:38.366501+0000","last_became_peered":"2024-09-17T05:07:38.366501+0000","last_unstale":"2024-09-17T05:07:39.416613+0000","last_undegraded":"2024-09-17T05:07:39.416613+0000","last_fullsized":"2024-09-17T05:07:39.416613+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:07.402424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362182+0000","last_change":"2024-09-17T05:07:38.364091+0000","last_active":"2024-09-17T05:07:39.362182+0000","last_peered":"2024-09-17T05:07:39.362182+0000","last_clean":"2024-09-17T05:07:39.362182+0000","last_became_active":"2024-09-17T05:07:38.362464+0000","last_became_peered":"2024-09-17T05:07:38.362464+0000","last_unstale":"2024-09-17T05:07:39.362182+0000","last_undegraded":"2024-09-17T05:07:39.362182+0000","last_fullsized":"2024-09-17T05:07:39.362182+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:53.530325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415217+0000","last_change":"2024-09-17T05:07:38.370784+0000","last_active":"2024-09-17T05:07:39.415217+0000","last_peered":"2024-09-17T05:07:39.415217+0000","last_clean":"2024-09-17T05:07:39.415217+0000","last_became_active":"2024-09-17T05:07:38.370468+0000","last_became_peered":"2024-09-17T05:07:38.370468+0000","last_unstale":"2024-09-17T05:07:39.415217+0000","last_undegraded":"2024-09-17T05:07:39.415217+0000","last_fullsized":"2024-09-17T05:07:39.415217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:49.572784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363817+0000","last_change":"2024-09-17T05:07:38.351295+0000","last_active":"2024-09-17T05:07:39.363817+0000","last_peered":"2024-09-17T05:07:39.363817+0000","last_clean":"2024-09-17T05:07:39.363817+0000","last_became_active":"2024-09-17T05:07:38.348971+0000","last_became_peered":"2024-09-17T05:07:38.348971+0000","last_unstale":"2024-09-17T05:07:39.363817+0000","last_undegraded":"2024-09-17T05:07:39.363817+0000","last_fullsized":"2024-09-17T05:07:39.363817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:12.675618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"99'4","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414897+0000","last_change":"2024-09-17T05:07:37.361640+0000","last_active":"2024-09-17T05:07:39.414897+0000","last_peered":"2024-09-17T05:07:39.414897+0000","last_clean":"2024-09-17T05:07:39.414897+0000","last_became_active":"2024-09-17T05:07:37.360159+0000","last_became_peered":"2024-09-17T05:07:37.360159+0000","last_unstale":"2024-09-17T05:07:39.414897+0000","last_undegraded":"2024-09-17T05:07:39.414897+0000","last_fullsized":"2024-09-17T05:07:39.414897+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:53.762127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414554+0000","last_change":"2024-09-17T05:07:38.382830+0000","last_active":"2024-09-17T05:07:39.414554+0000","last_peered":"2024-09-17T05:07:39.414554+0000","last_clean":"2024-09-17T05:07:39.414554+0000","last_became_active":"2024-09-17T05:07:38.382534+0000","last_became_peered":"2024-09-17T05:07:38.382534+0000","last_unstale":"2024-09-17T05:07:39.414554+0000","last_undegraded":"2024-09-17T05:07:39.414554+0000","last_fullsized":"2024-09-17T05:07:39.414554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:53.511630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364559+0000","last_change":"2024-09-17T05:07:37.367481+0000","last_active":"2024-09-17T05:07:39.364559+0000","last_peered":"2024-09-17T05:07:39.364559+0000","last_clean":"2024-09-17T05:07:39.364559+0000","last_became_active":"2024-09-17T05:07:37.367109+0000","last_became_peered":"2024-09-17T05:07:37.367109+0000","last_unstale":"2024-09-17T05:07:39.364559+0000","last_undegraded":"2024-09-17T05:07:39.364559+0000","last_fullsized":"2024-09-17T05:07:39.364559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:25.251480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415152+0000","last_change":"2024-09-17T05:07:38.370197+0000","last_active":"2024-09-17T05:07:39.415152+0000","last_peered":"2024-09-17T05:07:39.415152+0000","last_clean":"2024-09-17T05:07:39.415152+0000","last_became_active":"2024-09-17T05:07:38.369400+0000","last_became_peered":"2024-09-17T05:07:38.369400+0000","last_unstale":"2024-09-17T05:07:39.415152+0000","last_undegraded":"2024-09-17T05:07:39.415152+0000","last_fullsized":"2024-09-17T05:07:39.415152+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:34.270797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415529+0000","last_change":"2024-09-17T05:07:37.360648+0000","last_active":"2024-09-17T05:07:39.415529+0000","last_peered":"2024-09-17T05:07:39.415529+0000","last_clean":"2024-09-17T05:07:39.415529+0000","last_became_active":"2024-09-17T05:07:37.356573+0000","last_became_peered":"2024-09-17T05:07:37.356573+0000","last_unstale":"2024-09-17T05:07:39.415529+0000","last_undegraded":"2024-09-17T05:07:39.415529+0000","last_fullsized":"2024-09-17T05:07:39.415529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:05.220061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416821+0000","last_change":"2024-09-17T05:07:38.355606+0000","last_active":"2024-09-17T05:07:39.416821+0000","last_peered":"2024-09-17T05:07:39.416821+0000","last_clean":"2024-09-17T05:07:39.416821+0000","last_became_active":"2024-09-17T05:07:38.355214+0000","last_became_peered":"2024-09-17T05:07:38.355214+0000","last_unstale":"2024-09-17T05:07:39.416821+0000","last_undegraded":"2024-09-17T05:07:39.416821+0000","last_fullsized":"2024-09-17T05:07:39.416821+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:00.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414238+0000","last_change":"2024-09-17T05:07:37.364681+0000","last_active":"2024-09-17T05:07:39.414238+0000","last_peered":"2024-09-17T05:07:39.414238+0000","last_clean":"2024-09-17T05:07:39.414238+0000","last_became_active":"2024-09-17T05:07:37.364348+0000","last_became_peered":"2024-09-17T05:07:37.364348+0000","last_unstale":"2024-09-17T05:07:39.414238+0000","last_undegraded":"2024-09-17T05:07:39.414238+0000","last_fullsized":"2024-09-17T05:07:39.414238+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:30.250628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364659+0000","last_change":"2024-09-17T05:07:38.349727+0000","last_active":"2024-09-17T05:07:39.364659+0000","last_peered":"2024-09-17T05:07:39.364659+0000","last_clean":"2024-09-17T05:07:39.364659+0000","last_became_active":"2024-09-17T05:07:38.349358+0000","last_became_peered":"2024-09-17T05:07:38.349358+0000","last_unstale":"2024-09-17T05:07:39.364659+0000","last_undegraded":"2024-09-17T05:07:39.364659+0000","last_fullsized":"2024-09-17T05:07:39.364659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:21.290301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416825+0000","last_change":"2024-09-17T05:07:37.368233+0000","last_active":"2024-09-17T05:07:39.416825+0000","last_peered":"2024-09-17T05:07:39.416825+0000","last_clean":"2024-09-17T05:07:39.416825+0000","last_became_active":"2024-09-17T05:07:37.367904+0000","last_became_peered":"2024-09-17T05:07:37.367904+0000","last_unstale":"2024-09-17T05:07:39.416825+0000","last_undegraded":"2024-09-17T05:07:39.416825+0000","last_fullsized":"2024-09-17T05:07:39.416825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:40.022110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365963+0000","last_change":"2024-09-17T05:07:38.359562+0000","last_active":"2024-09-17T05:07:39.365963+0000","last_peered":"2024-09-17T05:07:39.365963+0000","last_clean":"2024-09-17T05:07:39.365963+0000","last_became_active":"2024-09-17T05:07:38.358891+0000","last_became_peered":"2024-09-17T05:07:38.358891+0000","last_unstale":"2024-09-17T05:07:39.365963+0000","last_undegraded":"2024-09-17T05:07:39.365963+0000","last_fullsized":"2024-09-17T05:07:39.365963+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:29.030523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416164+0000","last_change":"2024-09-17T05:07:37.368607+0000","last_active":"2024-09-17T05:07:39.416164+0000","last_peered":"2024-09-17T05:07:39.416164+0000","last_clean":"2024-09-17T05:07:39.416164+0000","last_became_active":"2024-09-17T05:07:37.368307+0000","last_became_peered":"2024-09-17T05:07:37.368307+0000","last_unstale":"2024-09-17T05:07:39.416164+0000","last_undegraded":"2024-09-17T05:07:39.416164+0000","last_fullsized":"2024-09-17T05:07:39.416164+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417454+0000","last_change":"2024-09-17T05:07:38.355984+0000","last_active":"2024-09-17T05:07:39.417454+0000","last_peered":"2024-09-17T05:07:39.417454+0000","last_clean":"2024-09-17T05:07:39.417454+0000","last_became_active":"2024-09-17T05:07:38.355541+0000","last_became_peered":"2024-09-17T05:07:38.355541+0000","last_unstale":"2024-09-17T05:07:39.417454+0000","last_undegraded":"2024-09-17T05:07:39.417454+0000","last_fullsized":"2024-09-17T05:07:39.417454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:25.686562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415772+0000","last_change":"2024-09-17T05:07:37.367847+0000","last_active":"2024-09-17T05:07:39.415772+0000","last_peered":"2024-09-17T05:07:39.415772+0000","last_clean":"2024-09-17T05:07:39.415772+0000","last_became_active":"2024-09-17T05:07:37.367465+0000","last_became_peered":"2024-09-17T05:07:37.367465+0000","last_unstale":"2024-09-17T05:07:39.415772+0000","last_undegraded":"2024-09-17T05:07:39.415772+0000","last_fullsized":"2024-09-17T05:07:39.415772+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:13.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365837+0000","last_change":"2024-09-17T05:07:38.362869+0000","last_active":"2024-09-17T05:07:39.365837+0000","last_peered":"2024-09-17T05:07:39.365837+0000","last_clean":"2024-09-17T05:07:39.365837+0000","last_became_active":"2024-09-17T05:07:38.362419+0000","last_became_peered":"2024-09-17T05:07:38.362419+0000","last_unstale":"2024-09-17T05:07:39.365837+0000","last_undegraded":"2024-09-17T05:07:39.365837+0000","last_fullsized":"2024-09-17T05:07:39.365837+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:48.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415369+0000","last_change":"2024-09-17T05:07:37.364825+0000","last_active":"2024-09-17T05:07:39.415369+0000","last_peered":"2024-09-17T05:07:39.415369+0000","last_clean":"2024-09-17T05:07:39.415369+0000","last_became_active":"2024-09-17T05:07:37.364481+0000","last_became_peered":"2024-09-17T05:07:37.364481+0000","last_unstale":"2024-09-17T05:07:39.415369+0000","last_undegraded":"2024-09-17T05:07:39.415369+0000","last_fullsized":"2024-09-17T05:07:39.415369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:40.978992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363791+0000","last_change":"2024-09-17T05:07:38.350775+0000","last_active":"2024-09-17T05:07:39.363791+0000","last_peered":"2024-09-17T05:07:39.363791+0000","last_clean":"2024-09-17T05:07:39.363791+0000","last_became_active":"2024-09-17T05:07:38.350028+0000","last_became_peered":"2024-09-17T05:07:38.350028+0000","last_unstale":"2024-09-17T05:07:39.363791+0000","last_undegraded":"2024-09-17T05:07:39.363791+0000","last_fullsized":"2024-09-17T05:07:39.363791+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:25.976813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363083+0000","last_change":"2024-09-17T05:07:37.362446+0000","last_active":"2024-09-17T05:07:39.363083+0000","last_peered":"2024-09-17T05:07:39.363083+0000","last_clean":"2024-09-17T05:07:39.363083+0000","last_became_active":"2024-09-17T05:07:37.362104+0000","last_became_peered":"2024-09-17T05:07:37.362104+0000","last_unstale":"2024-09-17T05:07:39.363083+0000","last_undegraded":"2024-09-17T05:07:39.363083+0000","last_fullsized":"2024-09-17T05:07:39.363083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:08.221672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364275+0000","last_change":"2024-09-17T05:07:38.365090+0000","last_active":"2024-09-17T05:07:39.364275+0000","last_peered":"2024-09-17T05:07:39.364275+0000","last_clean":"2024-09-17T05:07:39.364275+0000","last_became_active":"2024-09-17T05:07:38.364675+0000","last_became_peered":"2024-09-17T05:07:38.364675+0000","last_unstale":"2024-09-17T05:07:39.364275+0000","last_undegraded":"2024-09-17T05:07:39.364275+0000","last_fullsized":"2024-09-17T05:07:39.364275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:23.544324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414879+0000","last_change":"2024-09-17T05:07:37.359338+0000","last_active":"2024-09-17T05:07:39.414879+0000","last_peered":"2024-09-17T05:07:39.414879+0000","last_clean":"2024-09-17T05:07:39.414879+0000","last_became_active":"2024-09-17T05:07:37.359008+0000","last_became_peered":"2024-09-17T05:07:37.359008+0000","last_unstale":"2024-09-17T05:07:39.414879+0000","last_undegraded":"2024-09-17T05:07:39.414879+0000","last_fullsized":"2024-09-17T05:07:39.414879+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:31.114818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414670+0000","last_change":"2024-09-17T05:07:38.362780+0000","last_active":"2024-09-17T05:07:39.414670+0000","last_peered":"2024-09-17T05:07:39.414670+0000","last_clean":"2024-09-17T05:07:39.414670+0000","last_became_active":"2024-09-17T05:07:38.362473+0000","last_became_peered":"2024-09-17T05:07:38.362473+0000","last_unstale":"2024-09-17T05:07:39.414670+0000","last_undegraded":"2024-09-17T05:07:39.414670+0000","last_fullsized":"2024-09-17T05:07:39.414670+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:46.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"99'3","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362815+0000","last_change":"2024-09-17T05:07:37.362270+0000","last_active":"2024-09-17T05:07:39.362815+0000","last_peered":"2024-09-17T05:07:39.362815+0000","last_clean":"2024-09-17T05:07:39.362815+0000","last_became_active":"2024-09-17T05:07:37.361924+0000","last_became_peered":"2024-09-17T05:07:37.361924+0000","last_unstale":"2024-09-17T05:07:39.362815+0000","last_undegraded":"2024-09-17T05:07:39.362815+0000","last_fullsized":"2024-09-17T05:07:39.362815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:22.439861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415030+0000","last_change":"2024-09-17T05:07:38.374469+0000","last_active":"2024-09-17T05:07:39.415030+0000","last_peered":"2024-09-17T05:07:39.415030+0000","last_clean":"2024-09-17T05:07:39.415030+0000","last_became_active":"2024-09-17T05:07:38.374098+0000","last_became_peered":"2024-09-17T05:07:38.374098+0000","last_unstale":"2024-09-17T05:07:39.415030+0000","last_undegraded":"2024-09-17T05:07:39.415030+0000","last_fullsized":"2024-09-17T05:07:39.415030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:33.122152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416729+0000","last_change":"2024-09-17T05:07:37.365192+0000","last_active":"2024-09-17T05:07:39.416729+0000","last_peered":"2024-09-17T05:07:39.416729+0000","last_clean":"2024-09-17T05:07:39.416729+0000","last_became_active":"2024-09-17T05:07:37.364884+0000","last_became_peered":"2024-09-17T05:07:37.364884+0000","last_unstale":"2024-09-17T05:07:39.416729+0000","last_undegraded":"2024-09-17T05:07:39.416729+0000","last_fullsized":"2024-09-17T05:07:39.416729+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:42.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415661+0000","last_change":"2024-09-17T05:07:38.367518+0000","last_active":"2024-09-17T05:07:39.415661+0000","last_peered":"2024-09-17T05:07:39.415661+0000","last_clean":"2024-09-17T05:07:39.415661+0000","last_became_active":"2024-09-17T05:07:38.366758+0000","last_became_peered":"2024-09-17T05:07:38.366758+0000","last_unstale":"2024-09-17T05:07:39.415661+0000","last_undegraded":"2024-09-17T05:07:39.415661+0000","last_fullsized":"2024-09-17T05:07:39.415661+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:39.501480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365076+0000","last_change":"2024-09-17T05:07:37.358555+0000","last_active":"2024-09-17T05:07:39.365076+0000","last_peered":"2024-09-17T05:07:39.365076+0000","last_clean":"2024-09-17T05:07:39.365076+0000","last_became_active":"2024-09-17T05:07:37.358188+0000","last_became_peered":"2024-09-17T05:07:37.358188+0000","last_unstale":"2024-09-17T05:07:39.365076+0000","last_undegraded":"2024-09-17T05:07:39.365076+0000","last_fullsized":"2024-09-17T05:07:39.365076+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:04.804019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415466+0000","last_change":"2024-09-17T05:07:38.363561+0000","last_active":"2024-09-17T05:07:39.415466+0000","last_peered":"2024-09-17T05:07:39.415466+0000","last_clean":"2024-09-17T05:07:39.415466+0000","last_became_active":"2024-09-17T05:07:38.363042+0000","last_became_peered":"2024-09-17T05:07:38.363042+0000","last_unstale":"2024-09-17T05:07:39.415466+0000","last_undegraded":"2024-09-17T05:07:39.415466+0000","last_fullsized":"2024-09-17T05:07:39.415466+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:07.641855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414733+0000","last_change":"2024-09-17T05:07:37.363006+0000","last_active":"2024-09-17T05:07:39.414733+0000","last_peered":"2024-09-17T05:07:39.414733+0000","last_clean":"2024-09-17T05:07:39.414733+0000","last_became_active":"2024-09-17T05:07:37.362664+0000","last_became_peered":"2024-09-17T05:07:37.362664+0000","last_unstale":"2024-09-17T05:07:39.414733+0000","last_undegraded":"2024-09-17T05:07:39.414733+0000","last_fullsized":"2024-09-17T05:07:39.414733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:50.405118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414804+0000","last_change":"2024-09-17T05:07:38.375962+0000","last_active":"2024-09-17T05:07:39.414804+0000","last_peered":"2024-09-17T05:07:39.414804+0000","last_clean":"2024-09-17T05:07:39.414804+0000","last_became_active":"2024-09-17T05:07:38.375669+0000","last_became_peered":"2024-09-17T05:07:38.375669+0000","last_unstale":"2024-09-17T05:07:39.414804+0000","last_undegraded":"2024-09-17T05:07:39.414804+0000","last_fullsized":"2024-09-17T05:07:39.414804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:26.748990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364478+0000","last_change":"2024-09-17T05:07:37.365145+0000","last_active":"2024-09-17T05:07:39.364478+0000","last_peered":"2024-09-17T05:07:39.364478+0000","last_clean":"2024-09-17T05:07:39.364478+0000","last_became_active":"2024-09-17T05:07:37.364775+0000","last_became_peered":"2024-09-17T05:07:37.364775+0000","last_unstale":"2024-09-17T05:07:39.364478+0000","last_undegraded":"2024-09-17T05:07:39.364478+0000","last_fullsized":"2024-09-17T05:07:39.364478+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:18.797265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365284+0000","last_change":"2024-09-17T05:07:38.369937+0000","last_active":"2024-09-17T05:07:39.365284+0000","last_peered":"2024-09-17T05:07:39.365284+0000","last_clean":"2024-09-17T05:07:39.365284+0000","last_became_active":"2024-09-17T05:07:38.369450+0000","last_became_peered":"2024-09-17T05:07:38.369450+0000","last_unstale":"2024-09-17T05:07:39.365284+0000","last_undegraded":"2024-09-17T05:07:39.365284+0000","last_fullsized":"2024-09-17T05:07:39.365284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:59.251209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"99'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415327+0000","last_change":"2024-09-17T05:07:37.368862+0000","last_active":"2024-09-17T05:07:39.415327+0000","last_peered":"2024-09-17T05:07:39.415327+0000","last_clean":"2024-09-17T05:07:39.415327+0000","last_became_active":"2024-09-17T05:07:37.368509+0000","last_became_peered":"2024-09-17T05:07:37.368509+0000","last_unstale":"2024-09-17T05:07:39.415327+0000","last_undegraded":"2024-09-17T05:07:39.415327+0000","last_fullsized":"2024-09-17T05:07:39.415327+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.530129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414372+0000","last_change":"2024-09-17T05:07:38.365896+0000","last_active":"2024-09-17T05:07:39.414372+0000","last_peered":"2024-09-17T05:07:39.414372+0000","last_clean":"2024-09-17T05:07:39.414372+0000","last_became_active":"2024-09-17T05:07:38.365546+0000","last_became_peered":"2024-09-17T05:07:38.365546+0000","last_unstale":"2024-09-17T05:07:39.414372+0000","last_undegraded":"2024-09-17T05:07:39.414372+0000","last_fullsized":"2024-09-17T05:07:39.414372+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:30.766430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363124+0000","last_change":"2024-09-17T05:07:37.364495+0000","last_active":"2024-09-17T05:07:39.363124+0000","last_peered":"2024-09-17T05:07:39.363124+0000","last_clean":"2024-09-17T05:07:39.363124+0000","last_became_active":"2024-09-17T05:07:37.364156+0000","last_became_peered":"2024-09-17T05:07:37.364156+0000","last_unstale":"2024-09-17T05:07:39.363124+0000","last_undegraded":"2024-09-17T05:07:39.363124+0000","last_fullsized":"2024-09-17T05:07:39.363124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:14.240291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415949+0000","last_change":"2024-09-17T05:07:38.355046+0000","last_active":"2024-09-17T05:07:39.415949+0000","last_peered":"2024-09-17T05:07:39.415949+0000","last_clean":"2024-09-17T05:07:39.415949+0000","last_became_active":"2024-09-17T05:07:38.354647+0000","last_became_peered":"2024-09-17T05:07:38.354647+0000","last_unstale":"2024-09-17T05:07:39.415949+0000","last_undegraded":"2024-09-17T05:07:39.415949+0000","last_fullsized":"2024-09-17T05:07:39.415949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:09.954734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417200+0000","last_change":"2024-09-17T05:07:37.367221+0000","last_active":"2024-09-17T05:07:39.417200+0000","last_peered":"2024-09-17T05:07:39.417200+0000","last_clean":"2024-09-17T05:07:39.417200+0000","last_became_active":"2024-09-17T05:07:37.365939+0000","last_became_peered":"2024-09-17T05:07:37.365939+0000","last_unstale":"2024-09-17T05:07:39.417200+0000","last_undegraded":"2024-09-17T05:07:39.417200+0000","last_fullsized":"2024-09-17T05:07:39.417200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:03.504912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414423+0000","last_change":"2024-09-17T05:07:38.376306+0000","last_active":"2024-09-17T05:07:39.414423+0000","last_peered":"2024-09-17T05:07:39.414423+0000","last_clean":"2024-09-17T05:07:39.414423+0000","last_became_active":"2024-09-17T05:07:38.375958+0000","last_became_peered":"2024-09-17T05:07:38.375958+0000","last_unstale":"2024-09-17T05:07:39.414423+0000","last_undegraded":"2024-09-17T05:07:39.414423+0000","last_fullsized":"2024-09-17T05:07:39.414423+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:06.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362763+0000","last_change":"2024-09-17T05:07:37.367645+0000","last_active":"2024-09-17T05:07:39.362763+0000","last_peered":"2024-09-17T05:07:39.362763+0000","last_clean":"2024-09-17T05:07:39.362763+0000","last_became_active":"2024-09-17T05:07:37.367258+0000","last_became_peered":"2024-09-17T05:07:37.367258+0000","last_unstale":"2024-09-17T05:07:39.362763+0000","last_undegraded":"2024-09-17T05:07:39.362763+0000","last_fullsized":"2024-09-17T05:07:39.362763+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:50.405118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364416+0000","last_change":"2024-09-17T05:07:38.362637+0000","last_active":"2024-09-17T05:07:39.364416+0000","last_peered":"2024-09-17T05:07:39.364416+0000","last_clean":"2024-09-17T05:07:39.364416+0000","last_became_active":"2024-09-17T05:07:38.361414+0000","last_became_peered":"2024-09-17T05:07:38.361414+0000","last_unstale":"2024-09-17T05:07:39.364416+0000","last_undegraded":"2024-09-17T05:07:39.364416+0000","last_fullsized":"2024-09-17T05:07:39.364416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:09.501404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364438+0000","last_change":"2024-09-17T05:07:37.364934+0000","last_active":"2024-09-17T05:07:39.364438+0000","last_peered":"2024-09-17T05:07:39.364438+0000","last_clean":"2024-09-17T05:07:39.364438+0000","last_became_active":"2024-09-17T05:07:37.364564+0000","last_became_peered":"2024-09-17T05:07:37.364564+0000","last_unstale":"2024-09-17T05:07:39.364438+0000","last_undegraded":"2024-09-17T05:07:39.364438+0000","last_fullsized":"2024-09-17T05:07:39.364438+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:25.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415120+0000","last_change":"2024-09-17T05:07:38.369141+0000","last_active":"2024-09-17T05:07:39.415120+0000","last_peered":"2024-09-17T05:07:39.415120+0000","last_clean":"2024-09-17T05:07:39.415120+0000","last_became_active":"2024-09-17T05:07:38.368328+0000","last_became_peered":"2024-09-17T05:07:38.368328+0000","last_unstale":"2024-09-17T05:07:39.415120+0000","last_undegraded":"2024-09-17T05:07:39.415120+0000","last_fullsized":"2024-09-17T05:07:39.415120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:26.778140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363798+0000","last_change":"2024-09-17T05:07:37.359369+0000","last_active":"2024-09-17T05:07:39.363798+0000","last_peered":"2024-09-17T05:07:39.363798+0000","last_clean":"2024-09-17T05:07:39.363798+0000","last_became_active":"2024-09-17T05:07:37.359073+0000","last_became_peered":"2024-09-17T05:07:37.359073+0000","last_unstale":"2024-09-17T05:07:39.363798+0000","last_undegraded":"2024-09-17T05:07:39.363798+0000","last_fullsized":"2024-09-17T05:07:39.363798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:59.202139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414926+0000","last_change":"2024-09-17T05:07:38.374442+0000","last_active":"2024-09-17T05:07:39.414926+0000","last_peered":"2024-09-17T05:07:39.414926+0000","last_clean":"2024-09-17T05:07:39.414926+0000","last_became_active":"2024-09-17T05:07:38.374007+0000","last_became_peered":"2024-09-17T05:07:38.374007+0000","last_unstale":"2024-09-17T05:07:39.414926+0000","last_undegraded":"2024-09-17T05:07:39.414926+0000","last_fullsized":"2024-09-17T05:07:39.414926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:27.973016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362369+0000","last_change":"2024-09-17T05:07:37.361606+0000","last_active":"2024-09-17T05:07:39.362369+0000","last_peered":"2024-09-17T05:07:39.362369+0000","last_clean":"2024-09-17T05:07:39.362369+0000","last_became_active":"2024-09-17T05:07:37.361002+0000","last_became_peered":"2024-09-17T05:07:37.361002+0000","last_unstale":"2024-09-17T05:07:39.362369+0000","last_undegraded":"2024-09-17T05:07:39.362369+0000","last_fullsized":"2024-09-17T05:07:39.362369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:30.250628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365723+0000","last_change":"2024-09-17T05:07:38.363075+0000","last_active":"2024-09-17T05:07:39.365723+0000","last_peered":"2024-09-17T05:07:39.365723+0000","last_clean":"2024-09-17T05:07:39.365723+0000","last_became_active":"2024-09-17T05:07:38.362485+0000","last_became_peered":"2024-09-17T05:07:38.362485+0000","last_unstale":"2024-09-17T05:07:39.365723+0000","last_undegraded":"2024-09-17T05:07:39.365723+0000","last_fullsized":"2024-09-17T05:07:39.365723+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:02.455532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414342+0000","last_change":"2024-09-17T05:07:37.362489+0000","last_active":"2024-09-17T05:07:39.414342+0000","last_peered":"2024-09-17T05:07:39.414342+0000","last_clean":"2024-09-17T05:07:39.414342+0000","last_became_active":"2024-09-17T05:07:37.361729+0000","last_became_peered":"2024-09-17T05:07:37.361729+0000","last_unstale":"2024-09-17T05:07:39.414342+0000","last_undegraded":"2024-09-17T05:07:39.414342+0000","last_fullsized":"2024-09-17T05:07:39.414342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:51.258601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415184+0000","last_change":"2024-09-17T05:07:38.365765+0000","last_active":"2024-09-17T05:07:39.415184+0000","last_peered":"2024-09-17T05:07:39.415184+0000","last_clean":"2024-09-17T05:07:39.415184+0000","last_became_active":"2024-09-17T05:07:38.365452+0000","last_became_peered":"2024-09-17T05:07:38.365452+0000","last_unstale":"2024-09-17T05:07:39.415184+0000","last_undegraded":"2024-09-17T05:07:39.415184+0000","last_fullsized":"2024-09-17T05:07:39.415184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:34.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365507+0000","last_change":"2024-09-17T05:07:37.360081+0000","last_active":"2024-09-17T05:07:39.365507+0000","last_peered":"2024-09-17T05:07:39.365507+0000","last_clean":"2024-09-17T05:07:39.365507+0000","last_became_active":"2024-09-17T05:07:37.359720+0000","last_became_peered":"2024-09-17T05:07:37.359720+0000","last_unstale":"2024-09-17T05:07:39.365507+0000","last_undegraded":"2024-09-17T05:07:39.365507+0000","last_fullsized":"2024-09-17T05:07:39.365507+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:16.580036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415910+0000","last_change":"2024-09-17T05:07:38.358181+0000","last_active":"2024-09-17T05:07:39.415910+0000","last_peered":"2024-09-17T05:07:39.415910+0000","last_clean":"2024-09-17T05:07:39.415910+0000","last_became_active":"2024-09-17T05:07:38.357847+0000","last_became_peered":"2024-09-17T05:07:38.357847+0000","last_unstale":"2024-09-17T05:07:39.415910+0000","last_undegraded":"2024-09-17T05:07:39.415910+0000","last_fullsized":"2024-09-17T05:07:39.415910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:14.083379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.370455+0000","last_change":"2024-09-17T05:07:37.358084+0000","last_active":"2024-09-17T05:07:39.370455+0000","last_peered":"2024-09-17T05:07:39.370455+0000","last_clean":"2024-09-17T05:07:39.370455+0000","last_became_active":"2024-09-17T05:07:37.357773+0000","last_became_peered":"2024-09-17T05:07:37.357773+0000","last_unstale":"2024-09-17T05:07:39.370455+0000","last_undegraded":"2024-09-17T05:07:39.370455+0000","last_fullsized":"2024-09-17T05:07:39.370455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:02.583665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414402+0000","last_change":"2024-09-17T05:07:38.375508+0000","last_active":"2024-09-17T05:07:39.414402+0000","last_peered":"2024-09-17T05:07:39.414402+0000","last_clean":"2024-09-17T05:07:39.414402+0000","last_became_active":"2024-09-17T05:07:38.375074+0000","last_became_peered":"2024-09-17T05:07:38.375074+0000","last_unstale":"2024-09-17T05:07:39.414402+0000","last_undegraded":"2024-09-17T05:07:39.414402+0000","last_fullsized":"2024-09-17T05:07:39.414402+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:40.719087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417035+0000","last_change":"2024-09-17T05:07:37.350958+0000","last_active":"2024-09-17T05:07:39.417035+0000","last_peered":"2024-09-17T05:07:39.417035+0000","last_clean":"2024-09-17T05:07:39.417035+0000","last_became_active":"2024-09-17T05:07:37.349820+0000","last_became_peered":"2024-09-17T05:07:37.349820+0000","last_unstale":"2024-09-17T05:07:39.417035+0000","last_undegraded":"2024-09-17T05:07:39.417035+0000","last_fullsized":"2024-09-17T05:07:39.417035+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:43.480852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416520+0000","last_change":"2024-09-17T05:07:38.358326+0000","last_active":"2024-09-17T05:07:39.416520+0000","last_peered":"2024-09-17T05:07:39.416520+0000","last_clean":"2024-09-17T05:07:39.416520+0000","last_became_active":"2024-09-17T05:07:38.358002+0000","last_became_peered":"2024-09-17T05:07:38.358002+0000","last_unstale":"2024-09-17T05:07:39.416520+0000","last_undegraded":"2024-09-17T05:07:39.416520+0000","last_fullsized":"2024-09-17T05:07:39.416520+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:19.804778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364819+0000","last_change":"2024-09-17T05:07:37.368314+0000","last_active":"2024-09-17T05:07:39.364819+0000","last_peered":"2024-09-17T05:07:39.364819+0000","last_clean":"2024-09-17T05:07:39.364819+0000","last_became_active":"2024-09-17T05:07:37.367953+0000","last_became_peered":"2024-09-17T05:07:37.367953+0000","last_unstale":"2024-09-17T05:07:39.364819+0000","last_undegraded":"2024-09-17T05:07:39.364819+0000","last_fullsized":"2024-09-17T05:07:39.364819+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:52.522812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364275+0000","last_change":"2024-09-17T05:07:38.359694+0000","last_active":"2024-09-17T05:07:39.364275+0000","last_peered":"2024-09-17T05:07:39.364275+0000","last_clean":"2024-09-17T05:07:39.364275+0000","last_became_active":"2024-09-17T05:07:38.359388+0000","last_became_peered":"2024-09-17T05:07:38.359388+0000","last_unstale":"2024-09-17T05:07:39.364275+0000","last_undegraded":"2024-09-17T05:07:39.364275+0000","last_fullsized":"2024-09-17T05:07:39.364275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:43.873942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416949+0000","last_change":"2024-09-17T05:07:37.364116+0000","last_active":"2024-09-17T05:07:39.416949+0000","last_peered":"2024-09-17T05:07:39.416949+0000","last_clean":"2024-09-17T05:07:39.416949+0000","last_became_active":"2024-09-17T05:07:37.363791+0000","last_became_peered":"2024-09-17T05:07:37.363791+0000","last_unstale":"2024-09-17T05:07:39.416949+0000","last_undegraded":"2024-09-17T05:07:39.416949+0000","last_fullsized":"2024-09-17T05:07:39.416949+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:51.314268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365704+0000","last_change":"2024-09-17T05:07:38.359216+0000","last_active":"2024-09-17T05:07:39.365704+0000","last_peered":"2024-09-17T05:07:39.365704+0000","last_clean":"2024-09-17T05:07:39.365704+0000","last_became_active":"2024-09-17T05:07:38.358495+0000","last_became_peered":"2024-09-17T05:07:38.358495+0000","last_unstale":"2024-09-17T05:07:39.365704+0000","last_undegraded":"2024-09-17T05:07:39.365704+0000","last_fullsized":"2024-09-17T05:07:39.365704+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:11.941996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364119+0000","last_change":"2024-09-17T05:07:37.357614+0000","last_active":"2024-09-17T05:07:39.364119+0000","last_peered":"2024-09-17T05:07:39.364119+0000","last_clean":"2024-09-17T05:07:39.364119+0000","last_became_active":"2024-09-17T05:07:37.357276+0000","last_became_peered":"2024-09-17T05:07:37.357276+0000","last_unstale":"2024-09-17T05:07:39.364119+0000","last_undegraded":"2024-09-17T05:07:39.364119+0000","last_fullsized":"2024-09-17T05:07:39.364119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:39.232366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363028+0000","last_change":"2024-09-17T05:07:38.359254+0000","last_active":"2024-09-17T05:07:39.363028+0000","last_peered":"2024-09-17T05:07:39.363028+0000","last_clean":"2024-09-17T05:07:39.363028+0000","last_became_active":"2024-09-17T05:07:38.358849+0000","last_became_peered":"2024-09-17T05:07:38.358849+0000","last_unstale":"2024-09-17T05:07:39.363028+0000","last_undegraded":"2024-09-17T05:07:39.363028+0000","last_fullsized":"2024-09-17T05:07:39.363028+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:00.537642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415799+0000","last_change":"2024-09-17T05:07:37.364772+0000","last_active":"2024-09-17T05:07:39.415799+0000","last_peered":"2024-09-17T05:07:39.415799+0000","last_clean":"2024-09-17T05:07:39.415799+0000","last_became_active":"2024-09-17T05:07:37.364468+0000","last_became_peered":"2024-09-17T05:07:37.364468+0000","last_unstale":"2024-09-17T05:07:39.415799+0000","last_undegraded":"2024-09-17T05:07:39.415799+0000","last_fullsized":"2024-09-17T05:07:39.415799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:09.224428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416666+0000","last_change":"2024-09-17T05:07:38.378238+0000","last_active":"2024-09-17T05:07:39.416666+0000","last_peered":"2024-09-17T05:07:39.416666+0000","last_clean":"2024-09-17T05:07:39.416666+0000","last_became_active":"2024-09-17T05:07:38.377899+0000","last_became_peered":"2024-09-17T05:07:38.377899+0000","last_unstale":"2024-09-17T05:07:39.416666+0000","last_undegraded":"2024-09-17T05:07:39.416666+0000","last_fullsized":"2024-09-17T05:07:39.416666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:33.791006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363935+0000","last_change":"2024-09-17T05:07:37.355234+0000","last_active":"2024-09-17T05:07:39.363935+0000","last_peered":"2024-09-17T05:07:39.363935+0000","last_clean":"2024-09-17T05:07:39.363935+0000","last_became_active":"2024-09-17T05:07:37.354718+0000","last_became_peered":"2024-09-17T05:07:37.354718+0000","last_unstale":"2024-09-17T05:07:39.363935+0000","last_undegraded":"2024-09-17T05:07:39.363935+0000","last_fullsized":"2024-09-17T05:07:39.363935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.673587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362247+0000","last_change":"2024-09-17T05:07:38.368854+0000","last_active":"2024-09-17T05:07:39.362247+0000","last_peered":"2024-09-17T05:07:39.362247+0000","last_clean":"2024-09-17T05:07:39.362247+0000","last_became_active":"2024-09-17T05:07:38.368514+0000","last_became_peered":"2024-09-17T05:07:38.368514+0000","last_unstale":"2024-09-17T05:07:39.362247+0000","last_undegraded":"2024-09-17T05:07:39.362247+0000","last_fullsized":"2024-09-17T05:07:39.362247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:26.054302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364808+0000","last_change":"2024-09-17T05:07:37.363388+0000","last_active":"2024-09-17T05:07:39.364808+0000","last_peered":"2024-09-17T05:07:39.364808+0000","last_clean":"2024-09-17T05:07:39.364808+0000","last_became_active":"2024-09-17T05:07:37.362169+0000","last_became_peered":"2024-09-17T05:07:37.362169+0000","last_unstale":"2024-09-17T05:07:39.364808+0000","last_undegraded":"2024-09-17T05:07:39.364808+0000","last_fullsized":"2024-09-17T05:07:39.364808+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:08.947221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":264,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415767+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:07:39.415767+0000","last_peered":"2024-09-17T05:07:39.415767+0000","last_clean":"2024-09-17T05:07:39.415767+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:07:39.415767+0000","last_undegraded":"2024-09-17T05:07:39.415767+0000","last_fullsized":"2024-09-17T05:07:39.415767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":565809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":566769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509524,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1236992,"data_stored":565191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1245184,"data_stored":566769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509523,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1241088,"data_stored":565269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1835008,"data_stored":1155617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1236992,"data_stored":565739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1835008,"data_stored":1155649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:07:47.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:07:47.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 -- 172.21.15.16:0/2064873600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420073240 msgr2=0x7fe420073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 --2- 172.21.15.16:0/2064873600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420073240 0x7fe420073650 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe42006e960 tx=0x7fe41801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 -- 172.21.15.16:0/2064873600 shutdown_connections 2024-09-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 --2- 172.21.15.16:0/2064873600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe420073240 0x7fe420073650 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 --2- 172.21.15.16:0/2064873600 >> v2:172.21.15.104:3300/0 conn(0x7fe42015c270 0x7fe42015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 -- 172.21.15.16:0/2064873600 >> 172.21.15.16:0/2064873600 conn(0x7fe42006f900 msgr2=0x7fe4200b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 -- 172.21.15.16:0/2064873600 shutdown_connections 2024-09-17T05:07:47.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.467+0000 7fe425c00640 1 -- 172.21.15.16:0/2064873600 wait complete. 2024-09-17T05:07:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 Processor -- start 2024-09-17T05:07:47.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 -- start start 2024-09-17T05:07:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420073240 0x7fe4201448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4200b2f50 0x7fe42015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 0x7fe42015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe420168d10 con 0x7fe42015c270 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe420168b90 con 0x7fe420073240 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe425c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe420168e90 con 0x7fe4200b2f50 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 0x7fe42015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 0x7fe42015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60492/0 (socket says 172.21.15.16:60492) 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 -- 172.21.15.16:0/1091693976 learned_addr learned my addr 172.21.15.16:0/1091693976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41ea00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420073240 0x7fe4201448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe417e00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4200b2f50 0x7fe42015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 -- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4200b2f50 msgr2=0x7fe42015ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4200b2f50 0x7fe42015ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 -- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420073240 msgr2=0x7fe4201448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420073240 0x7fe4201448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 -- 172.21.15.16:0/1091693976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4201fc090 con 0x7fe42015c270 2024-09-17T05:07:47.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41ea00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420073240 0x7fe4201448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.471+0000 7fe41f400640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 0x7fe42015d200 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe418019900 tx=0x7fe4180092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180bb070 con 0x7fe42015c270 2024-09-17T05:07:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe418004d40 con 0x7fe42015c270 2024-09-17T05:07:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4180b2040 con 0x7fe42015c270 2024-09-17T05:07:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4201fc370 con 0x7fe42015c270 2024-09-17T05:07:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4201fc930 con 0x7fe42015c270 2024-09-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe420073650 con 0x7fe42015c270 2024-09-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4180ad850 con 0x7fe42015c270 2024-09-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe416a00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3fc073200 0x7fe3fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.475+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fe41812e140 con 0x7fe42015c270 2024-09-17T05:07:47.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.479+0000 7fe41ea00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3fc073200 0x7fe3fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.479+0000 7fe41ea00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3fc073200 0x7fe3fc075710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe4100016b0 tx=0x7fe410009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:47.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.479+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe4180f1db0 con 0x7fe42015c270 2024-09-17T05:07:47.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.655+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4200ad860 con 0x7fe3fc073200 2024-09-17T05:07:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.659+0000 7fe416a00640 1 -- 172.21.15.16:0/1091693976 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252166 (secure 0 0 0) 0x7fe4200ad860 con 0x7fe3fc073200 2024-09-17T05:07:47.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:47.662 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:07:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3fc073200 msgr2=0x7fe3fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3fc073200 0x7fe3fc075710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe4100016b0 tx=0x7fe410009040 comp rx=0 tx=0).stop 2024-09-17T05:07:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 msgr2=0x7fe42015d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 0x7fe42015d200 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe418019900 tx=0x7fe4180092c0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 shutdown_connections 2024-09-17T05:07:47.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe3fc073200 0x7fe3fc075710 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42015c270 0x7fe42015d200 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe4200b2f50 0x7fe42015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 --2- 172.21.15.16:0/1091693976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420073240 0x7fe4201448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 >> 172.21.15.16:0/1091693976 conn(0x7fe42006f900 msgr2=0x7fe4200d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 shutdown_connections 2024-09-17T05:07:47.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.663+0000 7fe425c00640 1 -- 172.21.15.16:0/1091693976 wait complete. 2024-09-17T05:07:47.682 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-17T05:07:47.572227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268712,"kb_used_data":10856,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706008,"statfs":{"total":773094113280,"available":772818952192,"internally_reserved":0,"allocated":11116544,"data_stored":5706812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.233801"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365181+0000","last_change":"2024-09-17T05:07:38.349474+0000","last_active":"2024-09-17T05:07:39.365181+0000","last_peered":"2024-09-17T05:07:39.365181+0000","last_clean":"2024-09-17T05:07:39.365181+0000","last_became_active":"2024-09-17T05:07:38.349166+0000","last_became_peered":"2024-09-17T05:07:38.349166+0000","last_unstale":"2024-09-17T05:07:39.365181+0000","last_undegraded":"2024-09-17T05:07:39.365181+0000","last_fullsized":"2024-09-17T05:07:39.365181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:03.183330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415351+0000","last_change":"2024-09-17T05:07:38.378880+0000","last_active":"2024-09-17T05:07:39.415351+0000","last_peered":"2024-09-17T05:07:39.415351+0000","last_clean":"2024-09-17T05:07:39.415351+0000","last_became_active":"2024-09-17T05:07:38.378565+0000","last_became_peered":"2024-09-17T05:07:38.378565+0000","last_unstale":"2024-09-17T05:07:39.415351+0000","last_undegraded":"2024-09-17T05:07:39.415351+0000","last_fullsized":"2024-09-17T05:07:39.415351+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:45.576630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365386+0000","last_change":"2024-09-17T05:07:38.373066+0000","last_active":"2024-09-17T05:07:39.365386+0000","last_peered":"2024-09-17T05:07:39.365386+0000","last_clean":"2024-09-17T05:07:39.365386+0000","last_became_active":"2024-09-17T05:07:38.372726+0000","last_became_peered":"2024-09-17T05:07:38.372726+0000","last_unstale":"2024-09-17T05:07:39.365386+0000","last_undegraded":"2024-09-17T05:07:39.365386+0000","last_fullsized":"2024-09-17T05:07:39.365386+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:37:40.211315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363993+0000","last_change":"2024-09-17T05:07:38.371262+0000","last_active":"2024-09-17T05:07:39.363993+0000","last_peered":"2024-09-17T05:07:39.363993+0000","last_clean":"2024-09-17T05:07:39.363993+0000","last_became_active":"2024-09-17T05:07:38.370812+0000","last_became_peered":"2024-09-17T05:07:38.370812+0000","last_unstale":"2024-09-17T05:07:39.363993+0000","last_undegraded":"2024-09-17T05:07:39.363993+0000","last_fullsized":"2024-09-17T05:07:39.363993+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:40.753264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364724+0000","last_change":"2024-09-17T05:07:38.364588+0000","last_active":"2024-09-17T05:07:39.364724+0000","last_peered":"2024-09-17T05:07:39.364724+0000","last_clean":"2024-09-17T05:07:39.364724+0000","last_became_active":"2024-09-17T05:07:38.364271+0000","last_became_peered":"2024-09-17T05:07:38.364271+0000","last_unstale":"2024-09-17T05:07:39.364724+0000","last_undegraded":"2024-09-17T05:07:39.364724+0000","last_fullsized":"2024-09-17T05:07:39.364724+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:36.712050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362349+0000","last_change":"2024-09-17T05:07:38.354213+0000","last_active":"2024-09-17T05:07:39.362349+0000","last_peered":"2024-09-17T05:07:39.362349+0000","last_clean":"2024-09-17T05:07:39.362349+0000","last_became_active":"2024-09-17T05:07:38.353898+0000","last_became_peered":"2024-09-17T05:07:38.353898+0000","last_unstale":"2024-09-17T05:07:39.362349+0000","last_undegraded":"2024-09-17T05:07:39.362349+0000","last_fullsized":"2024-09-17T05:07:39.362349+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:15.826789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362073+0000","last_change":"2024-09-17T05:07:38.352925+0000","last_active":"2024-09-17T05:07:39.362073+0000","last_peered":"2024-09-17T05:07:39.362073+0000","last_clean":"2024-09-17T05:07:39.362073+0000","last_became_active":"2024-09-17T05:07:38.352587+0000","last_became_peered":"2024-09-17T05:07:38.352587+0000","last_unstale":"2024-09-17T05:07:39.362073+0000","last_undegraded":"2024-09-17T05:07:39.362073+0000","last_fullsized":"2024-09-17T05:07:39.362073+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:05.811532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414053+0000","last_change":"2024-09-17T05:07:38.374955+0000","last_active":"2024-09-17T05:07:39.414053+0000","last_peered":"2024-09-17T05:07:39.414053+0000","last_clean":"2024-09-17T05:07:39.414053+0000","last_became_active":"2024-09-17T05:07:38.374642+0000","last_became_peered":"2024-09-17T05:07:38.374642+0000","last_unstale":"2024-09-17T05:07:39.414053+0000","last_undegraded":"2024-09-17T05:07:39.414053+0000","last_fullsized":"2024-09-17T05:07:39.414053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:03.975633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416424+0000","last_change":"2024-09-17T05:07:38.371399+0000","last_active":"2024-09-17T05:07:39.416424+0000","last_peered":"2024-09-17T05:07:39.416424+0000","last_clean":"2024-09-17T05:07:39.416424+0000","last_became_active":"2024-09-17T05:07:38.371082+0000","last_became_peered":"2024-09-17T05:07:38.371082+0000","last_unstale":"2024-09-17T05:07:39.416424+0000","last_undegraded":"2024-09-17T05:07:39.416424+0000","last_fullsized":"2024-09-17T05:07:39.416424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:10.098240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414665+0000","last_change":"2024-09-17T05:07:38.377561+0000","last_active":"2024-09-17T05:07:39.414665+0000","last_peered":"2024-09-17T05:07:39.414665+0000","last_clean":"2024-09-17T05:07:39.414665+0000","last_became_active":"2024-09-17T05:07:38.377253+0000","last_became_peered":"2024-09-17T05:07:38.377253+0000","last_unstale":"2024-09-17T05:07:39.414665+0000","last_undegraded":"2024-09-17T05:07:39.414665+0000","last_fullsized":"2024-09-17T05:07:39.414665+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:52.748201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416846+0000","last_change":"2024-09-17T05:07:38.367526+0000","last_active":"2024-09-17T05:07:39.416846+0000","last_peered":"2024-09-17T05:07:39.416846+0000","last_clean":"2024-09-17T05:07:39.416846+0000","last_became_active":"2024-09-17T05:07:38.367209+0000","last_became_peered":"2024-09-17T05:07:38.367209+0000","last_unstale":"2024-09-17T05:07:39.416846+0000","last_undegraded":"2024-09-17T05:07:39.416846+0000","last_fullsized":"2024-09-17T05:07:39.416846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:26.936939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362606+0000","last_change":"2024-09-17T05:07:38.375041+0000","last_active":"2024-09-17T05:07:39.362606+0000","last_peered":"2024-09-17T05:07:39.362606+0000","last_clean":"2024-09-17T05:07:39.362606+0000","last_became_active":"2024-09-17T05:07:38.372665+0000","last_became_peered":"2024-09-17T05:07:38.372665+0000","last_unstale":"2024-09-17T05:07:39.362606+0000","last_undegraded":"2024-09-17T05:07:39.362606+0000","last_fullsized":"2024-09-17T05:07:39.362606+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:39.425452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364112+0000","last_change":"2024-09-17T05:07:38.351889+0000","last_active":"2024-09-17T05:07:39.364112+0000","last_peered":"2024-09-17T05:07:39.364112+0000","last_clean":"2024-09-17T05:07:39.364112+0000","last_became_active":"2024-09-17T05:07:38.351576+0000","last_became_peered":"2024-09-17T05:07:38.351576+0000","last_unstale":"2024-09-17T05:07:39.364112+0000","last_undegraded":"2024-09-17T05:07:39.364112+0000","last_fullsized":"2024-09-17T05:07:39.364112+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:07.224493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362408+0000","last_change":"2024-09-17T05:07:38.371821+0000","last_active":"2024-09-17T05:07:39.362408+0000","last_peered":"2024-09-17T05:07:39.362408+0000","last_clean":"2024-09-17T05:07:39.362408+0000","last_became_active":"2024-09-17T05:07:38.371476+0000","last_became_peered":"2024-09-17T05:07:38.371476+0000","last_unstale":"2024-09-17T05:07:39.362408+0000","last_undegraded":"2024-09-17T05:07:39.362408+0000","last_fullsized":"2024-09-17T05:07:39.362408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:46.787438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415065+0000","last_change":"2024-09-17T05:07:38.368316+0000","last_active":"2024-09-17T05:07:39.415065+0000","last_peered":"2024-09-17T05:07:39.415065+0000","last_clean":"2024-09-17T05:07:39.415065+0000","last_became_active":"2024-09-17T05:07:38.367970+0000","last_became_peered":"2024-09-17T05:07:38.367970+0000","last_unstale":"2024-09-17T05:07:39.415065+0000","last_undegraded":"2024-09-17T05:07:39.415065+0000","last_fullsized":"2024-09-17T05:07:39.415065+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:13.001958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415658+0000","last_change":"2024-09-17T05:07:38.353082+0000","last_active":"2024-09-17T05:07:39.415658+0000","last_peered":"2024-09-17T05:07:39.415658+0000","last_clean":"2024-09-17T05:07:39.415658+0000","last_became_active":"2024-09-17T05:07:38.352602+0000","last_became_peered":"2024-09-17T05:07:38.352602+0000","last_unstale":"2024-09-17T05:07:39.415658+0000","last_undegraded":"2024-09-17T05:07:39.415658+0000","last_fullsized":"2024-09-17T05:07:39.415658+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:22.101853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365037+0000","last_change":"2024-09-17T05:07:38.369916+0000","last_active":"2024-09-17T05:07:39.365037+0000","last_peered":"2024-09-17T05:07:39.365037+0000","last_clean":"2024-09-17T05:07:39.365037+0000","last_became_active":"2024-09-17T05:07:38.369536+0000","last_became_peered":"2024-09-17T05:07:38.369536+0000","last_unstale":"2024-09-17T05:07:39.365037+0000","last_undegraded":"2024-09-17T05:07:39.365037+0000","last_fullsized":"2024-09-17T05:07:39.365037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:24:07.275670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416012+0000","last_change":"2024-09-17T05:07:38.365371+0000","last_active":"2024-09-17T05:07:39.416012+0000","last_peered":"2024-09-17T05:07:39.416012+0000","last_clean":"2024-09-17T05:07:39.416012+0000","last_became_active":"2024-09-17T05:07:38.365027+0000","last_became_peered":"2024-09-17T05:07:38.365027+0000","last_unstale":"2024-09-17T05:07:39.416012+0000","last_undegraded":"2024-09-17T05:07:39.416012+0000","last_fullsized":"2024-09-17T05:07:39.416012+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:13.836046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364710+0000","last_change":"2024-09-17T05:07:38.372948+0000","last_active":"2024-09-17T05:07:39.364710+0000","last_peered":"2024-09-17T05:07:39.364710+0000","last_clean":"2024-09-17T05:07:39.364710+0000","last_became_active":"2024-09-17T05:07:38.372526+0000","last_became_peered":"2024-09-17T05:07:38.372526+0000","last_unstale":"2024-09-17T05:07:39.364710+0000","last_undegraded":"2024-09-17T05:07:39.364710+0000","last_fullsized":"2024-09-17T05:07:39.364710+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:50.148108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417054+0000","last_change":"2024-09-17T05:07:38.366601+0000","last_active":"2024-09-17T05:07:39.417054+0000","last_peered":"2024-09-17T05:07:39.417054+0000","last_clean":"2024-09-17T05:07:39.417054+0000","last_became_active":"2024-09-17T05:07:38.366223+0000","last_became_peered":"2024-09-17T05:07:38.366223+0000","last_unstale":"2024-09-17T05:07:39.417054+0000","last_undegraded":"2024-09-17T05:07:39.417054+0000","last_fullsized":"2024-09-17T05:07:39.417054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:05.811532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364572+0000","last_change":"2024-09-17T05:07:38.365788+0000","last_active":"2024-09-17T05:07:39.364572+0000","last_peered":"2024-09-17T05:07:39.364572+0000","last_clean":"2024-09-17T05:07:39.364572+0000","last_became_active":"2024-09-17T05:07:38.364783+0000","last_became_peered":"2024-09-17T05:07:38.364783+0000","last_unstale":"2024-09-17T05:07:39.364572+0000","last_undegraded":"2024-09-17T05:07:39.364572+0000","last_fullsized":"2024-09-17T05:07:39.364572+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:32.081371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414574+0000","last_change":"2024-09-17T05:07:38.376620+0000","last_active":"2024-09-17T05:07:39.414574+0000","last_peered":"2024-09-17T05:07:39.414574+0000","last_clean":"2024-09-17T05:07:39.414574+0000","last_became_active":"2024-09-17T05:07:38.376316+0000","last_became_peered":"2024-09-17T05:07:38.376316+0000","last_unstale":"2024-09-17T05:07:39.414574+0000","last_undegraded":"2024-09-17T05:07:39.414574+0000","last_fullsized":"2024-09-17T05:07:39.414574+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:15.492636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415287+0000","last_change":"2024-09-17T05:07:38.384020+0000","last_active":"2024-09-17T05:07:39.415287+0000","last_peered":"2024-09-17T05:07:39.415287+0000","last_clean":"2024-09-17T05:07:39.415287+0000","last_became_active":"2024-09-17T05:07:38.383707+0000","last_became_peered":"2024-09-17T05:07:38.383707+0000","last_unstale":"2024-09-17T05:07:39.415287+0000","last_undegraded":"2024-09-17T05:07:39.415287+0000","last_fullsized":"2024-09-17T05:07:39.415287+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:25.182650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414053+0000","last_change":"2024-09-17T05:07:38.361287+0000","last_active":"2024-09-17T05:07:39.414053+0000","last_peered":"2024-09-17T05:07:39.414053+0000","last_clean":"2024-09-17T05:07:39.414053+0000","last_became_active":"2024-09-17T05:07:38.360906+0000","last_became_peered":"2024-09-17T05:07:38.360906+0000","last_unstale":"2024-09-17T05:07:39.414053+0000","last_undegraded":"2024-09-17T05:07:39.414053+0000","last_fullsized":"2024-09-17T05:07:39.414053+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:25.433058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417259+0000","last_change":"2024-09-17T05:07:38.360975+0000","last_active":"2024-09-17T05:07:39.417259+0000","last_peered":"2024-09-17T05:07:39.417259+0000","last_clean":"2024-09-17T05:07:39.417259+0000","last_became_active":"2024-09-17T05:07:38.360604+0000","last_became_peered":"2024-09-17T05:07:38.360604+0000","last_unstale":"2024-09-17T05:07:39.417259+0000","last_undegraded":"2024-09-17T05:07:39.417259+0000","last_fullsized":"2024-09-17T05:07:39.417259+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:26.054302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417368+0000","last_change":"2024-09-17T05:07:38.366400+0000","last_active":"2024-09-17T05:07:39.417368+0000","last_peered":"2024-09-17T05:07:39.417368+0000","last_clean":"2024-09-17T05:07:39.417368+0000","last_became_active":"2024-09-17T05:07:38.360850+0000","last_became_peered":"2024-09-17T05:07:38.360850+0000","last_unstale":"2024-09-17T05:07:39.417368+0000","last_undegraded":"2024-09-17T05:07:39.417368+0000","last_fullsized":"2024-09-17T05:07:39.417368+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:15.826789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417201+0000","last_change":"2024-09-17T05:07:38.365699+0000","last_active":"2024-09-17T05:07:39.417201+0000","last_peered":"2024-09-17T05:07:39.417201+0000","last_clean":"2024-09-17T05:07:39.417201+0000","last_became_active":"2024-09-17T05:07:38.364854+0000","last_became_peered":"2024-09-17T05:07:38.364854+0000","last_unstale":"2024-09-17T05:07:39.417201+0000","last_undegraded":"2024-09-17T05:07:39.417201+0000","last_fullsized":"2024-09-17T05:07:39.417201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:53.530325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362699+0000","last_change":"2024-09-17T05:07:38.358348+0000","last_active":"2024-09-17T05:07:39.362699+0000","last_peered":"2024-09-17T05:07:39.362699+0000","last_clean":"2024-09-17T05:07:39.362699+0000","last_became_active":"2024-09-17T05:07:38.358031+0000","last_became_peered":"2024-09-17T05:07:38.358031+0000","last_unstale":"2024-09-17T05:07:39.362699+0000","last_undegraded":"2024-09-17T05:07:39.362699+0000","last_fullsized":"2024-09-17T05:07:39.362699+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:57.952363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414835+0000","last_change":"2024-09-17T05:07:38.375974+0000","last_active":"2024-09-17T05:07:39.414835+0000","last_peered":"2024-09-17T05:07:39.414835+0000","last_clean":"2024-09-17T05:07:39.414835+0000","last_became_active":"2024-09-17T05:07:38.375278+0000","last_became_peered":"2024-09-17T05:07:38.375278+0000","last_unstale":"2024-09-17T05:07:39.414835+0000","last_undegraded":"2024-09-17T05:07:39.414835+0000","last_fullsized":"2024-09-17T05:07:39.414835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:44.488365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416613+0000","last_change":"2024-09-17T05:07:38.366829+0000","last_active":"2024-09-17T05:07:39.416613+0000","last_peered":"2024-09-17T05:07:39.416613+0000","last_clean":"2024-09-17T05:07:39.416613+0000","last_became_active":"2024-09-17T05:07:38.366501+0000","last_became_peered":"2024-09-17T05:07:38.366501+0000","last_unstale":"2024-09-17T05:07:39.416613+0000","last_undegraded":"2024-09-17T05:07:39.416613+0000","last_fullsized":"2024-09-17T05:07:39.416613+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:07.402424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362182+0000","last_change":"2024-09-17T05:07:38.364091+0000","last_active":"2024-09-17T05:07:39.362182+0000","last_peered":"2024-09-17T05:07:39.362182+0000","last_clean":"2024-09-17T05:07:39.362182+0000","last_became_active":"2024-09-17T05:07:38.362464+0000","last_became_peered":"2024-09-17T05:07:38.362464+0000","last_unstale":"2024-09-17T05:07:39.362182+0000","last_undegraded":"2024-09-17T05:07:39.362182+0000","last_fullsized":"2024-09-17T05:07:39.362182+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:53.530325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415217+0000","last_change":"2024-09-17T05:07:38.370784+0000","last_active":"2024-09-17T05:07:39.415217+0000","last_peered":"2024-09-17T05:07:39.415217+0000","last_clean":"2024-09-17T05:07:39.415217+0000","last_became_active":"2024-09-17T05:07:38.370468+0000","last_became_peered":"2024-09-17T05:07:38.370468+0000","last_unstale":"2024-09-17T05:07:39.415217+0000","last_undegraded":"2024-09-17T05:07:39.415217+0000","last_fullsized":"2024-09-17T05:07:39.415217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:49.572784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363817+0000","last_change":"2024-09-17T05:07:38.351295+0000","last_active":"2024-09-17T05:07:39.363817+0000","last_peered":"2024-09-17T05:07:39.363817+0000","last_clean":"2024-09-17T05:07:39.363817+0000","last_became_active":"2024-09-17T05:07:38.348971+0000","last_became_peered":"2024-09-17T05:07:38.348971+0000","last_unstale":"2024-09-17T05:07:39.363817+0000","last_undegraded":"2024-09-17T05:07:39.363817+0000","last_fullsized":"2024-09-17T05:07:39.363817+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:08:12.675618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"99'4","reported_seq":19,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414897+0000","last_change":"2024-09-17T05:07:37.361640+0000","last_active":"2024-09-17T05:07:39.414897+0000","last_peered":"2024-09-17T05:07:39.414897+0000","last_clean":"2024-09-17T05:07:39.414897+0000","last_became_active":"2024-09-17T05:07:37.360159+0000","last_became_peered":"2024-09-17T05:07:37.360159+0000","last_unstale":"2024-09-17T05:07:39.414897+0000","last_undegraded":"2024-09-17T05:07:39.414897+0000","last_fullsized":"2024-09-17T05:07:39.414897+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:53.762127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414554+0000","last_change":"2024-09-17T05:07:38.382830+0000","last_active":"2024-09-17T05:07:39.414554+0000","last_peered":"2024-09-17T05:07:39.414554+0000","last_clean":"2024-09-17T05:07:39.414554+0000","last_became_active":"2024-09-17T05:07:38.382534+0000","last_became_peered":"2024-09-17T05:07:38.382534+0000","last_unstale":"2024-09-17T05:07:39.414554+0000","last_undegraded":"2024-09-17T05:07:39.414554+0000","last_fullsized":"2024-09-17T05:07:39.414554+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:08:53.511630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364559+0000","last_change":"2024-09-17T05:07:37.367481+0000","last_active":"2024-09-17T05:07:39.364559+0000","last_peered":"2024-09-17T05:07:39.364559+0000","last_clean":"2024-09-17T05:07:39.364559+0000","last_became_active":"2024-09-17T05:07:37.367109+0000","last_became_peered":"2024-09-17T05:07:37.367109+0000","last_unstale":"2024-09-17T05:07:39.364559+0000","last_undegraded":"2024-09-17T05:07:39.364559+0000","last_fullsized":"2024-09-17T05:07:39.364559+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:25.251480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415152+0000","last_change":"2024-09-17T05:07:38.370197+0000","last_active":"2024-09-17T05:07:39.415152+0000","last_peered":"2024-09-17T05:07:39.415152+0000","last_clean":"2024-09-17T05:07:39.415152+0000","last_became_active":"2024-09-17T05:07:38.369400+0000","last_became_peered":"2024-09-17T05:07:38.369400+0000","last_unstale":"2024-09-17T05:07:39.415152+0000","last_undegraded":"2024-09-17T05:07:39.415152+0000","last_fullsized":"2024-09-17T05:07:39.415152+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:34.270797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415529+0000","last_change":"2024-09-17T05:07:37.360648+0000","last_active":"2024-09-17T05:07:39.415529+0000","last_peered":"2024-09-17T05:07:39.415529+0000","last_clean":"2024-09-17T05:07:39.415529+0000","last_became_active":"2024-09-17T05:07:37.356573+0000","last_became_peered":"2024-09-17T05:07:37.356573+0000","last_unstale":"2024-09-17T05:07:39.415529+0000","last_undegraded":"2024-09-17T05:07:39.415529+0000","last_fullsized":"2024-09-17T05:07:39.415529+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:05.220061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416821+0000","last_change":"2024-09-17T05:07:38.355606+0000","last_active":"2024-09-17T05:07:39.416821+0000","last_peered":"2024-09-17T05:07:39.416821+0000","last_clean":"2024-09-17T05:07:39.416821+0000","last_became_active":"2024-09-17T05:07:38.355214+0000","last_became_peered":"2024-09-17T05:07:38.355214+0000","last_unstale":"2024-09-17T05:07:39.416821+0000","last_undegraded":"2024-09-17T05:07:39.416821+0000","last_fullsized":"2024-09-17T05:07:39.416821+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:00.664431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414238+0000","last_change":"2024-09-17T05:07:37.364681+0000","last_active":"2024-09-17T05:07:39.414238+0000","last_peered":"2024-09-17T05:07:39.414238+0000","last_clean":"2024-09-17T05:07:39.414238+0000","last_became_active":"2024-09-17T05:07:37.364348+0000","last_became_peered":"2024-09-17T05:07:37.364348+0000","last_unstale":"2024-09-17T05:07:39.414238+0000","last_undegraded":"2024-09-17T05:07:39.414238+0000","last_fullsized":"2024-09-17T05:07:39.414238+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:30.250628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364659+0000","last_change":"2024-09-17T05:07:38.349727+0000","last_active":"2024-09-17T05:07:39.364659+0000","last_peered":"2024-09-17T05:07:39.364659+0000","last_clean":"2024-09-17T05:07:39.364659+0000","last_became_active":"2024-09-17T05:07:38.349358+0000","last_became_peered":"2024-09-17T05:07:38.349358+0000","last_unstale":"2024-09-17T05:07:39.364659+0000","last_undegraded":"2024-09-17T05:07:39.364659+0000","last_fullsized":"2024-09-17T05:07:39.364659+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:21.290301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416825+0000","last_change":"2024-09-17T05:07:37.368233+0000","last_active":"2024-09-17T05:07:39.416825+0000","last_peered":"2024-09-17T05:07:39.416825+0000","last_clean":"2024-09-17T05:07:39.416825+0000","last_became_active":"2024-09-17T05:07:37.367904+0000","last_became_peered":"2024-09-17T05:07:37.367904+0000","last_unstale":"2024-09-17T05:07:39.416825+0000","last_undegraded":"2024-09-17T05:07:39.416825+0000","last_fullsized":"2024-09-17T05:07:39.416825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:40.022110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365963+0000","last_change":"2024-09-17T05:07:38.359562+0000","last_active":"2024-09-17T05:07:39.365963+0000","last_peered":"2024-09-17T05:07:39.365963+0000","last_clean":"2024-09-17T05:07:39.365963+0000","last_became_active":"2024-09-17T05:07:38.358891+0000","last_became_peered":"2024-09-17T05:07:38.358891+0000","last_unstale":"2024-09-17T05:07:39.365963+0000","last_undegraded":"2024-09-17T05:07:39.365963+0000","last_fullsized":"2024-09-17T05:07:39.365963+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:29.030523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416164+0000","last_change":"2024-09-17T05:07:37.368607+0000","last_active":"2024-09-17T05:07:39.416164+0000","last_peered":"2024-09-17T05:07:39.416164+0000","last_clean":"2024-09-17T05:07:39.416164+0000","last_became_active":"2024-09-17T05:07:37.368307+0000","last_became_peered":"2024-09-17T05:07:37.368307+0000","last_unstale":"2024-09-17T05:07:39.416164+0000","last_undegraded":"2024-09-17T05:07:39.416164+0000","last_fullsized":"2024-09-17T05:07:39.416164+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:23.430017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417454+0000","last_change":"2024-09-17T05:07:38.355984+0000","last_active":"2024-09-17T05:07:39.417454+0000","last_peered":"2024-09-17T05:07:39.417454+0000","last_clean":"2024-09-17T05:07:39.417454+0000","last_became_active":"2024-09-17T05:07:38.355541+0000","last_became_peered":"2024-09-17T05:07:38.355541+0000","last_unstale":"2024-09-17T05:07:39.417454+0000","last_undegraded":"2024-09-17T05:07:39.417454+0000","last_fullsized":"2024-09-17T05:07:39.417454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:25.686562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415772+0000","last_change":"2024-09-17T05:07:37.367847+0000","last_active":"2024-09-17T05:07:39.415772+0000","last_peered":"2024-09-17T05:07:39.415772+0000","last_clean":"2024-09-17T05:07:39.415772+0000","last_became_active":"2024-09-17T05:07:37.367465+0000","last_became_peered":"2024-09-17T05:07:37.367465+0000","last_unstale":"2024-09-17T05:07:39.415772+0000","last_undegraded":"2024-09-17T05:07:39.415772+0000","last_fullsized":"2024-09-17T05:07:39.415772+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:13.428748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365837+0000","last_change":"2024-09-17T05:07:38.362869+0000","last_active":"2024-09-17T05:07:39.365837+0000","last_peered":"2024-09-17T05:07:39.365837+0000","last_clean":"2024-09-17T05:07:39.365837+0000","last_became_active":"2024-09-17T05:07:38.362419+0000","last_became_peered":"2024-09-17T05:07:38.362419+0000","last_unstale":"2024-09-17T05:07:39.365837+0000","last_undegraded":"2024-09-17T05:07:39.365837+0000","last_fullsized":"2024-09-17T05:07:39.365837+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:27:48.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415369+0000","last_change":"2024-09-17T05:07:37.364825+0000","last_active":"2024-09-17T05:07:39.415369+0000","last_peered":"2024-09-17T05:07:39.415369+0000","last_clean":"2024-09-17T05:07:39.415369+0000","last_became_active":"2024-09-17T05:07:37.364481+0000","last_became_peered":"2024-09-17T05:07:37.364481+0000","last_unstale":"2024-09-17T05:07:39.415369+0000","last_undegraded":"2024-09-17T05:07:39.415369+0000","last_fullsized":"2024-09-17T05:07:39.415369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:45:40.978992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363791+0000","last_change":"2024-09-17T05:07:38.350775+0000","last_active":"2024-09-17T05:07:39.363791+0000","last_peered":"2024-09-17T05:07:39.363791+0000","last_clean":"2024-09-17T05:07:39.363791+0000","last_became_active":"2024-09-17T05:07:38.350028+0000","last_became_peered":"2024-09-17T05:07:38.350028+0000","last_unstale":"2024-09-17T05:07:39.363791+0000","last_undegraded":"2024-09-17T05:07:39.363791+0000","last_fullsized":"2024-09-17T05:07:39.363791+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:25.976813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363083+0000","last_change":"2024-09-17T05:07:37.362446+0000","last_active":"2024-09-17T05:07:39.363083+0000","last_peered":"2024-09-17T05:07:39.363083+0000","last_clean":"2024-09-17T05:07:39.363083+0000","last_became_active":"2024-09-17T05:07:37.362104+0000","last_became_peered":"2024-09-17T05:07:37.362104+0000","last_unstale":"2024-09-17T05:07:39.363083+0000","last_undegraded":"2024-09-17T05:07:39.363083+0000","last_fullsized":"2024-09-17T05:07:39.363083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:08.221672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364275+0000","last_change":"2024-09-17T05:07:38.365090+0000","last_active":"2024-09-17T05:07:39.364275+0000","last_peered":"2024-09-17T05:07:39.364275+0000","last_clean":"2024-09-17T05:07:39.364275+0000","last_became_active":"2024-09-17T05:07:38.364675+0000","last_became_peered":"2024-09-17T05:07:38.364675+0000","last_unstale":"2024-09-17T05:07:39.364275+0000","last_undegraded":"2024-09-17T05:07:39.364275+0000","last_fullsized":"2024-09-17T05:07:39.364275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:23.544324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414879+0000","last_change":"2024-09-17T05:07:37.359338+0000","last_active":"2024-09-17T05:07:39.414879+0000","last_peered":"2024-09-17T05:07:39.414879+0000","last_clean":"2024-09-17T05:07:39.414879+0000","last_became_active":"2024-09-17T05:07:37.359008+0000","last_became_peered":"2024-09-17T05:07:37.359008+0000","last_unstale":"2024-09-17T05:07:39.414879+0000","last_undegraded":"2024-09-17T05:07:39.414879+0000","last_fullsized":"2024-09-17T05:07:39.414879+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:31.114818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414670+0000","last_change":"2024-09-17T05:07:38.362780+0000","last_active":"2024-09-17T05:07:39.414670+0000","last_peered":"2024-09-17T05:07:39.414670+0000","last_clean":"2024-09-17T05:07:39.414670+0000","last_became_active":"2024-09-17T05:07:38.362473+0000","last_became_peered":"2024-09-17T05:07:38.362473+0000","last_unstale":"2024-09-17T05:07:39.414670+0000","last_undegraded":"2024-09-17T05:07:39.414670+0000","last_fullsized":"2024-09-17T05:07:39.414670+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:46.658544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"99'3","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362815+0000","last_change":"2024-09-17T05:07:37.362270+0000","last_active":"2024-09-17T05:07:39.362815+0000","last_peered":"2024-09-17T05:07:39.362815+0000","last_clean":"2024-09-17T05:07:39.362815+0000","last_became_active":"2024-09-17T05:07:37.361924+0000","last_became_peered":"2024-09-17T05:07:37.361924+0000","last_unstale":"2024-09-17T05:07:39.362815+0000","last_undegraded":"2024-09-17T05:07:39.362815+0000","last_fullsized":"2024-09-17T05:07:39.362815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:22.439861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415030+0000","last_change":"2024-09-17T05:07:38.374469+0000","last_active":"2024-09-17T05:07:39.415030+0000","last_peered":"2024-09-17T05:07:39.415030+0000","last_clean":"2024-09-17T05:07:39.415030+0000","last_became_active":"2024-09-17T05:07:38.374098+0000","last_became_peered":"2024-09-17T05:07:38.374098+0000","last_unstale":"2024-09-17T05:07:39.415030+0000","last_undegraded":"2024-09-17T05:07:39.415030+0000","last_fullsized":"2024-09-17T05:07:39.415030+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:33.122152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"99'6","reported_seq":21,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416729+0000","last_change":"2024-09-17T05:07:37.365192+0000","last_active":"2024-09-17T05:07:39.416729+0000","last_peered":"2024-09-17T05:07:39.416729+0000","last_clean":"2024-09-17T05:07:39.416729+0000","last_became_active":"2024-09-17T05:07:37.364884+0000","last_became_peered":"2024-09-17T05:07:37.364884+0000","last_unstale":"2024-09-17T05:07:39.416729+0000","last_undegraded":"2024-09-17T05:07:39.416729+0000","last_fullsized":"2024-09-17T05:07:39.416729+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:42.265741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415661+0000","last_change":"2024-09-17T05:07:38.367518+0000","last_active":"2024-09-17T05:07:39.415661+0000","last_peered":"2024-09-17T05:07:39.415661+0000","last_clean":"2024-09-17T05:07:39.415661+0000","last_became_active":"2024-09-17T05:07:38.366758+0000","last_became_peered":"2024-09-17T05:07:38.366758+0000","last_unstale":"2024-09-17T05:07:39.415661+0000","last_undegraded":"2024-09-17T05:07:39.415661+0000","last_fullsized":"2024-09-17T05:07:39.415661+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:43:39.501480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365076+0000","last_change":"2024-09-17T05:07:37.358555+0000","last_active":"2024-09-17T05:07:39.365076+0000","last_peered":"2024-09-17T05:07:39.365076+0000","last_clean":"2024-09-17T05:07:39.365076+0000","last_became_active":"2024-09-17T05:07:37.358188+0000","last_became_peered":"2024-09-17T05:07:37.358188+0000","last_unstale":"2024-09-17T05:07:39.365076+0000","last_undegraded":"2024-09-17T05:07:39.365076+0000","last_fullsized":"2024-09-17T05:07:39.365076+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:04.804019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415466+0000","last_change":"2024-09-17T05:07:38.363561+0000","last_active":"2024-09-17T05:07:39.415466+0000","last_peered":"2024-09-17T05:07:39.415466+0000","last_clean":"2024-09-17T05:07:39.415466+0000","last_became_active":"2024-09-17T05:07:38.363042+0000","last_became_peered":"2024-09-17T05:07:38.363042+0000","last_unstale":"2024-09-17T05:07:39.415466+0000","last_undegraded":"2024-09-17T05:07:39.415466+0000","last_fullsized":"2024-09-17T05:07:39.415466+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:07.641855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414733+0000","last_change":"2024-09-17T05:07:37.363006+0000","last_active":"2024-09-17T05:07:39.414733+0000","last_peered":"2024-09-17T05:07:39.414733+0000","last_clean":"2024-09-17T05:07:39.414733+0000","last_became_active":"2024-09-17T05:07:37.362664+0000","last_became_peered":"2024-09-17T05:07:37.362664+0000","last_unstale":"2024-09-17T05:07:39.414733+0000","last_undegraded":"2024-09-17T05:07:39.414733+0000","last_fullsized":"2024-09-17T05:07:39.414733+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:50.405118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414804+0000","last_change":"2024-09-17T05:07:38.375962+0000","last_active":"2024-09-17T05:07:39.414804+0000","last_peered":"2024-09-17T05:07:39.414804+0000","last_clean":"2024-09-17T05:07:39.414804+0000","last_became_active":"2024-09-17T05:07:38.375669+0000","last_became_peered":"2024-09-17T05:07:38.375669+0000","last_unstale":"2024-09-17T05:07:39.414804+0000","last_undegraded":"2024-09-17T05:07:39.414804+0000","last_fullsized":"2024-09-17T05:07:39.414804+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:26.748990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364478+0000","last_change":"2024-09-17T05:07:37.365145+0000","last_active":"2024-09-17T05:07:39.364478+0000","last_peered":"2024-09-17T05:07:39.364478+0000","last_clean":"2024-09-17T05:07:39.364478+0000","last_became_active":"2024-09-17T05:07:37.364775+0000","last_became_peered":"2024-09-17T05:07:37.364775+0000","last_unstale":"2024-09-17T05:07:39.364478+0000","last_undegraded":"2024-09-17T05:07:39.364478+0000","last_fullsized":"2024-09-17T05:07:39.364478+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:18.797265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365284+0000","last_change":"2024-09-17T05:07:38.369937+0000","last_active":"2024-09-17T05:07:39.365284+0000","last_peered":"2024-09-17T05:07:39.365284+0000","last_clean":"2024-09-17T05:07:39.365284+0000","last_became_active":"2024-09-17T05:07:38.369450+0000","last_became_peered":"2024-09-17T05:07:38.369450+0000","last_unstale":"2024-09-17T05:07:39.365284+0000","last_undegraded":"2024-09-17T05:07:39.365284+0000","last_fullsized":"2024-09-17T05:07:39.365284+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:59.251209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"99'8","reported_seq":23,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415327+0000","last_change":"2024-09-17T05:07:37.368862+0000","last_active":"2024-09-17T05:07:39.415327+0000","last_peered":"2024-09-17T05:07:39.415327+0000","last_clean":"2024-09-17T05:07:39.415327+0000","last_became_active":"2024-09-17T05:07:37.368509+0000","last_became_peered":"2024-09-17T05:07:37.368509+0000","last_unstale":"2024-09-17T05:07:39.415327+0000","last_undegraded":"2024-09-17T05:07:39.415327+0000","last_fullsized":"2024-09-17T05:07:39.415327+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:59.530129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414372+0000","last_change":"2024-09-17T05:07:38.365896+0000","last_active":"2024-09-17T05:07:39.414372+0000","last_peered":"2024-09-17T05:07:39.414372+0000","last_clean":"2024-09-17T05:07:39.414372+0000","last_became_active":"2024-09-17T05:07:38.365546+0000","last_became_peered":"2024-09-17T05:07:38.365546+0000","last_unstale":"2024-09-17T05:07:39.414372+0000","last_undegraded":"2024-09-17T05:07:39.414372+0000","last_fullsized":"2024-09-17T05:07:39.414372+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:04:30.766430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363124+0000","last_change":"2024-09-17T05:07:37.364495+0000","last_active":"2024-09-17T05:07:39.363124+0000","last_peered":"2024-09-17T05:07:39.363124+0000","last_clean":"2024-09-17T05:07:39.363124+0000","last_became_active":"2024-09-17T05:07:37.364156+0000","last_became_peered":"2024-09-17T05:07:37.364156+0000","last_unstale":"2024-09-17T05:07:39.363124+0000","last_undegraded":"2024-09-17T05:07:39.363124+0000","last_fullsized":"2024-09-17T05:07:39.363124+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:16:14.240291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415949+0000","last_change":"2024-09-17T05:07:38.355046+0000","last_active":"2024-09-17T05:07:39.415949+0000","last_peered":"2024-09-17T05:07:39.415949+0000","last_clean":"2024-09-17T05:07:39.415949+0000","last_became_active":"2024-09-17T05:07:38.354647+0000","last_became_peered":"2024-09-17T05:07:38.354647+0000","last_unstale":"2024-09-17T05:07:39.415949+0000","last_undegraded":"2024-09-17T05:07:39.415949+0000","last_fullsized":"2024-09-17T05:07:39.415949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:09.954734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"99'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417200+0000","last_change":"2024-09-17T05:07:37.367221+0000","last_active":"2024-09-17T05:07:39.417200+0000","last_peered":"2024-09-17T05:07:39.417200+0000","last_clean":"2024-09-17T05:07:39.417200+0000","last_became_active":"2024-09-17T05:07:37.365939+0000","last_became_peered":"2024-09-17T05:07:37.365939+0000","last_unstale":"2024-09-17T05:07:39.417200+0000","last_undegraded":"2024-09-17T05:07:39.417200+0000","last_fullsized":"2024-09-17T05:07:39.417200+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:03.504912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414423+0000","last_change":"2024-09-17T05:07:38.376306+0000","last_active":"2024-09-17T05:07:39.414423+0000","last_peered":"2024-09-17T05:07:39.414423+0000","last_clean":"2024-09-17T05:07:39.414423+0000","last_became_active":"2024-09-17T05:07:38.375958+0000","last_became_peered":"2024-09-17T05:07:38.375958+0000","last_unstale":"2024-09-17T05:07:39.414423+0000","last_undegraded":"2024-09-17T05:07:39.414423+0000","last_fullsized":"2024-09-17T05:07:39.414423+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:06.227574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362763+0000","last_change":"2024-09-17T05:07:37.367645+0000","last_active":"2024-09-17T05:07:39.362763+0000","last_peered":"2024-09-17T05:07:39.362763+0000","last_clean":"2024-09-17T05:07:39.362763+0000","last_became_active":"2024-09-17T05:07:37.367258+0000","last_became_peered":"2024-09-17T05:07:37.367258+0000","last_unstale":"2024-09-17T05:07:39.362763+0000","last_undegraded":"2024-09-17T05:07:39.362763+0000","last_fullsized":"2024-09-17T05:07:39.362763+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:50.405118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364416+0000","last_change":"2024-09-17T05:07:38.362637+0000","last_active":"2024-09-17T05:07:39.364416+0000","last_peered":"2024-09-17T05:07:39.364416+0000","last_clean":"2024-09-17T05:07:39.364416+0000","last_became_active":"2024-09-17T05:07:38.361414+0000","last_became_peered":"2024-09-17T05:07:38.361414+0000","last_unstale":"2024-09-17T05:07:39.364416+0000","last_undegraded":"2024-09-17T05:07:39.364416+0000","last_fullsized":"2024-09-17T05:07:39.364416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:09.501404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364438+0000","last_change":"2024-09-17T05:07:37.364934+0000","last_active":"2024-09-17T05:07:39.364438+0000","last_peered":"2024-09-17T05:07:39.364438+0000","last_clean":"2024-09-17T05:07:39.364438+0000","last_became_active":"2024-09-17T05:07:37.364564+0000","last_became_peered":"2024-09-17T05:07:37.364564+0000","last_unstale":"2024-09-17T05:07:39.364438+0000","last_undegraded":"2024-09-17T05:07:39.364438+0000","last_fullsized":"2024-09-17T05:07:39.364438+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:25.876269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415120+0000","last_change":"2024-09-17T05:07:38.369141+0000","last_active":"2024-09-17T05:07:39.415120+0000","last_peered":"2024-09-17T05:07:39.415120+0000","last_clean":"2024-09-17T05:07:39.415120+0000","last_became_active":"2024-09-17T05:07:38.368328+0000","last_became_peered":"2024-09-17T05:07:38.368328+0000","last_unstale":"2024-09-17T05:07:39.415120+0000","last_undegraded":"2024-09-17T05:07:39.415120+0000","last_fullsized":"2024-09-17T05:07:39.415120+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:26.778140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363798+0000","last_change":"2024-09-17T05:07:37.359369+0000","last_active":"2024-09-17T05:07:39.363798+0000","last_peered":"2024-09-17T05:07:39.363798+0000","last_clean":"2024-09-17T05:07:39.363798+0000","last_became_active":"2024-09-17T05:07:37.359073+0000","last_became_peered":"2024-09-17T05:07:37.359073+0000","last_unstale":"2024-09-17T05:07:39.363798+0000","last_undegraded":"2024-09-17T05:07:39.363798+0000","last_fullsized":"2024-09-17T05:07:39.363798+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:59.202139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414926+0000","last_change":"2024-09-17T05:07:38.374442+0000","last_active":"2024-09-17T05:07:39.414926+0000","last_peered":"2024-09-17T05:07:39.414926+0000","last_clean":"2024-09-17T05:07:39.414926+0000","last_became_active":"2024-09-17T05:07:38.374007+0000","last_became_peered":"2024-09-17T05:07:38.374007+0000","last_unstale":"2024-09-17T05:07:39.414926+0000","last_undegraded":"2024-09-17T05:07:39.414926+0000","last_fullsized":"2024-09-17T05:07:39.414926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:27.973016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362369+0000","last_change":"2024-09-17T05:07:37.361606+0000","last_active":"2024-09-17T05:07:39.362369+0000","last_peered":"2024-09-17T05:07:39.362369+0000","last_clean":"2024-09-17T05:07:39.362369+0000","last_became_active":"2024-09-17T05:07:37.361002+0000","last_became_peered":"2024-09-17T05:07:37.361002+0000","last_unstale":"2024-09-17T05:07:39.362369+0000","last_undegraded":"2024-09-17T05:07:39.362369+0000","last_fullsized":"2024-09-17T05:07:39.362369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:30.250628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365723+0000","last_change":"2024-09-17T05:07:38.363075+0000","last_active":"2024-09-17T05:07:39.365723+0000","last_peered":"2024-09-17T05:07:39.365723+0000","last_clean":"2024-09-17T05:07:39.365723+0000","last_became_active":"2024-09-17T05:07:38.362485+0000","last_became_peered":"2024-09-17T05:07:38.362485+0000","last_unstale":"2024-09-17T05:07:39.365723+0000","last_undegraded":"2024-09-17T05:07:39.365723+0000","last_fullsized":"2024-09-17T05:07:39.365723+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:02.455532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414342+0000","last_change":"2024-09-17T05:07:37.362489+0000","last_active":"2024-09-17T05:07:39.414342+0000","last_peered":"2024-09-17T05:07:39.414342+0000","last_clean":"2024-09-17T05:07:39.414342+0000","last_became_active":"2024-09-17T05:07:37.361729+0000","last_became_peered":"2024-09-17T05:07:37.361729+0000","last_unstale":"2024-09-17T05:07:39.414342+0000","last_undegraded":"2024-09-17T05:07:39.414342+0000","last_fullsized":"2024-09-17T05:07:39.414342+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:51.258601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415184+0000","last_change":"2024-09-17T05:07:38.365765+0000","last_active":"2024-09-17T05:07:39.415184+0000","last_peered":"2024-09-17T05:07:39.415184+0000","last_clean":"2024-09-17T05:07:39.415184+0000","last_became_active":"2024-09-17T05:07:38.365452+0000","last_became_peered":"2024-09-17T05:07:38.365452+0000","last_unstale":"2024-09-17T05:07:39.415184+0000","last_undegraded":"2024-09-17T05:07:39.415184+0000","last_fullsized":"2024-09-17T05:07:39.415184+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:34.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365507+0000","last_change":"2024-09-17T05:07:37.360081+0000","last_active":"2024-09-17T05:07:39.365507+0000","last_peered":"2024-09-17T05:07:39.365507+0000","last_clean":"2024-09-17T05:07:39.365507+0000","last_became_active":"2024-09-17T05:07:37.359720+0000","last_became_peered":"2024-09-17T05:07:37.359720+0000","last_unstale":"2024-09-17T05:07:39.365507+0000","last_undegraded":"2024-09-17T05:07:39.365507+0000","last_fullsized":"2024-09-17T05:07:39.365507+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:16.580036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415910+0000","last_change":"2024-09-17T05:07:38.358181+0000","last_active":"2024-09-17T05:07:39.415910+0000","last_peered":"2024-09-17T05:07:39.415910+0000","last_clean":"2024-09-17T05:07:39.415910+0000","last_became_active":"2024-09-17T05:07:38.357847+0000","last_became_peered":"2024-09-17T05:07:38.357847+0000","last_unstale":"2024-09-17T05:07:39.415910+0000","last_undegraded":"2024-09-17T05:07:39.415910+0000","last_fullsized":"2024-09-17T05:07:39.415910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:14.083379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"100'1","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.370455+0000","last_change":"2024-09-17T05:07:37.358084+0000","last_active":"2024-09-17T05:07:39.370455+0000","last_peered":"2024-09-17T05:07:39.370455+0000","last_clean":"2024-09-17T05:07:39.370455+0000","last_became_active":"2024-09-17T05:07:37.357773+0000","last_became_peered":"2024-09-17T05:07:37.357773+0000","last_unstale":"2024-09-17T05:07:39.370455+0000","last_undegraded":"2024-09-17T05:07:39.370455+0000","last_fullsized":"2024-09-17T05:07:39.370455+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:02.583665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.414402+0000","last_change":"2024-09-17T05:07:38.375508+0000","last_active":"2024-09-17T05:07:39.414402+0000","last_peered":"2024-09-17T05:07:39.414402+0000","last_clean":"2024-09-17T05:07:39.414402+0000","last_became_active":"2024-09-17T05:07:38.375074+0000","last_became_peered":"2024-09-17T05:07:38.375074+0000","last_unstale":"2024-09-17T05:07:39.414402+0000","last_undegraded":"2024-09-17T05:07:39.414402+0000","last_fullsized":"2024-09-17T05:07:39.414402+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:40.719087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.417035+0000","last_change":"2024-09-17T05:07:37.350958+0000","last_active":"2024-09-17T05:07:39.417035+0000","last_peered":"2024-09-17T05:07:39.417035+0000","last_clean":"2024-09-17T05:07:39.417035+0000","last_became_active":"2024-09-17T05:07:37.349820+0000","last_became_peered":"2024-09-17T05:07:37.349820+0000","last_unstale":"2024-09-17T05:07:39.417035+0000","last_undegraded":"2024-09-17T05:07:39.417035+0000","last_fullsized":"2024-09-17T05:07:39.417035+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:43.480852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416520+0000","last_change":"2024-09-17T05:07:38.358326+0000","last_active":"2024-09-17T05:07:39.416520+0000","last_peered":"2024-09-17T05:07:39.416520+0000","last_clean":"2024-09-17T05:07:39.416520+0000","last_became_active":"2024-09-17T05:07:38.358002+0000","last_became_peered":"2024-09-17T05:07:38.358002+0000","last_unstale":"2024-09-17T05:07:39.416520+0000","last_undegraded":"2024-09-17T05:07:39.416520+0000","last_fullsized":"2024-09-17T05:07:39.416520+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:19.804778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364819+0000","last_change":"2024-09-17T05:07:37.368314+0000","last_active":"2024-09-17T05:07:39.364819+0000","last_peered":"2024-09-17T05:07:39.364819+0000","last_clean":"2024-09-17T05:07:39.364819+0000","last_became_active":"2024-09-17T05:07:37.367953+0000","last_became_peered":"2024-09-17T05:07:37.367953+0000","last_unstale":"2024-09-17T05:07:39.364819+0000","last_undegraded":"2024-09-17T05:07:39.364819+0000","last_fullsized":"2024-09-17T05:07:39.364819+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:52.522812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364275+0000","last_change":"2024-09-17T05:07:38.359694+0000","last_active":"2024-09-17T05:07:39.364275+0000","last_peered":"2024-09-17T05:07:39.364275+0000","last_clean":"2024-09-17T05:07:39.364275+0000","last_became_active":"2024-09-17T05:07:38.359388+0000","last_became_peered":"2024-09-17T05:07:38.359388+0000","last_unstale":"2024-09-17T05:07:39.364275+0000","last_undegraded":"2024-09-17T05:07:39.364275+0000","last_fullsized":"2024-09-17T05:07:39.364275+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:43.873942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416949+0000","last_change":"2024-09-17T05:07:37.364116+0000","last_active":"2024-09-17T05:07:39.416949+0000","last_peered":"2024-09-17T05:07:39.416949+0000","last_clean":"2024-09-17T05:07:39.416949+0000","last_became_active":"2024-09-17T05:07:37.363791+0000","last_became_peered":"2024-09-17T05:07:37.363791+0000","last_unstale":"2024-09-17T05:07:39.416949+0000","last_undegraded":"2024-09-17T05:07:39.416949+0000","last_fullsized":"2024-09-17T05:07:39.416949+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:51.314268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.365704+0000","last_change":"2024-09-17T05:07:38.359216+0000","last_active":"2024-09-17T05:07:39.365704+0000","last_peered":"2024-09-17T05:07:39.365704+0000","last_clean":"2024-09-17T05:07:39.365704+0000","last_became_active":"2024-09-17T05:07:38.358495+0000","last_became_peered":"2024-09-17T05:07:38.358495+0000","last_unstale":"2024-09-17T05:07:39.365704+0000","last_undegraded":"2024-09-17T05:07:39.365704+0000","last_fullsized":"2024-09-17T05:07:39.365704+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:11.941996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364119+0000","last_change":"2024-09-17T05:07:37.357614+0000","last_active":"2024-09-17T05:07:39.364119+0000","last_peered":"2024-09-17T05:07:39.364119+0000","last_clean":"2024-09-17T05:07:39.364119+0000","last_became_active":"2024-09-17T05:07:37.357276+0000","last_became_peered":"2024-09-17T05:07:37.357276+0000","last_unstale":"2024-09-17T05:07:39.364119+0000","last_undegraded":"2024-09-17T05:07:39.364119+0000","last_fullsized":"2024-09-17T05:07:39.364119+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:39.232366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363028+0000","last_change":"2024-09-17T05:07:38.359254+0000","last_active":"2024-09-17T05:07:39.363028+0000","last_peered":"2024-09-17T05:07:39.363028+0000","last_clean":"2024-09-17T05:07:39.363028+0000","last_became_active":"2024-09-17T05:07:38.358849+0000","last_became_peered":"2024-09-17T05:07:38.358849+0000","last_unstale":"2024-09-17T05:07:39.363028+0000","last_undegraded":"2024-09-17T05:07:39.363028+0000","last_fullsized":"2024-09-17T05:07:39.363028+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:00.537642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415799+0000","last_change":"2024-09-17T05:07:37.364772+0000","last_active":"2024-09-17T05:07:39.415799+0000","last_peered":"2024-09-17T05:07:39.415799+0000","last_clean":"2024-09-17T05:07:39.415799+0000","last_became_active":"2024-09-17T05:07:37.364468+0000","last_became_peered":"2024-09-17T05:07:37.364468+0000","last_unstale":"2024-09-17T05:07:39.415799+0000","last_undegraded":"2024-09-17T05:07:39.415799+0000","last_fullsized":"2024-09-17T05:07:39.415799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:56:09.224428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.416666+0000","last_change":"2024-09-17T05:07:38.378238+0000","last_active":"2024-09-17T05:07:39.416666+0000","last_peered":"2024-09-17T05:07:39.416666+0000","last_clean":"2024-09-17T05:07:39.416666+0000","last_became_active":"2024-09-17T05:07:38.377899+0000","last_became_peered":"2024-09-17T05:07:38.377899+0000","last_unstale":"2024-09-17T05:07:39.416666+0000","last_undegraded":"2024-09-17T05:07:39.416666+0000","last_fullsized":"2024-09-17T05:07:39.416666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:33.791006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.363935+0000","last_change":"2024-09-17T05:07:37.355234+0000","last_active":"2024-09-17T05:07:39.363935+0000","last_peered":"2024-09-17T05:07:39.363935+0000","last_clean":"2024-09-17T05:07:39.363935+0000","last_became_active":"2024-09-17T05:07:37.354718+0000","last_became_peered":"2024-09-17T05:07:37.354718+0000","last_unstale":"2024-09-17T05:07:39.363935+0000","last_undegraded":"2024-09-17T05:07:39.363935+0000","last_fullsized":"2024-09-17T05:07:39.363935+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:17.673587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.362247+0000","last_change":"2024-09-17T05:07:38.368854+0000","last_active":"2024-09-17T05:07:39.362247+0000","last_peered":"2024-09-17T05:07:39.362247+0000","last_clean":"2024-09-17T05:07:39.362247+0000","last_became_active":"2024-09-17T05:07:38.368514+0000","last_became_peered":"2024-09-17T05:07:38.368514+0000","last_unstale":"2024-09-17T05:07:39.362247+0000","last_undegraded":"2024-09-17T05:07:39.362247+0000","last_fullsized":"2024-09-17T05:07:39.362247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:37.326472+0000","last_clean_scrub_stamp":"2024-09-17T05:07:37.326472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:26.054302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"99'2","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.364808+0000","last_change":"2024-09-17T05:07:37.363388+0000","last_active":"2024-09-17T05:07:39.364808+0000","last_peered":"2024-09-17T05:07:39.364808+0000","last_clean":"2024-09-17T05:07:39.364808+0000","last_became_active":"2024-09-17T05:07:37.362169+0000","last_became_peered":"2024-09-17T05:07:37.362169+0000","last_unstale":"2024-09-17T05:07:39.364808+0000","last_undegraded":"2024-09-17T05:07:39.364808+0000","last_fullsized":"2024-09-17T05:07:39.364808+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:07:36.318960+0000","last_clean_scrub_stamp":"2024-09-17T05:07:36.318960+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:08.947221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":264,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-17T05:07:39.415767+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:07:39.415767+0000","last_peered":"2024-09-17T05:07:39.415767+0000","last_clean":"2024-09-17T05:07:39.415767+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:07:39.415767+0000","last_undegraded":"2024-09-17T05:07:39.415767+0000","last_fullsized":"2024-09-17T05:07:39.415767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":565809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":566769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509524,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1236992,"data_stored":565191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36572,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335268,"statfs":{"total":96636764160,"available":96599314432,"internally_reserved":0,"allocated":1245184,"data_stored":566769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509523,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1241088,"data_stored":565269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37148,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334692,"statfs":{"total":96636764160,"available":96598724608,"internally_reserved":0,"allocated":1835008,"data_stored":1155617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1236992,"data_stored":565739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1835008,"data_stored":1155649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:07:47.686 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:07:47.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:07:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 -- 172.21.15.16:0/3614278017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb80b1c40 msgr2=0x7fbcb80b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 --2- 172.21.15.16:0/3614278017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb80b1c40 0x7fbcb80b2050 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009900 tx=0x7fbca001c8b0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 -- 172.21.15.16:0/3614278017 shutdown_connections 2024-09-17T05:07:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 --2- 172.21.15.16:0/3614278017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb80b1c40 0x7fbcb80b2050 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 --2- 172.21.15.16:0/3614278017 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbcb815a050 0x7fbcb80cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 -- 172.21.15.16:0/3614278017 >> 172.21.15.16:0/3614278017 conn(0x7fbcb80d2240 msgr2=0x7fbcb80d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 -- 172.21.15.16:0/3614278017 shutdown_connections 2024-09-17T05:07:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 -- 172.21.15.16:0/3614278017 wait complete. 2024-09-17T05:07:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 Processor -- start 2024-09-17T05:07:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.895+0000 7fbcbe800640 1 -- start start 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcb80b1c40 0x7fbcb81433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbcb80b6a90 0x7fbcb80cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 0x7fbcb81fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8146190 con 0x7fbcb815a050 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8146010 con 0x7fbcb80b1c40 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbcb816a250 con 0x7fbcb80b6a90 2024-09-17T05:07:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 0x7fbcb81fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 0x7fbcb81fc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60504/0 (socket says 172.21.15.16:60504) 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 -- 172.21.15.16:0/2729225834 learned_addr learned my addr 172.21.15.16:0/2729225834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcaea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbcb80b6a90 0x7fbcb80cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7400640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcb80b1c40 0x7fbcb81433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 -- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbcb80b6a90 msgr2=0x7fbcb80cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbcb80b6a90 0x7fbcb80cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 -- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcb80b1c40 msgr2=0x7fbcb81433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcb80b1c40 0x7fbcb81433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 -- 172.21.15.16:0/2729225834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb81438f0 con 0x7fbcb815a050 2024-09-17T05:07:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcb7e00640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 0x7fbcb81fc6b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000c730 tx=0x7fbcb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0003b80 con 0x7fbcb815a050 2024-09-17T05:07:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb0003d40 con 0x7fbcb815a050 2024-09-17T05:07:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00052d0 con 0x7fbcb815a050 2024-09-17T05:07:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb80cd300 con 0x7fbcb815a050 2024-09-17T05:07:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb80cd8c0 con 0x7fbcb815a050 2024-09-17T05:07:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb80b2050 con 0x7fbcb815a050 2024-09-17T05:07:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.899+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbcb0005470 con 0x7fbcb815a050 2024-09-17T05:07:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.903+0000 7fbcafe00640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc94073200 0x7fbc94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.903+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fbcb00940c0 con 0x7fbcb815a050 2024-09-17T05:07:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.903+0000 7fbcb7400640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc94073200 0x7fbc94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.903+0000 7fbcb7400640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc94073200 0x7fbc94075710 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbca800ab40 tx=0x7fbca800a250 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:47.907+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbcb005e860 con 0x7fbcb815a050 2024-09-17T05:07:48.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.143+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbcb815be70 con 0x7fbcb815a050 2024-09-17T05:07:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.143+0000 7fbcafe00640 1 -- 172.21.15.16:0/2729225834 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7fbcb0063290 con 0x7fbcb815a050 2024-09-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:07:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc94073200 msgr2=0x7fbc94075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc94073200 0x7fbc94075710 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbca800ab40 tx=0x7fbca800a250 comp rx=0 tx=0).stop 2024-09-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 msgr2=0x7fbcb81fc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 0x7fbcb81fc6b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000c730 tx=0x7fbcb000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 shutdown_connections 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc94073200 0x7fbc94075710 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcb815a050 0x7fbcb81fc6b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbcb80b6a90 0x7fbcb80cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 --2- 172.21.15.16:0/2729225834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcb80b1c40 0x7fbcb81433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 >> 172.21.15.16:0/2729225834 conn(0x7fbcb80d2240 msgr2=0x7fbcb8140110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 shutdown_connections 2024-09-17T05:07:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.147+0000 7fbcbe800640 1 -- 172.21.15.16:0/2729225834 wait complete. 2024-09-17T05:07:48.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fceaca00640 1 -- 172.21.15.16:0/3763451811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c003a10 msgr2=0x7fce8c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fceaca00640 1 --2- 172.21.15.16:0/3763451811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c003a10 0x7fce8c003e80 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c4a0 tx=0x7fce9801cf80 comp rx=0 tx=0).stop 2024-09-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fcea7e00640 1 -- 172.21.15.16:0/3763451811 reap_dead start 2024-09-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fceaca00640 1 -- 172.21.15.16:0/3763451811 shutdown_connections 2024-09-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fceaca00640 1 -- 172.21.15.16:0/3763451811 >> 172.21.15.16:0/3763451811 conn(0x7fcea806f9c0 msgr2=0x7fcea80b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:48.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fceaca00640 1 -- 172.21.15.16:0/3763451811 shutdown_connections 2024-09-17T05:07:48.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.327+0000 7fceaca00640 1 -- 172.21.15.16:0/3763451811 wait complete. 2024-09-17T05:07:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 Processor -- start 2024-09-17T05:07:48.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 -- start start 2024-09-17T05:07:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce8c003a10 0x7fcea814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea8148140 0x7fcea81f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 0x7fcea8145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:48.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcea8166320 con 0x7fcea81fa170 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcea81661a0 con 0x7fcea8148140 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcea81664a0 con 0x7fce8c003a10 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 0x7fcea8145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 0x7fcea8145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60526/0 (socket says 172.21.15.16:60526) 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 -- 172.21.15.16:0/1513816308 learned_addr learned my addr 172.21.15.16:0/1513816308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7400640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce8c003a10 0x7fcea814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fce9fe00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea8148140 0x7fcea81f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 -- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce8c003a10 msgr2=0x7fcea814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce8c003a10 0x7fcea814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 -- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea8148140 msgr2=0x7fcea81f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea8148140 0x7fcea81f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 -- 172.21.15.16:0/1513816308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea81fa580 con 0x7fcea81fa170 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fce9fe00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea8148140 0x7fcea81f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fcea7e00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 0x7fcea8145480 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcea8072760 tx=0x7fcea0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea000ee70 con 0x7fcea81fa170 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea0015070 con 0x7fcea81fa170 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea8152fe0 con 0x7fcea81fa170 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0003d50 con 0x7fcea81fa170 2024-09-17T05:07:48.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.331+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea81556c0 con 0x7fcea81fa170 2024-09-17T05:07:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.335+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea814f870 con 0x7fcea81fa170 2024-09-17T05:07:48.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.335+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcea0004030 con 0x7fcea81fa170 2024-09-17T05:07:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.335+0000 7fce9f400640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce84073200 0x7fce84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.335+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7fcea0094de0 con 0x7fcea81fa170 2024-09-17T05:07:48.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.335+0000 7fcea7400640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce84073200 0x7fce84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:48.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.339+0000 7fcea7400640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce84073200 0x7fce84075710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce980ac000 tx=0x7fce8c005840 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.339+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcea81fa580 con 0x7fcea81fa170 2024-09-17T05:07:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.551+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcea8152d70 con 0x7fcea81fa170 2024-09-17T05:07:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.551+0000 7fce9f400640 1 -- 172.21.15.16:0/1513816308 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7fcea005fc60 con 0x7fcea81fa170 2024-09-17T05:07:48.551 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:48.551 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:07:37.774040+0000","modified":"2024-09-17T05:07:40.784518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2345541365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2345541365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2345541365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-17T05:07:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-09-17T05:07:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce84073200 msgr2=0x7fce84075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce84073200 0x7fce84075710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce980ac000 tx=0x7fce8c005840 comp rx=0 tx=0).stop 2024-09-17T05:07:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 msgr2=0x7fcea8145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 0x7fcea8145480 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcea8072760 tx=0x7fcea0010f40 comp rx=0 tx=0).stop 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 shutdown_connections 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce84073200 0x7fce84075710 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcea81fa170 0x7fcea8145480 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcea8148140 0x7fcea81f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 --2- 172.21.15.16:0/1513816308 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce8c003a10 0x7fcea814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 >> 172.21.15.16:0/1513816308 conn(0x7fcea806f9c0 msgr2=0x7fcea80cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 shutdown_connections 2024-09-17T05:07:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:48.555+0000 7fceaca00640 1 -- 172.21.15.16:0/1513816308 wait complete. 2024-09-17T05:07:48.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:07:37.774040+0000', 'modified': '2024-09-17T05:07:40.784518+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6993}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6993': {'gid': 6993, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/2345541365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2345541365}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2345541365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6993, 'qdb_cluster': [6993]} 2024-09-17T05:07:48.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:07:48.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T05:07:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 -- 172.21.15.16:0/3426626760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 msgr2=0x7f78a815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 --2- 172.21.15.16:0/3426626760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a815efe0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f78a80cca90 tx=0x7f789801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 -- 172.21.15.16:0/3426626760 shutdown_connections 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 --2- 172.21.15.16:0/3426626760 >> v2:172.21.15.104:3300/0 conn(0x7f78a8074e30 0x7f78a815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 --2- 172.21.15.16:0/3426626760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a815efe0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 -- 172.21.15.16:0/3426626760 >> 172.21.15.16:0/3426626760 conn(0x7f78a80b2250 msgr2=0x7f78a80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 -- 172.21.15.16:0/3426626760 shutdown_connections 2024-09-17T05:07:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 -- 172.21.15.16:0/3426626760 wait complete. 2024-09-17T05:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 Processor -- start 2024-09-17T05:07:49.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.143+0000 7f78ae000640 1 -- start start 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a81f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78a8074e30 0x7f78a81585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a815ba50 0x7f78a80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8168e80 con 0x7f78a8074a20 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8168d00 con 0x7f78a815ba50 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f78a8169000 con 0x7f78a8074e30 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a81f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a81f9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60548/0 (socket says 172.21.15.16:60548) 2024-09-17T05:07:49.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 -- 172.21.15.16:0/3646090009 learned_addr learned my addr 172.21.15.16:0/3646090009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 -- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78a8074e30 msgr2=0x7f78a81585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78a8074e30 0x7f78a81585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 -- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a815ba50 msgr2=0x7f78a80b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a815ba50 0x7f78a80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 -- 172.21.15.16:0/3646090009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a81535e0 con 0x7f78a8074a20 2024-09-17T05:07:49.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78a7400640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a81f9b40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7898009850 tx=0x7f78980a5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f789f400640 1 -- 172.21.15.16:0/3646090009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980bc070 con 0x7f78a8074a20 2024-09-17T05:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f789f400640 1 -- 172.21.15.16:0/3646090009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78980043b0 con 0x7f78a8074a20 2024-09-17T05:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a81538c0 con 0x7f78a8074a20 2024-09-17T05:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f789f400640 1 -- 172.21.15.16:0/3646090009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980b2040 con 0x7f78a8074a20 2024-09-17T05:07:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.147+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a8153e80 con 0x7f78a8074a20 2024-09-17T05:07:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f789f400640 1 -- 172.21.15.16:0/3646090009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7898005290 con 0x7f78a8074a20 2024-09-17T05:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f789f400640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7884073230 0x7f7884075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f789f400640 1 -- 172.21.15.16:0/3646090009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f789812dae0 con 0x7f78a8074a20 2024-09-17T05:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 Processor -- start 2024-09-17T05:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 -- start start 2024-09-17T05:07:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f789fe00640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7884073230 0x7f7884075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f789fe00640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7884073230 0x7f7884075740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78900058d0 tx=0x7f7890005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc15b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc145f950 0x560dc15b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 0x560dc15b1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560dc1485410 con 0x560dc1448ca0 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560dc13c2b40 con 0x560dc145f950 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.151+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x560dc13b1f70 con 0x560dc106f460 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 0x560dc15b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 0x560dc15b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60556/0 (socket says 172.21.15.16:60556) 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 -- 172.21.15.16:0/4249224820 learned_addr learned my addr 172.21.15.16:0/4249224820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7400640 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc15b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f789fe00640 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc145f950 0x560dc15b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 -- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 msgr2=0x560dc15b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc15b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 -- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc145f950 msgr2=0x560dc15b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc145f950 0x560dc15b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 -- 172.21.15.16:0/4249224820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc13795b0 con 0x560dc1448ca0 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78a7e00640 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 0x560dc15b1120 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f78a0001c80 tx=0x7f78a0002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2300af5f7ad174 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f7897400640 1 -- 172.21.15.16:0/4249224820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a000cbf0 con 0x560dc1448ca0 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f7897400640 1 -- 172.21.15.16:0/4249224820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78a0017070 con 0x560dc1448ca0 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f7897400640 1 -- 172.21.15.16:0/4249224820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0012420 con 0x560dc1448ca0 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 -- 172.21.15.16:0/4249224820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 msgr2=0x560dc15b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 0x560dc15b1120 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f78a0001c80 tx=0x7f78a0002140 comp rx=0 tx=0).stop 2024-09-17T05:07:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 -- 172.21.15.16:0/4249224820 shutdown_connections 2024-09-17T05:07:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc1448ca0 0x560dc15b1120 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc145f950 0x560dc15b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 --2- 172.21.15.16:0/4249224820 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc15b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 -- 172.21.15.16:0/4249224820 >> 172.21.15.16:0/4249224820 conn(0x560dc1535d90 msgr2=0x560dc1565690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 -- 172.21.15.16:0/4249224820 shutdown_connections 2024-09-17T05:07:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 -- 172.21.15.16:0/4249224820 wait complete. 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 -- start start 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.155+0000 7f78b55ca000 1 Processor -- start 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc15b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc15b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560dc134c5f0 con 0x560dc106f460 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560dc13b5d80 con 0x560dc1448ca0 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x560dc1358520 con 0x560dc145f950 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 10 client.0 _ll_register_callbacks cb 0x560dc135c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:07:49.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc15b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc15b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60560/0 (socket says 172.21.15.16:60560) 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 -- 172.21.15.16:0/1993758413 learned_addr learned my addr 172.21.15.16:0/1993758413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7e00640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc15b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f789fe00640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 -- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 msgr2=0x560dc15b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc15b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 -- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 msgr2=0x560dc15b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 -- 172.21.15.16:0/1993758413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc1350190 con 0x560dc106f460 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78a7400640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc15b7920 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f789801cda0 tx=0x7f78980f8290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d1a61be49b0d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 1 -- 172.21.15.16:0/1993758413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980f8a90 con 0x560dc106f460 2024-09-17T05:07:49.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 1 -- 172.21.15.16:0/1993758413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f7898101aa0 con 0x560dc106f460 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560dc134a110 con 0x560dc106f460 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 1 -- 172.21.15.16:0/1993758413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f78980f8390 con 0x560dc106f460 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 10 client.7140 fetch_fsmap learned FSMap version 66 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x560dc1436f10 con 0x560dc106f460 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 1 -- 172.21.15.16:0/1993758413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560dc134a110 con 0x560dc106f460 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 1 -- 172.21.15.16:0/1993758413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f78980ff0b0 con 0x560dc106f460 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 10 client.7140 fetch_fsmap finished waiting for FSMap version 66 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 10 client.7140 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 20 client.7140 populate_metadata read hostname 'smithi016' 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc10547d0 0x560dc15c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 4 client.7140 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:07:49.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 --> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dc13c4a90 con 0x560dc10547d0 2024-09-17T05:07:49.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f789fe00640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc10547d0 0x560dc15c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f788be00640 10 client.7140 ms_handle_connect on v2:172.21.15.16:6834/2345541365 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.159+0000 7f789fe00640 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc10547d0 0x560dc15c1d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2321f13c064216ae server_cookie=2237c2526c8ec819 in_seq=0 out_seq=0 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f788be00640 1 -- 172.21.15.16:0/1993758413 <== mds.0 v2:172.21.15.16:6834/2345541365 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560dc13c4a90 con 0x560dc10547d0 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f788be00640 10 client.7140 handle_command_reply: tid=0 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 client.7140 shutdown 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 msgr2=0x560dc15b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc15b7920 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f789801cda0 tx=0x7f78980f8290 comp rx=0 tx=0).stop 2024-09-17T05:07:49.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 shutdown_connections 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc10547d0 0x560dc15c1d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc15b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 --2- 172.21.15.16:0/1993758413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc15b7920 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f788be00640 0 client.7140 ms_handle_reset on v2:172.21.15.16:6834/2345541365 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 >> 172.21.15.16:0/1993758413 conn(0x560dc1535d90 msgr2=0x560dc156c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 shutdown_connections 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 1 -- 172.21.15.16:0/1993758413 wait complete. 2024-09-17T05:07:49.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.163+0000 7f78b55ca000 20 client.7140 trim_cache size 0 max 16384 2024-09-17T05:07:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 Processor -- start 2024-09-17T05:07:49.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- start start 2024-09-17T05:07:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc1564cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc1565210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc1565750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8205c50 con 0x560dc106f460 2024-09-17T05:07:49.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8204d40 con 0x560dc1448ca0 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f78a80b47b0 con 0x560dc145f950 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc1564cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc1564cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60566/0 (socket says 172.21.15.16:60566) 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 -- 172.21.15.16:0/3125552106 learned_addr learned my addr 172.21.15.16:0/3125552106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7e00640 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc1565750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f789fe00640 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc1565210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 -- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 msgr2=0x560dc1565750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc1565750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 -- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 msgr2=0x560dc1565210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc1565210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 -- 172.21.15.16:0/3125552106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc140ddb0 con 0x560dc106f460 2024-09-17T05:07:49.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78a7400640 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc1564cd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f78980f5600 tx=0x7f78980045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cf243395698cd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f7896a00640 1 -- 172.21.15.16:0/3125552106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980f7b00 con 0x560dc106f460 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f7896a00640 1 -- 172.21.15.16:0/3125552106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78980f8a90 con 0x560dc106f460 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f7896a00640 1 -- 172.21.15.16:0/3125552106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980fce70 con 0x560dc106f460 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- 172.21.15.16:0/3125552106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 msgr2=0x560dc1564cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc1564cd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f78980f5600 tx=0x7f78980045a0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- 172.21.15.16:0/3125552106 shutdown_connections 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc145f950 0x560dc1565750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc1565210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3125552106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc106f460 0x560dc1564cd0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- 172.21.15.16:0/3125552106 >> 172.21.15.16:0/3125552106 conn(0x560dc1535d90 msgr2=0x560dc15ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- 172.21.15.16:0/3125552106 shutdown_connections 2024-09-17T05:07:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.183+0000 7f78b55ca000 1 -- 172.21.15.16:0/3125552106 wait complete. 2024-09-17T05:07:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 -- start start 2024-09-17T05:07:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 Processor -- start 2024-09-17T05:07:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc1567db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 0x560dc15e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560dc13b5d80 con 0x560dc145f950 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560dc13c1910 con 0x560dc1448ca0 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x560dc137dee0 con 0x560dc106f460 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 10 client.0 _ll_register_callbacks cb 0x7f78980f4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 0x560dc15e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 0x560dc15e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60570/0 (socket says 172.21.15.16:60570) 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 -- 172.21.15.16:0/3822168934 learned_addr learned my addr 172.21.15.16:0/3822168934 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f789fe00640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7400640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc1567db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 -- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 msgr2=0x560dc1567db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc1567db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 -- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 msgr2=0x560dc15682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 -- 172.21.15.16:0/3822168934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc1378ab0 con 0x560dc145f950 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78a7e00640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 0x560dc15e2220 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f78a00012f0 tx=0x7f78a0006ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c83e030b9e2406 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f788be00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f788be00640 1 -- 172.21.15.16:0/3822168934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0003980 con 0x560dc145f950 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f788be00640 1 -- 172.21.15.16:0/3822168934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f78a0003b20 con 0x560dc145f950 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78980f8a90 con 0x560dc145f950 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f788be00640 1 -- 172.21.15.16:0/3822168934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f78a0003d20 con 0x560dc145f950 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 10 client.7146 fetch_fsmap learned FSMap version 66 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x560dc1357070 con 0x560dc145f950 2024-09-17T05:07:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.187+0000 7f788be00640 1 -- 172.21.15.16:0/3822168934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78980f8a90 con 0x560dc145f950 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f788be00640 1 -- 172.21.15.16:0/3822168934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f78a00050c0 con 0x560dc145f950 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 10 client.7146 fetch_fsmap finished waiting for FSMap version 66 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 10 client.7146 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 20 client.7146 populate_metadata read hostname 'smithi016' 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc1042830 0x560dc16262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 4 client.7146 mds_command: new command op to 6993 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 --> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] -- command(tid 0: {"prefix": "status"}) -- 0x560dc13c0540 con 0x560dc1042830 2024-09-17T05:07:49.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78a7400640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc1042830 0x560dc16262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78a7400640 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc1042830 0x560dc16262b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f26f5edf14401f server_cookie=8c11197cbcfe6eb4 in_seq=0 out_seq=0 2024-09-17T05:07:49.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f788be00640 10 client.7146 ms_handle_connect on v2:172.21.15.16:6834/2345541365 2024-09-17T05:07:49.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f788be00640 1 -- 172.21.15.16:0/3822168934 <== mds.0 v2:172.21.15.16:6834/2345541365 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560dc13c0540 con 0x560dc1042830 2024-09-17T05:07:49.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f788be00640 10 client.7146 handle_command_reply: tid=0 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 client.7146 shutdown 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 msgr2=0x560dc15e2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 0x560dc15e2220 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f78a00012f0 tx=0x7f78a0006ec0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 shutdown_connections 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:6834/2345541365,v1:172.21.15.16:6835/2345541365] conn(0x560dc1042830 0x560dc16262b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560dc145f950 0x560dc15e2220 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dc1448ca0 0x560dc15682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 --2- 172.21.15.16:0/3822168934 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x560dc106f460 0x560dc1567db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f788be00640 0 client.7146 ms_handle_reset on v2:172.21.15.16:6834/2345541365 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 >> 172.21.15.16:0/3822168934 conn(0x560dc1535d90 msgr2=0x560dc15c16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 shutdown_connections 2024-09-17T05:07:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 1 -- 172.21.15.16:0/3822168934 wait complete. 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78b55ca000 20 client.7146 trim_cache size 0 max 16384 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 6993, 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:07:49.194 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.832464937999999, 2024-09-17T05:07:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 66, 2024-09-17T05:07:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 100, 2024-09-17T05:07:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 100, 2024-09-17T05:07:49.195 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.235730512 2024-09-17T05:07:49.195 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7884073230 msgr2=0x7f7884075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7884073230 0x7f7884075740 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f78900058d0 tx=0x7f7890005860 comp rx=0 tx=0).stop 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 msgr2=0x7f78a81f9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a81f9b40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7898009850 tx=0x7f78980a5580 comp rx=0 tx=0).stop 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 shutdown_connections 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7884073230 0x7f7884075740 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a815ba50 0x7f78a80b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78a8074e30 0x7f78a81585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 --2- 172.21.15.16:0/3646090009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78a8074a20 0x7f78a81f9b40 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 >> 172.21.15.16:0/3646090009 conn(0x7f78a80b2250 msgr2=0x7f78a815cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 shutdown_connections 2024-09-17T05:07:49.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.191+0000 7f78ae000640 1 -- 172.21.15.16:0/3646090009 wait complete. 2024-09-17T05:07:49.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 -- 172.21.15.16:0/4049290604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74740b2f50 msgr2=0x7f747415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 --2- 172.21.15.16:0/4049290604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74740b2f50 0x7f747415bb60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7464009840 tx=0x7f746401c790 comp rx=0 tx=0).stop 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 -- 172.21.15.16:0/4049290604 shutdown_connections 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 --2- 172.21.15.16:0/4049290604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74740b2f50 0x7f747415bb60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 -- 172.21.15.16:0/4049290604 >> 172.21.15.16:0/4049290604 conn(0x7f747406f900 msgr2=0x7f74740b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 -- 172.21.15.16:0/4049290604 shutdown_connections 2024-09-17T05:07:49.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 -- 172.21.15.16:0/4049290604 wait complete. 2024-09-17T05:07:49.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 Processor -- start 2024-09-17T05:07:49.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.371+0000 7f7478e00640 1 -- start start 2024-09-17T05:07:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7474073240 0x7f747413e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74740b2f50 0x7f747413fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 0x7f7474140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7474169cc0 con 0x7f747415c0a0 2024-09-17T05:07:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7474169b40 con 0x7f7474073240 2024-09-17T05:07:49.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7474169e40 con 0x7f74740b2f50 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 0x7f7474140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 0x7f7474140320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60586/0 (socket says 172.21.15.16:60586) 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 -- 172.21.15.16:0/922294488 learned_addr learned my addr 172.21.15.16:0/922294488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7471e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7474073240 0x7f747413e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7469400640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74740b2f50 0x7f747413fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 -- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74740b2f50 msgr2=0x7f747413fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74740b2f50 0x7f747413fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 -- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7474073240 msgr2=0x7f747413e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7474073240 0x7f747413e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 -- 172.21.15.16:0/922294488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747413ed40 con 0x7f747415c0a0 2024-09-17T05:07:49.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7472800640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 0x7f7474140320 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c3f0 tx=0x7f746c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c017070 con 0x7f747415c0a0 2024-09-17T05:07:49.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f746c003600 con 0x7f747415c0a0 2024-09-17T05:07:49.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747413f020 con 0x7f747415c0a0 2024-09-17T05:07:49.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c003a50 con 0x7f747415c0a0 2024-09-17T05:07:49.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f747413f5e0 con 0x7f747415c0a0 2024-09-17T05:07:49.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.375+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74740b1f30 con 0x7f747415c0a0 2024-09-17T05:07:49.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.379+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f746c01a070 con 0x7f747415c0a0 2024-09-17T05:07:49.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.379+0000 7f746aa00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7450073200 0x7f7450075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:49.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.379+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6101+0+0 (secure 0 0 0) 0x7f746c093540 con 0x7f747415c0a0 2024-09-17T05:07:49.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.379+0000 7f7471e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7450073200 0x7f7450075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:49.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.379+0000 7f7471e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7450073200 0x7f7450075710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74640af000 tx=0x7f746401afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:49.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.383+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f746c05dd70 con 0x7f747415c0a0 2024-09-17T05:07:49.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74740b1c50 con 0x7f747415c0a0 2024-09-17T05:07:49.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f746aa00640 1 -- 172.21.15.16:0/922294488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f746c0627a0 con 0x7f747415c0a0 2024-09-17T05:07:49.593 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:07:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7450073200 msgr2=0x7f7450075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7450073200 0x7f7450075710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74640af000 tx=0x7f746401afd0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 msgr2=0x7f7474140320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 0x7f7474140320 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c3f0 tx=0x7f746c00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 shutdown_connections 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7450073200 0x7f7450075710 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f747415c0a0 0x7f7474140320 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f74740b2f50 0x7f747413fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 --2- 172.21.15.16:0/922294488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7474073240 0x7f747413e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 >> 172.21.15.16:0/922294488 conn(0x7f747406f900 msgr2=0x7f747406d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 shutdown_connections 2024-09-17T05:07:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49.595+0000 7f7478e00640 1 -- 172.21.15.16:0/922294488 wait complete. 2024-09-17T05:07:49.615 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:07:49.616 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:07:49.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:49.617 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:07:49.626 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42023sec preferred_lft 42023sec 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:07:49.627 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:07:49.628 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.629 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:07:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:07:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:07:49.629 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:49.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:49.630 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:07:49.690 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:07:49.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:49.692 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:07:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:07:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:07:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:07:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:07:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:07:49.750 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:07:49.750 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:07:49.751 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:07:49.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:07:49.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:07:49.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:07:49.751 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:07:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:07:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:07:49.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:49.893 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:07:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:07:49.902 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:07:49.903 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:07:49.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:49.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:07:49.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:07:49.968 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:07:50.020 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:07:50.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:07:50.021 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:07:50.072 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:07:50.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:50.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:07:50.177 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:07:50.177 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:07:50.177 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:07:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:07:50.195 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:07:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:07:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:07:50.196 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:07:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:07:50.211 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:07:50.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:50.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:07:50.232 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:07:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:07:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:07:50.340 INFO:teuthology.orchestra.run.smithi016.stdout:client7158 2024-09-17T05:07:50.341 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:50.341 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:50.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:07:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 -- 172.21.15.16:0/566890636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623015f150 msgr2=0x7f623015f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 --2- 172.21.15.16:0/566890636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623015f150 0x7f623015f560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f62300ccfb0 tx=0x7f622801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 -- 172.21.15.16:0/566890636 shutdown_connections 2024-09-17T05:07:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 --2- 172.21.15.16:0/566890636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623015f150 0x7f623015f560 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 --2- 172.21.15.16:0/566890636 >> v2:172.21.15.104:3300/0 conn(0x7f623015c0a0 0x7f623015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 --2- 172.21.15.16:0/566890636 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f62300b2f50 0x7f623015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 -- 172.21.15.16:0/566890636 >> 172.21.15.16:0/566890636 conn(0x7f623006f900 msgr2=0x7f62300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 -- 172.21.15.16:0/566890636 shutdown_connections 2024-09-17T05:07:50.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 -- 172.21.15.16:0/566890636 wait complete. 2024-09-17T05:07:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.551+0000 7f6237000640 1 Processor -- start 2024-09-17T05:07:50.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- start start 2024-09-17T05:07:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 0x7f62301f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f623015c0a0 0x7f62300d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f623015f150 0x7f62300d0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f623006e430 con 0x7f62300b2f50 2024-09-17T05:07:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f623006e2b0 con 0x7f623015f150 2024-09-17T05:07:50.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f623006e5b0 con 0x7f623015c0a0 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 0x7f62301f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 0x7f62301f9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60616/0 (socket says 172.21.15.16:60616) 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 -- 172.21.15.16:0/4002221048 learned_addr learned my addr 172.21.15.16:0/4002221048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6227e00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f623015c0a0 0x7f62300d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6235000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f623015f150 0x7f62300d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 -- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f623015c0a0 msgr2=0x7f62300d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:50.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f623015c0a0 0x7f62300d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 -- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f623015f150 msgr2=0x7f62300d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f623015f150 0x7f62300d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 -- 172.21.15.16:0/4002221048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6230136dc0 con 0x7f62300b2f50 2024-09-17T05:07:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6227e00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f623015c0a0 0x7f62300d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6235000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f623015f150 0x7f62300d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622fe00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 0x7f62301f9560 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f622000c940 tx=0x7f622000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220007bb0 con 0x7f62300b2f50 2024-09-17T05:07:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6220007e40 con 0x7f62300b2f50 2024-09-17T05:07:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62301370a0 con 0x7f62300b2f50 2024-09-17T05:07:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622000f690 con 0x7f62300b2f50 2024-09-17T05:07:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6230137660 con 0x7f62300b2f50 2024-09-17T05:07:50.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.555+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623015c4b0 con 0x7f62300b2f50 2024-09-17T05:07:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.559+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f622000f830 con 0x7f62300b2f50 2024-09-17T05:07:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.559+0000 7f622cc00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f620c073200 0x7f620c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.559+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6217+0+0 (secure 0 0 0) 0x7f6220094c10 con 0x7f62300b2f50 2024-09-17T05:07:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.559+0000 7f6227e00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f620c073200 0x7f620c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.559+0000 7f6227e00640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f620c073200 0x7f620c075710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f62301561a0 tx=0x7f6218005d20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.563+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6220014030 con 0x7f62300b2f50 2024-09-17T05:07:50.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f623014d810 con 0x7f62300b2f50 2024-09-17T05:07:50.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f622cc00640 1 -- 172.21.15.16:0/4002221048 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f622005f9a0 con 0x7f62300b2f50 2024-09-17T05:07:50.781 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:07:50.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f620c073200 msgr2=0x7f620c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:50.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f620c073200 0x7f620c075710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f62301561a0 tx=0x7f6218005d20 comp rx=0 tx=0).stop 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 msgr2=0x7f62301f9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 0x7f62301f9560 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f622000c940 tx=0x7f622000ce00 comp rx=0 tx=0).stop 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 shutdown_connections 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f620c073200 0x7f620c075710 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f623015f150 0x7f62300d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f623015c0a0 0x7f62300d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 --2- 172.21.15.16:0/4002221048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62300b2f50 0x7f62301f9560 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 >> 172.21.15.16:0/4002221048 conn(0x7f623006f900 msgr2=0x7f62300cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 shutdown_connections 2024-09-17T05:07:50.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:50.783+0000 7f6237000640 1 -- 172.21.15.16:0/4002221048 wait complete. 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:07:50.804 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:07:50.805 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:07:50.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:50.805 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:07:50.816 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:07:50.817 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.818 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:07:50.819 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:07:50.819 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:07:50.819 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:07:50.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:50.820 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:07:50.883 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:07:50.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:50.886 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:07:50.941 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:07:50.941 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:07:50.942 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:07:50.943 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:07:50.943 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:07:50.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:07:50.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:07:50.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:07:50.943 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:07:51.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:07:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:07:51.078 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:07:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:07:51.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:51.086 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:07:51.095 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:07:51.095 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:07:51.095 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:07:51.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:51.096 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:07:51.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:07:51.157 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:07:51.208 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:07:51.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:07:51.208 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:07:51.259 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:07:51.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:51.260 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:07:51.360 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:07:51.360 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:07:51.360 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:07:51.378 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:07:51.378 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:07:51.378 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:07:51.378 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:07:51.378 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:07:51.388 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:07:51.388 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:07:51.388 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:07:51.388 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:07:51.388 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:07:51.389 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:07:51.389 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:07:51.389 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:07:51.392 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:07:51.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:51.392 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:07:51.411 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:07:51.498 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:07:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:07:51.501 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:07:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:07:51.507 INFO:teuthology.orchestra.run.smithi104.stdout:client7167 2024-09-17T05:07:51.508 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:51.508 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:51.517 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:07:51.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:07:51.603 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:07:51.691 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:07:51.691 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:51.691 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:51.691 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:51.700 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:51.700 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:51.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 -- 172.21.15.16:0/2647085511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f768815ba50 msgr2=0x7f768815be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 --2- 172.21.15.16:0/2647085511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f768815ba50 0x7f768815be60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f7688072860 tx=0x7f768001cd80 comp rx=0 tx=0).stop 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 -- 172.21.15.16:0/2647085511 shutdown_connections 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 --2- 172.21.15.16:0/2647085511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f768815ba50 0x7f768815be60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 --2- 172.21.15.16:0/2647085511 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7688074e30 0x7f768815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 --2- 172.21.15.16:0/2647085511 >> v2:172.21.15.104:3300/0 conn(0x7f7688074a20 0x7f768815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:51.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 -- 172.21.15.16:0/2647085511 >> 172.21.15.16:0/2647085511 conn(0x7f76880b2250 msgr2=0x7f76880d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 -- 172.21.15.16:0/2647085511 shutdown_connections 2024-09-17T05:07:51.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.959+0000 7f7690400640 1 -- 172.21.15.16:0/2647085511 wait complete. 2024-09-17T05:07:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 Processor -- start 2024-09-17T05:07:51.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 -- start start 2024-09-17T05:07:51.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688074a20 0x7f76880c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:51.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 0x7f76880c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:51.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f768815ba50 0x7f76881516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:51.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76880cf310 con 0x7f7688074e30 2024-09-17T05:07:51.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76880cf190 con 0x7f7688074a20 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f7690400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f76881681d0 con 0x7f768815ba50 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f768d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 0x7f76880c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f768d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 0x7f76880c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60624/0 (socket says 172.21.15.16:60624) 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f768d000640 1 -- 172.21.15.16:0/2946597955 learned_addr learned my addr 172.21.15.16:0/2946597955 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f768e400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f768815ba50 0x7f76881516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f768da00640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688074a20 0x7f76880c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.963+0000 7f768d000640 1 -- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f768815ba50 msgr2=0x7f76881516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f768d000640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f768815ba50 0x7f76881516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f768d000640 1 -- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688074a20 msgr2=0x7f76880c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f768d000640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688074a20 0x7f76880c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:51.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f768d000640 1 -- 172.21.15.16:0/2946597955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76880c63d0 con 0x7f7688074e30 2024-09-17T05:07:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f768d000640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 0x7f76880c8040 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f767400d720 tx=0x7f767400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674017070 con 0x7f7688074e30 2024-09-17T05:07:51.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7674009cc0 con 0x7f7688074e30 2024-09-17T05:07:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674005280 con 0x7f7688074e30 2024-09-17T05:07:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7688150610 con 0x7f7688074e30 2024-09-17T05:07:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7688150bd0 con 0x7f7688074e30 2024-09-17T05:07:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768815be60 con 0x7f7688074e30 2024-09-17T05:07:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.967+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7674005420 con 0x7f7688074e30 2024-09-17T05:07:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.971+0000 7f7685600640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7668073230 0x7f7668075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.971+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f7674094830 con 0x7f7688074e30 2024-09-17T05:07:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.971+0000 7f768da00640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7668073230 0x7f7668075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.971+0000 7f768da00640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7668073230 0x7f7668075740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f767c010de0 tx=0x7f767c012040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:51.971+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f767405f070 con 0x7f7688074e30 2024-09-17T05:07:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.159+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7688155780 con 0x7f7688074e30 2024-09-17T05:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.159+0000 7f7685600640 1 -- 172.21.15.16:0/2946597955 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f7674063aa0 con 0x7f7688074e30 2024-09-17T05:07:52.159 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7668073230 msgr2=0x7f7668075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7668073230 0x7f7668075740 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f767c010de0 tx=0x7f767c012040 comp rx=0 tx=0).stop 2024-09-17T05:07:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 msgr2=0x7f76880c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 0x7f76880c8040 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f767400d720 tx=0x7f767400dbe0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 shutdown_connections 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7668073230 0x7f7668075740 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f768815ba50 0x7f76881516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7688074e30 0x7f76880c8040 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 --2- 172.21.15.16:0/2946597955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688074a20 0x7f76880c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 >> 172.21.15.16:0/2946597955 conn(0x7f76880b2250 msgr2=0x7f768815d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 shutdown_connections 2024-09-17T05:07:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.163+0000 7f7690400640 1 -- 172.21.15.16:0/2946597955 wait complete. 2024-09-17T05:07:52.182 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:07:52.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:52.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:07:52.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:07:52.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:52.208 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:07:52.260 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:52.261 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:52.261 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:52.270 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:52.270 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:52.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 -- 172.21.15.16:0/866239364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a00cce00 msgr2=0x7f90a00cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 --2- 172.21.15.16:0/866239364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a00cce00 0x7f90a00cd210 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9090009900 tx=0x7f909001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 -- 172.21.15.16:0/866239364 shutdown_connections 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 --2- 172.21.15.16:0/866239364 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 0x7f90a00741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 --2- 172.21.15.16:0/866239364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a00cce00 0x7f90a00cd210 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 -- 172.21.15.16:0/866239364 >> 172.21.15.16:0/866239364 conn(0x7f90a00b2dd0 msgr2=0x7f90a00b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 -- 172.21.15.16:0/866239364 shutdown_connections 2024-09-17T05:07:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 -- 172.21.15.16:0/866239364 wait complete. 2024-09-17T05:07:52.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 Processor -- start 2024-09-17T05:07:52.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.483+0000 7f90a7200640 1 -- start start 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 0x7f90a00b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a00cce00 0x7f90a01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a015f040 0x7f90a01555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0168d50 con 0x7f90a015f040 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0168bd0 con 0x7f90a00cce00 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f90a0168ed0 con 0x7f90a0073dc0 2024-09-17T05:07:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a015f040 0x7f90a01555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f909fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 0x7f90a00b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f909fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 0x7f90a00b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:35882/0 (socket says 172.21.15.16:35882) 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f909fe00640 1 -- 172.21.15.16:0/639204447 learned_addr learned my addr 172.21.15.16:0/639204447 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f9097e00640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a00cce00 0x7f90a01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 -- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 msgr2=0x7f90a00b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 0x7f90a00b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 -- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a00cce00 msgr2=0x7f90a01550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a00cce00 0x7f90a01550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 -- 172.21.15.16:0/639204447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a014ae20 con 0x7f90a015f040 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a5200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a015f040 0x7f90a01555e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f909800c650 tx=0x7f909800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098003b80 con 0x7f90a015f040 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9098003d20 con 0x7f90a015f040 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098005330 con 0x7f90a015f040 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a014b100 con 0x7f90a015f040 2024-09-17T05:07:52.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a014b6c0 con 0x7f90a015f040 2024-09-17T05:07:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.487+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a00741d0 con 0x7f90a015f040 2024-09-17T05:07:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.491+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f90980054d0 con 0x7f90a015f040 2024-09-17T05:07:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.491+0000 7f909cc00640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f907c073230 0x7f907c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.491+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f9098094a40 con 0x7f90a015f040 2024-09-17T05:07:52.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.491+0000 7f909fe00640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f907c073230 0x7f907c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.491+0000 7f909fe00640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f907c073230 0x7f907c075740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f909001af90 tx=0x7f909001b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:52.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.495+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f90a0074710 con 0x7f90a015f040 2024-09-17T05:07:52.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.687+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90a015f7d0 con 0x7f90a015f040 2024-09-17T05:07:52.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.687+0000 7f909cc00640 1 -- 172.21.15.16:0/639204447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) ==== 80+0+0 (secure 0 0 0) 0x7f909805f280 con 0x7f90a015f040 2024-09-17T05:07:52.687 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:07:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f907c073230 msgr2=0x7f907c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f907c073230 0x7f907c075740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f909001af90 tx=0x7f909001b120 comp rx=0 tx=0).stop 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a015f040 msgr2=0x7f90a01555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a015f040 0x7f90a01555e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f909800c650 tx=0x7f909800cb10 comp rx=0 tx=0).stop 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 shutdown_connections 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f907c073230 0x7f907c075740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a015f040 0x7f90a01555e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90a00cce00 0x7f90a01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 --2- 172.21.15.16:0/639204447 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f90a0073dc0 0x7f90a00b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 >> 172.21.15.16:0/639204447 conn(0x7f90a00b2dd0 msgr2=0x7f90a006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 shutdown_connections 2024-09-17T05:07:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:52.691+0000 7f90a7200640 1 -- 172.21.15.16:0/639204447 wait complete. 2024-09-17T05:07:52.709 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:07:52.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:52.710 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:07:52.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:07:52.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:07:52.738 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:07:52.789 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:52.789 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:07:52.789 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:52.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:07:52.801 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:07:52.801 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:07:52.801 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:07:52.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:07:52.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 -- 172.21.15.16:0/2164084420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 msgr2=0x7f137815e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 --2- 172.21.15.16:0/2164084420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f137815e7c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1368009900 tx=0x7f136801c850 comp rx=0 tx=0).stop 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 -- 172.21.15.16:0/2164084420 shutdown_connections 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 --2- 172.21.15.16:0/2164084420 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780b4980 0x7f13780738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 --2- 172.21.15.16:0/2164084420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f137815e7c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 -- 172.21.15.16:0/2164084420 >> 172.21.15.16:0/2164084420 conn(0x7f137806f900 msgr2=0x7f13780b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:53.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 -- 172.21.15.16:0/2164084420 shutdown_connections 2024-09-17T05:07:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.003+0000 7f137d800640 1 -- 172.21.15.16:0/2164084420 wait complete. 2024-09-17T05:07:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 Processor -- start 2024-09-17T05:07:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 -- start start 2024-09-17T05:07:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13780734e0 0x7f13781fb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f13781f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780aa970 0x7f13780a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1378165940 con 0x7f137815ac90 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13780cdf00 con 0x7f13780734e0 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1378165ac0 con 0x7f13780aa970 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f13781f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f13781f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60666/0 (socket says 172.21.15.16:60666) 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 -- 172.21.15.16:0/2404522551 learned_addr learned my addr 172.21.15.16:0/2404522551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f1377400640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780aa970 0x7f13780a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f1376a00640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13780734e0 0x7f13781fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 -- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780aa970 msgr2=0x7f13780a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780aa970 0x7f13780a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 -- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13780734e0 msgr2=0x7f13781fb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13780734e0 0x7f13781fb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 -- 172.21.15.16:0/2404522551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13781fc360 con 0x7f137815ac90 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f1377400640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780aa970 0x7f13780a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f1376a00640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13780734e0 0x7f13781fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136e000640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f13781f8da0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f136000c3c0 tx=0x7f136000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600052d0 con 0x7f137815ac90 2024-09-17T05:07:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1360007020 con 0x7f137815ac90 2024-09-17T05:07:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360007470 con 0x7f137815ac90 2024-09-17T05:07:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13781fc640 con 0x7f137815ac90 2024-09-17T05:07:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.007+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781f8880 con 0x7f137815ac90 2024-09-17T05:07:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.011+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13781fc0d0 con 0x7f137815ac90 2024-09-17T05:07:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.011+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13600067e0 con 0x7f137815ac90 2024-09-17T05:07:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.011+0000 7f136f400640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1354073200 0x7f1354075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.011+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f13600938e0 con 0x7f137815ac90 2024-09-17T05:07:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.011+0000 7f1376a00640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1354073200 0x7f1354075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.011+0000 7f1376a00640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1354073200 0x7f1354075710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13680af000 tx=0x7f136801b090 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:53.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.015+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f136005e700 con 0x7f137815ac90 2024-09-17T05:07:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.223+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13781f92e0 con 0x7f137815ac90 2024-09-17T05:07:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.223+0000 7f136f400640 1 -- 172.21.15.16:0/2404522551 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7f13781f92e0 con 0x7f137815ac90 2024-09-17T05:07:53.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:53.222 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:07:37.774040+0000","modified":"2024-09-17T05:07:40.784518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2345541365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2345541365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2345541365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-17T05:07:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1354073200 msgr2=0x7f1354075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1354073200 0x7f1354075710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13680af000 tx=0x7f136801b090 comp rx=0 tx=0).stop 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 msgr2=0x7f13781f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f13781f8da0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f136000c3c0 tx=0x7f136000c880 comp rx=0 tx=0).stop 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 shutdown_connections 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1354073200 0x7f1354075710 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f13780aa970 0x7f13780a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f137815ac90 0x7f13781f8da0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 --2- 172.21.15.16:0/2404522551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13780734e0 0x7f13781fb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 >> 172.21.15.16:0/2404522551 conn(0x7f137806f900 msgr2=0x7f13780b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 shutdown_connections 2024-09-17T05:07:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.227+0000 7f137d800640 1 -- 172.21.15.16:0/2404522551 wait complete. 2024-09-17T05:07:53.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 -- 172.21.15.16:0/530908523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 msgr2=0x7ff1040cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 --2- 172.21.15.16:0/530908523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff1040cf320 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009840 tx=0x7ff0f401c820 comp rx=0 tx=0).stop 2024-09-17T05:07:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 -- 172.21.15.16:0/530908523 shutdown_connections 2024-09-17T05:07:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 --2- 172.21.15.16:0/530908523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff1040cf320 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 -- 172.21.15.16:0/530908523 >> 172.21.15.16:0/530908523 conn(0x7ff1040d0860 msgr2=0x7ff1040d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 -- 172.21.15.16:0/530908523 shutdown_connections 2024-09-17T05:07:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 -- 172.21.15.16:0/530908523 wait complete. 2024-09-17T05:07:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.407+0000 7ff10ac00640 1 Processor -- start 2024-09-17T05:07:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- start start 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff104151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1040cf860 0x7ff10414f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff10415f750 0x7ff104154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff10416a510 con 0x7ff10406ce90 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040b5a80 con 0x7ff10415f750 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff10416a690 con 0x7ff1040cf860 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff104151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff104151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60686/0 (socket says 172.21.15.16:60686) 2024-09-17T05:07:53.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 -- 172.21.15.16:0/2241698648 learned_addr learned my addr 172.21.15.16:0/2241698648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff108c00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff10415f750 0x7ff104154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff0fbe00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1040cf860 0x7ff10414f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 -- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1040cf860 msgr2=0x7ff10414f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1040cf860 0x7ff10414f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 -- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff10415f750 msgr2=0x7ff104154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff10415f750 0x7ff104154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 -- 172.21.15.16:0/2241698648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff104152480 con 0x7ff10406ce90 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff108c00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff10415f750 0x7ff104154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff0fbe00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1040cf860 0x7ff10414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff103e00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff104151cc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009790 tx=0x7ff0f40a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4004040 con 0x7ff10406ce90 2024-09-17T05:07:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0f40042d0 con 0x7ff10406ce90 2024-09-17T05:07:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4005050 con 0x7ff10406ce90 2024-09-17T05:07:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff104152760 con 0x7ff10406ce90 2024-09-17T05:07:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff104158e50 con 0x7ff10406ce90 2024-09-17T05:07:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.411+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1040cf320 con 0x7ff10406ce90 2024-09-17T05:07:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.415+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff0f40c0030 con 0x7ff10406ce90 2024-09-17T05:07:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.415+0000 7ff100c00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff0e0073200 0x7ff0e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.415+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7ff0f412e820 con 0x7ff10406ce90 2024-09-17T05:07:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.415+0000 7ff0fbe00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff0e0073200 0x7ff0e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.415+0000 7ff0fbe00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff0e0073200 0x7ff0e0075710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec001400 tx=0x7ff0ec009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.419+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff0f40c7060 con 0x7ff10406ce90 2024-09-17T05:07:53.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.627+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff104154070 con 0x7ff10406ce90 2024-09-17T05:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.627+0000 7ff100c00640 1 -- 172.21.15.16:0/2241698648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7ff0f4016410 con 0x7ff10406ce90 2024-09-17T05:07:53.628 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:53.628 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:07:37.774040+0000","modified":"2024-09-17T05:07:40.784518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2345541365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2345541365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2345541365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-17T05:07:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-09-17T05:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff0e0073200 msgr2=0x7ff0e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff0e0073200 0x7ff0e0075710 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec001400 tx=0x7ff0ec009040 comp rx=0 tx=0).stop 2024-09-17T05:07:53.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 msgr2=0x7ff104151cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff104151cc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009790 tx=0x7ff0f40a65a0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 shutdown_connections 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff0e0073200 0x7ff0e0075710 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff10415f750 0x7ff104154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1040cf860 0x7ff10414f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 --2- 172.21.15.16:0/2241698648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff10406ce90 0x7ff104151cc0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 >> 172.21.15.16:0/2241698648 conn(0x7ff1040d0860 msgr2=0x7ff1040d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 shutdown_connections 2024-09-17T05:07:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.631+0000 7ff10ac00640 1 -- 172.21.15.16:0/2241698648 wait complete. 2024-09-17T05:07:53.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 -- 172.21.15.16:0/2810000590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4244004b70 msgr2=0x7f4244004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 --2- 172.21.15.16:0/2810000590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4244004b70 0x7f4244004fe0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4250006fd0 tx=0x7f425001cee0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 -- 172.21.15.16:0/2810000590 shutdown_connections 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 --2- 172.21.15.16:0/2810000590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4244004b70 0x7f4244004fe0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 -- 172.21.15.16:0/2810000590 >> 172.21.15.16:0/2810000590 conn(0x7f426006f900 msgr2=0x7f42600b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 -- 172.21.15.16:0/2810000590 shutdown_connections 2024-09-17T05:07:53.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.811+0000 7f4265600640 1 -- 172.21.15.16:0/2810000590 wait complete. 2024-09-17T05:07:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 Processor -- start 2024-09-17T05:07:53.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 -- start start 2024-09-17T05:07:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4244004b70 0x7f426014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42600cec70 0x7f42601f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 0x7f4260150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42600b37e0 con 0x7f426014fc40 2024-09-17T05:07:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42600b3660 con 0x7f42600cec70 2024-09-17T05:07:53.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f42600b3960 con 0x7f4244004b70 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 0x7f4260150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 0x7f4260150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60692/0 (socket says 172.21.15.16:60692) 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 -- 172.21.15.16:0/1377721433 learned_addr learned my addr 172.21.15.16:0/1377721433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425ea00640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4244004b70 0x7f426014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4257e00640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42600cec70 0x7f42601f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 -- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4244004b70 msgr2=0x7f426014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4244004b70 0x7f426014b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 -- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42600cec70 msgr2=0x7f42601f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42600cec70 0x7f42601f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 -- 172.21.15.16:0/1377721433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426014d0c0 con 0x7f426014fc40 2024-09-17T05:07:53.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f425f400640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 0x7f4260150050 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f42600b5380 tx=0x7f425800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258003d90 con 0x7f426014fc40 2024-09-17T05:07:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4258004b30 con 0x7f426014fc40 2024-09-17T05:07:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258004030 con 0x7f426014fc40 2024-09-17T05:07:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4260150590 con 0x7f426014fc40 2024-09-17T05:07:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.815+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426014cde0 con 0x7f426014fc40 2024-09-17T05:07:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.819+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426015a630 con 0x7f426014fc40 2024-09-17T05:07:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.819+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f42580041d0 con 0x7f426014fc40 2024-09-17T05:07:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.819+0000 7f4256a00640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f423c073230 0x7f423c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.819+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f42580951b0 con 0x7f426014fc40 2024-09-17T05:07:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.819+0000 7f425ea00640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f423c073230 0x7f423c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.823+0000 7f425ea00640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f423c073230 0x7f423c075740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42500ab000 tx=0x7f4250019d20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.823+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4258020020 con 0x7f426014fc40 2024-09-17T05:07:54.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:53.999+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4260069770 con 0x7f426014fc40 2024-09-17T05:07:54.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4256a00640 1 -- 172.21.15.16:0/1377721433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+17135 (secure 0 0 0) 0x7f4260150590 con 0x7f426014fc40 2024-09-17T05:07:54.001 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:54.001 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":102,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:50.638383+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:07:35.523800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T05:07:36.731373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.c","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.37","mappings":[{"from":4,"to":6}]},{"pgid":"16.3a","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:54.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f423c073230 msgr2=0x7f423c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f423c073230 0x7f423c075740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42500ab000 tx=0x7f4250019d20 comp rx=0 tx=0).stop 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 msgr2=0x7f4260150050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 0x7f4260150050 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f42600b5380 tx=0x7f425800f790 comp rx=0 tx=0).stop 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 shutdown_connections 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f423c073230 0x7f423c075740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f426014fc40 0x7f4260150050 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42600cec70 0x7f42601f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 --2- 172.21.15.16:0/1377721433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4244004b70 0x7f426014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 >> 172.21.15.16:0/1377721433 conn(0x7f426006f900 msgr2=0x7f4260150f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:54.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 shutdown_connections 2024-09-17T05:07:54.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.003+0000 7f4265600640 1 -- 172.21.15.16:0/1377721433 wait complete. 2024-09-17T05:07:54.025 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:07:54.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 -- 172.21.15.16:0/3774327400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff0003c20 msgr2=0x7f4ff0004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 --2- 172.21.15.16:0/3774327400 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ff0003c20 0x7f4ff0004070 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc006fa0 tx=0x7f4ffc01ced0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5010a00640 1 -- 172.21.15.16:0/3774327400 reap_dead start 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 -- 172.21.15.16:0/3774327400 shutdown_connections 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 -- 172.21.15.16:0/3774327400 >> 172.21.15.16:0/3774327400 conn(0x7f500c0d10f0 msgr2=0x7f500c0d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 -- 172.21.15.16:0/3774327400 shutdown_connections 2024-09-17T05:07:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 -- 172.21.15.16:0/3774327400 wait complete. 2024-09-17T05:07:54.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 Processor -- start 2024-09-17T05:07:54.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.187+0000 7f5011400640 1 -- start start 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ff0003c20 0x7f500c1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f500c154940 0x7f500c1648a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 0x7f500c164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f500c0b4470 con 0x7f500c1fb460 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f500c15fb50 con 0x7f4ff0003c20 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f500c0b45f0 con 0x7f500c154940 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 0x7f500c164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 0x7f500c164de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60696/0 (socket says 172.21.15.16:60696) 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 -- 172.21.15.16:0/2365881615 learned_addr learned my addr 172.21.15.16:0/2365881615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5003e00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f500c154940 0x7f500c1648a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f500be00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ff0003c20 0x7f500c1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 -- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f500c154940 msgr2=0x7f500c1648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f500c154940 0x7f500c1648a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 -- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ff0003c20 msgr2=0x7f500c1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ff0003c20 0x7f500c1535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 -- 172.21.15.16:0/2365881615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c1f8160 con 0x7f500c1fb460 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5010a00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 0x7f500c164de0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5004006fd0 tx=0x7f5004006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004011900 con 0x7f500c1fb460 2024-09-17T05:07:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5004018070 con 0x7f500c1fb460 2024-09-17T05:07:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500c1f7e70 con 0x7f500c1fb460 2024-09-17T05:07:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004011d60 con 0x7f500c1fb460 2024-09-17T05:07:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f500c1f8720 con 0x7f500c1fb460 2024-09-17T05:07:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f500c1f8de0 con 0x7f500c1fb460 2024-09-17T05:07:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.191+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f500401d460 con 0x7f500c1fb460 2024-09-17T05:07:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.195+0000 7f5008c00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fe8073230 0x7f4fe8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.195+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f5004094d60 con 0x7f500c1fb460 2024-09-17T05:07:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.195+0000 7f500be00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fe8073230 0x7f4fe8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.195+0000 7f500be00640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fe8073230 0x7f4fe8075740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0ac000 tx=0x7f4ffc019d10 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.199+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f500c1f8de0 con 0x7f500c1fb460 2024-09-17T05:07:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.411+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f500c071210 con 0x7f500c1fb460 2024-09-17T05:07:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.411+0000 7f5008c00640 1 -- 172.21.15.16:0/2365881615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7f500405fb70 con 0x7f500c1fb460 2024-09-17T05:07:54.412 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:54.412 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:07:37.774040+0000","modified":"2024-09-17T05:07:40.784518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2345541365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2345541365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2345541365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-17T05:07:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fe8073230 msgr2=0x7f4fe8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fe8073230 0x7f4fe8075740 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc0ac000 tx=0x7f4ffc019d10 comp rx=0 tx=0).stop 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 msgr2=0x7f500c164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 0x7f500c164de0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f5004006fd0 tx=0x7f5004006430 comp rx=0 tx=0).stop 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 shutdown_connections 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fe8073230 0x7f4fe8075740 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f500c1fb460 0x7f500c164de0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f500c154940 0x7f500c1648a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 --2- 172.21.15.16:0/2365881615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ff0003c20 0x7f500c1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 >> 172.21.15.16:0/2365881615 conn(0x7f500c0d10f0 msgr2=0x7f500c15d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 shutdown_connections 2024-09-17T05:07:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.415+0000 7f5011400640 1 -- 172.21.15.16:0/2365881615 wait complete. 2024-09-17T05:07:54.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:07:54.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 -- 172.21.15.16:0/3733574543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc15c0a0 msgr2=0x7f24fc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 --2- 172.21.15.16:0/3733574543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc15c0a0 0x7f24fc15c4b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f24fc071ce0 tx=0x7f24e401c830 comp rx=0 tx=0).stop 2024-09-17T05:07:54.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 -- 172.21.15.16:0/3733574543 shutdown_connections 2024-09-17T05:07:54.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 --2- 172.21.15.16:0/3733574543 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24fc15f150 0x7f24fc15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 --2- 172.21.15.16:0/3733574543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc15c0a0 0x7f24fc15c4b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 --2- 172.21.15.16:0/3733574543 >> v2:172.21.15.104:3300/0 conn(0x7f24fc0b2f50 0x7f24fc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 -- 172.21.15.16:0/3733574543 >> 172.21.15.16:0/3733574543 conn(0x7f24fc06f900 msgr2=0x7f24fc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:54.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 -- 172.21.15.16:0/3733574543 shutdown_connections 2024-09-17T05:07:54.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 -- 172.21.15.16:0/3733574543 wait complete. 2024-09-17T05:07:54.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 Processor -- start 2024-09-17T05:07:54.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.595+0000 7f2503800640 1 -- start start 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 0x7f24fc164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24fc15c0a0 0x7f24fc150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24fc15f150 0x7f24fc163f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc06e3d0 con 0x7f24fc0b2f50 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f24fc06e250 con 0x7f24fc15f150 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f24fc06e550 con 0x7f24fc15c0a0 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 0x7f24fc164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 0x7f24fc164d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60710/0 (socket says 172.21.15.16:60710) 2024-09-17T05:07:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 -- 172.21.15.16:0/2557718296 learned_addr learned my addr 172.21.15.16:0/2557718296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f24f3e00640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24fc15c0a0 0x7f24fc150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2501800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24fc15f150 0x7f24fc163f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 -- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24fc15c0a0 msgr2=0x7f24fc150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24fc15c0a0 0x7f24fc150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 -- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24fc15f150 msgr2=0x7f24fc163f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24fc15f150 0x7f24fc163f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 -- 172.21.15.16:0/2557718296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24fc161850 con 0x7f24fc0b2f50 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2500e00640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 0x7f24fc164d40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005950 tx=0x7f24ec00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec007bb0 con 0x7f24fc0b2f50 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24ec007e40 con 0x7f24fc0b2f50 2024-09-17T05:07:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24fc161b30 con 0x7f24fc0b2f50 2024-09-17T05:07:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00f690 con 0x7f24fc0b2f50 2024-09-17T05:07:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24fc162700 con 0x7f24fc0b2f50 2024-09-17T05:07:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.599+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24fc162240 con 0x7f24fc0b2f50 2024-09-17T05:07:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.603+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24ec00f830 con 0x7f24fc0b2f50 2024-09-17T05:07:54.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.603+0000 7f24f9600640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f24d8073230 0x7f24d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.603+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f24ec094b20 con 0x7f24fc0b2f50 2024-09-17T05:07:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.603+0000 7f24f3e00640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f24d8073230 0x7f24d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.603+0000 7f24f3e00640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f24d8073230 0x7f24d8075740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f24e40af000 tx=0x7f24e401b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:54.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.607+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f24fc162240 con 0x7f24fc0b2f50 2024-09-17T05:07:54.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24fc15c4b0 con 0x7f24fc0b2f50 2024-09-17T05:07:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f24f9600640 1 -- 172.21.15.16:0/2557718296 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5231 (secure 0 0 0) 0x7f24ec014030 con 0x7f24fc0b2f50 2024-09-17T05:07:54.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:54.817 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"btime":"2024-09-17T05:07:41:368078+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:07:37.774040+0000","modified":"2024-09-17T05:07:40.784518+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6993},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6993":{"gid":6993,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2345541365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2345541365},{"type":"v1","addr":"172.21.15.16:6835","nonce":2345541365}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6993,"qdb_cluster":[6993]},"id":7}]} 2024-09-17T05:07:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 66 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f24d8073230 msgr2=0x7f24d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f24d8073230 0x7f24d8075740 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f24e40af000 tx=0x7f24e401b070 comp rx=0 tx=0).stop 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 msgr2=0x7f24fc164d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 0x7f24fc164d40 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f24ec005950 tx=0x7f24ec00cdd0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 shutdown_connections 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f24d8073230 0x7f24d8075740 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24fc15f150 0x7f24fc163f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f24fc15c0a0 0x7f24fc150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 --2- 172.21.15.16:0/2557718296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24fc0b2f50 0x7f24fc164d40 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 >> 172.21.15.16:0/2557718296 conn(0x7f24fc06f900 msgr2=0x7f24fc0ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 shutdown_connections 2024-09-17T05:07:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.819+0000 7f2503800640 1 -- 172.21.15.16:0/2557718296 wait complete. 2024-09-17T05:07:54.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 -- 172.21.15.16:0/3194942648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b40cf860 msgr2=0x7f79b415f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 --2- 172.21.15.16:0/3194942648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b40cf860 0x7f79b415f040 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f79b4072540 tx=0x7f79a401c890 comp rx=0 tx=0).stop 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 -- 172.21.15.16:0/3194942648 shutdown_connections 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 --2- 172.21.15.16:0/3194942648 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b415f580 0x7f79b415f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 --2- 172.21.15.16:0/3194942648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b40cf860 0x7f79b415f040 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 --2- 172.21.15.16:0/3194942648 >> v2:172.21.15.104:3300/0 conn(0x7f79b406ce90 0x7f79b40cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 -- 172.21.15.16:0/3194942648 >> 172.21.15.16:0/3194942648 conn(0x7f79b40d0860 msgr2=0x7f79b40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 -- 172.21.15.16:0/3194942648 shutdown_connections 2024-09-17T05:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:54.999+0000 7f79bc800640 1 -- 172.21.15.16:0/3194942648 wait complete. 2024-09-17T05:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 Processor -- start 2024-09-17T05:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 -- start start 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b406ce90 0x7f79b4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b40cf860 0x7f79b415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 0x7f79b41ee120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40b59f0 con 0x7f79b415f580 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79b40b5870 con 0x7f79b406ce90 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f79b40b4330 con 0x7f79b40cf860 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 0x7f79b41ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 0x7f79b41ee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60736/0 (socket says 172.21.15.16:60736) 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 -- 172.21.15.16:0/2964900566 learned_addr learned my addr 172.21.15.16:0/2964900566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79b9400640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b40cf860 0x7f79b415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79b9e00640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b406ce90 0x7f79b4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 -- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b40cf860 msgr2=0x7f79b415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b40cf860 0x7f79b415e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 -- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b406ce90 msgr2=0x7f79b4154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b406ce90 0x7f79b4154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 -- 172.21.15.16:0/2964900566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b41ed080 con 0x7f79b415f580 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ba800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 0x7f79b41ee120 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f79b000e970 tx=0x7f79b000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0004d30 con 0x7f79b415f580 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79b000cb20 con 0x7f79b415f580 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b41ed360 con 0x7f79b415f580 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b0005020 con 0x7f79b415f580 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.003+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b41ed920 con 0x7f79b415f580 2024-09-17T05:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.007+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b40b48c0 con 0x7f79b415f580 2024-09-17T05:07:55.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.007+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f79b00051c0 con 0x7f79b415f580 2024-09-17T05:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.007+0000 7f79ae000640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7998073230 0x7f7998075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.007+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f79b0094b80 con 0x7f79b415f580 2024-09-17T05:07:55.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.007+0000 7f79b9e00640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7998073230 0x7f7998075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.011+0000 7f79b9e00640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7998073230 0x7f7998075740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79a800bb50 tx=0x7f79a800b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:55.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.011+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f79b0016030 con 0x7f79b415f580 2024-09-17T05:07:55.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.187+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79b41658b0 con 0x7f79b415f580 2024-09-17T05:07:55.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.187+0000 7f79ae000640 1 -- 172.21.15.16:0/2964900566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+17135 (secure 0 0 0) 0x7f79b005f350 con 0x7f79b415f580 2024-09-17T05:07:55.187 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:07:55.187 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":102,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:50.638383+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:07:35.523800+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T05:07:36.731373+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.c","mappings":[{"from":4,"to":3}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.37","mappings":[{"from":4,"to":6}]},{"pgid":"16.3a","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:07:55.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7998073230 msgr2=0x7f7998075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7998073230 0x7f7998075740 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f79a800bb50 tx=0x7f79a800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 msgr2=0x7f79b41ee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 0x7f79b41ee120 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f79b000e970 tx=0x7f79b000ee30 comp rx=0 tx=0).stop 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 shutdown_connections 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7998073230 0x7f7998075740 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79b415f580 0x7f79b41ee120 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f79b40cf860 0x7f79b415e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 --2- 172.21.15.16:0/2964900566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79b406ce90 0x7f79b4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 >> 172.21.15.16:0/2964900566 conn(0x7f79b40d0860 msgr2=0x7f79b4164560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 shutdown_connections 2024-09-17T05:07:55.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.191+0000 7f79bc800640 1 -- 172.21.15.16:0/2964900566 wait complete. 2024-09-17T05:07:55.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 -- 172.21.15.16:0/3707666104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 msgr2=0x7f66701590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 --2- 172.21.15.16:0/3707666104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f66701590a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f66700cff70 tx=0x7f666401c740 comp rx=0 tx=0).stop 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 -- 172.21.15.16:0/3707666104 shutdown_connections 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 --2- 172.21.15.16:0/3707666104 >> v2:172.21.15.104:3300/0 conn(0x7f66700ca6b0 0x7f66700caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 --2- 172.21.15.16:0/3707666104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f66701590a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 -- 172.21.15.16:0/3707666104 >> 172.21.15.16:0/3707666104 conn(0x7f66700b2250 msgr2=0x7f66700d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 -- 172.21.15.16:0/3707666104 shutdown_connections 2024-09-17T05:07:55.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 -- 172.21.15.16:0/3707666104 wait complete. 2024-09-17T05:07:55.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 Processor -- start 2024-09-17T05:07:55.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.371+0000 7f6678200640 1 -- start start 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f66700ca6b0 0x7f66701fac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f6670156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667015f100 0x7f66700b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6670168e10 con 0x7f66700d0940 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6670168c90 con 0x7f667015f100 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6670168f90 con 0x7f66700ca6b0 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f6670156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f6670156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60744/0 (socket says 172.21.15.16:60744) 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 -- 172.21.15.16:0/1654515337 learned_addr learned my addr 172.21.15.16:0/1654515337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6675800640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f66700ca6b0 0x7f66701fac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6676200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667015f100 0x7f66700b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 -- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f66700ca6b0 msgr2=0x7f66701fac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f66700ca6b0 0x7f66701fac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 -- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667015f100 msgr2=0x7f66700b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667015f100 0x7f66700b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 -- 172.21.15.16:0/1654515337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701fb150 con 0x7f66700d0940 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6675800640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f66700ca6b0 0x7f66701fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6674e00640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f6670156220 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f665c00d6a0 tx=0x7f665c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c017070 con 0x7f66700d0940 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f665c004310 con 0x7f66700d0940 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c005280 con 0x7f66700d0940 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667015dae0 con 0x7f66700d0940 2024-09-17T05:07:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66700c6cb0 con 0x7f66700d0940 2024-09-17T05:07:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.375+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66700caac0 con 0x7f66700d0940 2024-09-17T05:07:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.379+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f665c005420 con 0x7f66700d0940 2024-09-17T05:07:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.379+0000 7f666d600640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6650073200 0x7f6650075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.379+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6217+0+0 (secure 0 0 0) 0x7f665c094850 con 0x7f66700d0940 2024-09-17T05:07:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.379+0000 7f6675800640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6650073200 0x7f6650075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.379+0000 7f6675800640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6650073200 0x7f6650075710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f66640af000 tx=0x7f666401af80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.383+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f665c020020 con 0x7f66700d0940 2024-09-17T05:07:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.595+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f66700c6a40 con 0x7f66700d0940 2024-09-17T05:07:55.593 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:07:55.595+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:07:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.595+0000 7f666d600640 1 -- 172.21.15.16:0/1654515337 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7f66701fb150 con 0x7f66700d0940 2024-09-17T05:07:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6650073200 msgr2=0x7f6650075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6650073200 0x7f6650075710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f66640af000 tx=0x7f666401af80 comp rx=0 tx=0).stop 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 msgr2=0x7f6670156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f6670156220 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f665c00d6a0 tx=0x7f665c00db60 comp rx=0 tx=0).stop 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 shutdown_connections 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6650073200 0x7f6650075710 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f667015f100 0x7f66700b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66700d0940 0x7f6670156220 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 --2- 172.21.15.16:0/1654515337 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f66700ca6b0 0x7f66701fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 >> 172.21.15.16:0/1654515337 conn(0x7f66700b2250 msgr2=0x7f66700ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 shutdown_connections 2024-09-17T05:07:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.599+0000 7f6678200640 1 -- 172.21.15.16:0/1654515337 wait complete. 2024-09-17T05:07:55.605 INFO:tasks.ceph.mds.c.smithi016.stderr: -2361> 2024-09-17T05:07:40.327+0000 7f3664000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:07:55.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:07:55.723 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:07:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 -- 172.21.15.16:0/3095103731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc06e1d0 msgr2=0x7f30cc15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 --2- 172.21.15.16:0/3095103731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc06e1d0 0x7f30cc15e780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009900 tx=0x7f30c001c850 comp rx=0 tx=0).stop 2024-09-17T05:07:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 -- 172.21.15.16:0/3095103731 shutdown_connections 2024-09-17T05:07:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 --2- 172.21.15.16:0/3095103731 >> v2:172.21.15.104:3300/0 conn(0x7f30cc15b850 0x7f30cc15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 --2- 172.21.15.16:0/3095103731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc06e1d0 0x7f30cc15e780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 -- 172.21.15.16:0/3095103731 >> 172.21.15.16:0/3095103731 conn(0x7f30cc0b2250 msgr2=0x7f30cc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 -- 172.21.15.16:0/3095103731 shutdown_connections 2024-09-17T05:07:55.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.787+0000 7f30d4600640 1 -- 172.21.15.16:0/3095103731 wait complete. 2024-09-17T05:07:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 Processor -- start 2024-09-17T05:07:55.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 -- start start 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 0x7f30cc144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc15b850 0x7f30cc152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30cc15ecc0 0x7f30cc152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc0ce8a0 con 0x7f30cc06e1d0 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc0ce720 con 0x7f30cc15ecc0 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f30cc0cea20 con 0x7f30cc15b850 2024-09-17T05:07:55.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 0x7f30cc144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 0x7f30cc144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60780/0 (socket says 172.21.15.16:60780) 2024-09-17T05:07:55.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 -- 172.21.15.16:0/2863135223 learned_addr learned my addr 172.21.15.16:0/2863135223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:55.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1200640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc15b850 0x7f30cc152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d2600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30cc15ecc0 0x7f30cc152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 -- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc15b850 msgr2=0x7f30cc152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc15b850 0x7f30cc152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 -- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30cc15ecc0 msgr2=0x7f30cc152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30cc15ecc0 0x7f30cc152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 -- 172.21.15.16:0/2863135223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc1456a0 con 0x7f30cc06e1d0 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d1c00640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 0x7f30cc144880 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f30c00b0040 tx=0x7f30c00a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c00b2040 con 0x7f30cc06e1d0 2024-09-17T05:07:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30c0004320 con 0x7f30cc06e1d0 2024-09-17T05:07:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0005280 con 0x7f30cc06e1d0 2024-09-17T05:07:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc14bf00 con 0x7f30cc06e1d0 2024-09-17T05:07:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.791+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc144f20 con 0x7f30cc06e1d0 2024-09-17T05:07:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.795+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc145160 con 0x7f30cc06e1d0 2024-09-17T05:07:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.795+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30c0005420 con 0x7f30cc06e1d0 2024-09-17T05:07:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.795+0000 7f30c9600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30ac073230 0x7f30ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.795+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6303+0+0 (secure 0 0 0) 0x7f30c012d640 con 0x7f30cc06e1d0 2024-09-17T05:07:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.795+0000 7f30d1200640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30ac073230 0x7f30ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:55.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.795+0000 7f30d1200640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30ac073230 0x7f30ac075740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0b59d0 tx=0x7f30b8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:55.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:55.799+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30cc145160 con 0x7f30cc06e1d0 2024-09-17T05:07:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.007+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f30cc15e780 con 0x7f30cc06e1d0 2024-09-17T05:07:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.671+0000 7f30c9600640 1 -- 172.21.15.16:0/2863135223 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) ==== 104+0+0 (secure 0 0 0) 0x7f30c00164d0 con 0x7f30cc06e1d0 2024-09-17T05:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30ac073230 msgr2=0x7f30ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30ac073230 0x7f30ac075740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0b59d0 tx=0x7f30b8009040 comp rx=0 tx=0).stop 2024-09-17T05:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 msgr2=0x7f30cc144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 0x7f30cc144880 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f30c00b0040 tx=0x7f30c00a6f60 comp rx=0 tx=0).stop 2024-09-17T05:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 shutdown_connections 2024-09-17T05:07:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f30ac073230 0x7f30ac075740 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f30cc15ecc0 0x7f30cc152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f30cc15b850 0x7f30cc152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 --2- 172.21.15.16:0/2863135223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30cc06e1d0 0x7f30cc144880 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 >> 172.21.15.16:0/2863135223 conn(0x7f30cc0b2250 msgr2=0x7f30cc14b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 shutdown_connections 2024-09-17T05:07:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.675+0000 7f30d4600640 1 -- 172.21.15.16:0/2863135223 wait complete. 2024-09-17T05:07:56.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 -- 172.21.15.16:0/1140918318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9480cea50 msgr2=0x7fb9480ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1140918318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9480cea50 0x7fb9480ccfb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009870 tx=0x7fb93c01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 -- 172.21.15.16:0/1140918318 shutdown_connections 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1140918318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9480cea50 0x7fb9480ccfb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 -- 172.21.15.16:0/1140918318 >> 172.21.15.16:0/1140918318 conn(0x7fb9480cdf00 msgr2=0x7fb9480ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 -- 172.21.15.16:0/1140918318 shutdown_connections 2024-09-17T05:07:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 -- 172.21.15.16:0/1140918318 wait complete. 2024-09-17T05:07:56.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 Processor -- start 2024-09-17T05:07:56.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.867+0000 7fb94ec00640 1 -- start start 2024-09-17T05:07:56.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 0x7fb9480cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb94815c040 0x7fb9481472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9481fa170 0x7fb9481fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb94816a210 con 0x7fb9480cea50 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb94816a090 con 0x7fb9481fa170 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb94816a390 con 0x7fb94815c040 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 0x7fb9480cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 0x7fb9480cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60794/0 (socket says 172.21.15.16:60794) 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 -- 172.21.15.16:0/1534489083 learned_addr learned my addr 172.21.15.16:0/1534489083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94e200640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9481fa170 0x7fb9481fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ce00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb94815c040 0x7fb9481472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 -- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb94815c040 msgr2=0x7fb9481472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb94815c040 0x7fb9481472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 -- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9481fa170 msgr2=0x7fb9481fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9481fa170 0x7fb9481fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 -- 172.21.15.16:0/1534489083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481faac0 con 0x7fb9480cea50 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94d800640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 0x7fb9480cac60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0193e0 tx=0x7fb93c0a6530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c005040 con 0x7fb9480cea50 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb93c0051e0 con 0x7fb9480cea50 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c005630 con 0x7fb9480cea50 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9480cb720 con 0x7fb9480cea50 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9480cbce0 con 0x7fb9480cea50 2024-09-17T05:07:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.871+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9480ccfb0 con 0x7fb9480cea50 2024-09-17T05:07:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.875+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb93c0046d0 con 0x7fb9480cea50 2024-09-17T05:07:56.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.875+0000 7fb945600640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb928073230 0x7fb928075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:56.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.875+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6303+0+0 (secure 0 0 0) 0x7fb93c12e340 con 0x7fb9480cea50 2024-09-17T05:07:56.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.875+0000 7fb94ce00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb928073230 0x7fb928075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:56.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.875+0000 7fb94ce00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb928073230 0x7fb928075740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb934007fd0 tx=0x7fb934009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:56.879+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb9481faac0 con 0x7fb9480cea50 2024-09-17T05:07:57.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.055+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb9480b5a40 con 0x7fb9480cea50 2024-09-17T05:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.675+0000 7fb945600640 1 -- 172.21.15.16:0/1534489083 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7fb93c0ac030 con 0x7fb9480cea50 2024-09-17T05:07:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:07:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.679+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb928073230 msgr2=0x7fb928075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.679+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb928073230 0x7fb928075740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb934007fd0 tx=0x7fb934009040 comp rx=0 tx=0).stop 2024-09-17T05:07:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.679+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 msgr2=0x7fb9480cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.679+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 0x7fb9480cac60 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0193e0 tx=0x7fb93c0a6530 comp rx=0 tx=0).stop 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 shutdown_connections 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb928073230 0x7fb928075740 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9481fa170 0x7fb9481fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb94815c040 0x7fb9481472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 --2- 172.21.15.16:0/1534489083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9480cea50 0x7fb9480cac60 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 >> 172.21.15.16:0/1534489083 conn(0x7fb9480cdf00 msgr2=0x7fb9480b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 shutdown_connections 2024-09-17T05:07:57.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.683+0000 7fb94ec00640 1 -- 172.21.15.16:0/1534489083 wait complete. 2024-09-17T05:07:57.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:07:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d8e200640 1 -- 172.21.15.16:0/1872952760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c004da0 msgr2=0x7f2d6c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d8e200640 1 --2- 172.21.15.16:0/1872952760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d6c004da0 0x7f2d6c005230 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009ee0 tx=0x7f2d8001d610 comp rx=0 tx=0).stop 2024-09-17T05:07:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d87e00640 1 -- 172.21.15.16:0/1872952760 reap_dead start 2024-09-17T05:07:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d8e200640 1 -- 172.21.15.16:0/1872952760 shutdown_connections 2024-09-17T05:07:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d8e200640 1 -- 172.21.15.16:0/1872952760 >> 172.21.15.16:0/1872952760 conn(0x7f2d880715e0 msgr2=0x7f2d880719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:57.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d8e200640 1 -- 172.21.15.16:0/1872952760 shutdown_connections 2024-09-17T05:07:57.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.879+0000 7f2d8e200640 1 -- 172.21.15.16:0/1872952760 wait complete. 2024-09-17T05:07:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 Processor -- start 2024-09-17T05:07:57.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 -- start start 2024-09-17T05:07:57.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c004da0 0x7f2d880b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 0x7f2d881f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d880cb250 0x7f2d880cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8815d460 con 0x7f2d88149780 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8815d2e0 con 0x7f2d6c004da0 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8815d5e0 con 0x7f2d880cb250 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 0x7f2d881f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 0x7f2d881f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55672/0 (socket says 172.21.15.16:55672) 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 -- 172.21.15.16:0/305468401 learned_addr learned my addr 172.21.15.16:0/305468401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d87400640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c004da0 0x7f2d880b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d87e00640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d880cb250 0x7f2d880cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:57.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 -- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d880cb250 msgr2=0x7f2d880cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d880cb250 0x7f2d880cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 -- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c004da0 msgr2=0x7f2d880b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c004da0 0x7f2d880b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 -- 172.21.15.16:0/305468401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d880cbba0 con 0x7f2d88149780 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d87400640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c004da0 0x7f2d880b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7fe00640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 0x7f2d881f6510 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001a40 tx=0x7f2d700129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:57.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70013920 con 0x7f2d88149780 2024-09-17T05:07:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d70013ae0 con 0x7f2d88149780 2024-09-17T05:07:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70010480 con 0x7f2d88149780 2024-09-17T05:07:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d88149e90 con 0x7f2d88149780 2024-09-17T05:07:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.883+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8814a450 con 0x7f2d88149780 2024-09-17T05:07:57.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.887+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d880b6ca0 con 0x7f2d88149780 2024-09-17T05:07:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.887+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2d70010620 con 0x7f2d88149780 2024-09-17T05:07:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.887+0000 7f2d7f400640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2d64073200 0x7f2d64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:57.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.887+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5865+0+0 (secure 0 0 0) 0x7f2d70098070 con 0x7f2d88149780 2024-09-17T05:07:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.887+0000 7f2d87400640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2d64073200 0x7f2d64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:57.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.891+0000 7f2d87400640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2d64073200 0x7f2d64075710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:57.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:57.891+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2d880cbba0 con 0x7f2d88149780 2024-09-17T05:07:58.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.067+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d88149be0 con 0x7f2d88149780 2024-09-17T05:07:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.679+0000 7f2d7f400640 1 -- 172.21.15.16:0/305468401 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f2d880d0200 con 0x7f2d88149780 2024-09-17T05:07:58.678 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:07:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2d64073200 msgr2=0x7f2d64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2d64073200 0x7f2d64075710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78006b20 comp rx=0 tx=0).stop 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 msgr2=0x7f2d881f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 0x7f2d881f6510 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001a40 tx=0x7f2d700129e0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 shutdown_connections 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2d64073200 0x7f2d64075710 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2d880cb250 0x7f2d880cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d88149780 0x7f2d881f6510 secure :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2d70001a40 tx=0x7f2d700129e0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 --2- 172.21.15.16:0/305468401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c004da0 0x7f2d880b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 >> 172.21.15.16:0/305468401 conn(0x7f2d880715e0 msgr2=0x7f2d88159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 shutdown_connections 2024-09-17T05:07:58.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.683+0000 7f2d8e200640 1 -- 172.21.15.16:0/305468401 wait complete. 2024-09-17T05:07:58.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 -- 172.21.15.16:0/3828563338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 msgr2=0x7fe74415efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 --2- 172.21.15.16:0/3828563338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74415efe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe744156030 tx=0x7fe73801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 -- 172.21.15.16:0/3828563338 shutdown_connections 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 --2- 172.21.15.16:0/3828563338 >> v2:172.21.15.104:3300/0 conn(0x7fe74415f520 0x7fe74415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 --2- 172.21.15.16:0/3828563338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74415efe0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 -- 172.21.15.16:0/3828563338 >> 172.21.15.16:0/3828563338 conn(0x7fe7440b2250 msgr2=0x7fe7440d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 -- 172.21.15.16:0/3828563338 shutdown_connections 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.883+0000 7fe74c000640 1 -- 172.21.15.16:0/3828563338 wait complete. 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 Processor -- start 2024-09-17T05:07:58.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 -- start start 2024-09-17T05:07:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe744074e30 0x7fe744154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:58.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe74415f520 0x7fe744154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe744168e20 con 0x7fe744074a20 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe744168ca0 con 0x7fe744074e30 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74c000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe744168fa0 con 0x7fe74415f520 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74414edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55684/0 (socket says 172.21.15.16:55684) 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 -- 172.21.15.16:0/3143694329 learned_addr learned my addr 172.21.15.16:0/3143694329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe748c00640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe744074e30 0x7fe744154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74a000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe74415f520 0x7fe744154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 -- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe74415f520 msgr2=0x7fe744154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe74415f520 0x7fe744154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 -- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe744074e30 msgr2=0x7fe744154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe744074e30 0x7fe744154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:58.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 -- 172.21.15.16:0/3143694329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7440c9780 con 0x7fe744074a20 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe74a000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe74415f520 0x7fe744154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.887+0000 7fe749600640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74414edd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe7380b0040 tx=0x7fe738005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7380bd070 con 0x7fe744074a20 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe738004330 con 0x7fe744074a20 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe744153d60 con 0x7fe744074a20 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7380b2040 con 0x7fe744074a20 2024-09-17T05:07:58.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74415a630 con 0x7fe744074a20 2024-09-17T05:07:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74415efe0 con 0x7fe744074a20 2024-09-17T05:07:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe738005290 con 0x7fe744074a20 2024-09-17T05:07:58.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe741600640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe724073230 0x7fe724075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.891+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7fe73812ddd0 con 0x7fe744074a20 2024-09-17T05:07:58.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.895+0000 7fe748c00640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe724073230 0x7fe724075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:58.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.895+0000 7fe748c00640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe724073230 0x7fe724075740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe7440b6070 tx=0x7fe730009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:58.899+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe7380164b0 con 0x7fe744074a20 2024-09-17T05:07:59.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.071+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fe74414f310 con 0x7fe744074a20 2024-09-17T05:07:59.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.075+0000 7fe741600640 1 -- 172.21.15.16:0/3143694329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7fe7380f8900 con 0x7fe744074a20 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe724073230 msgr2=0x7fe724075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe724073230 0x7fe724075740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe7440b6070 tx=0x7fe730009040 comp rx=0 tx=0).stop 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 msgr2=0x7fe74414edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74414edd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe7380b0040 tx=0x7fe738005e90 comp rx=0 tx=0).stop 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 shutdown_connections 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe724073230 0x7fe724075740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe74415f520 0x7fe744154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe744074e30 0x7fe744154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 --2- 172.21.15.16:0/3143694329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe744074a20 0x7fe74414edd0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 >> 172.21.15.16:0/3143694329 conn(0x7fe7440b2250 msgr2=0x7fe744158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 shutdown_connections 2024-09-17T05:07:59.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.079+0000 7fe74c000640 1 -- 172.21.15.16:0/3143694329 wait complete. 2024-09-17T05:07:59.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-09-17T05:07:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 -- 172.21.15.16:0/886715221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 msgr2=0x7fc92815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 --2- 172.21.15.16:0/886715221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815f560 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc9280ccfb0 tx=0x7fc92001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 -- 172.21.15.16:0/886715221 shutdown_connections 2024-09-17T05:07:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 --2- 172.21.15.16:0/886715221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815f560 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 --2- 172.21.15.16:0/886715221 >> v2:172.21.15.104:3300/0 conn(0x7fc92815c0a0 0x7fc92815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 --2- 172.21.15.16:0/886715221 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9280b2f50 0x7fc92815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 -- 172.21.15.16:0/886715221 >> 172.21.15.16:0/886715221 conn(0x7fc92806f900 msgr2=0x7fc9280b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 -- 172.21.15.16:0/886715221 shutdown_connections 2024-09-17T05:07:59.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.255+0000 7fc92fc00640 1 -- 172.21.15.16:0/886715221 wait complete. 2024-09-17T05:07:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 Processor -- start 2024-09-17T05:07:59.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 -- start start 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280b2f50 0x7fc9280b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc92815c0a0 0x7fc9281fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc928073f50 con 0x7fc92815f150 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc928073dd0 con 0x7fc9280b2f50 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.259+0000 7fc92fc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc9280740d0 con 0x7fc92815c0a0 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55706/0 (socket says 172.21.15.16:55706) 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 -- 172.21.15.16:0/439696798 learned_addr learned my addr 172.21.15.16:0/439696798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc91fe00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc92815c0a0 0x7fc9281fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92d200640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280b2f50 0x7fc9280b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 -- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc92815c0a0 msgr2=0x7fc9281fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc92815c0a0 0x7fc9281fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 -- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280b2f50 msgr2=0x7fc9280b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280b2f50 0x7fc9280b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 -- 172.21.15.16:0/439696798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9281f8a40 con 0x7fc92815f150 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92d200640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280b2f50 0x7fc9280b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:07:59.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92dc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815a630 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc920009bd0 tx=0x7fc920005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9200bb070 con 0x7fc92815f150 2024-09-17T05:07:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc920004d40 con 0x7fc92815f150 2024-09-17T05:07:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9280b1e10 con 0x7fc92815f150 2024-09-17T05:07:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9200b2040 con 0x7fc92815f150 2024-09-17T05:07:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281fcf00 con 0x7fc92815f150 2024-09-17T05:07:59.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc92815c4b0 con 0x7fc92815f150 2024-09-17T05:07:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.263+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc9200be070 con 0x7fc92815f150 2024-09-17T05:07:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.267+0000 7fc925600640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc904073200 0x7fc904075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.267+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7fc92012df10 con 0x7fc92815f150 2024-09-17T05:07:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.267+0000 7fc92d200640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc904073200 0x7fc904075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.267+0000 7fc92d200640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc904073200 0x7fc904075710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc918005d40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:59.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.271+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc9281f8a40 con 0x7fc92815f150 2024-09-17T05:07:59.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.519+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7fc928152c80 con 0x7fc92815f150 2024-09-17T05:07:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc925600640 1 -- 172.21.15.16:0/439696798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v343) ==== 150+0+0 (secure 0 0 0) 0x7fc920016970 con 0x7fc92815f150 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc904073200 msgr2=0x7fc904075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc904073200 0x7fc904075710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc918005fd0 tx=0x7fc918005d40 comp rx=0 tx=0).stop 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 msgr2=0x7fc92815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815a630 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc920009bd0 tx=0x7fc920005b00 comp rx=0 tx=0).stop 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 shutdown_connections 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc904073200 0x7fc904075710 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc92815f150 0x7fc92815a630 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc92815c0a0 0x7fc9281fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 --2- 172.21.15.16:0/439696798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280b2f50 0x7fc9280b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 >> 172.21.15.16:0/439696798 conn(0x7fc92806f900 msgr2=0x7fc928165a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 shutdown_connections 2024-09-17T05:07:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.695+0000 7fc92fc00640 1 -- 172.21.15.16:0/439696798 wait complete. 2024-09-17T05:07:59.715 INFO:tasks.cephfs_test_runner:When a client does not advance its oldest tid, the MDS should notice that ... skipped 'Require FUSE client to inject client release failure' 2024-09-17T05:07:59.715 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:07:59.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:07:59.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 -- 172.21.15.16:0/1122250706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 msgr2=0x7f4ee80d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 --2- 172.21.15.16:0/1122250706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee80d07c0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009bc0 tx=0x7f4ed001cae0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 -- 172.21.15.16:0/1122250706 shutdown_connections 2024-09-17T05:07:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 --2- 172.21.15.16:0/1122250706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee80d07c0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 -- 172.21.15.16:0/1122250706 >> 172.21.15.16:0/1122250706 conn(0x7f4ee8073330 msgr2=0x7f4ee80b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:07:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 -- 172.21.15.16:0/1122250706 shutdown_connections 2024-09-17T05:07:59.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.875+0000 7f4eece00640 1 -- 172.21.15.16:0/1122250706 wait complete. 2024-09-17T05:07:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 Processor -- start 2024-09-17T05:07:59.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 -- start start 2024-09-17T05:07:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ee80744f0 0x7f4ee8099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee80d0d00 0x7f4ee8155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8169ad0 con 0x7f4ee80d03b0 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8169950 con 0x7f4ee80d0d00 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4eece00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee8169c50 con 0x7f4ee80744f0 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee8155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55722/0 (socket says 172.21.15.16:55722) 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 -- 172.21.15.16:0/3296662691 learned_addr learned my addr 172.21.15.16:0/3296662691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4ee7400640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ee80744f0 0x7f4ee8099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4ee7e00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee80d0d00 0x7f4ee8155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 -- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ee80744f0 msgr2=0x7f4ee8099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ee80744f0 0x7f4ee8099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 -- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee80d0d00 msgr2=0x7f4ee8155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee80d0d00 0x7f4ee8155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:07:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.879+0000 7f4edea00640 1 -- 172.21.15.16:0/3296662691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee8166d50 con 0x7f4ee80d03b0 2024-09-17T05:07:59.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4edea00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee8155a50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009b10 tx=0x7f4ed00a59b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00b2040 con 0x7f4ee80d03b0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ed00a5e50 con 0x7f4ee80d03b0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee8158bf0 con 0x7f4ee80d03b0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed00045d0 con 0x7f4ee80d03b0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81591b0 con 0x7f4ee80d03b0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee80d07c0 con 0x7f4ee80d03b0 2024-09-17T05:07:59.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4ed00bd060 con 0x7f4ee80d03b0 2024-09-17T05:07:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.883+0000 7f4edfe00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ec4073200 0x7f4ec4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:07:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.887+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7f4ed012d9d0 con 0x7f4ee80d03b0 2024-09-17T05:07:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.887+0000 7f4ee7400640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ec4073200 0x7f4ec4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:07:59.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.887+0000 7f4ee7400640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ec4073200 0x7f4ec4075710 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007fd0 tx=0x7f4ed8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:07:59.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:07:59.891+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4ed0016790 con 0x7f4ee80d03b0 2024-09-17T05:08:00.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.067+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ee80b0750 con 0x7f4ee80d03b0 2024-09-17T05:08:00.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.067+0000 7f4edfe00640 1 -- 172.21.15.16:0/3296662691 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+13298 (secure 0 0 0) 0x7f4ed00f8590 con 0x7f4ee80d03b0 2024-09-17T05:08:00.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:00.066 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":105,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:07:58.679405+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/2345541365":"2024-09-18T05:07:55.596943+0000","172.21.15.16:6834/2345541365":"2024-09-18T05:07:55.596943+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ec4073200 msgr2=0x7f4ec4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ec4073200 0x7f4ec4075710 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8007fd0 tx=0x7f4ed8009040 comp rx=0 tx=0).stop 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 msgr2=0x7f4ee8155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee8155a50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009b10 tx=0x7f4ed00a59b0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 shutdown_connections 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4ec4073200 0x7f4ec4075710 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee80d0d00 0x7f4ee8155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee80d03b0 0x7f4ee8155a50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 --2- 172.21.15.16:0/3296662691 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4ee80744f0 0x7f4ee8099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 >> 172.21.15.16:0/3296662691 conn(0x7f4ee8073330 msgr2=0x7f4ee80aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 shutdown_connections 2024-09-17T05:08:00.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.071+0000 7f4eece00640 1 -- 172.21.15.16:0/3296662691 wait complete. 2024-09-17T05:08:00.091 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:08:00.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:08:00.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 -- 172.21.15.16:0/3414277458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c074a20 msgr2=0x7f3a5c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 --2- 172.21.15.16:0/3414277458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c074a20 0x7f3a5c15efe0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0ce2c0 tx=0x7f3a4c01cda0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 -- 172.21.15.16:0/3414277458 shutdown_connections 2024-09-17T05:08:00.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 --2- 172.21.15.16:0/3414277458 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a5c15f520 0x7f3a5c15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 --2- 172.21.15.16:0/3414277458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c074a20 0x7f3a5c15efe0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 -- 172.21.15.16:0/3414277458 >> 172.21.15.16:0/3414277458 conn(0x7f3a5c0b2250 msgr2=0x7f3a5c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:00.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 -- 172.21.15.16:0/3414277458 shutdown_connections 2024-09-17T05:08:00.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.251+0000 7f3a62000640 1 -- 172.21.15.16:0/3414277458 wait complete. 2024-09-17T05:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 Processor -- start 2024-09-17T05:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 -- start start 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a5c074a20 0x7f3a5c1f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a5c074e30 0x7f3a5c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 0x7f3a5c15e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c168e80 con 0x7f3a5c15f520 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c168d00 con 0x7f3a5c074a20 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a62000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c169000 con 0x7f3a5c074e30 2024-09-17T05:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 0x7f3a5c15e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a5b400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a5c074a20 0x7f3a5c1f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.255+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 0x7f3a5c15e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55746/0 (socket says 172.21.15.16:55746) 2024-09-17T05:08:00.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 -- 172.21.15.16:0/2966210069 learned_addr learned my addr 172.21.15.16:0/2966210069 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:00.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a52a00640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a5c074e30 0x7f3a5c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 -- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a5c074e30 msgr2=0x7f3a5c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a5c074e30 0x7f3a5c14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 -- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a5c074a20 msgr2=0x7f3a5c1f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a5c074a20 0x7f3a5c1f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 -- 172.21.15.16:0/2966210069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c1f9220 con 0x7f3a5c15f520 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a5be00640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 0x7f3a5c15e000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d630 tx=0x7f3a5400daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a5c15f520 2024-09-17T05:08:00.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a54005310 con 0x7f3a5c15f520 2024-09-17T05:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c1f9730 con 0x7f3a5c15f520 2024-09-17T05:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a540047b0 con 0x7f3a5c15f520 2024-09-17T05:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c150200 con 0x7f3a5c15f520 2024-09-17T05:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c15efe0 con 0x7f3a5c15f520 2024-09-17T05:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.259+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a54004950 con 0x7f3a5c15f520 2024-09-17T05:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.263+0000 7f3a53e00640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a38073230 0x7f3a38075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.263+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7f3a54093fe0 con 0x7f3a5c15f520 2024-09-17T05:08:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.263+0000 7f3a5b400640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a38073230 0x7f3a38075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.263+0000 7f3a5b400640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a38073230 0x7f3a38075740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0af000 tx=0x7f3a4c01b5e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.267+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a5405eb30 con 0x7f3a5c15f520 2024-09-17T05:08:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.439+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a5c159350 con 0x7f3a5c15f520 2024-09-17T05:08:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a53e00640 1 -- 172.21.15.16:0/2966210069 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f3a54063560 con 0x7f3a5c15f520 2024-09-17T05:08:00.441 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a38073230 msgr2=0x7f3a38075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a38073230 0x7f3a38075740 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0af000 tx=0x7f3a4c01b5e0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 msgr2=0x7f3a5c15e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 0x7f3a5c15e000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d630 tx=0x7f3a5400daf0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 shutdown_connections 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a38073230 0x7f3a38075740 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c15f520 0x7f3a5c15e000 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a5c074e30 0x7f3a5c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 --2- 172.21.15.16:0/2966210069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a5c074a20 0x7f3a5c1f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 >> 172.21.15.16:0/2966210069 conn(0x7f3a5c0b2250 msgr2=0x7f3a5c0ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 shutdown_connections 2024-09-17T05:08:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.443+0000 7f3a62000640 1 -- 172.21.15.16:0/2966210069 wait complete. 2024-09-17T05:08:00.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-09-17T05:08:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 -- 172.21.15.16:0/4205257696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d415f8c0 msgr2=0x7fb1d40b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 --2- 172.21.15.16:0/4205257696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d415f8c0 0x7fb1d40b2b30 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40cfd90 tx=0x7fb1cc01ccf0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 -- 172.21.15.16:0/4205257696 shutdown_connections 2024-09-17T05:08:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 --2- 172.21.15.16:0/4205257696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d415f8c0 0x7fb1d40b2b30 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 --2- 172.21.15.16:0/4205257696 >> v2:172.21.15.104:3300/0 conn(0x7fb1d406e170 0x7fb1d406e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 -- 172.21.15.16:0/4205257696 >> 172.21.15.16:0/4205257696 conn(0x7fb1d40d2270 msgr2=0x7fb1d40d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 -- 172.21.15.16:0/4205257696 shutdown_connections 2024-09-17T05:08:00.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.619+0000 7fb1db800640 1 -- 172.21.15.16:0/4205257696 wait complete. 2024-09-17T05:08:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 Processor -- start 2024-09-17T05:08:00.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 -- start start 2024-09-17T05:08:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 0x7fb1d41fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1d40b2010 0x7fb1d414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb1d415f8c0 0x7fb1d4152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d416c2c0 con 0x7fb1d406e170 2024-09-17T05:08:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d416c140 con 0x7fb1d40b2010 2024-09-17T05:08:00.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1db800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d416c440 con 0x7fb1d415f8c0 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1d8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 0x7fb1d41fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.623+0000 7fb1d8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 0x7fb1d41fd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55752/0 (socket says 172.21.15.16:55752) 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 -- 172.21.15.16:0/1234401689 learned_addr learned my addr 172.21.15.16:0/1234401689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 -- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb1d415f8c0 msgr2=0x7fb1d4152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb1d415f8c0 0x7fb1d4152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 -- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1d40b2010 msgr2=0x7fb1d414f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d3e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1d40b2010 0x7fb1d414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1d40b2010 0x7fb1d414f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 -- 172.21.15.16:0/1234401689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d414fa70 con 0x7fb1d406e170 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d3e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1d40b2010 0x7fb1d414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d8e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 0x7fb1d41fd100 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800c9f0 tx=0x7fb1c800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8007bb0 con 0x7fb1d406e170 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1c8007e40 con 0x7fb1d406e170 2024-09-17T05:08:00.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800f690 con 0x7fb1d406e170 2024-09-17T05:08:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d4161e00 con 0x7fb1d406e170 2024-09-17T05:08:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d41589a0 con 0x7fb1d406e170 2024-09-17T05:08:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d4162300 con 0x7fb1d406e170 2024-09-17T05:08:00.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.627+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb1c800f830 con 0x7fb1d406e170 2024-09-17T05:08:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.631+0000 7fb1d0c00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb1b4073200 0x7fb1b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:00.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.631+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7fb1c80947b0 con 0x7fb1d406e170 2024-09-17T05:08:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.631+0000 7fb1d3e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb1b4073200 0x7fb1b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.631+0000 7fb1d3e00640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb1b4073200 0x7fb1b4075710 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0001630 tx=0x7fb1c0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:00.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.635+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb1c8014030 con 0x7fb1d406e170 2024-09-17T05:08:00.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:00.871+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7fb1d406e580 con 0x7fb1d406e170 2024-09-17T05:08:01.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:08:01.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:08:01.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.699+0000 7fb1d0c00640 1 -- 172.21.15.16:0/1234401689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v345) ==== 146+0+0 (secure 0 0 0) 0x7fb1c805f9c0 con 0x7fb1d406e170 2024-09-17T05:08:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb1b4073200 msgr2=0x7fb1b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb1b4073200 0x7fb1b4075710 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0001630 tx=0x7fb1c0009040 comp rx=0 tx=0).stop 2024-09-17T05:08:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 msgr2=0x7fb1d41fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 0x7fb1d41fd100 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800c9f0 tx=0x7fb1c800ceb0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 shutdown_connections 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb1b4073200 0x7fb1b4075710 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb1d415f8c0 0x7fb1d4152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1d40b2010 0x7fb1d414f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 --2- 172.21.15.16:0/1234401689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1d406e170 0x7fb1d41fd100 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 >> 172.21.15.16:0/1234401689 conn(0x7fb1d40d2270 msgr2=0x7fb1d40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.703+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 shutdown_connections 2024-09-17T05:08:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.707+0000 7fb1db800640 1 -- 172.21.15.16:0/1234401689 wait complete. 2024-09-17T05:08:01.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:08:01.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafdd000640 1 -- 172.21.15.16:0/721054347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0052e0 msgr2=0x7fafbc005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:01.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafdd000640 1 --2- 172.21.15.16:0/721054347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0052e0 0x7fafbc005920 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fafc4001800 tx=0x7fafc401f040 comp rx=0 tx=0).stop 2024-09-17T05:08:01.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafd6800640 1 -- 172.21.15.16:0/721054347 reap_dead start 2024-09-17T05:08:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafdd000640 1 -- 172.21.15.16:0/721054347 shutdown_connections 2024-09-17T05:08:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafdd000640 1 -- 172.21.15.16:0/721054347 >> 172.21.15.16:0/721054347 conn(0x7fafd806f900 msgr2=0x7fafd80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafdd000640 1 -- 172.21.15.16:0/721054347 shutdown_connections 2024-09-17T05:08:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.891+0000 7fafdd000640 1 -- 172.21.15.16:0/721054347 wait complete. 2024-09-17T05:08:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 Processor -- start 2024-09-17T05:08:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 -- start start 2024-09-17T05:08:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafbc0052e0 0x7fafd80cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafd8150f90 0x7fafd80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 0x7fafd8153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafd815bf90 con 0x7fafd8153990 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fafd815be10 con 0x7fafd8150f90 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafdd000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fafd815c110 con 0x7fafbc0052e0 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafd6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 0x7fafd8153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafd6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 0x7fafd8153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55770/0 (socket says 172.21.15.16:55770) 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.895+0000 7fafd6800640 1 -- 172.21.15.16:0/1923773922 learned_addr learned my addr 172.21.15.16:0/1923773922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd5e00640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafbc0052e0 0x7fafd80cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd5400640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafd8150f90 0x7fafd80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd6800640 1 -- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafbc0052e0 msgr2=0x7fafd80cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd6800640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafbc0052e0 0x7fafd80cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd6800640 1 -- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafd8150f90 msgr2=0x7fafd80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd6800640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafd8150f90 0x7fafd80cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd6800640 1 -- 172.21.15.16:0/1923773922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd806a6c0 con 0x7fafd8153990 2024-09-17T05:08:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd5400640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafd8150f90 0x7fafd80cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafd6800640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 0x7fafd8153da0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fafd80b2960 tx=0x7fafd000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0010890 con 0x7fafd8153990 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafd0018070 con 0x7fafd8153990 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81513a0 con 0x7fafd8153990 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0010dc0 con 0x7fafd8153990 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd80cf450 con 0x7fafd8153990 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd815eb00 con 0x7fafd8153990 2024-09-17T05:08:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.899+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fafd001d460 con 0x7fafd8153990 2024-09-17T05:08:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.903+0000 7fafce000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafb8073200 0x7fafb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.903+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7fafd00945a0 con 0x7fafd8153990 2024-09-17T05:08:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.903+0000 7fafd5e00640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafb8073200 0x7fafb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.903+0000 7fafd5e00640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafb8073200 0x7fafb8075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009bc0 tx=0x7fafc80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:01.907+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fafd005f750 con 0x7fafd8153990 2024-09-17T05:08:02.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.079+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fafd806e7b0 con 0x7fafd8153990 2024-09-17T05:08:02.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.079+0000 7fafce000640 1 -- 172.21.15.16:0/1923773922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+2294 (secure 0 0 0) 0x7fafd005f050 con 0x7fafd8153990 2024-09-17T05:08:02.079 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:08:02.079 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:08:02.079 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:08:02.080 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:08:02.081 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:08:02.082 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:08:02.082 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:08:02.082 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafb8073200 msgr2=0x7fafb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafb8073200 0x7fafb8075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009bc0 tx=0x7fafc80091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 msgr2=0x7fafd8153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 0x7fafd8153da0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fafd80b2960 tx=0x7fafd000ea20 comp rx=0 tx=0).stop 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 shutdown_connections 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafb8073200 0x7fafb8075710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafd8153990 0x7fafd8153da0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafd8150f90 0x7fafd80cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 --2- 172.21.15.16:0/1923773922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafbc0052e0 0x7fafd80cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 >> 172.21.15.16:0/1923773922 conn(0x7fafd806f900 msgr2=0x7fafd815a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 shutdown_connections 2024-09-17T05:08:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.083+0000 7fafdd000640 1 -- 172.21.15.16:0/1923773922 wait complete. 2024-09-17T05:08:02.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:08:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 -- 172.21.15.16:0/4274694680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 msgr2=0x7f014c160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 --2- 172.21.15.16:0/4274694680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c160130 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f014c071ce0 tx=0x7f013801c890 comp rx=0 tx=0).stop 2024-09-17T05:08:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 -- 172.21.15.16:0/4274694680 shutdown_connections 2024-09-17T05:08:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 --2- 172.21.15.16:0/4274694680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c160130 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 --2- 172.21.15.16:0/4274694680 >> v2:172.21.15.104:3300/0 conn(0x7f014c0733e0 0x7f014c15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 -- 172.21.15.16:0/4274694680 >> 172.21.15.16:0/4274694680 conn(0x7f014c0b2250 msgr2=0x7f014c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 -- 172.21.15.16:0/4274694680 shutdown_connections 2024-09-17T05:08:02.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.263+0000 7f0151800640 1 -- 172.21.15.16:0/4274694680 wait complete. 2024-09-17T05:08:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 Processor -- start 2024-09-17T05:08:02.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 -- start start 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f014c0733e0 0x7f014c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f014c160670 0x7f014c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f014c16a2d0 con 0x7f014c15d680 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f014c0cae90 con 0x7f014c160670 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0151800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f014c16a450 con 0x7f014c0733e0 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55788/0 (socket says 172.21.15.16:55788) 2024-09-17T05:08:02.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 -- 172.21.15.16:0/3733434196 learned_addr learned my addr 172.21.15.16:0/3733434196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014aa00640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f014c0733e0 0x7f014c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014b400640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f014c160670 0x7f014c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 -- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f014c0733e0 msgr2=0x7f014c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f014c0733e0 0x7f014c158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 -- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f014c160670 msgr2=0x7f014c15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f014c160670 0x7f014c15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 -- 172.21.15.16:0/3733434196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c1fac80 con 0x7f014c15d680 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014aa00640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f014c0733e0 0x7f014c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014b400640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f014c160670 0x7f014c15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f014a000640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c152ab0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f01380b0040 tx=0x7f0138004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380b2040 con 0x7f014c15d680 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0138005310 con 0x7f014c15d680 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.267+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01380046d0 con 0x7f014c15d680 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c15ae90 con 0x7f014c15d680 2024-09-17T05:08:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c15b450 con 0x7f014c15d680 2024-09-17T05:08:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c15ac50 con 0x7f014c15d680 2024-09-17T05:08:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0138004870 con 0x7f014c15d680 2024-09-17T05:08:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f0142a00640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f012c073230 0x7f012c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7f013812e280 con 0x7f014c15d680 2024-09-17T05:08:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.271+0000 7f014aa00640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f012c073230 0x7f012c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.275+0000 7f014aa00640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f012c073230 0x7f012c075740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f013c005fd0 tx=0x7f013c005dc0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:02.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.275+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f014c15ac50 con 0x7f014c15d680 2024-09-17T05:08:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.451+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f014c15d140 con 0x7f014c15d680 2024-09-17T05:08:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0142a00640 1 -- 172.21.15.16:0/3733434196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f01380164b0 con 0x7f014c15d680 2024-09-17T05:08:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f012c073230 msgr2=0x7f012c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f012c073230 0x7f012c075740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f013c005fd0 tx=0x7f013c005dc0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 msgr2=0x7f014c152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c152ab0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f01380b0040 tx=0x7f0138004600 comp rx=0 tx=0).stop 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 shutdown_connections 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f012c073230 0x7f012c075740 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f014c160670 0x7f014c15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f014c15d680 0x7f014c152ab0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 --2- 172.21.15.16:0/3733434196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f014c0733e0 0x7f014c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 >> 172.21.15.16:0/3733434196 conn(0x7f014c0b2250 msgr2=0x7f014c155e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 shutdown_connections 2024-09-17T05:08:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.455+0000 7f0151800640 1 -- 172.21.15.16:0/3733434196 wait complete. 2024-09-17T05:08:02.476 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:08:02.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:08:02.566 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:08:02.663 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:08:02.663 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:02.663 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:02.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:08:02.672 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:02.673 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:02.682 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:08:02.682 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:02.682 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:02.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:08:02.724 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:02.724 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:02.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:08:02.734 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:08:02.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 -- 172.21.15.16:0/835257248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180cc8c0 msgr2=0x7f691815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 --2- 172.21.15.16:0/835257248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180cc8c0 0x7f691815cab0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6908009840 tx=0x7f690801c820 comp rx=0 tx=0).stop 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 -- 172.21.15.16:0/835257248 shutdown_connections 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 --2- 172.21.15.16:0/835257248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180cc8c0 0x7f691815cab0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 -- 172.21.15.16:0/835257248 >> 172.21.15.16:0/835257248 conn(0x7f69180d2270 msgr2=0x7f69180d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 -- 172.21.15.16:0/835257248 shutdown_connections 2024-09-17T05:08:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 -- 172.21.15.16:0/835257248 wait complete. 2024-09-17T05:08:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 Processor -- start 2024-09-17T05:08:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.927+0000 7f691e000640 1 -- start start 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 0x7f69181fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f69180cc8c0 0x7f6918153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69180d1580 0x7f69181f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691816b000 con 0x7f69180c9440 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f691816ae80 con 0x7f69180d1580 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f691816b180 con 0x7f69180cc8c0 2024-09-17T05:08:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 0x7f69181fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 0x7f69181fcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55802/0 (socket says 172.21.15.16:55802) 2024-09-17T05:08:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 -- 172.21.15.16:0/2604361143 learned_addr learned my addr 172.21.15.16:0/2604361143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917e00640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69180d1580 0x7f69181f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f690fe00640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f69180cc8c0 0x7f6918153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 -- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f69180cc8c0 msgr2=0x7f6918153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f69180cc8c0 0x7f6918153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 -- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69180d1580 msgr2=0x7f69181f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69180d1580 0x7f69181f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 -- 172.21.15.16:0/2604361143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69181541e0 con 0x7f69180c9440 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f690fe00640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f69180cc8c0 0x7f6918153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f6917400640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 0x7f69181fcb00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f690801aed0 tx=0x7f6908004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080bb070 con 0x7f69180c9440 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6908005030 con 0x7f69180c9440 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080b2040 con 0x7f69180c9440 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69181f9610 con 0x7f69180c9440 2024-09-17T05:08:02.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69181fc300 con 0x7f69180c9440 2024-09-17T05:08:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.931+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691815cab0 con 0x7f69180c9440 2024-09-17T05:08:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.935+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f69080046d0 con 0x7f69180c9440 2024-09-17T05:08:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.935+0000 7f690f400640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68f4073230 0x7f68f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:02.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.935+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7f690812da60 con 0x7f69180c9440 2024-09-17T05:08:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.935+0000 7f690fe00640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68f4073230 0x7f68f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.935+0000 7f690fe00640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68f4073230 0x7f68f4075740 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f69000046c0 tx=0x7f6900009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:02.939+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f69080f1db0 con 0x7f69180c9440 2024-09-17T05:08:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.151+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69180d0e80 con 0x7f69180c9440 2024-09-17T05:08:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.151+0000 7f690f400640 1 -- 172.21.15.16:0/2604361143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+3917 (secure 0 0 0) 0x7f6908016410 con 0x7f69180c9440 2024-09-17T05:08:03.151 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:03.151 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"btime":"2024-09-17T05:07:56:667167+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2133460308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2133460308},{"type":"v1","addr":"172.21.15.104:6837","nonce":2133460308}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6709,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/2272477536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":2272477536},{"type":"v1","addr":"172.21.15.104:6835","nonce":2272477536}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1584364815","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1584364815},{"type":"v1","addr":"172.21.15.16:6835","nonce":1584364815}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":6984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/2858762360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2858762360},{"type":"v1","addr":"172.21.15.16:6833","nonce":2858762360}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-09-17T05:08:03.151 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 68 2024-09-17T05:08:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68f4073230 msgr2=0x7f68f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68f4073230 0x7f68f4075740 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f69000046c0 tx=0x7f6900009040 comp rx=0 tx=0).stop 2024-09-17T05:08:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 msgr2=0x7f69181fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 0x7f69181fcb00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f690801aed0 tx=0x7f6908004040 comp rx=0 tx=0).stop 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 shutdown_connections 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68f4073230 0x7f68f4075740 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69180d1580 0x7f69181f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f69180cc8c0 0x7f6918153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 --2- 172.21.15.16:0/2604361143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69180c9440 0x7f69181fcb00 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 >> 172.21.15.16:0/2604361143 conn(0x7f69180d2270 msgr2=0x7f69180c5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 shutdown_connections 2024-09-17T05:08:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.155+0000 7f691e000640 1 -- 172.21.15.16:0/2604361143 wait complete. 2024-09-17T05:08:03.173 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:08:03.173 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:08:03.173 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:08:03.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:03.174 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:08:03.174 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:08:03.174 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:08:03.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:03.174 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:08:03.174 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:08:03.175 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:08:03.175 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:03.175 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:08:03.175 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:08:03.175 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:08:03.175 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:03.186 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:08:03.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:08:03.190 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:08:03.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:08:03.191 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:08:03.192 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:08:03.194 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:08:03.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:08:03.235 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:08:03.235 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:03.236 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:08:03.237 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:03.242 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:08:03.243 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:03.245 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:08:03.245 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:03.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:08:03.338 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:08:03.338 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:08:03.341 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:08:03.345 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:08:03.396 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T05:08:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 -- 172.21.15.16:0/4203442065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff924159f70 msgr2=0x7ff9240d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 --2- 172.21.15.16:0/4203442065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff924159f70 0x7ff9240d1b80 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91401c790 comp rx=0 tx=0).stop 2024-09-17T05:08:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 -- 172.21.15.16:0/4203442065 shutdown_connections 2024-09-17T05:08:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 --2- 172.21.15.16:0/4203442065 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9240d02d0 0x7ff9240cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 --2- 172.21.15.16:0/4203442065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff924159f70 0x7ff9240d1b80 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 -- 172.21.15.16:0/4203442065 >> 172.21.15.16:0/4203442065 conn(0x7ff9240b42d0 msgr2=0x7ff9240b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:03.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 -- 172.21.15.16:0/4203442065 shutdown_connections 2024-09-17T05:08:03.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 -- 172.21.15.16:0/4203442065 wait complete. 2024-09-17T05:08:03.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.411+0000 7ff92aa00640 1 Processor -- start 2024-09-17T05:08:03.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 -- start start 2024-09-17T05:08:03.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9240cf680 0x7ff924147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 0x7ff9241fc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924159f70 0x7ff9241fca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff924169e20 con 0x7ff9240d02d0 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff924169ca0 con 0x7ff924159f70 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff924169fa0 con 0x7ff9240cf680 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 0x7ff9241fc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 0x7ff9241fc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55854/0 (socket says 172.21.15.16:55854) 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 -- 172.21.15.16:0/2293250656 learned_addr learned my addr 172.21.15.16:0/2293250656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff923e00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9240cf680 0x7ff924147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff928a00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924159f70 0x7ff9241fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 -- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9240cf680 msgr2=0x7ff924147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9240cf680 0x7ff924147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 -- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924159f70 msgr2=0x7ff9241fca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924159f70 0x7ff9241fca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 -- 172.21.15.16:0/2293250656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9241fd2d0 con 0x7ff9240d02d0 2024-09-17T05:08:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff928a00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924159f70 0x7ff9241fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff91be00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 0x7ff9241fc550 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00e770 tx=0x7ff90c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:03.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00ccc0 con 0x7ff9240d02d0 2024-09-17T05:08:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff90c00ee80 con 0x7ff9240d02d0 2024-09-17T05:08:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92406f090 con 0x7ff9240d02d0 2024-09-17T05:08:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00ce60 con 0x7ff9240d02d0 2024-09-17T05:08:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.415+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92406f650 con 0x7ff9240d02d0 2024-09-17T05:08:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.419+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9240cf140 con 0x7ff9240d02d0 2024-09-17T05:08:03.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.419+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff90c005280 con 0x7ff9240d02d0 2024-09-17T05:08:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.419+0000 7ff920c00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff900073230 0x7ff900075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.419+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5335+0+0 (secure 0 0 0) 0x7ff90c094820 con 0x7ff9240d02d0 2024-09-17T05:08:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.419+0000 7ff923e00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff900073230 0x7ff900075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.419+0000 7ff923e00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff900073230 0x7ff900075740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9140af000 tx=0x7ff91401afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:03.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.423+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff90c05fad0 con 0x7ff9240d02d0 2024-09-17T05:08:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.611+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff9241fd060 con 0x7ff9240d02d0 2024-09-17T05:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff920c00640 1 -- 172.21.15.16:0/2293250656 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) ==== 97+0+0 (secure 0 0 0) 0x7ff90c05f3d0 con 0x7ff9240d02d0 2024-09-17T05:08:03.714 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff900073230 msgr2=0x7ff900075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff900073230 0x7ff900075740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff9140af000 tx=0x7ff91401afd0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 msgr2=0x7ff9241fc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 0x7ff9241fc550 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00e770 tx=0x7ff90c00ec30 comp rx=0 tx=0).stop 2024-09-17T05:08:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 shutdown_connections 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff900073230 0x7ff900075740 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924159f70 0x7ff9241fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.715+0000 7ff92aa00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9240d02d0 0x7ff9241fc550 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.719+0000 7ff92aa00640 1 --2- 172.21.15.16:0/2293250656 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9240cf680 0x7ff924147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.719+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 >> 172.21.15.16:0/2293250656 conn(0x7ff9240b42d0 msgr2=0x7ff9241591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.719+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 shutdown_connections 2024-09-17T05:08:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.719+0000 7ff92aa00640 1 -- 172.21.15.16:0/2293250656 wait complete. 2024-09-17T05:08:03.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:08:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 -- 172.21.15.16:0/1223636301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a8015c270 msgr2=0x7f8a8015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 --2- 172.21.15.16:0/1223636301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a8015c270 0x7f8a8015c680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009870 tx=0x7f8a6801c6f0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 -- 172.21.15.16:0/1223636301 shutdown_connections 2024-09-17T05:08:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 --2- 172.21.15.16:0/1223636301 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a8015c270 0x7f8a8015c680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 -- 172.21.15.16:0/1223636301 >> 172.21.15.16:0/1223636301 conn(0x7f8a8006f900 msgr2=0x7f8a800b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:03.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 -- 172.21.15.16:0/1223636301 shutdown_connections 2024-09-17T05:08:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 -- 172.21.15.16:0/1223636301 wait complete. 2024-09-17T05:08:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.899+0000 7f8a87200640 1 Processor -- start 2024-09-17T05:08:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 -- start start 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a80073240 0x7f8a8014f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 0x7f8a8015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8015c270 0x7f8a8015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80169db0 con 0x7f8a8015c270 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a80169c30 con 0x7f8a80073240 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8a80169f30 con 0x7f8a800b2f50 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a7fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a80073240 0x7f8a8014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 0x7f8a8015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a85200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8015c270 0x7f8a8015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 0x7f8a8015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:58420/0 (socket says 172.21.15.16:58420) 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a7fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a80073240 0x7f8a8014f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:32914/0 (socket says 172.21.15.16:32914) 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 -- 172.21.15.16:0/1907328727 learned_addr learned my addr 172.21.15.16:0/1907328727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 -- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a80073240 msgr2=0x7f8a8014f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a80073240 0x7f8a8014f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 -- 172.21.15.16:0/1907328727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8015c270 msgr2=0x7f8a8015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8015c270 0x7f8a8015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 -- 172.21.15.16:0/1907328727 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a801fccd0 con 0x7f8a800b2f50 2024-09-17T05:08:03.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a85200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8015c270 0x7f8a8015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:08:03.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a77e00640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 0x7f8a8015a600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a6801ada0 tx=0x7f8a68004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a680bb070 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a680053e0 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a680b2040 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a800afdd0 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.903+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a801fc9c0 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.907+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a800afbc0 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.907+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a680a5a50 con 0x7f8a800b2f50 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.907+0000 7f8a7cc00640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a5c073200 0x7f8a5c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:03.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.907+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f8a6812d910 con 0x7f8a800b2f50 2024-09-17T05:08:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.907+0000 7f8a7fe00640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a5c073200 0x7f8a5c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:03.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.907+0000 7f8a7fe00640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a5c073200 0x7f8a5c075710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007fd0 tx=0x7f8a70009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:03.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:03.911+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8a680c4020 con 0x7f8a800b2f50 2024-09-17T05:08:04.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.147+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8a8015c680 con 0x7f8a800b2f50 2024-09-17T05:08:04.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.147+0000 7f8a7cc00640 1 -- 172.21.15.16:0/1907328727 <== mon.2 v2:172.21.15.104:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) ==== 80+0+5304 (secure 0 0 0) 0x7f8a68016370 con 0x7f8a800b2f50 2024-09-17T05:08:04.147 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:04.147 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:08:04.147 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:08:04.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.147 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:08:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:08:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.149 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.149 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.149 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:08:04.149 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:08:04.150 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.150 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.150 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:04.150 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:04.150 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.150 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.151 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.151 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:08:04.151 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:08:04.151 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.151 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.152 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:04.152 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:04.152 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.152 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.153 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:08:04.153 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:08:04.153 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.153 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.153 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:04.154 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:04.154 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.154 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.154 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:08:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:08:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.155 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.156 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.156 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:08:04.156 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:08:04.157 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.157 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.157 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.157 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.157 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.158 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.158 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:08:04.158 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:08:04.158 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.159 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.159 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.159 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.159 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.160 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:08:04.160 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:08:04.160 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.160 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.160 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.160 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:08:04.161 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.162 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.163 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:08:04.163 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:08:04.163 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.163 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.163 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.163 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.164 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.164 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.164 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:08:04.164 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:08:04.164 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.165 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.165 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.165 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.165 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.165 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.167 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.167 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:08:04.167 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:08:04.167 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.167 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.168 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:08:04.168 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:08:04.168 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.168 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.168 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:08:04.169 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:08:04.169 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.169 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.169 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:08:04.169 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:08:04.169 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:08:04.170 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.170 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:08:04.170 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:08:04.171 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.172 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:08:04.173 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.174 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:08:04.175 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:08:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a5c073200 msgr2=0x7f8a5c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a5c073200 0x7f8a5c075710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8a70007fd0 tx=0x7f8a70009040 comp rx=0 tx=0).stop 2024-09-17T05:08:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 msgr2=0x7f8a8015a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 0x7f8a8015a600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8a6801ada0 tx=0x7f8a68004600 comp rx=0 tx=0).stop 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 shutdown_connections 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a5c073200 0x7f8a5c075710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8015c270 0x7f8a8015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8a800b2f50 0x7f8a8015a600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 --2- 172.21.15.16:0/1907328727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a80073240 0x7f8a8014f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 >> 172.21.15.16:0/1907328727 conn(0x7f8a8006f900 msgr2=0x7f8a800cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 shutdown_connections 2024-09-17T05:08:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.151+0000 7f8a87200640 1 -- 172.21.15.16:0/1907328727 wait complete. 2024-09-17T05:08:04.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:08:04.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 -- 172.21.15.16:0/1410977053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb18005260 msgr2=0x7feb1800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 --2- 172.21.15.16:0/1410977053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb18005260 0x7feb1800b9d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7feb28009ec0 tx=0x7feb2801d610 comp rx=0 tx=0).stop 2024-09-17T05:08:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 -- 172.21.15.16:0/1410977053 shutdown_connections 2024-09-17T05:08:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 --2- 172.21.15.16:0/1410977053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb18005260 0x7feb1800b9d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 -- 172.21.15.16:0/1410977053 >> 172.21.15.16:0/1410977053 conn(0x7feb300d2270 msgr2=0x7feb300d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 -- 172.21.15.16:0/1410977053 shutdown_connections 2024-09-17T05:08:04.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 -- 172.21.15.16:0/1410977053 wait complete. 2024-09-17T05:08:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 Processor -- start 2024-09-17T05:08:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.339+0000 7feb37a00640 1 -- start start 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb18005260 0x7feb3015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 0x7feb300b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb3015a890 0x7feb300b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb300d1700 con 0x7feb301f5120 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb300d1580 con 0x7feb18005260 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7feb300d1880 con 0x7feb3015a890 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 0x7feb300b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 0x7feb300b66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55878/0 (socket says 172.21.15.16:55878) 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 -- 172.21.15.16:0/399024484 learned_addr learned my addr 172.21.15.16:0/399024484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb35000640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb18005260 0x7feb3015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb35a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb3015a890 0x7feb300b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 -- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb3015a890 msgr2=0x7feb300b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb3015a890 0x7feb300b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 -- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb18005260 msgr2=0x7feb3015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb18005260 0x7feb3015fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 -- 172.21.15.16:0/399024484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb300b50c0 con 0x7feb301f5120 2024-09-17T05:08:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2fe00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 0x7feb300b66e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7feb1c004e00 tx=0x7feb1c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c015020 con 0x7feb301f5120 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb1c011960 con 0x7feb301f5120 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301f59e0 con 0x7feb301f5120 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c011d90 con 0x7feb301f5120 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb301f5fa0 con 0x7feb301f5120 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.343+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb301f5750 con 0x7feb301f5120 2024-09-17T05:08:04.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.347+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb1c018020 con 0x7feb301f5120 2024-09-17T05:08:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.347+0000 7feb2cc00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb10073230 0x7feb10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.347+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7feb1c0949a0 con 0x7feb301f5120 2024-09-17T05:08:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.347+0000 7feb35000640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb10073230 0x7feb10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.347+0000 7feb35000640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb10073230 0x7feb10075740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feb300b1bc0 tx=0x7feb240067b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.351+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb1c01f020 con 0x7feb301f5120 2024-09-17T05:08:04.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feb300d1d70 con 0x7feb301f5120 2024-09-17T05:08:04.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb2cc00640 1 -- 172.21.15.16:0/399024484 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) ==== 84+0+65 (secure 0 0 0) 0x7feb1c05f490 con 0x7feb301f5120 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:04.526 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb10073230 msgr2=0x7feb10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb10073230 0x7feb10075740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feb300b1bc0 tx=0x7feb240067b0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 msgr2=0x7feb300b66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 0x7feb300b66e0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7feb1c004e00 tx=0x7feb1c001640 comp rx=0 tx=0).stop 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 shutdown_connections 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7feb10073230 0x7feb10075740 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7feb3015a890 0x7feb300b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb301f5120 0x7feb300b66e0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 --2- 172.21.15.16:0/399024484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb18005260 0x7feb3015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 >> 172.21.15.16:0/399024484 conn(0x7feb300d2270 msgr2=0x7feb3015b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:04.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 shutdown_connections 2024-09-17T05:08:04.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.527+0000 7feb37a00640 1 -- 172.21.15.16:0/399024484 wait complete. 2024-09-17T05:08:04.548 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:08:04.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 -- 172.21.15.16:0/1929199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 msgr2=0x7f68fc0d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 --2- 172.21.15.16:0/1929199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc0d1ae0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009900 tx=0x7f68ec01c8a0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 -- 172.21.15.16:0/1929199478 shutdown_connections 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 --2- 172.21.15.16:0/1929199478 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b7050 0x7f68fc160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 --2- 172.21.15.16:0/1929199478 >> v2:172.21.15.104:3300/0 conn(0x7f68fc0b6c40 0x7f68fc15ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 --2- 172.21.15.16:0/1929199478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc0d1ae0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 -- 172.21.15.16:0/1929199478 >> 172.21.15.16:0/1929199478 conn(0x7f68fc0d2240 msgr2=0x7f68fc0d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 -- 172.21.15.16:0/1929199478 shutdown_connections 2024-09-17T05:08:04.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.715+0000 7f6901c00640 1 -- 172.21.15.16:0/1929199478 wait complete. 2024-09-17T05:08:04.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 Processor -- start 2024-09-17T05:08:04.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 -- start start 2024-09-17T05:08:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 0x7f68fc149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc0b7050 0x7f68fc15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc15d560 con 0x7f68fc153490 2024-09-17T05:08:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc15d3e0 con 0x7f68fc0b7050 2024-09-17T05:08:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f6901c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc15d6e0 con 0x7f68fc0b6c40 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68faa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 0x7f68fc149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68faa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 0x7f68fc149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:58470/0 (socket says 172.21.15.16:58470) 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68faa00640 1 -- 172.21.15.16:0/1902378356 learned_addr learned my addr 172.21.15.16:0/1902378356 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68f2000640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc0b7050 0x7f68fc15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68fb400640 1 -- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 msgr2=0x7f68fc149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68fb400640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 0x7f68fc149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68fb400640 1 -- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc0b7050 msgr2=0x7f68fc15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68fb400640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc0b7050 0x7f68fc15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68fb400640 1 -- 172.21.15.16:0/1902378356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc1f6e50 con 0x7f68fc153490 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.719+0000 7f68faa00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 0x7f68fc149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f68fb400640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc15f460 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d950 tx=0x7f68f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4017030 con 0x7f68fc153490 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68f4004c20 con 0x7f68fc153490 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f400bd70 con 0x7f68fc153490 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc1f7130 con 0x7f68fc153490 2024-09-17T05:08:04.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1f76f0 con 0x7f68fc153490 2024-09-17T05:08:04.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc0d1ae0 con 0x7f68fc153490 2024-09-17T05:08:04.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.723+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68f4003980 con 0x7f68fc153490 2024-09-17T05:08:04.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.727+0000 7f68f3400640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:04.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.727+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5249+0+0 (secure 0 0 0) 0x7f68f4094380 con 0x7f68fc153490 2024-09-17T05:08:04.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.727+0000 7f68faa00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:04.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.727+0000 7f68faa00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f68ec01d140 tx=0x7f68ec0a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.731+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f68f4003c70 con 0x7f68fc153490 2024-09-17T05:08:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:04.907+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f68fc152480 con 0x7f68fc153490 2024-09-17T05:08:05.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.723+0000 7f68f3400640 1 -- 172.21.15.16:0/1902378356 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) ==== 138+0+0 (secure 0 0 0) 0x7f68f405f4c0 con 0x7f68fc153490 2024-09-17T05:08:05.724 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 msgr2=0x7f68d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f68ec01d140 tx=0x7f68ec0a7040 comp rx=0 tx=0).stop 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 msgr2=0x7f68fc15f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc15f460 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f68f400d950 tx=0x7f68f400de10 comp rx=0 tx=0).stop 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 shutdown_connections 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc153490 0x7f68fc15f460 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc0b7050 0x7f68fc15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 --2- 172.21.15.16:0/1902378356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b6c40 0x7f68fc149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 >> 172.21.15.16:0/1902378356 conn(0x7f68fc0d2240 msgr2=0x7f68fc14e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:05.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 shutdown_connections 2024-09-17T05:08:05.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.727+0000 7f6901c00640 1 -- 172.21.15.16:0/1902378356 wait complete. 2024-09-17T05:08:05.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:08:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 -- 172.21.15.16:0/4104091682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb0152710 msgr2=0x7fafb006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 --2- 172.21.15.16:0/4104091682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb0152710 0x7fafb006dfa0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fafb0074d70 tx=0x7faf9c01c830 comp rx=0 tx=0).stop 2024-09-17T05:08:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 -- 172.21.15.16:0/4104091682 shutdown_connections 2024-09-17T05:08:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 --2- 172.21.15.16:0/4104091682 >> v2:172.21.15.104:3300/0 conn(0x7fafb00b5050 0x7fafb006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 --2- 172.21.15.16:0/4104091682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb0152710 0x7fafb006dfa0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 --2- 172.21.15.16:0/4104091682 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 0x7fafb00b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 -- 172.21.15.16:0/4104091682 >> 172.21.15.16:0/4104091682 conn(0x7fafb006f9c0 msgr2=0x7fafb00b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 -- 172.21.15.16:0/4104091682 shutdown_connections 2024-09-17T05:08:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.963+0000 7fafb6a00640 1 -- 172.21.15.16:0/4104091682 wait complete. 2024-09-17T05:08:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 Processor -- start 2024-09-17T05:08:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 -- start start 2024-09-17T05:08:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 0x7fafb01553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 0x7fafb01f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafb0152710 0x7fafb01f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:05.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafb015c000 con 0x7fafb00b5050 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fafb015be80 con 0x7fafb0152710 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fafb015c180 con 0x7fafb0152300 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 0x7fafb01553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 0x7fafb01553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55904/0 (socket says 172.21.15.16:55904) 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 -- 172.21.15.16:0/1397417599 learned_addr learned my addr 172.21.15.16:0/1397417599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb4c00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 0x7fafb01f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 -- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 msgr2=0x7fafb01f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6000640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafb0152710 0x7fafb01f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 0x7fafb01f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 -- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafb0152710 msgr2=0x7fafb01f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:05.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafb0152710 0x7fafb01f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 -- 172.21.15.16:0/1397417599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb014bfc0 con 0x7fafb00b5050 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb4c00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 0x7fafb01f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb6000640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafb0152710 0x7fafb01f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafb5600640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 0x7fafb01553d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d750 tx=0x7fafa400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.967+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4017070 con 0x7fafb00b5050 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafa4009d20 con 0x7fafb00b5050 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa4005280 con 0x7fafb00b5050 2024-09-17T05:08:05.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb014c2a0 con 0x7fafb00b5050 2024-09-17T05:08:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb014c860 con 0x7fafb00b5050 2024-09-17T05:08:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb015c6c0 con 0x7fafb00b5050 2024-09-17T05:08:05.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fafa4005420 con 0x7fafb00b5050 2024-09-17T05:08:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafad600640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf90073200 0x7faf90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.971+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5619+0+0 (secure 0 0 0) 0x7fafa40947a0 con 0x7fafb00b5050 2024-09-17T05:08:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.975+0000 7fafb4c00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf90073200 0x7faf90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:05.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.975+0000 7fafb4c00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf90073200 0x7faf90075710 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0af000 tx=0x7faf9c01b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:05.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:05.979+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fafa400de60 con 0x7fafb00b5050 2024-09-17T05:08:06.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.155+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fafb006fdd0 con 0x7fafb00b5050 2024-09-17T05:08:06.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.727+0000 7fafad600640 1 -- 172.21.15.16:0/1397417599 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) ==== 170+0+0 (secure 0 0 0) 0x7fafa405f880 con 0x7fafb00b5050 2024-09-17T05:08:06.727 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf90073200 msgr2=0x7faf90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf90073200 0x7faf90075710 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faf9c0af000 tx=0x7faf9c01b070 comp rx=0 tx=0).stop 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 msgr2=0x7fafb01553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 0x7fafb01553d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fafa400d750 tx=0x7fafa400dc10 comp rx=0 tx=0).stop 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 shutdown_connections 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7faf90073200 0x7faf90075710 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafb0152710 0x7fafb01f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafb0152300 0x7fafb01f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 --2- 172.21.15.16:0/1397417599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafb00b5050 0x7fafb01553d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 >> 172.21.15.16:0/1397417599 conn(0x7fafb006f9c0 msgr2=0x7fafb014d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:06.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 shutdown_connections 2024-09-17T05:08:06.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.731+0000 7fafb6a00640 1 -- 172.21.15.16:0/1397417599 wait complete. 2024-09-17T05:08:06.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:08:06.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3491449063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac0744d0 msgr2=0x7fbeac0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:06.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3491449063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac0744d0 0x7fbeac0cd210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009840 tx=0x7fbe9801c740 comp rx=0 tx=0).stop 2024-09-17T05:08:06.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3491449063 shutdown_connections 2024-09-17T05:08:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3491449063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac0744d0 0x7fbeac0cd210 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3491449063 >> 172.21.15.16:0/3491449063 conn(0x7fbeac0b2dd0 msgr2=0x7fbeac0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3491449063 shutdown_connections 2024-09-17T05:08:06.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.935+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3491449063 wait complete. 2024-09-17T05:08:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 Processor -- start 2024-09-17T05:08:06.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 -- start start 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac06dfa0 0x7fbeac14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 0x7fbeac0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbeac14e600 0x7fbeac14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac16a030 con 0x7fbeac0cce00 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac169eb0 con 0x7fbeac06dfa0 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac16a1b0 con 0x7fbeac14e600 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 0x7fbeac0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 0x7fbeac0b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55906/0 (socket says 172.21.15.16:55906) 2024-09-17T05:08:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 -- 172.21.15.16:0/3429933731 learned_addr learned my addr 172.21.15.16:0/3429933731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb1000640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac06dfa0 0x7fbeac14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb1a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbeac14e600 0x7fbeac14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 -- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbeac14e600 msgr2=0x7fbeac14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbeac14e600 0x7fbeac14ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 -- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac06dfa0 msgr2=0x7fbeac14fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac06dfa0 0x7fbeac14fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 -- 172.21.15.16:0/3429933731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac0b5670 con 0x7fbeac0cce00 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb1a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbeac14e600 0x7fbeac14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeabe00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 0x7fbeac0b5130 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbe9801cc20 tx=0x7fbe98005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:06.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980be070 con 0x7fbeac0cce00 2024-09-17T05:08:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe980b0860 con 0x7fbeac0cce00 2024-09-17T05:08:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac1437d0 con 0x7fbeac0cce00 2024-09-17T05:08:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980b0c10 con 0x7fbeac0cce00 2024-09-17T05:08:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.939+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac0b4c10 con 0x7fbeac0cce00 2024-09-17T05:08:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.943+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac14f1f0 con 0x7fbeac0cce00 2024-09-17T05:08:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.943+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe98003600 con 0x7fbeac0cce00 2024-09-17T05:08:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.943+0000 7fbea8c00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe8c073200 0x7fbe8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.943+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6001+0+0 (secure 0 0 0) 0x7fbe9812e790 con 0x7fbeac0cce00 2024-09-17T05:08:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.943+0000 7fbeb1000640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe8c073200 0x7fbe8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.943+0000 7fbeb1000640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe8c073200 0x7fbe8c075710 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbea0007fd0 tx=0x7fbea0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:06.947+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe980ac030 con 0x7fbeac0cce00 2024-09-17T05:08:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.159+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbeac0cd210 con 0x7fbeac0cce00 2024-09-17T05:08:07.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.763+0000 7fbea8c00640 1 -- 172.21.15.16:0/3429933731 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v71) ==== 174+0+0 (secure 0 0 0) 0x7fbe98016330 con 0x7fbeac0cce00 2024-09-17T05:08:07.773 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 17 and data pool 18 2024-09-17T05:08:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe8c073200 msgr2=0x7fbe8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe8c073200 0x7fbe8c075710 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbea0007fd0 tx=0x7fbea0009040 comp rx=0 tx=0).stop 2024-09-17T05:08:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 msgr2=0x7fbeac0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 0x7fbeac0b5130 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fbe9801cc20 tx=0x7fbe98005d90 comp rx=0 tx=0).stop 2024-09-17T05:08:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 shutdown_connections 2024-09-17T05:08:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe8c073200 0x7fbe8c075710 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbeac14e600 0x7fbeac14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeac0cce00 0x7fbeac0b5130 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 --2- 172.21.15.16:0/3429933731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac06dfa0 0x7fbeac14fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.779+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 >> 172.21.15.16:0/3429933731 conn(0x7fbeac0b2dd0 msgr2=0x7fbeac06e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.783+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 shutdown_connections 2024-09-17T05:08:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.783+0000 7fbeb3a00640 1 -- 172.21.15.16:0/3429933731 wait complete. 2024-09-17T05:08:07.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 -- 172.21.15.16:0/2169911044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380d0010 msgr2=0x7fc9380d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 --2- 172.21.15.16:0/2169911044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380d0010 0x7fc9380d0420 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc9380c92f0 tx=0x7fc93001cd80 comp rx=0 tx=0).stop 2024-09-17T05:08:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 -- 172.21.15.16:0/2169911044 shutdown_connections 2024-09-17T05:08:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 --2- 172.21.15.16:0/2169911044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380d0010 0x7fc9380d0420 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 --2- 172.21.15.16:0/2169911044 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9380d1580 0x7fc9380d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 --2- 172.21.15.16:0/2169911044 >> v2:172.21.15.104:3300/0 conn(0x7fc9380cc8c0 0x7fc93815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 -- 172.21.15.16:0/2169911044 >> 172.21.15.16:0/2169911044 conn(0x7fc9380d2270 msgr2=0x7fc9380d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 -- 172.21.15.16:0/2169911044 shutdown_connections 2024-09-17T05:08:07.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.979+0000 7fc940400640 1 -- 172.21.15.16:0/2169911044 wait complete. 2024-09-17T05:08:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 Processor -- start 2024-09-17T05:08:07.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 -- start start 2024-09-17T05:08:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 0x7fc93806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:07.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9380d0010 0x7fc9380ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9380d1580 0x7fc9381f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380ca120 con 0x7fc9380cc8c0 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9380c9fa0 con 0x7fc9380d1580 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc940400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc9380ca2a0 con 0x7fc9380d0010 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc93da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 0x7fc93806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc93da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 0x7fc93806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33966/0 (socket says 172.21.15.16:33966) 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.983+0000 7fc93da00640 1 -- 172.21.15.16:0/2056658862 learned_addr learned my addr 172.21.15.16:0/2056658862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93d000640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9380d0010 0x7fc9380ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93e400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9380d1580 0x7fc9381f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93da00640 1 -- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9380d0010 msgr2=0x7fc9380ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93da00640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9380d0010 0x7fc9380ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93da00640 1 -- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9380d1580 msgr2=0x7fc9381f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93da00640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9380d1580 0x7fc9381f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93da00640 1 -- 172.21.15.16:0/2056658862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9380b0450 con 0x7fc9380cc8c0 2024-09-17T05:08:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc93da00640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 0x7fc93806f280 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e770 tx=0x7fc92c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:07.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00ccc0 con 0x7fc9380cc8c0 2024-09-17T05:08:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc92c00ee80 con 0x7fc9380cc8c0 2024-09-17T05:08:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00ce60 con 0x7fc9380cc8c0 2024-09-17T05:08:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9380b0fd0 con 0x7fc9380cc8c0 2024-09-17T05:08:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9380b0a10 con 0x7fc9380cc8c0 2024-09-17T05:08:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9380d0420 con 0x7fc9380cc8c0 2024-09-17T05:08:07.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.987+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc92c004260 con 0x7fc9380cc8c0 2024-09-17T05:08:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.991+0000 7fc935600640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc918073200 0x7fc918075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.991+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7fc92c094cb0 con 0x7fc9380cc8c0 2024-09-17T05:08:07.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.991+0000 7fc93d000640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc918073200 0x7fc918075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:07.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.991+0000 7fc93d000640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc918073200 0x7fc918075710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc938070700 tx=0x7fc9240091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:07.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:07.995+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc92c05f450 con 0x7fc9380cc8c0 2024-09-17T05:08:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc938071dd0 con 0x7fc9380cc8c0 2024-09-17T05:08:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc935600640 1 -- 172.21.15.16:0/2056658862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17017 (secure 0 0 0) 0x7fc92c063e80 con 0x7fc9380cc8c0 2024-09-17T05:08:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:08.170 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":112,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:07.160649+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:04.918539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:06.167190+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc918073200 msgr2=0x7fc918075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc918073200 0x7fc918075710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc938070700 tx=0x7fc9240091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 msgr2=0x7fc93806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 0x7fc93806f280 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00e770 tx=0x7fc92c00ec30 comp rx=0 tx=0).stop 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 shutdown_connections 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc918073200 0x7fc918075710 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9380d1580 0x7fc9381f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.171+0000 7fc940400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9380d0010 0x7fc9380ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.175+0000 7fc940400640 1 --2- 172.21.15.16:0/2056658862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9380cc8c0 0x7fc93806f280 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.175+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 >> 172.21.15.16:0/2056658862 conn(0x7fc9380d2270 msgr2=0x7fc93815a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.175+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 shutdown_connections 2024-09-17T05:08:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.175+0000 7fc940400640 1 -- 172.21.15.16:0/2056658862 wait complete. 2024-09-17T05:08:08.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 -- 172.21.15.16:0/2586875972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0160880 msgr2=0x7fc9f0167500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2586875972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0160880 0x7fc9f0167500 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0072860 tx=0x7fc9e801cb80 comp rx=0 tx=0).stop 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 -- 172.21.15.16:0/2586875972 shutdown_connections 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2586875972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f0160880 0x7fc9f0167500 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2586875972 >> v2:172.21.15.104:3300/0 conn(0x7fc9f015d600 0x7fc9f00cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 -- 172.21.15.16:0/2586875972 >> 172.21.15.16:0/2586875972 conn(0x7fc9f006f900 msgr2=0x7fc9f00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 -- 172.21.15.16:0/2586875972 shutdown_connections 2024-09-17T05:08:08.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 -- 172.21.15.16:0/2586875972 wait complete. 2024-09-17T05:08:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 Processor -- start 2024-09-17T05:08:08.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.355+0000 7fc9f7400640 1 -- start start 2024-09-17T05:08:08.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f00cf390 0x7fc9f00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 0x7fc9f00aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9f0160880 0x7fc9f01598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f016a260 con 0x7fc9f015d600 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00b5280 con 0x7fc9f00cf390 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f016a3e0 con 0x7fc9f0160880 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 0x7fc9f00aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 0x7fc9f00aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33978/0 (socket says 172.21.15.16:33978) 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 -- 172.21.15.16:0/2125989939 learned_addr learned my addr 172.21.15.16:0/2125989939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f4a00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f00cf390 0x7fc9f00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f5400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9f0160880 0x7fc9f01598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 -- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9f0160880 msgr2=0x7fc9f01598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9f0160880 0x7fc9f01598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 -- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f00cf390 msgr2=0x7fc9f00ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f00cf390 0x7fc9f00ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 -- 172.21.15.16:0/2125989939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f0074a50 con 0x7fc9f015d600 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f5400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9f0160880 0x7fc9f01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9efe00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 0x7fc9f00aedf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c3f0 tx=0x7fc9dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0068a0 con 0x7fc9f015d600 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9dc006b50 con 0x7fc9f015d600 2024-09-17T05:08:08.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f0158e00 con 0x7fc9f015d600 2024-09-17T05:08:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc007020 con 0x7fc9f015d600 2024-09-17T05:08:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f0074d30 con 0x7fc9f015d600 2024-09-17T05:08:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.359+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f00cee50 con 0x7fc9f015d600 2024-09-17T05:08:08.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.363+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc9dc019020 con 0x7fc9f015d600 2024-09-17T05:08:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.363+0000 7fc9ecc00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc9d0073200 0x7fc9d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.363+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7fc9dc093a50 con 0x7fc9f015d600 2024-09-17T05:08:08.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.363+0000 7fc9f4a00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc9d0073200 0x7fc9d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.363+0000 7fc9f4a00640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc9d0073200 0x7fc9d0075710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00afb90 tx=0x7fc9e40091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:08.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.367+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc9dc05e8d0 con 0x7fc9f015d600 2024-09-17T05:08:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9f0146140 con 0x7fc9f015d600 2024-09-17T05:08:08.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9ecc00640 1 -- 172.21.15.16:0/2125989939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+17017 (secure 0 0 0) 0x7fc9dc05e1d0 con 0x7fc9f015d600 2024-09-17T05:08:08.542 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:08.542 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":112,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:07.160649+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:04.918539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:06.167190+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc9d0073200 msgr2=0x7fc9d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc9d0073200 0x7fc9d0075710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00afb90 tx=0x7fc9e40091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 msgr2=0x7fc9f00aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 0x7fc9f00aedf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c3f0 tx=0x7fc9dc00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 shutdown_connections 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc9d0073200 0x7fc9d0075710 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc9f0160880 0x7fc9f01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f015d600 0x7fc9f00aedf0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 --2- 172.21.15.16:0/2125989939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9f00cf390 0x7fc9f00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 >> 172.21.15.16:0/2125989939 conn(0x7fc9f006f900 msgr2=0x7fc9f0145250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.543+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 shutdown_connections 2024-09-17T05:08:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.547+0000 7fc9f7400640 1 -- 172.21.15.16:0/2125989939 wait complete. 2024-09-17T05:08:08.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:08:08.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f7804c00640 1 -- 172.21.15.16:0/3382744557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 msgr2=0x7f77e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f7804c00640 1 --2- 172.21.15.16:0/3382744557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f77e4004810 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009b40 tx=0x7f77e801d270 comp rx=0 tx=0).stop 2024-09-17T05:08:08.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f77fe800640 1 -- 172.21.15.16:0/3382744557 reap_dead start 2024-09-17T05:08:08.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f7804c00640 1 -- 172.21.15.16:0/3382744557 shutdown_connections 2024-09-17T05:08:08.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f7804c00640 1 -- 172.21.15.16:0/3382744557 >> 172.21.15.16:0/3382744557 conn(0x7f78000b2250 msgr2=0x7f78000d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:08.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f7804c00640 1 -- 172.21.15.16:0/3382744557 shutdown_connections 2024-09-17T05:08:08.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.727+0000 7f7804c00640 1 -- 172.21.15.16:0/3382744557 wait complete. 2024-09-17T05:08:08.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 Processor -- start 2024-09-17T05:08:08.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 -- start start 2024-09-17T05:08:08.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f78001f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7800146430 0x7f78000c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78000c9d30 0x7f78000c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78000737f0 con 0x7f77e4004010 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78000cc890 con 0x7f78000c9d30 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f7804c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7800073970 con 0x7f7800146430 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f78001f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f78001f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34008/0 (socket says 172.21.15.16:34008) 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 -- 172.21.15.16:0/3926064136 learned_addr learned my addr 172.21.15.16:0/3926064136 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fe800640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78000c9d30 0x7f78000c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77f7e00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7800146430 0x7f78000c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 -- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7800146430 msgr2=0x7f78000c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7800146430 0x7f78000c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 -- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78000c9d30 msgr2=0x7f78000c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78000c9d30 0x7f78000c3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 -- 172.21.15.16:0/3926064136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78001f7b40 con 0x7f77e4004010 2024-09-17T05:08:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fe800640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78000c9d30 0x7f78000c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.731+0000 7f77fde00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f78001f7600 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f77f0004f10 tx=0x7f77f0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0015020 con 0x7f77e4004010 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77f0011960 con 0x7f77e4004010 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78000ca140 con 0x7f77e4004010 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011d90 con 0x7f77e4004010 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78000c5110 con 0x7f77e4004010 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780006d130 con 0x7f77e4004010 2024-09-17T05:08:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.735+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77f0018020 con 0x7f77e4004010 2024-09-17T05:08:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.739+0000 7f77f6000640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f77dc073200 0x7f77dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.739+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 6101+0+0 (secure 0 0 0) 0x7f77f0094fb0 con 0x7f77e4004010 2024-09-17T05:08:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.739+0000 7f77f7e00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f77dc073200 0x7f77dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:08.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.739+0000 7f77f7e00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f77dc073200 0x7f77dc075710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f77e80ad000 tx=0x7f77e801a010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.743+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77f001f020 con 0x7f77e4004010 2024-09-17T05:08:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:08.959+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f78000af970 con 0x7f77e4004010 2024-09-17T05:08:09.725 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-17T05:08:09.727+0000 7fca8c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:08:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f77f6000640 1 -- 172.21.15.16:0/3926064136 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) ==== 118+0+0 (secure 0 0 0) 0x7f78000ca140 con 0x7f77e4004010 2024-09-17T05:08:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f77dc073200 msgr2=0x7f77dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f77dc073200 0x7f77dc075710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f77e80ad000 tx=0x7f77e801a010 comp rx=0 tx=0).stop 2024-09-17T05:08:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 msgr2=0x7f78001f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f78001f7600 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f77f0004f10 tx=0x7f77f0001640 comp rx=0 tx=0).stop 2024-09-17T05:08:09.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 shutdown_connections 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f77dc073200 0x7f77dc075710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78000c9d30 0x7f78000c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7800146430 0x7f78000c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 --2- 172.21.15.16:0/3926064136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77e4004010 0x7f78001f7600 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 >> 172.21.15.16:0/3926064136 conn(0x7f78000b2250 msgr2=0x7f780015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 shutdown_connections 2024-09-17T05:08:09.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.775+0000 7f7804c00640 1 -- 172.21.15.16:0/3926064136 wait complete. 2024-09-17T05:08:09.801 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:08:09.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 -- 172.21.15.16:0/4128339619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074a20 msgr2=0x7fcbac15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 --2- 172.21.15.16:0/4128339619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074a20 0x7fcbac15efe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009840 tx=0x7fcb9c01c820 comp rx=0 tx=0).stop 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 -- 172.21.15.16:0/4128339619 shutdown_connections 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 --2- 172.21.15.16:0/4128339619 >> v2:172.21.15.104:3300/0 conn(0x7fcbac074e30 0x7fcbac15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 --2- 172.21.15.16:0/4128339619 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074a20 0x7fcbac15efe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 -- 172.21.15.16:0/4128339619 >> 172.21.15.16:0/4128339619 conn(0x7fcbac0b2250 msgr2=0x7fcbac0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 -- 172.21.15.16:0/4128339619 shutdown_connections 2024-09-17T05:08:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 -- 172.21.15.16:0/4128339619 wait complete. 2024-09-17T05:08:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 Processor -- start 2024-09-17T05:08:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.971+0000 7fcbb1800640 1 -- start start 2024-09-17T05:08:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac074a20 0x7fcbac14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074e30 0x7fcbac150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 0x7fcbac15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac168e80 con 0x7fcbac15ba50 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac168d00 con 0x7fcbac074a20 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcbac169000 con 0x7fcbac074e30 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 0x7fcbac15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 0x7fcbac15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34016/0 (socket says 172.21.15.16:34016) 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 -- 172.21.15.16:0/3603906532 learned_addr learned my addr 172.21.15.16:0/3603906532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbaaa00640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac074a20 0x7fcbac14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcba3e00640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074e30 0x7fcbac150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 -- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074e30 msgr2=0x7fcbac150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074e30 0x7fcbac150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 -- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac074a20 msgr2=0x7fcbac14d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac074a20 0x7fcbac14d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 -- 172.21.15.16:0/3603906532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac152740 con 0x7fcbac15ba50 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbab400640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 0x7fcbac15a190 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcba400cad0 tx=0x7fcba400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4016020 con 0x7fcbac15ba50 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcba400a950 con 0x7fcbac15ba50 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac1f8ca0 con 0x7fcbac15ba50 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba400ada0 con 0x7fcbac15ba50 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac1f9260 con 0x7fcbac15ba50 2024-09-17T05:08:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.975+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbac14f530 con 0x7fcbac15ba50 2024-09-17T05:08:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.979+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcba4019020 con 0x7fcbac15ba50 2024-09-17T05:08:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.979+0000 7fcba2a00640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb88073200 0x7fcb88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.979+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fcba4094520 con 0x7fcbac15ba50 2024-09-17T05:08:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.979+0000 7fcbaaa00640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb88073200 0x7fcb88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.979+0000 7fcbaaa00640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb88073200 0x7fcb88075710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0af000 tx=0x7fcb9c01b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:09.983+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcba4020020 con 0x7fcbac15ba50 2024-09-17T05:08:10.110 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T05:08:10.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.199+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcbac15be60 con 0x7fcbac15ba50 2024-09-17T05:08:10.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.778+0000 7fcba2a00640 1 -- 172.21.15.16:0/3603906532 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) ==== 122+0+0 (secure 0 0 0) 0x7fcba407ee20 con 0x7fcbac15ba50 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb88073200 msgr2=0x7fcb88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb88073200 0x7fcb88075710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0af000 tx=0x7fcb9c01b060 comp rx=0 tx=0).stop 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 msgr2=0x7fcbac15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 0x7fcbac15a190 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcba400cad0 tx=0x7fcba400cbc0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 shutdown_connections 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb88073200 0x7fcb88075710 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac15ba50 0x7fcbac15a190 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcbac074e30 0x7fcbac150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 --2- 172.21.15.16:0/3603906532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac074a20 0x7fcbac14d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 >> 172.21.15.16:0/3603906532 conn(0x7fcbac0b2250 msgr2=0x7fcbac0cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:10.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 shutdown_connections 2024-09-17T05:08:10.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.782+0000 7fcbb1800640 1 -- 172.21.15.16:0/3603906532 wait complete. 2024-09-17T05:08:10.802 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:08:10.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 -- 172.21.15.16:0/1907024697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c001606b0 msgr2=0x7f3c00160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:10.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 --2- 172.21.15.16:0/1907024697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c001606b0 0x7f3c00160ac0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3c00070f40 tx=0x7f3bf801cd80 comp rx=0 tx=0).stop 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 -- 172.21.15.16:0/1907024697 shutdown_connections 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 --2- 172.21.15.16:0/1907024697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c001606b0 0x7f3c00160ac0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 --2- 172.21.15.16:0/1907024697 >> v2:172.21.15.104:3300/0 conn(0x7f3c000cf390 0x7f3c00160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 --2- 172.21.15.16:0/1907024697 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c0015d600 0x7f3c000cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 -- 172.21.15.16:0/1907024697 >> 172.21.15.16:0/1907024697 conn(0x7f3c0006f900 msgr2=0x7f3c000b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 -- 172.21.15.16:0/1907024697 shutdown_connections 2024-09-17T05:08:10.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.962+0000 7f3c06600640 1 -- 172.21.15.16:0/1907024697 wait complete. 2024-09-17T05:08:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 Processor -- start 2024-09-17T05:08:10.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 -- start start 2024-09-17T05:08:10.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 0x7f3c001422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:10.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c0015d600 0x7f3c000a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:10.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c001606b0 0x7f3c000b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c000b4320 con 0x7f3c000cf390 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c000b41a0 con 0x7f3c0015d600 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3c000b44a0 con 0x7f3c001606b0 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 0x7f3c001422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 0x7f3c001422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34032/0 (socket says 172.21.15.16:34032) 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 -- 172.21.15.16:0/3649957690 learned_addr learned my addr 172.21.15.16:0/3649957690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bf7e00640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c0015d600 0x7f3c000a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bffe00640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c001606b0 0x7f3c000b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 -- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c001606b0 msgr2=0x7f3c000b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c001606b0 0x7f3c000b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 -- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c0015d600 msgr2=0x7f3c000a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:10.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c0015d600 0x7f3c000a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 -- 172.21.15.16:0/3649957690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c00142a80 con 0x7f3c000cf390 2024-09-17T05:08:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bffe00640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c001606b0 0x7f3c000b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bf7e00640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c0015d600 0x7f3c000a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:10.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bff400640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 0x7f3c001422d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000e7c0 tx=0x7f3bf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000ccc0 con 0x7f3c000cf390 2024-09-17T05:08:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bf000ce80 con 0x7f3c000cf390 2024-09-17T05:08:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c00144bb0 con 0x7f3c000cf390 2024-09-17T05:08:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0004290 con 0x7f3c000cf390 2024-09-17T05:08:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.966+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c00145170 con 0x7f3c000cf390 2024-09-17T05:08:10.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.970+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c00142c50 con 0x7f3c000cf390 2024-09-17T05:08:10.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.970+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bf0003600 con 0x7f3c000cf390 2024-09-17T05:08:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.970+0000 7f3bf7400640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bdc073230 0x7f3bdc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:10.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.970+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3bf0095290 con 0x7f3c000cf390 2024-09-17T05:08:10.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.970+0000 7f3bf7e00640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bdc073230 0x7f3bdc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:10.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.974+0000 7f3bf7e00640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bdc073230 0x7f3bdc075740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3c001579f0 tx=0x7f3be8005d50 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:10.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:10.974+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3bf0060100 con 0x7f3c000cf390 2024-09-17T05:08:11.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.182+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c000cee50 con 0x7f3c000cf390 2024-09-17T05:08:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.182+0000 7f3bf7400640 1 -- 172.21.15.16:0/3649957690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7f3bf005fa00 con 0x7f3c000cf390 2024-09-17T05:08:11.184 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:11.184 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"btime":"2024-09-17T05:08:10:776523+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:10.202627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:11.184 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 75 2024-09-17T05:08:11.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bdc073230 msgr2=0x7f3bdc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bdc073230 0x7f3bdc075740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3c001579f0 tx=0x7f3be8005d50 comp rx=0 tx=0).stop 2024-09-17T05:08:11.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 msgr2=0x7f3c001422d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 0x7f3c001422d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000e7c0 tx=0x7f3bf000ec80 comp rx=0 tx=0).stop 2024-09-17T05:08:11.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 shutdown_connections 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bdc073230 0x7f3bdc075740 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3c001606b0 0x7f3c000b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c0015d600 0x7f3c000a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 --2- 172.21.15.16:0/3649957690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c000cf390 0x7f3c001422d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 >> 172.21.15.16:0/3649957690 conn(0x7f3c0006f900 msgr2=0x7f3c0015ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 shutdown_connections 2024-09-17T05:08:11.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.186+0000 7f3c06600640 1 -- 172.21.15.16:0/3649957690 wait complete. 2024-09-17T05:08:11.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 -- 172.21.15.16:0/730648866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd8071920 msgr2=0x7f3bd80b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 --2- 172.21.15.16:0/730648866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd8071920 0x7f3bd80b61e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009900 tx=0x7f3bc401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 -- 172.21.15.16:0/730648866 shutdown_connections 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 --2- 172.21.15.16:0/730648866 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd806e030 0x7f3bd806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 --2- 172.21.15.16:0/730648866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd8071920 0x7f3bd80b61e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 -- 172.21.15.16:0/730648866 >> 172.21.15.16:0/730648866 conn(0x7f3bd80d10f0 msgr2=0x7f3bd80d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 -- 172.21.15.16:0/730648866 shutdown_connections 2024-09-17T05:08:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.362+0000 7f3bde400640 1 -- 172.21.15.16:0/730648866 wait complete. 2024-09-17T05:08:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 Processor -- start 2024-09-17T05:08:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 -- start start 2024-09-17T05:08:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bd806e030 0x7f3bd8148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd8152740 0x7f3bd815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 0x7f3bd815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd8168f90 con 0x7f3bd814b690 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd8168e10 con 0x7f3bd806e030 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bde400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd8169110 con 0x7f3bd8152740 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bdda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 0x7f3bd815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bd806e030 0x7f3bd8148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bdda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 0x7f3bd815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34046/0 (socket says 172.21.15.16:34046) 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.366+0000 7f3bdda00640 1 -- 172.21.15.16:0/2994516202 learned_addr learned my addr 172.21.15.16:0/2994516202 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bd7e00640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd8152740 0x7f3bd815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bdda00640 1 -- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd8152740 msgr2=0x7f3bd815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bdda00640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd8152740 0x7f3bd815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bdda00640 1 -- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bd806e030 msgr2=0x7f3bd8148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bdda00640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bd806e030 0x7f3bd8148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bdda00640 1 -- 172.21.15.16:0/2994516202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd80d04a0 con 0x7f3bd814b690 2024-09-17T05:08:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bdda00640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 0x7f3bd815a9b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e910 tx=0x7f3bd000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0004d30 con 0x7f3bd814b690 2024-09-17T05:08:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd000cb20 con 0x7f3bd814b690 2024-09-17T05:08:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd80b2b90 con 0x7f3bd814b690 2024-09-17T05:08:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0005020 con 0x7f3bd814b690 2024-09-17T05:08:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd8149460 con 0x7f3bd814b690 2024-09-17T05:08:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd8148ef0 con 0x7f3bd814b690 2024-09-17T05:08:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.370+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bd00051c0 con 0x7f3bd814b690 2024-09-17T05:08:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.374+0000 7f3bd4c00640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb8073200 0x7f3bb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.374+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3bd0094bc0 con 0x7f3bd814b690 2024-09-17T05:08:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.374+0000 7f3bdd000640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb8073200 0x7f3bb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.374+0000 7f3bdd000640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb8073200 0x7f3bb8075710 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc007fd0 tx=0x7f3bcc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:11.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.378+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3bd0016030 con 0x7f3bd814b690 2024-09-17T05:08:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.550+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bd806e440 con 0x7f3bd814b690 2024-09-17T05:08:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bd4c00640 1 -- 172.21.15.16:0/2994516202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17004 (secure 0 0 0) 0x7f3bd005f3f0 con 0x7f3bd814b690 2024-09-17T05:08:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:11.553 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":113,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:08.760898+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:04.918539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:06.167190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb8073200 msgr2=0x7f3bb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb8073200 0x7f3bb8075710 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc007fd0 tx=0x7f3bcc009040 comp rx=0 tx=0).stop 2024-09-17T05:08:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 msgr2=0x7f3bd815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 0x7f3bd815a9b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e910 tx=0x7f3bd000edd0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 shutdown_connections 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3bb8073200 0x7f3bb8075710 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3bd814b690 0x7f3bd815a9b0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3bd8152740 0x7f3bd815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 --2- 172.21.15.16:0/2994516202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bd806e030 0x7f3bd8148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 >> 172.21.15.16:0/2994516202 conn(0x7f3bd80d10f0 msgr2=0x7f3bd80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 shutdown_connections 2024-09-17T05:08:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.554+0000 7f3bde400640 1 -- 172.21.15.16:0/2994516202 wait complete. 2024-09-17T05:08:11.576 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:08:11.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 -- 172.21.15.16:0/1331677404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 msgr2=0x7fdce0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1331677404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce0071210 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009c80 tx=0x7fdcd001cd10 comp rx=0 tx=0).stop 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 -- 172.21.15.16:0/1331677404 shutdown_connections 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1331677404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce015bc20 0x7fdce0071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1331677404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce0071210 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 -- 172.21.15.16:0/1331677404 >> 172.21.15.16:0/1331677404 conn(0x7fdce00d1030 msgr2=0x7fdce00d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 -- 172.21.15.16:0/1331677404 shutdown_connections 2024-09-17T05:08:11.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.734+0000 7fdce4c00640 1 -- 172.21.15.16:0/1331677404 wait complete. 2024-09-17T05:08:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 Processor -- start 2024-09-17T05:08:11.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 -- start start 2024-09-17T05:08:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce006dfa0 0x7fdce014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce00b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce015bc20 0x7fdce0164340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0169060 con 0x7fdce015b810 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0168ee0 con 0x7fdce015bc20 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdce4c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdce01691e0 con 0x7fdce006dfa0 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdcd6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce00b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdcd6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce00b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34068/0 (socket says 172.21.15.16:34068) 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1270738545 learned_addr learned my addr 172.21.15.16:0/1270738545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdcdf400640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce006dfa0 0x7fdce014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.738+0000 7fdcdfe00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce015bc20 0x7fdce0164340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce006dfa0 msgr2=0x7fdce014c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd6a00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce006dfa0 0x7fdce014c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce015bc20 msgr2=0x7fdce0164340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd6a00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce015bc20 0x7fdce0164340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd6a00640 1 -- 172.21.15.16:0/1270738545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce00b19a0 con 0x7fdce015b810 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcdfe00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce015bc20 0x7fdce0164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd6a00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce00b1460 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d700 tx=0x7fdcc800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8017070 con 0x7fdce015b810 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc8009ab0 con 0x7fdce015b810 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8010460 con 0x7fdce015b810 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce01fcc30 con 0x7fdce015b810 2024-09-17T05:08:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce01fd1f0 con 0x7fdce015b810 2024-09-17T05:08:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce006e3b0 con 0x7fdce015b810 2024-09-17T05:08:11.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdcc801a070 con 0x7fdce015b810 2024-09-17T05:08:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.742+0000 7fdcd7e00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073230 0x7fdcbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.746+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fdcc8094400 con 0x7fdce015b810 2024-09-17T05:08:11.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.746+0000 7fdcdf400640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073230 0x7fdcbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.746+0000 7fdcdf400640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073230 0x7fdcbc075740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00af000 tx=0x7fdcd00a5f90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.750+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdcc805ec30 con 0x7fdce015b810 2024-09-17T05:08:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.922+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdce0151cc0 con 0x7fdcbc073230 2024-09-17T05:08:11.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.926+0000 7fdcd7e00640 1 -- 172.21.15.16:0/1270738545 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248047 (secure 0 0 0) 0x7fdce0151cc0 con 0x7fdcbc073230 2024-09-17T05:08:11.930 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:11.932 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:08:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073230 msgr2=0x7fdcbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073230 0x7fdcbc075740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00af000 tx=0x7fdcd00a5f90 comp rx=0 tx=0).stop 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 msgr2=0x7fdce00b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce00b1460 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d700 tx=0x7fdcc800dbc0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 shutdown_connections 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdcbc073230 0x7fdcbc075740 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce015bc20 0x7fdce0164340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdce015b810 0x7fdce00b1460 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 --2- 172.21.15.16:0/1270738545 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdce006dfa0 0x7fdce014c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 >> 172.21.15.16:0/1270738545 conn(0x7fdce00d1030 msgr2=0x7fdce015ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 shutdown_connections 2024-09-17T05:08:11.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:11.930+0000 7fdce4c00640 1 -- 172.21.15.16:0/1270738545 wait complete. 2024-09-17T05:08:11.952 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-17T05:08:11.578325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269780,"kb_used_data":11936,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704940,"statfs":{"total":773094113280,"available":772817858560,"internally_reserved":0,"allocated":12222464,"data_stored":6246212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.851674"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743530+0000","last_change":"2024-09-17T05:08:06.743530+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743530+0000","last_undegraded":"2024-09-17T05:08:06.743530+0000","last_fullsized":"2024-09-17T05:08:06.743530+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774172+0000","last_change":"2024-09-17T05:08:07.761565+0000","last_active":"2024-09-17T05:08:08.774172+0000","last_peered":"2024-09-17T05:08:08.774172+0000","last_clean":"2024-09-17T05:08:08.774172+0000","last_became_active":"2024-09-17T05:08:07.761261+0000","last_became_peered":"2024-09-17T05:08:07.761261+0000","last_unstale":"2024-09-17T05:08:08.774172+0000","last_undegraded":"2024-09-17T05:08:08.774172+0000","last_fullsized":"2024-09-17T05:08:08.774172+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:41.539621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743447+0000","last_change":"2024-09-17T05:08:06.743447+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743447+0000","last_undegraded":"2024-09-17T05:08:06.743447+0000","last_fullsized":"2024-09-17T05:08:06.743447+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748147+0000","last_change":"2024-09-17T05:08:06.748147+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748147+0000","last_undegraded":"2024-09-17T05:08:06.748147+0000","last_fullsized":"2024-09-17T05:08:06.748147+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738577+0000","last_change":"2024-09-17T05:08:06.738577+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738577+0000","last_undegraded":"2024-09-17T05:08:06.738577+0000","last_fullsized":"2024-09-17T05:08:06.738577+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773432+0000","last_change":"2024-09-17T05:08:07.758709+0000","last_active":"2024-09-17T05:08:08.773432+0000","last_peered":"2024-09-17T05:08:08.773432+0000","last_clean":"2024-09-17T05:08:08.773432+0000","last_became_active":"2024-09-17T05:08:07.758382+0000","last_became_peered":"2024-09-17T05:08:07.758382+0000","last_unstale":"2024-09-17T05:08:08.773432+0000","last_undegraded":"2024-09-17T05:08:08.773432+0000","last_fullsized":"2024-09-17T05:08:08.773432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:11.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773005+0000","last_change":"2024-09-17T05:08:07.787949+0000","last_active":"2024-09-17T05:08:08.773005+0000","last_peered":"2024-09-17T05:08:08.773005+0000","last_clean":"2024-09-17T05:08:08.773005+0000","last_became_active":"2024-09-17T05:08:07.787577+0000","last_became_peered":"2024-09-17T05:08:07.787577+0000","last_unstale":"2024-09-17T05:08:08.773005+0000","last_undegraded":"2024-09-17T05:08:08.773005+0000","last_fullsized":"2024-09-17T05:08:08.773005+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:33.927797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740861+0000","last_change":"2024-09-17T05:08:06.740861+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740861+0000","last_undegraded":"2024-09-17T05:08:06.740861+0000","last_fullsized":"2024-09-17T05:08:06.740861+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748951+0000","last_change":"2024-09-17T05:08:06.748951+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748951+0000","last_undegraded":"2024-09-17T05:08:06.748951+0000","last_fullsized":"2024-09-17T05:08:06.748951+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774347+0000","last_change":"2024-09-17T05:08:07.777536+0000","last_active":"2024-09-17T05:08:08.774347+0000","last_peered":"2024-09-17T05:08:08.774347+0000","last_clean":"2024-09-17T05:08:08.774347+0000","last_became_active":"2024-09-17T05:08:07.777182+0000","last_became_peered":"2024-09-17T05:08:07.777182+0000","last_unstale":"2024-09-17T05:08:08.774347+0000","last_undegraded":"2024-09-17T05:08:08.774347+0000","last_fullsized":"2024-09-17T05:08:08.774347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:26.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742789+0000","last_change":"2024-09-17T05:08:06.742789+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742789+0000","last_undegraded":"2024-09-17T05:08:06.742789+0000","last_fullsized":"2024-09-17T05:08:06.742789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.749238+0000","last_change":"2024-09-17T05:08:06.749238+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.749238+0000","last_undegraded":"2024-09-17T05:08:06.749238+0000","last_fullsized":"2024-09-17T05:08:06.749238+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773816+0000","last_change":"2024-09-17T05:08:07.776167+0000","last_active":"2024-09-17T05:08:08.773816+0000","last_peered":"2024-09-17T05:08:08.773816+0000","last_clean":"2024-09-17T05:08:08.773816+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:08.773816+0000","last_undegraded":"2024-09-17T05:08:08.773816+0000","last_fullsized":"2024-09-17T05:08:08.773816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:33.590100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.745703+0000","last_change":"2024-09-17T05:08:06.745703+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.745703+0000","last_undegraded":"2024-09-17T05:08:06.745703+0000","last_fullsized":"2024-09-17T05:08:06.745703+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742487+0000","last_change":"2024-09-17T05:08:06.742487+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742487+0000","last_undegraded":"2024-09-17T05:08:06.742487+0000","last_fullsized":"2024-09-17T05:08:06.742487+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746658+0000","last_change":"2024-09-17T05:08:06.746658+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746658+0000","last_undegraded":"2024-09-17T05:08:06.746658+0000","last_fullsized":"2024-09-17T05:08:06.746658+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.750232+0000","last_change":"2024-09-17T05:08:06.750232+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.750232+0000","last_undegraded":"2024-09-17T05:08:06.750232+0000","last_fullsized":"2024-09-17T05:08:06.750232+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.750393+0000","last_change":"2024-09-17T05:08:06.750393+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.750393+0000","last_undegraded":"2024-09-17T05:08:06.750393+0000","last_fullsized":"2024-09-17T05:08:06.750393+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737941+0000","last_change":"2024-09-17T05:08:06.737941+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737941+0000","last_undegraded":"2024-09-17T05:08:06.737941+0000","last_fullsized":"2024-09-17T05:08:06.737941+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773372+0000","last_change":"2024-09-17T05:08:07.774967+0000","last_active":"2024-09-17T05:08:08.773372+0000","last_peered":"2024-09-17T05:08:08.773372+0000","last_clean":"2024-09-17T05:08:08.773372+0000","last_became_active":"2024-09-17T05:08:07.774314+0000","last_became_peered":"2024-09-17T05:08:07.774314+0000","last_unstale":"2024-09-17T05:08:08.773372+0000","last_undegraded":"2024-09-17T05:08:08.773372+0000","last_fullsized":"2024-09-17T05:08:08.773372+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:57.062162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738056+0000","last_change":"2024-09-17T05:08:06.738056+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738056+0000","last_undegraded":"2024-09-17T05:08:06.738056+0000","last_fullsized":"2024-09-17T05:08:06.738056+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744125+0000","last_change":"2024-09-17T05:08:06.744125+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744125+0000","last_undegraded":"2024-09-17T05:08:06.744125+0000","last_fullsized":"2024-09-17T05:08:06.744125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741899+0000","last_change":"2024-09-17T05:08:06.741899+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741899+0000","last_undegraded":"2024-09-17T05:08:06.741899+0000","last_fullsized":"2024-09-17T05:08:06.741899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.749357+0000","last_change":"2024-09-17T05:08:06.749357+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.749357+0000","last_undegraded":"2024-09-17T05:08:06.749357+0000","last_fullsized":"2024-09-17T05:08:06.749357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743023+0000","last_change":"2024-09-17T05:08:06.743023+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743023+0000","last_undegraded":"2024-09-17T05:08:06.743023+0000","last_fullsized":"2024-09-17T05:08:06.743023+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742252+0000","last_change":"2024-09-17T05:08:06.742252+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742252+0000","last_undegraded":"2024-09-17T05:08:06.742252+0000","last_fullsized":"2024-09-17T05:08:06.742252+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737421+0000","last_change":"2024-09-17T05:08:06.737421+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737421+0000","last_undegraded":"2024-09-17T05:08:06.737421+0000","last_fullsized":"2024-09-17T05:08:06.737421+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748659+0000","last_change":"2024-09-17T05:08:06.748659+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748659+0000","last_undegraded":"2024-09-17T05:08:06.748659+0000","last_fullsized":"2024-09-17T05:08:06.748659+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741360+0000","last_change":"2024-09-17T05:08:06.741360+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741360+0000","last_undegraded":"2024-09-17T05:08:06.741360+0000","last_fullsized":"2024-09-17T05:08:06.741360+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742132+0000","last_change":"2024-09-17T05:08:06.742132+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742132+0000","last_undegraded":"2024-09-17T05:08:06.742132+0000","last_fullsized":"2024-09-17T05:08:06.742132+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742237+0000","last_change":"2024-09-17T05:08:06.742237+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742237+0000","last_undegraded":"2024-09-17T05:08:06.742237+0000","last_fullsized":"2024-09-17T05:08:06.742237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742081+0000","last_change":"2024-09-17T05:08:06.742081+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742081+0000","last_undegraded":"2024-09-17T05:08:06.742081+0000","last_fullsized":"2024-09-17T05:08:06.742081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740105+0000","last_change":"2024-09-17T05:08:06.740105+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740105+0000","last_undegraded":"2024-09-17T05:08:06.740105+0000","last_fullsized":"2024-09-17T05:08:06.740105+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763330+0000","last_change":"2024-09-17T05:08:06.757535+0000","last_active":"2024-09-17T05:08:06.763330+0000","last_peered":"2024-09-17T05:08:06.763330+0000","last_clean":"2024-09-17T05:08:06.763330+0000","last_became_active":"2024-09-17T05:08:06.757229+0000","last_became_peered":"2024-09-17T05:08:06.757229+0000","last_unstale":"2024-09-17T05:08:06.763330+0000","last_undegraded":"2024-09-17T05:08:06.763330+0000","last_fullsized":"2024-09-17T05:08:06.763330+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:06.277691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773515+0000","last_change":"2024-09-17T05:08:07.767395+0000","last_active":"2024-09-17T05:08:08.773515+0000","last_peered":"2024-09-17T05:08:08.773515+0000","last_clean":"2024-09-17T05:08:08.773515+0000","last_became_active":"2024-09-17T05:08:07.767090+0000","last_became_peered":"2024-09-17T05:08:07.767090+0000","last_unstale":"2024-09-17T05:08:08.773515+0000","last_undegraded":"2024-09-17T05:08:08.773515+0000","last_fullsized":"2024-09-17T05:08:08.773515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:54.194851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.761917+0000","last_change":"2024-09-17T05:08:06.758493+0000","last_active":"2024-09-17T05:08:06.761917+0000","last_peered":"2024-09-17T05:08:06.761917+0000","last_clean":"2024-09-17T05:08:06.761917+0000","last_became_active":"2024-09-17T05:08:06.758182+0000","last_became_peered":"2024-09-17T05:08:06.758182+0000","last_unstale":"2024-09-17T05:08:06.761917+0000","last_undegraded":"2024-09-17T05:08:06.761917+0000","last_fullsized":"2024-09-17T05:08:06.761917+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:42.476520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741598+0000","last_change":"2024-09-17T05:08:06.741598+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741598+0000","last_undegraded":"2024-09-17T05:08:06.741598+0000","last_fullsized":"2024-09-17T05:08:06.741598+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762999+0000","last_change":"2024-09-17T05:08:06.761067+0000","last_active":"2024-09-17T05:08:06.762999+0000","last_peered":"2024-09-17T05:08:06.762999+0000","last_clean":"2024-09-17T05:08:06.762999+0000","last_became_active":"2024-09-17T05:08:06.760761+0000","last_became_peered":"2024-09-17T05:08:06.760761+0000","last_unstale":"2024-09-17T05:08:06.762999+0000","last_undegraded":"2024-09-17T05:08:06.762999+0000","last_fullsized":"2024-09-17T05:08:06.762999+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:09.422764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748461+0000","last_change":"2024-09-17T05:08:06.748461+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748461+0000","last_undegraded":"2024-09-17T05:08:06.748461+0000","last_fullsized":"2024-09-17T05:08:06.748461+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773564+0000","last_change":"2024-09-17T05:08:06.761480+0000","last_active":"2024-09-17T05:08:08.773564+0000","last_peered":"2024-09-17T05:08:08.773564+0000","last_clean":"2024-09-17T05:08:08.773564+0000","last_became_active":"2024-09-17T05:08:06.761119+0000","last_became_peered":"2024-09-17T05:08:06.761119+0000","last_unstale":"2024-09-17T05:08:08.773564+0000","last_undegraded":"2024-09-17T05:08:08.773564+0000","last_fullsized":"2024-09-17T05:08:08.773564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740276+0000","last_change":"2024-09-17T05:08:06.740276+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740276+0000","last_undegraded":"2024-09-17T05:08:06.740276+0000","last_fullsized":"2024-09-17T05:08:06.740276+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.756045+0000","last_change":"2024-09-17T05:08:06.749915+0000","last_active":"2024-09-17T05:08:06.756045+0000","last_peered":"2024-09-17T05:08:06.756045+0000","last_clean":"2024-09-17T05:08:06.756045+0000","last_became_active":"2024-09-17T05:08:06.749590+0000","last_became_peered":"2024-09-17T05:08:06.749590+0000","last_unstale":"2024-09-17T05:08:06.756045+0000","last_undegraded":"2024-09-17T05:08:06.756045+0000","last_fullsized":"2024-09-17T05:08:06.756045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:40.105827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741257+0000","last_change":"2024-09-17T05:08:06.741257+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741257+0000","last_undegraded":"2024-09-17T05:08:06.741257+0000","last_fullsized":"2024-09-17T05:08:06.741257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763468+0000","last_change":"2024-09-17T05:08:06.761320+0000","last_active":"2024-09-17T05:08:06.763468+0000","last_peered":"2024-09-17T05:08:06.763468+0000","last_clean":"2024-09-17T05:08:06.763468+0000","last_became_active":"2024-09-17T05:08:06.760045+0000","last_became_peered":"2024-09-17T05:08:06.760045+0000","last_unstale":"2024-09-17T05:08:06.763468+0000","last_undegraded":"2024-09-17T05:08:06.763468+0000","last_fullsized":"2024-09-17T05:08:06.763468+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:07.884401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741592+0000","last_change":"2024-09-17T05:08:06.741592+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741592+0000","last_undegraded":"2024-09-17T05:08:06.741592+0000","last_fullsized":"2024-09-17T05:08:06.741592+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764389+0000","last_change":"2024-09-17T05:08:06.762962+0000","last_active":"2024-09-17T05:08:06.764389+0000","last_peered":"2024-09-17T05:08:06.764389+0000","last_clean":"2024-09-17T05:08:06.764389+0000","last_became_active":"2024-09-17T05:08:06.762619+0000","last_became_peered":"2024-09-17T05:08:06.762619+0000","last_unstale":"2024-09-17T05:08:06.764389+0000","last_undegraded":"2024-09-17T05:08:06.764389+0000","last_fullsized":"2024-09-17T05:08:06.764389+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.226520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741671+0000","last_change":"2024-09-17T05:08:06.741671+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741671+0000","last_undegraded":"2024-09-17T05:08:06.741671+0000","last_fullsized":"2024-09-17T05:08:06.741671+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760010+0000","last_change":"2024-09-17T05:08:06.757901+0000","last_active":"2024-09-17T05:08:06.760010+0000","last_peered":"2024-09-17T05:08:06.760010+0000","last_clean":"2024-09-17T05:08:06.760010+0000","last_became_active":"2024-09-17T05:08:06.757576+0000","last_became_peered":"2024-09-17T05:08:06.757576+0000","last_unstale":"2024-09-17T05:08:06.760010+0000","last_undegraded":"2024-09-17T05:08:06.760010+0000","last_fullsized":"2024-09-17T05:08:06.760010+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:38.491381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744703+0000","last_change":"2024-09-17T05:08:06.744703+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744703+0000","last_undegraded":"2024-09-17T05:08:06.744703+0000","last_fullsized":"2024-09-17T05:08:06.744703+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758358+0000","last_change":"2024-09-17T05:08:06.756508+0000","last_active":"2024-09-17T05:08:06.758358+0000","last_peered":"2024-09-17T05:08:06.758358+0000","last_clean":"2024-09-17T05:08:06.758358+0000","last_became_active":"2024-09-17T05:08:06.756117+0000","last_became_peered":"2024-09-17T05:08:06.756117+0000","last_unstale":"2024-09-17T05:08:06.758358+0000","last_undegraded":"2024-09-17T05:08:06.758358+0000","last_fullsized":"2024-09-17T05:08:06.758358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:09.980821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.747209+0000","last_change":"2024-09-17T05:08:06.747209+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.747209+0000","last_undegraded":"2024-09-17T05:08:06.747209+0000","last_fullsized":"2024-09-17T05:08:06.747209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764654+0000","last_change":"2024-09-17T05:08:06.762648+0000","last_active":"2024-09-17T05:08:06.764654+0000","last_peered":"2024-09-17T05:08:06.764654+0000","last_clean":"2024-09-17T05:08:06.764654+0000","last_became_active":"2024-09-17T05:08:06.762351+0000","last_became_peered":"2024-09-17T05:08:06.762351+0000","last_unstale":"2024-09-17T05:08:06.764654+0000","last_undegraded":"2024-09-17T05:08:06.764654+0000","last_fullsized":"2024-09-17T05:08:06.764654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:05.490303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744234+0000","last_change":"2024-09-17T05:08:06.744234+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744234+0000","last_undegraded":"2024-09-17T05:08:06.744234+0000","last_fullsized":"2024-09-17T05:08:06.744234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.759349+0000","last_change":"2024-09-17T05:08:06.756469+0000","last_active":"2024-09-17T05:08:06.759349+0000","last_peered":"2024-09-17T05:08:06.759349+0000","last_clean":"2024-09-17T05:08:06.759349+0000","last_became_active":"2024-09-17T05:08:06.756119+0000","last_became_peered":"2024-09-17T05:08:06.756119+0000","last_unstale":"2024-09-17T05:08:06.759349+0000","last_undegraded":"2024-09-17T05:08:06.759349+0000","last_fullsized":"2024-09-17T05:08:06.759349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:34.202346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738887+0000","last_change":"2024-09-17T05:08:06.738887+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738887+0000","last_undegraded":"2024-09-17T05:08:06.738887+0000","last_fullsized":"2024-09-17T05:08:06.738887+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":21,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773530+0000","last_change":"2024-09-17T05:08:06.751740+0000","last_active":"2024-09-17T05:08:08.773530+0000","last_peered":"2024-09-17T05:08:08.773530+0000","last_clean":"2024-09-17T05:08:08.773530+0000","last_became_active":"2024-09-17T05:08:06.751425+0000","last_became_peered":"2024-09-17T05:08:06.751425+0000","last_unstale":"2024-09-17T05:08:08.773530+0000","last_undegraded":"2024-09-17T05:08:08.773530+0000","last_fullsized":"2024-09-17T05:08:08.773530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:19.025078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737240+0000","last_change":"2024-09-17T05:08:06.737240+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737240+0000","last_undegraded":"2024-09-17T05:08:06.737240+0000","last_fullsized":"2024-09-17T05:08:06.737240+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758041+0000","last_change":"2024-09-17T05:08:06.755780+0000","last_active":"2024-09-17T05:08:06.758041+0000","last_peered":"2024-09-17T05:08:06.758041+0000","last_clean":"2024-09-17T05:08:06.758041+0000","last_became_active":"2024-09-17T05:08:06.754229+0000","last_became_peered":"2024-09-17T05:08:06.754229+0000","last_unstale":"2024-09-17T05:08:06.758041+0000","last_undegraded":"2024-09-17T05:08:06.758041+0000","last_fullsized":"2024-09-17T05:08:06.758041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:32.056062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773646+0000","last_change":"2024-09-17T05:08:07.765055+0000","last_active":"2024-09-17T05:08:08.773646+0000","last_peered":"2024-09-17T05:08:08.773646+0000","last_clean":"2024-09-17T05:08:08.773646+0000","last_became_active":"2024-09-17T05:08:07.764271+0000","last_became_peered":"2024-09-17T05:08:07.764271+0000","last_unstale":"2024-09-17T05:08:08.773646+0000","last_undegraded":"2024-09-17T05:08:08.773646+0000","last_fullsized":"2024-09-17T05:08:08.773646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.819206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758439+0000","last_change":"2024-09-17T05:08:06.754758+0000","last_active":"2024-09-17T05:08:06.758439+0000","last_peered":"2024-09-17T05:08:06.758439+0000","last_clean":"2024-09-17T05:08:06.758439+0000","last_became_active":"2024-09-17T05:08:06.754461+0000","last_became_peered":"2024-09-17T05:08:06.754461+0000","last_unstale":"2024-09-17T05:08:06.758439+0000","last_undegraded":"2024-09-17T05:08:06.758439+0000","last_fullsized":"2024-09-17T05:08:06.758439+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:07.894622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744709+0000","last_change":"2024-09-17T05:08:06.744709+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744709+0000","last_undegraded":"2024-09-17T05:08:06.744709+0000","last_fullsized":"2024-09-17T05:08:06.744709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.759715+0000","last_change":"2024-09-17T05:08:06.756328+0000","last_active":"2024-09-17T05:08:06.759715+0000","last_peered":"2024-09-17T05:08:06.759715+0000","last_clean":"2024-09-17T05:08:06.759715+0000","last_became_active":"2024-09-17T05:08:06.756035+0000","last_became_peered":"2024-09-17T05:08:06.756035+0000","last_unstale":"2024-09-17T05:08:06.759715+0000","last_undegraded":"2024-09-17T05:08:06.759715+0000","last_fullsized":"2024-09-17T05:08:06.759715+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:27.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746007+0000","last_change":"2024-09-17T05:08:06.746007+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746007+0000","last_undegraded":"2024-09-17T05:08:06.746007+0000","last_fullsized":"2024-09-17T05:08:06.746007+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763592+0000","last_change":"2024-09-17T05:08:06.760008+0000","last_active":"2024-09-17T05:08:06.763592+0000","last_peered":"2024-09-17T05:08:06.763592+0000","last_clean":"2024-09-17T05:08:06.763592+0000","last_became_active":"2024-09-17T05:08:06.759701+0000","last_became_peered":"2024-09-17T05:08:06.759701+0000","last_unstale":"2024-09-17T05:08:06.763592+0000","last_undegraded":"2024-09-17T05:08:06.763592+0000","last_fullsized":"2024-09-17T05:08:06.763592+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:29.057572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741247+0000","last_change":"2024-09-17T05:08:06.741247+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741247+0000","last_undegraded":"2024-09-17T05:08:06.741247+0000","last_fullsized":"2024-09-17T05:08:06.741247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.765021+0000","last_change":"2024-09-17T05:08:06.763335+0000","last_active":"2024-09-17T05:08:06.765021+0000","last_peered":"2024-09-17T05:08:06.765021+0000","last_clean":"2024-09-17T05:08:06.765021+0000","last_became_active":"2024-09-17T05:08:06.762302+0000","last_became_peered":"2024-09-17T05:08:06.762302+0000","last_unstale":"2024-09-17T05:08:06.765021+0000","last_undegraded":"2024-09-17T05:08:06.765021+0000","last_fullsized":"2024-09-17T05:08:06.765021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:55.330080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773938+0000","last_change":"2024-09-17T05:08:07.775183+0000","last_active":"2024-09-17T05:08:08.773938+0000","last_peered":"2024-09-17T05:08:08.773938+0000","last_clean":"2024-09-17T05:08:08.773938+0000","last_became_active":"2024-09-17T05:08:07.774107+0000","last_became_peered":"2024-09-17T05:08:07.774107+0000","last_unstale":"2024-09-17T05:08:08.773938+0000","last_undegraded":"2024-09-17T05:08:08.773938+0000","last_fullsized":"2024-09-17T05:08:08.773938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:50.882298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763223+0000","last_change":"2024-09-17T05:08:06.756451+0000","last_active":"2024-09-17T05:08:06.763223+0000","last_peered":"2024-09-17T05:08:06.763223+0000","last_clean":"2024-09-17T05:08:06.763223+0000","last_became_active":"2024-09-17T05:08:06.756108+0000","last_became_peered":"2024-09-17T05:08:06.756108+0000","last_unstale":"2024-09-17T05:08:06.763223+0000","last_undegraded":"2024-09-17T05:08:06.763223+0000","last_fullsized":"2024-09-17T05:08:06.763223+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:25.086196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738350+0000","last_change":"2024-09-17T05:08:06.738350+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738350+0000","last_undegraded":"2024-09-17T05:08:06.738350+0000","last_fullsized":"2024-09-17T05:08:06.738350+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763533+0000","last_change":"2024-09-17T05:08:06.759565+0000","last_active":"2024-09-17T05:08:06.763533+0000","last_peered":"2024-09-17T05:08:06.763533+0000","last_clean":"2024-09-17T05:08:06.763533+0000","last_became_active":"2024-09-17T05:08:06.759251+0000","last_became_peered":"2024-09-17T05:08:06.759251+0000","last_unstale":"2024-09-17T05:08:06.763533+0000","last_undegraded":"2024-09-17T05:08:06.763533+0000","last_fullsized":"2024-09-17T05:08:06.763533+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:40.973680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773295+0000","last_change":"2024-09-17T05:08:07.766627+0000","last_active":"2024-09-17T05:08:08.773295+0000","last_peered":"2024-09-17T05:08:08.773295+0000","last_clean":"2024-09-17T05:08:08.773295+0000","last_became_active":"2024-09-17T05:08:07.766265+0000","last_became_peered":"2024-09-17T05:08:07.766265+0000","last_unstale":"2024-09-17T05:08:08.773295+0000","last_undegraded":"2024-09-17T05:08:08.773295+0000","last_fullsized":"2024-09-17T05:08:08.773295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:16.381763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762217+0000","last_change":"2024-09-17T05:08:06.758648+0000","last_active":"2024-09-17T05:08:06.762217+0000","last_peered":"2024-09-17T05:08:06.762217+0000","last_clean":"2024-09-17T05:08:06.762217+0000","last_became_active":"2024-09-17T05:08:06.756522+0000","last_became_peered":"2024-09-17T05:08:06.756522+0000","last_unstale":"2024-09-17T05:08:06.762217+0000","last_undegraded":"2024-09-17T05:08:06.762217+0000","last_fullsized":"2024-09-17T05:08:06.762217+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:35.795566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746537+0000","last_change":"2024-09-17T05:08:06.746537+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746537+0000","last_undegraded":"2024-09-17T05:08:06.746537+0000","last_fullsized":"2024-09-17T05:08:06.746537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762568+0000","last_change":"2024-09-17T05:08:06.759894+0000","last_active":"2024-09-17T05:08:06.762568+0000","last_peered":"2024-09-17T05:08:06.762568+0000","last_clean":"2024-09-17T05:08:06.762568+0000","last_became_active":"2024-09-17T05:08:06.759190+0000","last_became_peered":"2024-09-17T05:08:06.759190+0000","last_unstale":"2024-09-17T05:08:06.762568+0000","last_undegraded":"2024-09-17T05:08:06.762568+0000","last_fullsized":"2024-09-17T05:08:06.762568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:11.666395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740954+0000","last_change":"2024-09-17T05:08:06.740954+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740954+0000","last_undegraded":"2024-09-17T05:08:06.740954+0000","last_fullsized":"2024-09-17T05:08:06.740954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760351+0000","last_change":"2024-09-17T05:08:06.757737+0000","last_active":"2024-09-17T05:08:06.760351+0000","last_peered":"2024-09-17T05:08:06.760351+0000","last_clean":"2024-09-17T05:08:06.760351+0000","last_became_active":"2024-09-17T05:08:06.757394+0000","last_became_peered":"2024-09-17T05:08:06.757394+0000","last_unstale":"2024-09-17T05:08:06.760351+0000","last_undegraded":"2024-09-17T05:08:06.760351+0000","last_fullsized":"2024-09-17T05:08:06.760351+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:42.229187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741382+0000","last_change":"2024-09-17T05:08:06.741382+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741382+0000","last_undegraded":"2024-09-17T05:08:06.741382+0000","last_fullsized":"2024-09-17T05:08:06.741382+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758100+0000","last_change":"2024-09-17T05:08:06.752439+0000","last_active":"2024-09-17T05:08:06.758100+0000","last_peered":"2024-09-17T05:08:06.758100+0000","last_clean":"2024-09-17T05:08:06.758100+0000","last_became_active":"2024-09-17T05:08:06.752134+0000","last_became_peered":"2024-09-17T05:08:06.752134+0000","last_unstale":"2024-09-17T05:08:06.758100+0000","last_undegraded":"2024-09-17T05:08:06.758100+0000","last_fullsized":"2024-09-17T05:08:06.758100+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:51.840516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.747995+0000","last_change":"2024-09-17T05:08:06.747995+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.747995+0000","last_undegraded":"2024-09-17T05:08:06.747995+0000","last_fullsized":"2024-09-17T05:08:06.747995+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773884+0000","last_change":"2024-09-17T05:08:06.758852+0000","last_active":"2024-09-17T05:08:08.773884+0000","last_peered":"2024-09-17T05:08:08.773884+0000","last_clean":"2024-09-17T05:08:08.773884+0000","last_became_active":"2024-09-17T05:08:06.757319+0000","last_became_peered":"2024-09-17T05:08:06.757319+0000","last_unstale":"2024-09-17T05:08:08.773884+0000","last_undegraded":"2024-09-17T05:08:08.773884+0000","last_fullsized":"2024-09-17T05:08:08.773884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:35.769440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742980+0000","last_change":"2024-09-17T05:08:06.742980+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742980+0000","last_undegraded":"2024-09-17T05:08:06.742980+0000","last_fullsized":"2024-09-17T05:08:06.742980+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764906+0000","last_change":"2024-09-17T05:08:06.763961+0000","last_active":"2024-09-17T05:08:06.764906+0000","last_peered":"2024-09-17T05:08:06.764906+0000","last_clean":"2024-09-17T05:08:06.764906+0000","last_became_active":"2024-09-17T05:08:06.763384+0000","last_became_peered":"2024-09-17T05:08:06.763384+0000","last_unstale":"2024-09-17T05:08:06.764906+0000","last_undegraded":"2024-09-17T05:08:06.764906+0000","last_fullsized":"2024-09-17T05:08:06.764906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:38.347875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746304+0000","last_change":"2024-09-17T05:08:06.746304+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746304+0000","last_undegraded":"2024-09-17T05:08:06.746304+0000","last_fullsized":"2024-09-17T05:08:06.746304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763446+0000","last_change":"2024-09-17T05:08:06.758429+0000","last_active":"2024-09-17T05:08:06.763446+0000","last_peered":"2024-09-17T05:08:06.763446+0000","last_clean":"2024-09-17T05:08:06.763446+0000","last_became_active":"2024-09-17T05:08:06.758102+0000","last_became_peered":"2024-09-17T05:08:06.758102+0000","last_unstale":"2024-09-17T05:08:06.763446+0000","last_undegraded":"2024-09-17T05:08:06.763446+0000","last_fullsized":"2024-09-17T05:08:06.763446+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:50.494994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742504+0000","last_change":"2024-09-17T05:08:06.742504+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742504+0000","last_undegraded":"2024-09-17T05:08:06.742504+0000","last_fullsized":"2024-09-17T05:08:06.742504+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760605+0000","last_change":"2024-09-17T05:08:06.754339+0000","last_active":"2024-09-17T05:08:06.760605+0000","last_peered":"2024-09-17T05:08:06.760605+0000","last_clean":"2024-09-17T05:08:06.760605+0000","last_became_active":"2024-09-17T05:08:06.753263+0000","last_became_peered":"2024-09-17T05:08:06.753263+0000","last_unstale":"2024-09-17T05:08:06.760605+0000","last_undegraded":"2024-09-17T05:08:06.760605+0000","last_fullsized":"2024-09-17T05:08:06.760605+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:57.117338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.745993+0000","last_change":"2024-09-17T05:08:06.745993+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.745993+0000","last_undegraded":"2024-09-17T05:08:06.745993+0000","last_fullsized":"2024-09-17T05:08:06.745993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.756524+0000","last_change":"2024-09-17T05:08:06.750182+0000","last_active":"2024-09-17T05:08:06.756524+0000","last_peered":"2024-09-17T05:08:06.756524+0000","last_clean":"2024-09-17T05:08:06.756524+0000","last_became_active":"2024-09-17T05:08:06.749803+0000","last_became_peered":"2024-09-17T05:08:06.749803+0000","last_unstale":"2024-09-17T05:08:06.756524+0000","last_undegraded":"2024-09-17T05:08:06.756524+0000","last_fullsized":"2024-09-17T05:08:06.756524+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:00.515472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773077+0000","last_change":"2024-09-17T05:08:07.765395+0000","last_active":"2024-09-17T05:08:08.773077+0000","last_peered":"2024-09-17T05:08:08.773077+0000","last_clean":"2024-09-17T05:08:08.773077+0000","last_became_active":"2024-09-17T05:08:07.764733+0000","last_became_peered":"2024-09-17T05:08:07.764733+0000","last_unstale":"2024-09-17T05:08:08.773077+0000","last_undegraded":"2024-09-17T05:08:08.773077+0000","last_fullsized":"2024-09-17T05:08:08.773077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:38.846186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762599+0000","last_change":"2024-09-17T05:08:06.760241+0000","last_active":"2024-09-17T05:08:06.762599+0000","last_peered":"2024-09-17T05:08:06.762599+0000","last_clean":"2024-09-17T05:08:06.762599+0000","last_became_active":"2024-09-17T05:08:06.759577+0000","last_became_peered":"2024-09-17T05:08:06.759577+0000","last_unstale":"2024-09-17T05:08:06.762599+0000","last_undegraded":"2024-09-17T05:08:06.762599+0000","last_fullsized":"2024-09-17T05:08:06.762599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:12.881506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737087+0000","last_change":"2024-09-17T05:08:06.737087+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737087+0000","last_undegraded":"2024-09-17T05:08:06.737087+0000","last_fullsized":"2024-09-17T05:08:06.737087+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773378+0000","last_change":"2024-09-17T05:08:06.756365+0000","last_active":"2024-09-17T05:08:08.773378+0000","last_peered":"2024-09-17T05:08:08.773378+0000","last_clean":"2024-09-17T05:08:08.773378+0000","last_became_active":"2024-09-17T05:08:06.756054+0000","last_became_peered":"2024-09-17T05:08:06.756054+0000","last_unstale":"2024-09-17T05:08:08.773378+0000","last_undegraded":"2024-09-17T05:08:08.773378+0000","last_fullsized":"2024-09-17T05:08:08.773378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.455026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741219+0000","last_change":"2024-09-17T05:08:06.741219+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741219+0000","last_undegraded":"2024-09-17T05:08:06.741219+0000","last_fullsized":"2024-09-17T05:08:06.741219+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763256+0000","last_change":"2024-09-17T05:08:06.756541+0000","last_active":"2024-09-17T05:08:06.763256+0000","last_peered":"2024-09-17T05:08:06.763256+0000","last_clean":"2024-09-17T05:08:06.763256+0000","last_became_active":"2024-09-17T05:08:06.756230+0000","last_became_peered":"2024-09-17T05:08:06.756230+0000","last_unstale":"2024-09-17T05:08:06.763256+0000","last_undegraded":"2024-09-17T05:08:06.763256+0000","last_fullsized":"2024-09-17T05:08:06.763256+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:46.413879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.739124+0000","last_change":"2024-09-17T05:08:06.739124+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.739124+0000","last_undegraded":"2024-09-17T05:08:06.739124+0000","last_fullsized":"2024-09-17T05:08:06.739124+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760586+0000","last_change":"2024-09-17T05:08:06.756186+0000","last_active":"2024-09-17T05:08:06.760586+0000","last_peered":"2024-09-17T05:08:06.760586+0000","last_clean":"2024-09-17T05:08:06.760586+0000","last_became_active":"2024-09-17T05:08:06.755300+0000","last_became_peered":"2024-09-17T05:08:06.755300+0000","last_unstale":"2024-09-17T05:08:06.760586+0000","last_undegraded":"2024-09-17T05:08:06.760586+0000","last_fullsized":"2024-09-17T05:08:06.760586+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:29.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":284,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.750214+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:06.750214+0000","last_peered":"2024-09-17T05:08:06.750214+0000","last_clean":"2024-09-17T05:08:06.750214+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:06.750214+0000","last_undegraded":"2024-09-17T05:08:06.750214+0000","last_fullsized":"2024-09-17T05:08:06.750214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":625417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509529,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1222452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1237923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:08:11.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 -- 172.21.15.16:0/1463913901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 msgr2=0x7f1ec015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/1463913901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec015c4b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0071ce0 tx=0x7f1ea801c830 comp rx=0 tx=0).stop 2024-09-17T05:08:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 -- 172.21.15.16:0/1463913901 shutdown_connections 2024-09-17T05:08:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/1463913901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec015c4b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/1463913901 >> v2:172.21.15.104:3300/0 conn(0x7f1ec00b2f50 0x7f1ec015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 -- 172.21.15.16:0/1463913901 >> 172.21.15.16:0/1463913901 conn(0x7f1ec006f900 msgr2=0x7f1ec00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 -- 172.21.15.16:0/1463913901 shutdown_connections 2024-09-17T05:08:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.158+0000 7f1ec6a00640 1 -- 172.21.15.16:0/1463913901 wait complete. 2024-09-17T05:08:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 Processor -- start 2024-09-17T05:08:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 -- start start 2024-09-17T05:08:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00b2f50 0x7f1ec01fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec0154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ec015f150 0x7f1ec014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0169070 con 0x7f1ec015c0a0 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec0168ef0 con 0x7f1ec00b2f50 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1ec01691f0 con 0x7f1ec015f150 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec0154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec0154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34092/0 (socket says 172.21.15.16:34092) 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ebfe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00b2f50 0x7f1ec01fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 -- 172.21.15.16:0/113529660 learned_addr learned my addr 172.21.15.16:0/113529660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec4a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ec015f150 0x7f1ec014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 -- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ec015f150 msgr2=0x7f1ec014d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ec015f150 0x7f1ec014d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 -- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00b2f50 msgr2=0x7f1ec01fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00b2f50 0x7f1ec01fd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 -- 172.21.15.16:0/113529660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec014de10 con 0x7f1ec015c0a0 2024-09-17T05:08:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1eb7e00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec0154360 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f1ea801aee0 tx=0x7f1ea80a5830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80ba020 con 0x7f1ec015c0a0 2024-09-17T05:08:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ea80044b0 con 0x7f1ec015c0a0 2024-09-17T05:08:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80b2040 con 0x7f1ec015c0a0 2024-09-17T05:08:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec0154a00 con 0x7f1ec015c0a0 2024-09-17T05:08:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.162+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec01fc980 con 0x7f1ec015c0a0 2024-09-17T05:08:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.166+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ec015bb60 con 0x7f1ec015c0a0 2024-09-17T05:08:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.166+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1ea8005290 con 0x7f1ec015c0a0 2024-09-17T05:08:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.166+0000 7f1ebcc00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e9c073200 0x7f1e9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.166+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f1ea812e290 con 0x7f1ec015c0a0 2024-09-17T05:08:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.166+0000 7f1ebfe00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e9c073200 0x7f1e9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.166+0000 7f1ebfe00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e9c073200 0x7f1e9c075710 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0004630 tx=0x7f1eb000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.170+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1ea80c4060 con 0x7f1ec015c0a0 2024-09-17T05:08:12.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.346+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ec015f970 con 0x7f1e9c073200 2024-09-17T05:08:12.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.350+0000 7f1ebcc00640 1 -- 172.21.15.16:0/113529660 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248047 (secure 0 0 0) 0x7f1ec015f970 con 0x7f1e9c073200 2024-09-17T05:08:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e9c073200 msgr2=0x7f1e9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e9c073200 0x7f1e9c075710 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0004630 tx=0x7f1eb000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 msgr2=0x7f1ec0154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec0154360 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f1ea801aee0 tx=0x7f1ea80a5830 comp rx=0 tx=0).stop 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 shutdown_connections 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e9c073200 0x7f1e9c075710 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ec015f150 0x7f1ec014d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec015c0a0 0x7f1ec0154360 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 --2- 172.21.15.16:0/113529660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ec00b2f50 0x7f1ec01fd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 >> 172.21.15.16:0/113529660 conn(0x7f1ec006f900 msgr2=0x7f1ec015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 shutdown_connections 2024-09-17T05:08:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.354+0000 7f1ec6a00640 1 -- 172.21.15.16:0/113529660 wait complete. 2024-09-17T05:08:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-17T05:08:11.578325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269780,"kb_used_data":11936,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704940,"statfs":{"total":773094113280,"available":772817858560,"internally_reserved":0,"allocated":12222464,"data_stored":6246212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.851674"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743530+0000","last_change":"2024-09-17T05:08:06.743530+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743530+0000","last_undegraded":"2024-09-17T05:08:06.743530+0000","last_fullsized":"2024-09-17T05:08:06.743530+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774172+0000","last_change":"2024-09-17T05:08:07.761565+0000","last_active":"2024-09-17T05:08:08.774172+0000","last_peered":"2024-09-17T05:08:08.774172+0000","last_clean":"2024-09-17T05:08:08.774172+0000","last_became_active":"2024-09-17T05:08:07.761261+0000","last_became_peered":"2024-09-17T05:08:07.761261+0000","last_unstale":"2024-09-17T05:08:08.774172+0000","last_undegraded":"2024-09-17T05:08:08.774172+0000","last_fullsized":"2024-09-17T05:08:08.774172+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:41.539621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743447+0000","last_change":"2024-09-17T05:08:06.743447+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743447+0000","last_undegraded":"2024-09-17T05:08:06.743447+0000","last_fullsized":"2024-09-17T05:08:06.743447+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748147+0000","last_change":"2024-09-17T05:08:06.748147+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748147+0000","last_undegraded":"2024-09-17T05:08:06.748147+0000","last_fullsized":"2024-09-17T05:08:06.748147+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738577+0000","last_change":"2024-09-17T05:08:06.738577+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738577+0000","last_undegraded":"2024-09-17T05:08:06.738577+0000","last_fullsized":"2024-09-17T05:08:06.738577+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773432+0000","last_change":"2024-09-17T05:08:07.758709+0000","last_active":"2024-09-17T05:08:08.773432+0000","last_peered":"2024-09-17T05:08:08.773432+0000","last_clean":"2024-09-17T05:08:08.773432+0000","last_became_active":"2024-09-17T05:08:07.758382+0000","last_became_peered":"2024-09-17T05:08:07.758382+0000","last_unstale":"2024-09-17T05:08:08.773432+0000","last_undegraded":"2024-09-17T05:08:08.773432+0000","last_fullsized":"2024-09-17T05:08:08.773432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:11.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773005+0000","last_change":"2024-09-17T05:08:07.787949+0000","last_active":"2024-09-17T05:08:08.773005+0000","last_peered":"2024-09-17T05:08:08.773005+0000","last_clean":"2024-09-17T05:08:08.773005+0000","last_became_active":"2024-09-17T05:08:07.787577+0000","last_became_peered":"2024-09-17T05:08:07.787577+0000","last_unstale":"2024-09-17T05:08:08.773005+0000","last_undegraded":"2024-09-17T05:08:08.773005+0000","last_fullsized":"2024-09-17T05:08:08.773005+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:33.927797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740861+0000","last_change":"2024-09-17T05:08:06.740861+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740861+0000","last_undegraded":"2024-09-17T05:08:06.740861+0000","last_fullsized":"2024-09-17T05:08:06.740861+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748951+0000","last_change":"2024-09-17T05:08:06.748951+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748951+0000","last_undegraded":"2024-09-17T05:08:06.748951+0000","last_fullsized":"2024-09-17T05:08:06.748951+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774347+0000","last_change":"2024-09-17T05:08:07.777536+0000","last_active":"2024-09-17T05:08:08.774347+0000","last_peered":"2024-09-17T05:08:08.774347+0000","last_clean":"2024-09-17T05:08:08.774347+0000","last_became_active":"2024-09-17T05:08:07.777182+0000","last_became_peered":"2024-09-17T05:08:07.777182+0000","last_unstale":"2024-09-17T05:08:08.774347+0000","last_undegraded":"2024-09-17T05:08:08.774347+0000","last_fullsized":"2024-09-17T05:08:08.774347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:26.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742789+0000","last_change":"2024-09-17T05:08:06.742789+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742789+0000","last_undegraded":"2024-09-17T05:08:06.742789+0000","last_fullsized":"2024-09-17T05:08:06.742789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.749238+0000","last_change":"2024-09-17T05:08:06.749238+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.749238+0000","last_undegraded":"2024-09-17T05:08:06.749238+0000","last_fullsized":"2024-09-17T05:08:06.749238+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773816+0000","last_change":"2024-09-17T05:08:07.776167+0000","last_active":"2024-09-17T05:08:08.773816+0000","last_peered":"2024-09-17T05:08:08.773816+0000","last_clean":"2024-09-17T05:08:08.773816+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:08.773816+0000","last_undegraded":"2024-09-17T05:08:08.773816+0000","last_fullsized":"2024-09-17T05:08:08.773816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:33.590100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.745703+0000","last_change":"2024-09-17T05:08:06.745703+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.745703+0000","last_undegraded":"2024-09-17T05:08:06.745703+0000","last_fullsized":"2024-09-17T05:08:06.745703+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742487+0000","last_change":"2024-09-17T05:08:06.742487+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742487+0000","last_undegraded":"2024-09-17T05:08:06.742487+0000","last_fullsized":"2024-09-17T05:08:06.742487+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746658+0000","last_change":"2024-09-17T05:08:06.746658+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746658+0000","last_undegraded":"2024-09-17T05:08:06.746658+0000","last_fullsized":"2024-09-17T05:08:06.746658+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.750232+0000","last_change":"2024-09-17T05:08:06.750232+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.750232+0000","last_undegraded":"2024-09-17T05:08:06.750232+0000","last_fullsized":"2024-09-17T05:08:06.750232+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.750393+0000","last_change":"2024-09-17T05:08:06.750393+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.750393+0000","last_undegraded":"2024-09-17T05:08:06.750393+0000","last_fullsized":"2024-09-17T05:08:06.750393+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737941+0000","last_change":"2024-09-17T05:08:06.737941+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737941+0000","last_undegraded":"2024-09-17T05:08:06.737941+0000","last_fullsized":"2024-09-17T05:08:06.737941+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773372+0000","last_change":"2024-09-17T05:08:07.774967+0000","last_active":"2024-09-17T05:08:08.773372+0000","last_peered":"2024-09-17T05:08:08.773372+0000","last_clean":"2024-09-17T05:08:08.773372+0000","last_became_active":"2024-09-17T05:08:07.774314+0000","last_became_peered":"2024-09-17T05:08:07.774314+0000","last_unstale":"2024-09-17T05:08:08.773372+0000","last_undegraded":"2024-09-17T05:08:08.773372+0000","last_fullsized":"2024-09-17T05:08:08.773372+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:57.062162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738056+0000","last_change":"2024-09-17T05:08:06.738056+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738056+0000","last_undegraded":"2024-09-17T05:08:06.738056+0000","last_fullsized":"2024-09-17T05:08:06.738056+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744125+0000","last_change":"2024-09-17T05:08:06.744125+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744125+0000","last_undegraded":"2024-09-17T05:08:06.744125+0000","last_fullsized":"2024-09-17T05:08:06.744125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741899+0000","last_change":"2024-09-17T05:08:06.741899+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741899+0000","last_undegraded":"2024-09-17T05:08:06.741899+0000","last_fullsized":"2024-09-17T05:08:06.741899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.749357+0000","last_change":"2024-09-17T05:08:06.749357+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.749357+0000","last_undegraded":"2024-09-17T05:08:06.749357+0000","last_fullsized":"2024-09-17T05:08:06.749357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743023+0000","last_change":"2024-09-17T05:08:06.743023+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743023+0000","last_undegraded":"2024-09-17T05:08:06.743023+0000","last_fullsized":"2024-09-17T05:08:06.743023+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742252+0000","last_change":"2024-09-17T05:08:06.742252+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742252+0000","last_undegraded":"2024-09-17T05:08:06.742252+0000","last_fullsized":"2024-09-17T05:08:06.742252+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737421+0000","last_change":"2024-09-17T05:08:06.737421+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737421+0000","last_undegraded":"2024-09-17T05:08:06.737421+0000","last_fullsized":"2024-09-17T05:08:06.737421+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748659+0000","last_change":"2024-09-17T05:08:06.748659+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748659+0000","last_undegraded":"2024-09-17T05:08:06.748659+0000","last_fullsized":"2024-09-17T05:08:06.748659+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741360+0000","last_change":"2024-09-17T05:08:06.741360+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741360+0000","last_undegraded":"2024-09-17T05:08:06.741360+0000","last_fullsized":"2024-09-17T05:08:06.741360+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742132+0000","last_change":"2024-09-17T05:08:06.742132+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742132+0000","last_undegraded":"2024-09-17T05:08:06.742132+0000","last_fullsized":"2024-09-17T05:08:06.742132+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742237+0000","last_change":"2024-09-17T05:08:06.742237+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742237+0000","last_undegraded":"2024-09-17T05:08:06.742237+0000","last_fullsized":"2024-09-17T05:08:06.742237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742081+0000","last_change":"2024-09-17T05:08:06.742081+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742081+0000","last_undegraded":"2024-09-17T05:08:06.742081+0000","last_fullsized":"2024-09-17T05:08:06.742081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740105+0000","last_change":"2024-09-17T05:08:06.740105+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740105+0000","last_undegraded":"2024-09-17T05:08:06.740105+0000","last_fullsized":"2024-09-17T05:08:06.740105+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763330+0000","last_change":"2024-09-17T05:08:06.757535+0000","last_active":"2024-09-17T05:08:06.763330+0000","last_peered":"2024-09-17T05:08:06.763330+0000","last_clean":"2024-09-17T05:08:06.763330+0000","last_became_active":"2024-09-17T05:08:06.757229+0000","last_became_peered":"2024-09-17T05:08:06.757229+0000","last_unstale":"2024-09-17T05:08:06.763330+0000","last_undegraded":"2024-09-17T05:08:06.763330+0000","last_fullsized":"2024-09-17T05:08:06.763330+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:06.277691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773515+0000","last_change":"2024-09-17T05:08:07.767395+0000","last_active":"2024-09-17T05:08:08.773515+0000","last_peered":"2024-09-17T05:08:08.773515+0000","last_clean":"2024-09-17T05:08:08.773515+0000","last_became_active":"2024-09-17T05:08:07.767090+0000","last_became_peered":"2024-09-17T05:08:07.767090+0000","last_unstale":"2024-09-17T05:08:08.773515+0000","last_undegraded":"2024-09-17T05:08:08.773515+0000","last_fullsized":"2024-09-17T05:08:08.773515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:54.194851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.761917+0000","last_change":"2024-09-17T05:08:06.758493+0000","last_active":"2024-09-17T05:08:06.761917+0000","last_peered":"2024-09-17T05:08:06.761917+0000","last_clean":"2024-09-17T05:08:06.761917+0000","last_became_active":"2024-09-17T05:08:06.758182+0000","last_became_peered":"2024-09-17T05:08:06.758182+0000","last_unstale":"2024-09-17T05:08:06.761917+0000","last_undegraded":"2024-09-17T05:08:06.761917+0000","last_fullsized":"2024-09-17T05:08:06.761917+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:42.476520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741598+0000","last_change":"2024-09-17T05:08:06.741598+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741598+0000","last_undegraded":"2024-09-17T05:08:06.741598+0000","last_fullsized":"2024-09-17T05:08:06.741598+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762999+0000","last_change":"2024-09-17T05:08:06.761067+0000","last_active":"2024-09-17T05:08:06.762999+0000","last_peered":"2024-09-17T05:08:06.762999+0000","last_clean":"2024-09-17T05:08:06.762999+0000","last_became_active":"2024-09-17T05:08:06.760761+0000","last_became_peered":"2024-09-17T05:08:06.760761+0000","last_unstale":"2024-09-17T05:08:06.762999+0000","last_undegraded":"2024-09-17T05:08:06.762999+0000","last_fullsized":"2024-09-17T05:08:06.762999+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:09.422764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748461+0000","last_change":"2024-09-17T05:08:06.748461+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748461+0000","last_undegraded":"2024-09-17T05:08:06.748461+0000","last_fullsized":"2024-09-17T05:08:06.748461+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773564+0000","last_change":"2024-09-17T05:08:06.761480+0000","last_active":"2024-09-17T05:08:08.773564+0000","last_peered":"2024-09-17T05:08:08.773564+0000","last_clean":"2024-09-17T05:08:08.773564+0000","last_became_active":"2024-09-17T05:08:06.761119+0000","last_became_peered":"2024-09-17T05:08:06.761119+0000","last_unstale":"2024-09-17T05:08:08.773564+0000","last_undegraded":"2024-09-17T05:08:08.773564+0000","last_fullsized":"2024-09-17T05:08:08.773564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740276+0000","last_change":"2024-09-17T05:08:06.740276+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740276+0000","last_undegraded":"2024-09-17T05:08:06.740276+0000","last_fullsized":"2024-09-17T05:08:06.740276+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.756045+0000","last_change":"2024-09-17T05:08:06.749915+0000","last_active":"2024-09-17T05:08:06.756045+0000","last_peered":"2024-09-17T05:08:06.756045+0000","last_clean":"2024-09-17T05:08:06.756045+0000","last_became_active":"2024-09-17T05:08:06.749590+0000","last_became_peered":"2024-09-17T05:08:06.749590+0000","last_unstale":"2024-09-17T05:08:06.756045+0000","last_undegraded":"2024-09-17T05:08:06.756045+0000","last_fullsized":"2024-09-17T05:08:06.756045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:40.105827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741257+0000","last_change":"2024-09-17T05:08:06.741257+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741257+0000","last_undegraded":"2024-09-17T05:08:06.741257+0000","last_fullsized":"2024-09-17T05:08:06.741257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763468+0000","last_change":"2024-09-17T05:08:06.761320+0000","last_active":"2024-09-17T05:08:06.763468+0000","last_peered":"2024-09-17T05:08:06.763468+0000","last_clean":"2024-09-17T05:08:06.763468+0000","last_became_active":"2024-09-17T05:08:06.760045+0000","last_became_peered":"2024-09-17T05:08:06.760045+0000","last_unstale":"2024-09-17T05:08:06.763468+0000","last_undegraded":"2024-09-17T05:08:06.763468+0000","last_fullsized":"2024-09-17T05:08:06.763468+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:07.884401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741592+0000","last_change":"2024-09-17T05:08:06.741592+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741592+0000","last_undegraded":"2024-09-17T05:08:06.741592+0000","last_fullsized":"2024-09-17T05:08:06.741592+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764389+0000","last_change":"2024-09-17T05:08:06.762962+0000","last_active":"2024-09-17T05:08:06.764389+0000","last_peered":"2024-09-17T05:08:06.764389+0000","last_clean":"2024-09-17T05:08:06.764389+0000","last_became_active":"2024-09-17T05:08:06.762619+0000","last_became_peered":"2024-09-17T05:08:06.762619+0000","last_unstale":"2024-09-17T05:08:06.764389+0000","last_undegraded":"2024-09-17T05:08:06.764389+0000","last_fullsized":"2024-09-17T05:08:06.764389+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.226520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741671+0000","last_change":"2024-09-17T05:08:06.741671+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741671+0000","last_undegraded":"2024-09-17T05:08:06.741671+0000","last_fullsized":"2024-09-17T05:08:06.741671+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760010+0000","last_change":"2024-09-17T05:08:06.757901+0000","last_active":"2024-09-17T05:08:06.760010+0000","last_peered":"2024-09-17T05:08:06.760010+0000","last_clean":"2024-09-17T05:08:06.760010+0000","last_became_active":"2024-09-17T05:08:06.757576+0000","last_became_peered":"2024-09-17T05:08:06.757576+0000","last_unstale":"2024-09-17T05:08:06.760010+0000","last_undegraded":"2024-09-17T05:08:06.760010+0000","last_fullsized":"2024-09-17T05:08:06.760010+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:38.491381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744703+0000","last_change":"2024-09-17T05:08:06.744703+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744703+0000","last_undegraded":"2024-09-17T05:08:06.744703+0000","last_fullsized":"2024-09-17T05:08:06.744703+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758358+0000","last_change":"2024-09-17T05:08:06.756508+0000","last_active":"2024-09-17T05:08:06.758358+0000","last_peered":"2024-09-17T05:08:06.758358+0000","last_clean":"2024-09-17T05:08:06.758358+0000","last_became_active":"2024-09-17T05:08:06.756117+0000","last_became_peered":"2024-09-17T05:08:06.756117+0000","last_unstale":"2024-09-17T05:08:06.758358+0000","last_undegraded":"2024-09-17T05:08:06.758358+0000","last_fullsized":"2024-09-17T05:08:06.758358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:09.980821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.747209+0000","last_change":"2024-09-17T05:08:06.747209+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.747209+0000","last_undegraded":"2024-09-17T05:08:06.747209+0000","last_fullsized":"2024-09-17T05:08:06.747209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764654+0000","last_change":"2024-09-17T05:08:06.762648+0000","last_active":"2024-09-17T05:08:06.764654+0000","last_peered":"2024-09-17T05:08:06.764654+0000","last_clean":"2024-09-17T05:08:06.764654+0000","last_became_active":"2024-09-17T05:08:06.762351+0000","last_became_peered":"2024-09-17T05:08:06.762351+0000","last_unstale":"2024-09-17T05:08:06.764654+0000","last_undegraded":"2024-09-17T05:08:06.764654+0000","last_fullsized":"2024-09-17T05:08:06.764654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:05.490303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744234+0000","last_change":"2024-09-17T05:08:06.744234+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744234+0000","last_undegraded":"2024-09-17T05:08:06.744234+0000","last_fullsized":"2024-09-17T05:08:06.744234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.759349+0000","last_change":"2024-09-17T05:08:06.756469+0000","last_active":"2024-09-17T05:08:06.759349+0000","last_peered":"2024-09-17T05:08:06.759349+0000","last_clean":"2024-09-17T05:08:06.759349+0000","last_became_active":"2024-09-17T05:08:06.756119+0000","last_became_peered":"2024-09-17T05:08:06.756119+0000","last_unstale":"2024-09-17T05:08:06.759349+0000","last_undegraded":"2024-09-17T05:08:06.759349+0000","last_fullsized":"2024-09-17T05:08:06.759349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:34.202346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738887+0000","last_change":"2024-09-17T05:08:06.738887+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738887+0000","last_undegraded":"2024-09-17T05:08:06.738887+0000","last_fullsized":"2024-09-17T05:08:06.738887+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":21,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773530+0000","last_change":"2024-09-17T05:08:06.751740+0000","last_active":"2024-09-17T05:08:08.773530+0000","last_peered":"2024-09-17T05:08:08.773530+0000","last_clean":"2024-09-17T05:08:08.773530+0000","last_became_active":"2024-09-17T05:08:06.751425+0000","last_became_peered":"2024-09-17T05:08:06.751425+0000","last_unstale":"2024-09-17T05:08:08.773530+0000","last_undegraded":"2024-09-17T05:08:08.773530+0000","last_fullsized":"2024-09-17T05:08:08.773530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:19.025078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737240+0000","last_change":"2024-09-17T05:08:06.737240+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737240+0000","last_undegraded":"2024-09-17T05:08:06.737240+0000","last_fullsized":"2024-09-17T05:08:06.737240+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758041+0000","last_change":"2024-09-17T05:08:06.755780+0000","last_active":"2024-09-17T05:08:06.758041+0000","last_peered":"2024-09-17T05:08:06.758041+0000","last_clean":"2024-09-17T05:08:06.758041+0000","last_became_active":"2024-09-17T05:08:06.754229+0000","last_became_peered":"2024-09-17T05:08:06.754229+0000","last_unstale":"2024-09-17T05:08:06.758041+0000","last_undegraded":"2024-09-17T05:08:06.758041+0000","last_fullsized":"2024-09-17T05:08:06.758041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:32.056062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773646+0000","last_change":"2024-09-17T05:08:07.765055+0000","last_active":"2024-09-17T05:08:08.773646+0000","last_peered":"2024-09-17T05:08:08.773646+0000","last_clean":"2024-09-17T05:08:08.773646+0000","last_became_active":"2024-09-17T05:08:07.764271+0000","last_became_peered":"2024-09-17T05:08:07.764271+0000","last_unstale":"2024-09-17T05:08:08.773646+0000","last_undegraded":"2024-09-17T05:08:08.773646+0000","last_fullsized":"2024-09-17T05:08:08.773646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.819206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758439+0000","last_change":"2024-09-17T05:08:06.754758+0000","last_active":"2024-09-17T05:08:06.758439+0000","last_peered":"2024-09-17T05:08:06.758439+0000","last_clean":"2024-09-17T05:08:06.758439+0000","last_became_active":"2024-09-17T05:08:06.754461+0000","last_became_peered":"2024-09-17T05:08:06.754461+0000","last_unstale":"2024-09-17T05:08:06.758439+0000","last_undegraded":"2024-09-17T05:08:06.758439+0000","last_fullsized":"2024-09-17T05:08:06.758439+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:07.894622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744709+0000","last_change":"2024-09-17T05:08:06.744709+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744709+0000","last_undegraded":"2024-09-17T05:08:06.744709+0000","last_fullsized":"2024-09-17T05:08:06.744709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.759715+0000","last_change":"2024-09-17T05:08:06.756328+0000","last_active":"2024-09-17T05:08:06.759715+0000","last_peered":"2024-09-17T05:08:06.759715+0000","last_clean":"2024-09-17T05:08:06.759715+0000","last_became_active":"2024-09-17T05:08:06.756035+0000","last_became_peered":"2024-09-17T05:08:06.756035+0000","last_unstale":"2024-09-17T05:08:06.759715+0000","last_undegraded":"2024-09-17T05:08:06.759715+0000","last_fullsized":"2024-09-17T05:08:06.759715+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:27.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746007+0000","last_change":"2024-09-17T05:08:06.746007+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746007+0000","last_undegraded":"2024-09-17T05:08:06.746007+0000","last_fullsized":"2024-09-17T05:08:06.746007+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763592+0000","last_change":"2024-09-17T05:08:06.760008+0000","last_active":"2024-09-17T05:08:06.763592+0000","last_peered":"2024-09-17T05:08:06.763592+0000","last_clean":"2024-09-17T05:08:06.763592+0000","last_became_active":"2024-09-17T05:08:06.759701+0000","last_became_peered":"2024-09-17T05:08:06.759701+0000","last_unstale":"2024-09-17T05:08:06.763592+0000","last_undegraded":"2024-09-17T05:08:06.763592+0000","last_fullsized":"2024-09-17T05:08:06.763592+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:29.057572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741247+0000","last_change":"2024-09-17T05:08:06.741247+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741247+0000","last_undegraded":"2024-09-17T05:08:06.741247+0000","last_fullsized":"2024-09-17T05:08:06.741247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.765021+0000","last_change":"2024-09-17T05:08:06.763335+0000","last_active":"2024-09-17T05:08:06.765021+0000","last_peered":"2024-09-17T05:08:06.765021+0000","last_clean":"2024-09-17T05:08:06.765021+0000","last_became_active":"2024-09-17T05:08:06.762302+0000","last_became_peered":"2024-09-17T05:08:06.762302+0000","last_unstale":"2024-09-17T05:08:06.765021+0000","last_undegraded":"2024-09-17T05:08:06.765021+0000","last_fullsized":"2024-09-17T05:08:06.765021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:55.330080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773938+0000","last_change":"2024-09-17T05:08:07.775183+0000","last_active":"2024-09-17T05:08:08.773938+0000","last_peered":"2024-09-17T05:08:08.773938+0000","last_clean":"2024-09-17T05:08:08.773938+0000","last_became_active":"2024-09-17T05:08:07.774107+0000","last_became_peered":"2024-09-17T05:08:07.774107+0000","last_unstale":"2024-09-17T05:08:08.773938+0000","last_undegraded":"2024-09-17T05:08:08.773938+0000","last_fullsized":"2024-09-17T05:08:08.773938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:50.882298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763223+0000","last_change":"2024-09-17T05:08:06.756451+0000","last_active":"2024-09-17T05:08:06.763223+0000","last_peered":"2024-09-17T05:08:06.763223+0000","last_clean":"2024-09-17T05:08:06.763223+0000","last_became_active":"2024-09-17T05:08:06.756108+0000","last_became_peered":"2024-09-17T05:08:06.756108+0000","last_unstale":"2024-09-17T05:08:06.763223+0000","last_undegraded":"2024-09-17T05:08:06.763223+0000","last_fullsized":"2024-09-17T05:08:06.763223+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:25.086196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738350+0000","last_change":"2024-09-17T05:08:06.738350+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738350+0000","last_undegraded":"2024-09-17T05:08:06.738350+0000","last_fullsized":"2024-09-17T05:08:06.738350+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763533+0000","last_change":"2024-09-17T05:08:06.759565+0000","last_active":"2024-09-17T05:08:06.763533+0000","last_peered":"2024-09-17T05:08:06.763533+0000","last_clean":"2024-09-17T05:08:06.763533+0000","last_became_active":"2024-09-17T05:08:06.759251+0000","last_became_peered":"2024-09-17T05:08:06.759251+0000","last_unstale":"2024-09-17T05:08:06.763533+0000","last_undegraded":"2024-09-17T05:08:06.763533+0000","last_fullsized":"2024-09-17T05:08:06.763533+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:40.973680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773295+0000","last_change":"2024-09-17T05:08:07.766627+0000","last_active":"2024-09-17T05:08:08.773295+0000","last_peered":"2024-09-17T05:08:08.773295+0000","last_clean":"2024-09-17T05:08:08.773295+0000","last_became_active":"2024-09-17T05:08:07.766265+0000","last_became_peered":"2024-09-17T05:08:07.766265+0000","last_unstale":"2024-09-17T05:08:08.773295+0000","last_undegraded":"2024-09-17T05:08:08.773295+0000","last_fullsized":"2024-09-17T05:08:08.773295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:16.381763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762217+0000","last_change":"2024-09-17T05:08:06.758648+0000","last_active":"2024-09-17T05:08:06.762217+0000","last_peered":"2024-09-17T05:08:06.762217+0000","last_clean":"2024-09-17T05:08:06.762217+0000","last_became_active":"2024-09-17T05:08:06.756522+0000","last_became_peered":"2024-09-17T05:08:06.756522+0000","last_unstale":"2024-09-17T05:08:06.762217+0000","last_undegraded":"2024-09-17T05:08:06.762217+0000","last_fullsized":"2024-09-17T05:08:06.762217+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:35.795566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746537+0000","last_change":"2024-09-17T05:08:06.746537+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746537+0000","last_undegraded":"2024-09-17T05:08:06.746537+0000","last_fullsized":"2024-09-17T05:08:06.746537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762568+0000","last_change":"2024-09-17T05:08:06.759894+0000","last_active":"2024-09-17T05:08:06.762568+0000","last_peered":"2024-09-17T05:08:06.762568+0000","last_clean":"2024-09-17T05:08:06.762568+0000","last_became_active":"2024-09-17T05:08:06.759190+0000","last_became_peered":"2024-09-17T05:08:06.759190+0000","last_unstale":"2024-09-17T05:08:06.762568+0000","last_undegraded":"2024-09-17T05:08:06.762568+0000","last_fullsized":"2024-09-17T05:08:06.762568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:11.666395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740954+0000","last_change":"2024-09-17T05:08:06.740954+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740954+0000","last_undegraded":"2024-09-17T05:08:06.740954+0000","last_fullsized":"2024-09-17T05:08:06.740954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760351+0000","last_change":"2024-09-17T05:08:06.757737+0000","last_active":"2024-09-17T05:08:06.760351+0000","last_peered":"2024-09-17T05:08:06.760351+0000","last_clean":"2024-09-17T05:08:06.760351+0000","last_became_active":"2024-09-17T05:08:06.757394+0000","last_became_peered":"2024-09-17T05:08:06.757394+0000","last_unstale":"2024-09-17T05:08:06.760351+0000","last_undegraded":"2024-09-17T05:08:06.760351+0000","last_fullsized":"2024-09-17T05:08:06.760351+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:42.229187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741382+0000","last_change":"2024-09-17T05:08:06.741382+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741382+0000","last_undegraded":"2024-09-17T05:08:06.741382+0000","last_fullsized":"2024-09-17T05:08:06.741382+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758100+0000","last_change":"2024-09-17T05:08:06.752439+0000","last_active":"2024-09-17T05:08:06.758100+0000","last_peered":"2024-09-17T05:08:06.758100+0000","last_clean":"2024-09-17T05:08:06.758100+0000","last_became_active":"2024-09-17T05:08:06.752134+0000","last_became_peered":"2024-09-17T05:08:06.752134+0000","last_unstale":"2024-09-17T05:08:06.758100+0000","last_undegraded":"2024-09-17T05:08:06.758100+0000","last_fullsized":"2024-09-17T05:08:06.758100+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:51.840516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.747995+0000","last_change":"2024-09-17T05:08:06.747995+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.747995+0000","last_undegraded":"2024-09-17T05:08:06.747995+0000","last_fullsized":"2024-09-17T05:08:06.747995+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773884+0000","last_change":"2024-09-17T05:08:06.758852+0000","last_active":"2024-09-17T05:08:08.773884+0000","last_peered":"2024-09-17T05:08:08.773884+0000","last_clean":"2024-09-17T05:08:08.773884+0000","last_became_active":"2024-09-17T05:08:06.757319+0000","last_became_peered":"2024-09-17T05:08:06.757319+0000","last_unstale":"2024-09-17T05:08:08.773884+0000","last_undegraded":"2024-09-17T05:08:08.773884+0000","last_fullsized":"2024-09-17T05:08:08.773884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:35.769440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742980+0000","last_change":"2024-09-17T05:08:06.742980+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742980+0000","last_undegraded":"2024-09-17T05:08:06.742980+0000","last_fullsized":"2024-09-17T05:08:06.742980+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764906+0000","last_change":"2024-09-17T05:08:06.763961+0000","last_active":"2024-09-17T05:08:06.764906+0000","last_peered":"2024-09-17T05:08:06.764906+0000","last_clean":"2024-09-17T05:08:06.764906+0000","last_became_active":"2024-09-17T05:08:06.763384+0000","last_became_peered":"2024-09-17T05:08:06.763384+0000","last_unstale":"2024-09-17T05:08:06.764906+0000","last_undegraded":"2024-09-17T05:08:06.764906+0000","last_fullsized":"2024-09-17T05:08:06.764906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:38.347875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746304+0000","last_change":"2024-09-17T05:08:06.746304+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746304+0000","last_undegraded":"2024-09-17T05:08:06.746304+0000","last_fullsized":"2024-09-17T05:08:06.746304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763446+0000","last_change":"2024-09-17T05:08:06.758429+0000","last_active":"2024-09-17T05:08:06.763446+0000","last_peered":"2024-09-17T05:08:06.763446+0000","last_clean":"2024-09-17T05:08:06.763446+0000","last_became_active":"2024-09-17T05:08:06.758102+0000","last_became_peered":"2024-09-17T05:08:06.758102+0000","last_unstale":"2024-09-17T05:08:06.763446+0000","last_undegraded":"2024-09-17T05:08:06.763446+0000","last_fullsized":"2024-09-17T05:08:06.763446+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:50.494994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742504+0000","last_change":"2024-09-17T05:08:06.742504+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742504+0000","last_undegraded":"2024-09-17T05:08:06.742504+0000","last_fullsized":"2024-09-17T05:08:06.742504+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760605+0000","last_change":"2024-09-17T05:08:06.754339+0000","last_active":"2024-09-17T05:08:06.760605+0000","last_peered":"2024-09-17T05:08:06.760605+0000","last_clean":"2024-09-17T05:08:06.760605+0000","last_became_active":"2024-09-17T05:08:06.753263+0000","last_became_peered":"2024-09-17T05:08:06.753263+0000","last_unstale":"2024-09-17T05:08:06.760605+0000","last_undegraded":"2024-09-17T05:08:06.760605+0000","last_fullsized":"2024-09-17T05:08:06.760605+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:57.117338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.745993+0000","last_change":"2024-09-17T05:08:06.745993+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.745993+0000","last_undegraded":"2024-09-17T05:08:06.745993+0000","last_fullsized":"2024-09-17T05:08:06.745993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.756524+0000","last_change":"2024-09-17T05:08:06.750182+0000","last_active":"2024-09-17T05:08:06.756524+0000","last_peered":"2024-09-17T05:08:06.756524+0000","last_clean":"2024-09-17T05:08:06.756524+0000","last_became_active":"2024-09-17T05:08:06.749803+0000","last_became_peered":"2024-09-17T05:08:06.749803+0000","last_unstale":"2024-09-17T05:08:06.756524+0000","last_undegraded":"2024-09-17T05:08:06.756524+0000","last_fullsized":"2024-09-17T05:08:06.756524+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:00.515472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773077+0000","last_change":"2024-09-17T05:08:07.765395+0000","last_active":"2024-09-17T05:08:08.773077+0000","last_peered":"2024-09-17T05:08:08.773077+0000","last_clean":"2024-09-17T05:08:08.773077+0000","last_became_active":"2024-09-17T05:08:07.764733+0000","last_became_peered":"2024-09-17T05:08:07.764733+0000","last_unstale":"2024-09-17T05:08:08.773077+0000","last_undegraded":"2024-09-17T05:08:08.773077+0000","last_fullsized":"2024-09-17T05:08:08.773077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:38.846186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762599+0000","last_change":"2024-09-17T05:08:06.760241+0000","last_active":"2024-09-17T05:08:06.762599+0000","last_peered":"2024-09-17T05:08:06.762599+0000","last_clean":"2024-09-17T05:08:06.762599+0000","last_became_active":"2024-09-17T05:08:06.759577+0000","last_became_peered":"2024-09-17T05:08:06.759577+0000","last_unstale":"2024-09-17T05:08:06.762599+0000","last_undegraded":"2024-09-17T05:08:06.762599+0000","last_fullsized":"2024-09-17T05:08:06.762599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:12.881506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737087+0000","last_change":"2024-09-17T05:08:06.737087+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737087+0000","last_undegraded":"2024-09-17T05:08:06.737087+0000","last_fullsized":"2024-09-17T05:08:06.737087+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773378+0000","last_change":"2024-09-17T05:08:06.756365+0000","last_active":"2024-09-17T05:08:08.773378+0000","last_peered":"2024-09-17T05:08:08.773378+0000","last_clean":"2024-09-17T05:08:08.773378+0000","last_became_active":"2024-09-17T05:08:06.756054+0000","last_became_peered":"2024-09-17T05:08:06.756054+0000","last_unstale":"2024-09-17T05:08:08.773378+0000","last_undegraded":"2024-09-17T05:08:08.773378+0000","last_fullsized":"2024-09-17T05:08:08.773378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.455026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741219+0000","last_change":"2024-09-17T05:08:06.741219+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741219+0000","last_undegraded":"2024-09-17T05:08:06.741219+0000","last_fullsized":"2024-09-17T05:08:06.741219+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763256+0000","last_change":"2024-09-17T05:08:06.756541+0000","last_active":"2024-09-17T05:08:06.763256+0000","last_peered":"2024-09-17T05:08:06.763256+0000","last_clean":"2024-09-17T05:08:06.763256+0000","last_became_active":"2024-09-17T05:08:06.756230+0000","last_became_peered":"2024-09-17T05:08:06.756230+0000","last_unstale":"2024-09-17T05:08:06.763256+0000","last_undegraded":"2024-09-17T05:08:06.763256+0000","last_fullsized":"2024-09-17T05:08:06.763256+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:46.413879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.739124+0000","last_change":"2024-09-17T05:08:06.739124+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.739124+0000","last_undegraded":"2024-09-17T05:08:06.739124+0000","last_fullsized":"2024-09-17T05:08:06.739124+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760586+0000","last_change":"2024-09-17T05:08:06.756186+0000","last_active":"2024-09-17T05:08:06.760586+0000","last_peered":"2024-09-17T05:08:06.760586+0000","last_clean":"2024-09-17T05:08:06.760586+0000","last_became_active":"2024-09-17T05:08:06.755300+0000","last_became_peered":"2024-09-17T05:08:06.755300+0000","last_unstale":"2024-09-17T05:08:06.760586+0000","last_undegraded":"2024-09-17T05:08:06.760586+0000","last_fullsized":"2024-09-17T05:08:06.760586+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:29.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":284,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.750214+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:06.750214+0000","last_peered":"2024-09-17T05:08:06.750214+0000","last_clean":"2024-09-17T05:08:06.750214+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:06.750214+0000","last_undegraded":"2024-09-17T05:08:06.750214+0000","last_fullsized":"2024-09-17T05:08:06.750214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":625417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509529,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1222452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1237923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:08:12.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 -- 172.21.15.16:0/2713821423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c8003a60 msgr2=0x7ff1c8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 --2- 172.21.15.16:0/2713821423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c8003a60 0x7ff1c8003ed0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400c290 tx=0x7ff1d401cf60 comp rx=0 tx=0).stop 2024-09-17T05:08:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 -- 172.21.15.16:0/2713821423 shutdown_connections 2024-09-17T05:08:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 --2- 172.21.15.16:0/2713821423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1c8003a60 0x7ff1c8003ed0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 -- 172.21.15.16:0/2713821423 >> 172.21.15.16:0/2713821423 conn(0x7ff1e406f900 msgr2=0x7ff1e40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 -- 172.21.15.16:0/2713821423 shutdown_connections 2024-09-17T05:08:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.578+0000 7ff1ea400640 1 -- 172.21.15.16:0/2713821423 wait complete. 2024-09-17T05:08:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 Processor -- start 2024-09-17T05:08:12.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 -- start start 2024-09-17T05:08:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1c8003a60 0x7ff1e41fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1e415a800 0x7ff1e415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 0x7ff1e413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e415ff30 con 0x7ff1e413f0d0 2024-09-17T05:08:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e415fdb0 con 0x7ff1c8003a60 2024-09-17T05:08:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e4073240 con 0x7ff1e415a800 2024-09-17T05:08:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 0x7ff1e413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 0x7ff1e413f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34118/0 (socket says 172.21.15.16:34118) 2024-09-17T05:08:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 -- 172.21.15.16:0/3089976834 learned_addr learned my addr 172.21.15.16:0/3089976834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1c8003a60 0x7ff1e41fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1daa00640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1e415a800 0x7ff1e415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 -- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1e415a800 msgr2=0x7ff1e415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1e415a800 0x7ff1e415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 -- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1c8003a60 msgr2=0x7ff1e41fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1c8003a60 0x7ff1e41fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 -- 172.21.15.16:0/3089976834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e413fa20 con 0x7ff1e413f0d0 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1c8003a60 0x7ff1e41fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:12.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1e3e00640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 0x7ff1e413f4e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40cd7d0 tx=0x7ff1dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:12.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004230 con 0x7ff1e413f0d0 2024-09-17T05:08:12.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1dc0044c0 con 0x7ff1e413f0d0 2024-09-17T05:08:12.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00fa90 con 0x7ff1e413f0d0 2024-09-17T05:08:12.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e4142cb0 con 0x7ff1e413f0d0 2024-09-17T05:08:12.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.582+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e41421d0 con 0x7ff1e413f0d0 2024-09-17T05:08:12.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.586+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e40737a0 con 0x7ff1e413f0d0 2024-09-17T05:08:12.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.586+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff1dc00fc30 con 0x7ff1e413f0d0 2024-09-17T05:08:12.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.586+0000 7ff1dbe00640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1c0073200 0x7ff1c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.586+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7ff1dc094cf0 con 0x7ff1e413f0d0 2024-09-17T05:08:12.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.586+0000 7ff1e3400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1c0073200 0x7ff1c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.590+0000 7ff1e3400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1c0073200 0x7ff1c0075710 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40ab000 tx=0x7ff1d4019da0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:12.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.590+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff1e413fa20 con 0x7ff1e413f0d0 2024-09-17T05:08:12.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.762+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff1e40b19e0 con 0x7ff1c0073200 2024-09-17T05:08:12.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.770+0000 7ff1dbe00640 1 -- 172.21.15.16:0/3089976834 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248047 (secure 0 0 0) 0x7ff1e40b19e0 con 0x7ff1c0073200 2024-09-17T05:08:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:12.772 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1c0073200 msgr2=0x7ff1c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1c0073200 0x7ff1c0075710 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40ab000 tx=0x7ff1d4019da0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 msgr2=0x7ff1e413f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 0x7ff1e413f4e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40cd7d0 tx=0x7ff1dc011aa0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 shutdown_connections 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff1c0073200 0x7ff1c0075710 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1e413f0d0 0x7ff1e413f4e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1e415a800 0x7ff1e415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 --2- 172.21.15.16:0/3089976834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1c8003a60 0x7ff1e41fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 >> 172.21.15.16:0/3089976834 conn(0x7ff1e406f900 msgr2=0x7ff1e415cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:12.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 shutdown_connections 2024-09-17T05:08:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.774+0000 7ff1ea400640 1 -- 172.21.15.16:0/3089976834 wait complete. 2024-09-17T05:08:12.793 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-17T05:08:11.578325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":113,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269780,"kb_used_data":11936,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704940,"statfs":{"total":773094113280,"available":772817858560,"internally_reserved":0,"allocated":12222464,"data_stored":6246212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.851674"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743530+0000","last_change":"2024-09-17T05:08:06.743530+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743530+0000","last_undegraded":"2024-09-17T05:08:06.743530+0000","last_fullsized":"2024-09-17T05:08:06.743530+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774172+0000","last_change":"2024-09-17T05:08:07.761565+0000","last_active":"2024-09-17T05:08:08.774172+0000","last_peered":"2024-09-17T05:08:08.774172+0000","last_clean":"2024-09-17T05:08:08.774172+0000","last_became_active":"2024-09-17T05:08:07.761261+0000","last_became_peered":"2024-09-17T05:08:07.761261+0000","last_unstale":"2024-09-17T05:08:08.774172+0000","last_undegraded":"2024-09-17T05:08:08.774172+0000","last_fullsized":"2024-09-17T05:08:08.774172+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:41.539621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743447+0000","last_change":"2024-09-17T05:08:06.743447+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743447+0000","last_undegraded":"2024-09-17T05:08:06.743447+0000","last_fullsized":"2024-09-17T05:08:06.743447+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748147+0000","last_change":"2024-09-17T05:08:06.748147+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748147+0000","last_undegraded":"2024-09-17T05:08:06.748147+0000","last_fullsized":"2024-09-17T05:08:06.748147+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738577+0000","last_change":"2024-09-17T05:08:06.738577+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738577+0000","last_undegraded":"2024-09-17T05:08:06.738577+0000","last_fullsized":"2024-09-17T05:08:06.738577+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773432+0000","last_change":"2024-09-17T05:08:07.758709+0000","last_active":"2024-09-17T05:08:08.773432+0000","last_peered":"2024-09-17T05:08:08.773432+0000","last_clean":"2024-09-17T05:08:08.773432+0000","last_became_active":"2024-09-17T05:08:07.758382+0000","last_became_peered":"2024-09-17T05:08:07.758382+0000","last_unstale":"2024-09-17T05:08:08.773432+0000","last_undegraded":"2024-09-17T05:08:08.773432+0000","last_fullsized":"2024-09-17T05:08:08.773432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:11.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773005+0000","last_change":"2024-09-17T05:08:07.787949+0000","last_active":"2024-09-17T05:08:08.773005+0000","last_peered":"2024-09-17T05:08:08.773005+0000","last_clean":"2024-09-17T05:08:08.773005+0000","last_became_active":"2024-09-17T05:08:07.787577+0000","last_became_peered":"2024-09-17T05:08:07.787577+0000","last_unstale":"2024-09-17T05:08:08.773005+0000","last_undegraded":"2024-09-17T05:08:08.773005+0000","last_fullsized":"2024-09-17T05:08:08.773005+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:33.927797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740861+0000","last_change":"2024-09-17T05:08:06.740861+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740861+0000","last_undegraded":"2024-09-17T05:08:06.740861+0000","last_fullsized":"2024-09-17T05:08:06.740861+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748951+0000","last_change":"2024-09-17T05:08:06.748951+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748951+0000","last_undegraded":"2024-09-17T05:08:06.748951+0000","last_fullsized":"2024-09-17T05:08:06.748951+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774347+0000","last_change":"2024-09-17T05:08:07.777536+0000","last_active":"2024-09-17T05:08:08.774347+0000","last_peered":"2024-09-17T05:08:08.774347+0000","last_clean":"2024-09-17T05:08:08.774347+0000","last_became_active":"2024-09-17T05:08:07.777182+0000","last_became_peered":"2024-09-17T05:08:07.777182+0000","last_unstale":"2024-09-17T05:08:08.774347+0000","last_undegraded":"2024-09-17T05:08:08.774347+0000","last_fullsized":"2024-09-17T05:08:08.774347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:26.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742789+0000","last_change":"2024-09-17T05:08:06.742789+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742789+0000","last_undegraded":"2024-09-17T05:08:06.742789+0000","last_fullsized":"2024-09-17T05:08:06.742789+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.749238+0000","last_change":"2024-09-17T05:08:06.749238+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.749238+0000","last_undegraded":"2024-09-17T05:08:06.749238+0000","last_fullsized":"2024-09-17T05:08:06.749238+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773816+0000","last_change":"2024-09-17T05:08:07.776167+0000","last_active":"2024-09-17T05:08:08.773816+0000","last_peered":"2024-09-17T05:08:08.773816+0000","last_clean":"2024-09-17T05:08:08.773816+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:08.773816+0000","last_undegraded":"2024-09-17T05:08:08.773816+0000","last_fullsized":"2024-09-17T05:08:08.773816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:33.590100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.745703+0000","last_change":"2024-09-17T05:08:06.745703+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.745703+0000","last_undegraded":"2024-09-17T05:08:06.745703+0000","last_fullsized":"2024-09-17T05:08:06.745703+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742487+0000","last_change":"2024-09-17T05:08:06.742487+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742487+0000","last_undegraded":"2024-09-17T05:08:06.742487+0000","last_fullsized":"2024-09-17T05:08:06.742487+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746658+0000","last_change":"2024-09-17T05:08:06.746658+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746658+0000","last_undegraded":"2024-09-17T05:08:06.746658+0000","last_fullsized":"2024-09-17T05:08:06.746658+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.750232+0000","last_change":"2024-09-17T05:08:06.750232+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.750232+0000","last_undegraded":"2024-09-17T05:08:06.750232+0000","last_fullsized":"2024-09-17T05:08:06.750232+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.750393+0000","last_change":"2024-09-17T05:08:06.750393+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.750393+0000","last_undegraded":"2024-09-17T05:08:06.750393+0000","last_fullsized":"2024-09-17T05:08:06.750393+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737941+0000","last_change":"2024-09-17T05:08:06.737941+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737941+0000","last_undegraded":"2024-09-17T05:08:06.737941+0000","last_fullsized":"2024-09-17T05:08:06.737941+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773372+0000","last_change":"2024-09-17T05:08:07.774967+0000","last_active":"2024-09-17T05:08:08.773372+0000","last_peered":"2024-09-17T05:08:08.773372+0000","last_clean":"2024-09-17T05:08:08.773372+0000","last_became_active":"2024-09-17T05:08:07.774314+0000","last_became_peered":"2024-09-17T05:08:07.774314+0000","last_unstale":"2024-09-17T05:08:08.773372+0000","last_undegraded":"2024-09-17T05:08:08.773372+0000","last_fullsized":"2024-09-17T05:08:08.773372+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:57.062162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738056+0000","last_change":"2024-09-17T05:08:06.738056+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738056+0000","last_undegraded":"2024-09-17T05:08:06.738056+0000","last_fullsized":"2024-09-17T05:08:06.738056+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744125+0000","last_change":"2024-09-17T05:08:06.744125+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744125+0000","last_undegraded":"2024-09-17T05:08:06.744125+0000","last_fullsized":"2024-09-17T05:08:06.744125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741899+0000","last_change":"2024-09-17T05:08:06.741899+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741899+0000","last_undegraded":"2024-09-17T05:08:06.741899+0000","last_fullsized":"2024-09-17T05:08:06.741899+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.749357+0000","last_change":"2024-09-17T05:08:06.749357+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.749357+0000","last_undegraded":"2024-09-17T05:08:06.749357+0000","last_fullsized":"2024-09-17T05:08:06.749357+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.743023+0000","last_change":"2024-09-17T05:08:06.743023+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.743023+0000","last_undegraded":"2024-09-17T05:08:06.743023+0000","last_fullsized":"2024-09-17T05:08:06.743023+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742252+0000","last_change":"2024-09-17T05:08:06.742252+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742252+0000","last_undegraded":"2024-09-17T05:08:06.742252+0000","last_fullsized":"2024-09-17T05:08:06.742252+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737421+0000","last_change":"2024-09-17T05:08:06.737421+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737421+0000","last_undegraded":"2024-09-17T05:08:06.737421+0000","last_fullsized":"2024-09-17T05:08:06.737421+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748659+0000","last_change":"2024-09-17T05:08:06.748659+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748659+0000","last_undegraded":"2024-09-17T05:08:06.748659+0000","last_fullsized":"2024-09-17T05:08:06.748659+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741360+0000","last_change":"2024-09-17T05:08:06.741360+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741360+0000","last_undegraded":"2024-09-17T05:08:06.741360+0000","last_fullsized":"2024-09-17T05:08:06.741360+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742132+0000","last_change":"2024-09-17T05:08:06.742132+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742132+0000","last_undegraded":"2024-09-17T05:08:06.742132+0000","last_fullsized":"2024-09-17T05:08:06.742132+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742237+0000","last_change":"2024-09-17T05:08:06.742237+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742237+0000","last_undegraded":"2024-09-17T05:08:06.742237+0000","last_fullsized":"2024-09-17T05:08:06.742237+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742081+0000","last_change":"2024-09-17T05:08:06.742081+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742081+0000","last_undegraded":"2024-09-17T05:08:06.742081+0000","last_fullsized":"2024-09-17T05:08:06.742081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740105+0000","last_change":"2024-09-17T05:08:06.740105+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740105+0000","last_undegraded":"2024-09-17T05:08:06.740105+0000","last_fullsized":"2024-09-17T05:08:06.740105+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763330+0000","last_change":"2024-09-17T05:08:06.757535+0000","last_active":"2024-09-17T05:08:06.763330+0000","last_peered":"2024-09-17T05:08:06.763330+0000","last_clean":"2024-09-17T05:08:06.763330+0000","last_became_active":"2024-09-17T05:08:06.757229+0000","last_became_peered":"2024-09-17T05:08:06.757229+0000","last_unstale":"2024-09-17T05:08:06.763330+0000","last_undegraded":"2024-09-17T05:08:06.763330+0000","last_fullsized":"2024-09-17T05:08:06.763330+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:06.277691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773515+0000","last_change":"2024-09-17T05:08:07.767395+0000","last_active":"2024-09-17T05:08:08.773515+0000","last_peered":"2024-09-17T05:08:08.773515+0000","last_clean":"2024-09-17T05:08:08.773515+0000","last_became_active":"2024-09-17T05:08:07.767090+0000","last_became_peered":"2024-09-17T05:08:07.767090+0000","last_unstale":"2024-09-17T05:08:08.773515+0000","last_undegraded":"2024-09-17T05:08:08.773515+0000","last_fullsized":"2024-09-17T05:08:08.773515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:54.194851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.761917+0000","last_change":"2024-09-17T05:08:06.758493+0000","last_active":"2024-09-17T05:08:06.761917+0000","last_peered":"2024-09-17T05:08:06.761917+0000","last_clean":"2024-09-17T05:08:06.761917+0000","last_became_active":"2024-09-17T05:08:06.758182+0000","last_became_peered":"2024-09-17T05:08:06.758182+0000","last_unstale":"2024-09-17T05:08:06.761917+0000","last_undegraded":"2024-09-17T05:08:06.761917+0000","last_fullsized":"2024-09-17T05:08:06.761917+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:42.476520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741598+0000","last_change":"2024-09-17T05:08:06.741598+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741598+0000","last_undegraded":"2024-09-17T05:08:06.741598+0000","last_fullsized":"2024-09-17T05:08:06.741598+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762999+0000","last_change":"2024-09-17T05:08:06.761067+0000","last_active":"2024-09-17T05:08:06.762999+0000","last_peered":"2024-09-17T05:08:06.762999+0000","last_clean":"2024-09-17T05:08:06.762999+0000","last_became_active":"2024-09-17T05:08:06.760761+0000","last_became_peered":"2024-09-17T05:08:06.760761+0000","last_unstale":"2024-09-17T05:08:06.762999+0000","last_undegraded":"2024-09-17T05:08:06.762999+0000","last_fullsized":"2024-09-17T05:08:06.762999+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:09.422764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.748461+0000","last_change":"2024-09-17T05:08:06.748461+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.748461+0000","last_undegraded":"2024-09-17T05:08:06.748461+0000","last_fullsized":"2024-09-17T05:08:06.748461+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773564+0000","last_change":"2024-09-17T05:08:06.761480+0000","last_active":"2024-09-17T05:08:08.773564+0000","last_peered":"2024-09-17T05:08:08.773564+0000","last_clean":"2024-09-17T05:08:08.773564+0000","last_became_active":"2024-09-17T05:08:06.761119+0000","last_became_peered":"2024-09-17T05:08:06.761119+0000","last_unstale":"2024-09-17T05:08:08.773564+0000","last_undegraded":"2024-09-17T05:08:08.773564+0000","last_fullsized":"2024-09-17T05:08:08.773564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740276+0000","last_change":"2024-09-17T05:08:06.740276+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740276+0000","last_undegraded":"2024-09-17T05:08:06.740276+0000","last_fullsized":"2024-09-17T05:08:06.740276+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.756045+0000","last_change":"2024-09-17T05:08:06.749915+0000","last_active":"2024-09-17T05:08:06.756045+0000","last_peered":"2024-09-17T05:08:06.756045+0000","last_clean":"2024-09-17T05:08:06.756045+0000","last_became_active":"2024-09-17T05:08:06.749590+0000","last_became_peered":"2024-09-17T05:08:06.749590+0000","last_unstale":"2024-09-17T05:08:06.756045+0000","last_undegraded":"2024-09-17T05:08:06.756045+0000","last_fullsized":"2024-09-17T05:08:06.756045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:40.105827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741257+0000","last_change":"2024-09-17T05:08:06.741257+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741257+0000","last_undegraded":"2024-09-17T05:08:06.741257+0000","last_fullsized":"2024-09-17T05:08:06.741257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763468+0000","last_change":"2024-09-17T05:08:06.761320+0000","last_active":"2024-09-17T05:08:06.763468+0000","last_peered":"2024-09-17T05:08:06.763468+0000","last_clean":"2024-09-17T05:08:06.763468+0000","last_became_active":"2024-09-17T05:08:06.760045+0000","last_became_peered":"2024-09-17T05:08:06.760045+0000","last_unstale":"2024-09-17T05:08:06.763468+0000","last_undegraded":"2024-09-17T05:08:06.763468+0000","last_fullsized":"2024-09-17T05:08:06.763468+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:07.884401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741592+0000","last_change":"2024-09-17T05:08:06.741592+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741592+0000","last_undegraded":"2024-09-17T05:08:06.741592+0000","last_fullsized":"2024-09-17T05:08:06.741592+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764389+0000","last_change":"2024-09-17T05:08:06.762962+0000","last_active":"2024-09-17T05:08:06.764389+0000","last_peered":"2024-09-17T05:08:06.764389+0000","last_clean":"2024-09-17T05:08:06.764389+0000","last_became_active":"2024-09-17T05:08:06.762619+0000","last_became_peered":"2024-09-17T05:08:06.762619+0000","last_unstale":"2024-09-17T05:08:06.764389+0000","last_undegraded":"2024-09-17T05:08:06.764389+0000","last_fullsized":"2024-09-17T05:08:06.764389+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:24:43.226520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741671+0000","last_change":"2024-09-17T05:08:06.741671+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741671+0000","last_undegraded":"2024-09-17T05:08:06.741671+0000","last_fullsized":"2024-09-17T05:08:06.741671+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760010+0000","last_change":"2024-09-17T05:08:06.757901+0000","last_active":"2024-09-17T05:08:06.760010+0000","last_peered":"2024-09-17T05:08:06.760010+0000","last_clean":"2024-09-17T05:08:06.760010+0000","last_became_active":"2024-09-17T05:08:06.757576+0000","last_became_peered":"2024-09-17T05:08:06.757576+0000","last_unstale":"2024-09-17T05:08:06.760010+0000","last_undegraded":"2024-09-17T05:08:06.760010+0000","last_fullsized":"2024-09-17T05:08:06.760010+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:38.491381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744703+0000","last_change":"2024-09-17T05:08:06.744703+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744703+0000","last_undegraded":"2024-09-17T05:08:06.744703+0000","last_fullsized":"2024-09-17T05:08:06.744703+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758358+0000","last_change":"2024-09-17T05:08:06.756508+0000","last_active":"2024-09-17T05:08:06.758358+0000","last_peered":"2024-09-17T05:08:06.758358+0000","last_clean":"2024-09-17T05:08:06.758358+0000","last_became_active":"2024-09-17T05:08:06.756117+0000","last_became_peered":"2024-09-17T05:08:06.756117+0000","last_unstale":"2024-09-17T05:08:06.758358+0000","last_undegraded":"2024-09-17T05:08:06.758358+0000","last_fullsized":"2024-09-17T05:08:06.758358+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:09.980821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.747209+0000","last_change":"2024-09-17T05:08:06.747209+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.747209+0000","last_undegraded":"2024-09-17T05:08:06.747209+0000","last_fullsized":"2024-09-17T05:08:06.747209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764654+0000","last_change":"2024-09-17T05:08:06.762648+0000","last_active":"2024-09-17T05:08:06.764654+0000","last_peered":"2024-09-17T05:08:06.764654+0000","last_clean":"2024-09-17T05:08:06.764654+0000","last_became_active":"2024-09-17T05:08:06.762351+0000","last_became_peered":"2024-09-17T05:08:06.762351+0000","last_unstale":"2024-09-17T05:08:06.764654+0000","last_undegraded":"2024-09-17T05:08:06.764654+0000","last_fullsized":"2024-09-17T05:08:06.764654+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:05.490303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744234+0000","last_change":"2024-09-17T05:08:06.744234+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744234+0000","last_undegraded":"2024-09-17T05:08:06.744234+0000","last_fullsized":"2024-09-17T05:08:06.744234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.759349+0000","last_change":"2024-09-17T05:08:06.756469+0000","last_active":"2024-09-17T05:08:06.759349+0000","last_peered":"2024-09-17T05:08:06.759349+0000","last_clean":"2024-09-17T05:08:06.759349+0000","last_became_active":"2024-09-17T05:08:06.756119+0000","last_became_peered":"2024-09-17T05:08:06.756119+0000","last_unstale":"2024-09-17T05:08:06.759349+0000","last_undegraded":"2024-09-17T05:08:06.759349+0000","last_fullsized":"2024-09-17T05:08:06.759349+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:34.202346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738887+0000","last_change":"2024-09-17T05:08:06.738887+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738887+0000","last_undegraded":"2024-09-17T05:08:06.738887+0000","last_fullsized":"2024-09-17T05:08:06.738887+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":21,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773530+0000","last_change":"2024-09-17T05:08:06.751740+0000","last_active":"2024-09-17T05:08:08.773530+0000","last_peered":"2024-09-17T05:08:08.773530+0000","last_clean":"2024-09-17T05:08:08.773530+0000","last_became_active":"2024-09-17T05:08:06.751425+0000","last_became_peered":"2024-09-17T05:08:06.751425+0000","last_unstale":"2024-09-17T05:08:08.773530+0000","last_undegraded":"2024-09-17T05:08:08.773530+0000","last_fullsized":"2024-09-17T05:08:08.773530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:19.025078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737240+0000","last_change":"2024-09-17T05:08:06.737240+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737240+0000","last_undegraded":"2024-09-17T05:08:06.737240+0000","last_fullsized":"2024-09-17T05:08:06.737240+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758041+0000","last_change":"2024-09-17T05:08:06.755780+0000","last_active":"2024-09-17T05:08:06.758041+0000","last_peered":"2024-09-17T05:08:06.758041+0000","last_clean":"2024-09-17T05:08:06.758041+0000","last_became_active":"2024-09-17T05:08:06.754229+0000","last_became_peered":"2024-09-17T05:08:06.754229+0000","last_unstale":"2024-09-17T05:08:06.758041+0000","last_undegraded":"2024-09-17T05:08:06.758041+0000","last_fullsized":"2024-09-17T05:08:06.758041+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:32.056062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773646+0000","last_change":"2024-09-17T05:08:07.765055+0000","last_active":"2024-09-17T05:08:08.773646+0000","last_peered":"2024-09-17T05:08:08.773646+0000","last_clean":"2024-09-17T05:08:08.773646+0000","last_became_active":"2024-09-17T05:08:07.764271+0000","last_became_peered":"2024-09-17T05:08:07.764271+0000","last_unstale":"2024-09-17T05:08:08.773646+0000","last_undegraded":"2024-09-17T05:08:08.773646+0000","last_fullsized":"2024-09-17T05:08:08.773646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.819206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758439+0000","last_change":"2024-09-17T05:08:06.754758+0000","last_active":"2024-09-17T05:08:06.758439+0000","last_peered":"2024-09-17T05:08:06.758439+0000","last_clean":"2024-09-17T05:08:06.758439+0000","last_became_active":"2024-09-17T05:08:06.754461+0000","last_became_peered":"2024-09-17T05:08:06.754461+0000","last_unstale":"2024-09-17T05:08:06.758439+0000","last_undegraded":"2024-09-17T05:08:06.758439+0000","last_fullsized":"2024-09-17T05:08:06.758439+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:44:07.894622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.744709+0000","last_change":"2024-09-17T05:08:06.744709+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.744709+0000","last_undegraded":"2024-09-17T05:08:06.744709+0000","last_fullsized":"2024-09-17T05:08:06.744709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.759715+0000","last_change":"2024-09-17T05:08:06.756328+0000","last_active":"2024-09-17T05:08:06.759715+0000","last_peered":"2024-09-17T05:08:06.759715+0000","last_clean":"2024-09-17T05:08:06.759715+0000","last_became_active":"2024-09-17T05:08:06.756035+0000","last_became_peered":"2024-09-17T05:08:06.756035+0000","last_unstale":"2024-09-17T05:08:06.759715+0000","last_undegraded":"2024-09-17T05:08:06.759715+0000","last_fullsized":"2024-09-17T05:08:06.759715+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:27.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746007+0000","last_change":"2024-09-17T05:08:06.746007+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746007+0000","last_undegraded":"2024-09-17T05:08:06.746007+0000","last_fullsized":"2024-09-17T05:08:06.746007+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763592+0000","last_change":"2024-09-17T05:08:06.760008+0000","last_active":"2024-09-17T05:08:06.763592+0000","last_peered":"2024-09-17T05:08:06.763592+0000","last_clean":"2024-09-17T05:08:06.763592+0000","last_became_active":"2024-09-17T05:08:06.759701+0000","last_became_peered":"2024-09-17T05:08:06.759701+0000","last_unstale":"2024-09-17T05:08:06.763592+0000","last_undegraded":"2024-09-17T05:08:06.763592+0000","last_fullsized":"2024-09-17T05:08:06.763592+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:29.057572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741247+0000","last_change":"2024-09-17T05:08:06.741247+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741247+0000","last_undegraded":"2024-09-17T05:08:06.741247+0000","last_fullsized":"2024-09-17T05:08:06.741247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.765021+0000","last_change":"2024-09-17T05:08:06.763335+0000","last_active":"2024-09-17T05:08:06.765021+0000","last_peered":"2024-09-17T05:08:06.765021+0000","last_clean":"2024-09-17T05:08:06.765021+0000","last_became_active":"2024-09-17T05:08:06.762302+0000","last_became_peered":"2024-09-17T05:08:06.762302+0000","last_unstale":"2024-09-17T05:08:06.765021+0000","last_undegraded":"2024-09-17T05:08:06.765021+0000","last_fullsized":"2024-09-17T05:08:06.765021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:55.330080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773938+0000","last_change":"2024-09-17T05:08:07.775183+0000","last_active":"2024-09-17T05:08:08.773938+0000","last_peered":"2024-09-17T05:08:08.773938+0000","last_clean":"2024-09-17T05:08:08.773938+0000","last_became_active":"2024-09-17T05:08:07.774107+0000","last_became_peered":"2024-09-17T05:08:07.774107+0000","last_unstale":"2024-09-17T05:08:08.773938+0000","last_undegraded":"2024-09-17T05:08:08.773938+0000","last_fullsized":"2024-09-17T05:08:08.773938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:50.882298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763223+0000","last_change":"2024-09-17T05:08:06.756451+0000","last_active":"2024-09-17T05:08:06.763223+0000","last_peered":"2024-09-17T05:08:06.763223+0000","last_clean":"2024-09-17T05:08:06.763223+0000","last_became_active":"2024-09-17T05:08:06.756108+0000","last_became_peered":"2024-09-17T05:08:06.756108+0000","last_unstale":"2024-09-17T05:08:06.763223+0000","last_undegraded":"2024-09-17T05:08:06.763223+0000","last_fullsized":"2024-09-17T05:08:06.763223+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:25.086196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.738350+0000","last_change":"2024-09-17T05:08:06.738350+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.738350+0000","last_undegraded":"2024-09-17T05:08:06.738350+0000","last_fullsized":"2024-09-17T05:08:06.738350+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763533+0000","last_change":"2024-09-17T05:08:06.759565+0000","last_active":"2024-09-17T05:08:06.763533+0000","last_peered":"2024-09-17T05:08:06.763533+0000","last_clean":"2024-09-17T05:08:06.763533+0000","last_became_active":"2024-09-17T05:08:06.759251+0000","last_became_peered":"2024-09-17T05:08:06.759251+0000","last_unstale":"2024-09-17T05:08:06.763533+0000","last_undegraded":"2024-09-17T05:08:06.763533+0000","last_fullsized":"2024-09-17T05:08:06.763533+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:40.973680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773295+0000","last_change":"2024-09-17T05:08:07.766627+0000","last_active":"2024-09-17T05:08:08.773295+0000","last_peered":"2024-09-17T05:08:08.773295+0000","last_clean":"2024-09-17T05:08:08.773295+0000","last_became_active":"2024-09-17T05:08:07.766265+0000","last_became_peered":"2024-09-17T05:08:07.766265+0000","last_unstale":"2024-09-17T05:08:08.773295+0000","last_undegraded":"2024-09-17T05:08:08.773295+0000","last_fullsized":"2024-09-17T05:08:08.773295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:16.381763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762217+0000","last_change":"2024-09-17T05:08:06.758648+0000","last_active":"2024-09-17T05:08:06.762217+0000","last_peered":"2024-09-17T05:08:06.762217+0000","last_clean":"2024-09-17T05:08:06.762217+0000","last_became_active":"2024-09-17T05:08:06.756522+0000","last_became_peered":"2024-09-17T05:08:06.756522+0000","last_unstale":"2024-09-17T05:08:06.762217+0000","last_undegraded":"2024-09-17T05:08:06.762217+0000","last_fullsized":"2024-09-17T05:08:06.762217+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:35.795566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746537+0000","last_change":"2024-09-17T05:08:06.746537+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746537+0000","last_undegraded":"2024-09-17T05:08:06.746537+0000","last_fullsized":"2024-09-17T05:08:06.746537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762568+0000","last_change":"2024-09-17T05:08:06.759894+0000","last_active":"2024-09-17T05:08:06.762568+0000","last_peered":"2024-09-17T05:08:06.762568+0000","last_clean":"2024-09-17T05:08:06.762568+0000","last_became_active":"2024-09-17T05:08:06.759190+0000","last_became_peered":"2024-09-17T05:08:06.759190+0000","last_unstale":"2024-09-17T05:08:06.762568+0000","last_undegraded":"2024-09-17T05:08:06.762568+0000","last_fullsized":"2024-09-17T05:08:06.762568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:11.666395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.740954+0000","last_change":"2024-09-17T05:08:06.740954+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.740954+0000","last_undegraded":"2024-09-17T05:08:06.740954+0000","last_fullsized":"2024-09-17T05:08:06.740954+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760351+0000","last_change":"2024-09-17T05:08:06.757737+0000","last_active":"2024-09-17T05:08:06.760351+0000","last_peered":"2024-09-17T05:08:06.760351+0000","last_clean":"2024-09-17T05:08:06.760351+0000","last_became_active":"2024-09-17T05:08:06.757394+0000","last_became_peered":"2024-09-17T05:08:06.757394+0000","last_unstale":"2024-09-17T05:08:06.760351+0000","last_undegraded":"2024-09-17T05:08:06.760351+0000","last_fullsized":"2024-09-17T05:08:06.760351+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:42.229187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741382+0000","last_change":"2024-09-17T05:08:06.741382+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741382+0000","last_undegraded":"2024-09-17T05:08:06.741382+0000","last_fullsized":"2024-09-17T05:08:06.741382+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.758100+0000","last_change":"2024-09-17T05:08:06.752439+0000","last_active":"2024-09-17T05:08:06.758100+0000","last_peered":"2024-09-17T05:08:06.758100+0000","last_clean":"2024-09-17T05:08:06.758100+0000","last_became_active":"2024-09-17T05:08:06.752134+0000","last_became_peered":"2024-09-17T05:08:06.752134+0000","last_unstale":"2024-09-17T05:08:06.758100+0000","last_undegraded":"2024-09-17T05:08:06.758100+0000","last_fullsized":"2024-09-17T05:08:06.758100+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:51.840516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.747995+0000","last_change":"2024-09-17T05:08:06.747995+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.747995+0000","last_undegraded":"2024-09-17T05:08:06.747995+0000","last_fullsized":"2024-09-17T05:08:06.747995+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773884+0000","last_change":"2024-09-17T05:08:06.758852+0000","last_active":"2024-09-17T05:08:08.773884+0000","last_peered":"2024-09-17T05:08:08.773884+0000","last_clean":"2024-09-17T05:08:08.773884+0000","last_became_active":"2024-09-17T05:08:06.757319+0000","last_became_peered":"2024-09-17T05:08:06.757319+0000","last_unstale":"2024-09-17T05:08:08.773884+0000","last_undegraded":"2024-09-17T05:08:08.773884+0000","last_fullsized":"2024-09-17T05:08:08.773884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:35.769440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742980+0000","last_change":"2024-09-17T05:08:06.742980+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742980+0000","last_undegraded":"2024-09-17T05:08:06.742980+0000","last_fullsized":"2024-09-17T05:08:06.742980+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.764906+0000","last_change":"2024-09-17T05:08:06.763961+0000","last_active":"2024-09-17T05:08:06.764906+0000","last_peered":"2024-09-17T05:08:06.764906+0000","last_clean":"2024-09-17T05:08:06.764906+0000","last_became_active":"2024-09-17T05:08:06.763384+0000","last_became_peered":"2024-09-17T05:08:06.763384+0000","last_unstale":"2024-09-17T05:08:06.764906+0000","last_undegraded":"2024-09-17T05:08:06.764906+0000","last_fullsized":"2024-09-17T05:08:06.764906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:38.347875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.746304+0000","last_change":"2024-09-17T05:08:06.746304+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.746304+0000","last_undegraded":"2024-09-17T05:08:06.746304+0000","last_fullsized":"2024-09-17T05:08:06.746304+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763446+0000","last_change":"2024-09-17T05:08:06.758429+0000","last_active":"2024-09-17T05:08:06.763446+0000","last_peered":"2024-09-17T05:08:06.763446+0000","last_clean":"2024-09-17T05:08:06.763446+0000","last_became_active":"2024-09-17T05:08:06.758102+0000","last_became_peered":"2024-09-17T05:08:06.758102+0000","last_unstale":"2024-09-17T05:08:06.763446+0000","last_undegraded":"2024-09-17T05:08:06.763446+0000","last_fullsized":"2024-09-17T05:08:06.763446+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:50.494994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.742504+0000","last_change":"2024-09-17T05:08:06.742504+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.742504+0000","last_undegraded":"2024-09-17T05:08:06.742504+0000","last_fullsized":"2024-09-17T05:08:06.742504+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760605+0000","last_change":"2024-09-17T05:08:06.754339+0000","last_active":"2024-09-17T05:08:06.760605+0000","last_peered":"2024-09-17T05:08:06.760605+0000","last_clean":"2024-09-17T05:08:06.760605+0000","last_became_active":"2024-09-17T05:08:06.753263+0000","last_became_peered":"2024-09-17T05:08:06.753263+0000","last_unstale":"2024-09-17T05:08:06.760605+0000","last_undegraded":"2024-09-17T05:08:06.760605+0000","last_fullsized":"2024-09-17T05:08:06.760605+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:57.117338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.745993+0000","last_change":"2024-09-17T05:08:06.745993+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.745993+0000","last_undegraded":"2024-09-17T05:08:06.745993+0000","last_fullsized":"2024-09-17T05:08:06.745993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.756524+0000","last_change":"2024-09-17T05:08:06.750182+0000","last_active":"2024-09-17T05:08:06.756524+0000","last_peered":"2024-09-17T05:08:06.756524+0000","last_clean":"2024-09-17T05:08:06.756524+0000","last_became_active":"2024-09-17T05:08:06.749803+0000","last_became_peered":"2024-09-17T05:08:06.749803+0000","last_unstale":"2024-09-17T05:08:06.756524+0000","last_undegraded":"2024-09-17T05:08:06.756524+0000","last_fullsized":"2024-09-17T05:08:06.756524+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:00.515472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773077+0000","last_change":"2024-09-17T05:08:07.765395+0000","last_active":"2024-09-17T05:08:08.773077+0000","last_peered":"2024-09-17T05:08:08.773077+0000","last_clean":"2024-09-17T05:08:08.773077+0000","last_became_active":"2024-09-17T05:08:07.764733+0000","last_became_peered":"2024-09-17T05:08:07.764733+0000","last_unstale":"2024-09-17T05:08:08.773077+0000","last_undegraded":"2024-09-17T05:08:08.773077+0000","last_fullsized":"2024-09-17T05:08:08.773077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:38.846186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.762599+0000","last_change":"2024-09-17T05:08:06.760241+0000","last_active":"2024-09-17T05:08:06.762599+0000","last_peered":"2024-09-17T05:08:06.762599+0000","last_clean":"2024-09-17T05:08:06.762599+0000","last_became_active":"2024-09-17T05:08:06.759577+0000","last_became_peered":"2024-09-17T05:08:06.759577+0000","last_unstale":"2024-09-17T05:08:06.762599+0000","last_undegraded":"2024-09-17T05:08:06.762599+0000","last_fullsized":"2024-09-17T05:08:06.762599+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:12.881506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.737087+0000","last_change":"2024-09-17T05:08:06.737087+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.737087+0000","last_undegraded":"2024-09-17T05:08:06.737087+0000","last_fullsized":"2024-09-17T05:08:06.737087+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773378+0000","last_change":"2024-09-17T05:08:06.756365+0000","last_active":"2024-09-17T05:08:08.773378+0000","last_peered":"2024-09-17T05:08:08.773378+0000","last_clean":"2024-09-17T05:08:08.773378+0000","last_became_active":"2024-09-17T05:08:06.756054+0000","last_became_peered":"2024-09-17T05:08:06.756054+0000","last_unstale":"2024-09-17T05:08:08.773378+0000","last_undegraded":"2024-09-17T05:08:08.773378+0000","last_fullsized":"2024-09-17T05:08:08.773378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.455026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.741219+0000","last_change":"2024-09-17T05:08:06.741219+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.741219+0000","last_undegraded":"2024-09-17T05:08:06.741219+0000","last_fullsized":"2024-09-17T05:08:06.741219+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.763256+0000","last_change":"2024-09-17T05:08:06.756541+0000","last_active":"2024-09-17T05:08:06.763256+0000","last_peered":"2024-09-17T05:08:06.763256+0000","last_clean":"2024-09-17T05:08:06.763256+0000","last_became_active":"2024-09-17T05:08:06.756230+0000","last_became_peered":"2024-09-17T05:08:06.756230+0000","last_unstale":"2024-09-17T05:08:06.763256+0000","last_undegraded":"2024-09-17T05:08:06.763256+0000","last_fullsized":"2024-09-17T05:08:06.763256+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:46.413879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":111,"state":"creating+peering","last_fresh":"2024-09-17T05:08:06.739124+0000","last_change":"2024-09-17T05:08:06.739124+0000","last_active":"2024-09-17T05:08:06.727230+0000","last_peered":"2024-09-17T05:08:06.727230+0000","last_clean":"2024-09-17T05:08:06.727230+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T05:08:06.739124+0000","last_undegraded":"2024-09-17T05:08:06.739124+0000","last_fullsized":"2024-09-17T05:08:06.739124+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.760586+0000","last_change":"2024-09-17T05:08:06.756186+0000","last_active":"2024-09-17T05:08:06.760586+0000","last_peered":"2024-09-17T05:08:06.760586+0000","last_clean":"2024-09-17T05:08:06.760586+0000","last_became_active":"2024-09-17T05:08:06.755300+0000","last_became_peered":"2024-09-17T05:08:06.755300+0000","last_unstale":"2024-09-17T05:08:06.760586+0000","last_undegraded":"2024-09-17T05:08:06.760586+0000","last_fullsized":"2024-09-17T05:08:06.760586+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:29.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":284,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-17T05:08:06.750214+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:06.750214+0000","last_peered":"2024-09-17T05:08:06.750214+0000","last_clean":"2024-09-17T05:08:06.750214+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:06.750214+0000","last_undegraded":"2024-09-17T05:08:06.750214+0000","last_fullsized":"2024-09-17T05:08:06.750214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509530,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":625417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509529,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":1222452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1376256,"data_stored":632084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1237923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:08:12.797 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3f is not active+clean 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.743530+0000', 'last_change': '2024-09-17T05:08:06.743530+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.743530+0000', 'last_undegraded': '2024-09-17T05:08:06.743530+0000', 'last_fullsized': '2024-09-17T05:08:06.743530+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3d is not active+clean 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.743447+0000', 'last_change': '2024-09-17T05:08:06.743447+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.743447+0000', 'last_undegraded': '2024-09-17T05:08:06.743447+0000', 'last_fullsized': '2024-09-17T05:08:06.743447+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3c is not active+clean 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.748147+0000', 'last_change': '2024-09-17T05:08:06.748147+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.748147+0000', 'last_undegraded': '2024-09-17T05:08:06.748147+0000', 'last_fullsized': '2024-09-17T05:08:06.748147+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3b is not active+clean 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.738577+0000', 'last_change': '2024-09-17T05:08:06.738577+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.738577+0000', 'last_undegraded': '2024-09-17T05:08:06.738577+0000', 'last_fullsized': '2024-09-17T05:08:06.738577+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:PG 18.38 is not active+clean 2024-09-17T05:08:12.798 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.740861+0000', 'last_change': '2024-09-17T05:08:06.740861+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.740861+0000', 'last_undegraded': '2024-09-17T05:08:06.740861+0000', 'last_fullsized': '2024-09-17T05:08:06.740861+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:PG 18.37 is not active+clean 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.748951+0000', 'last_change': '2024-09-17T05:08:06.748951+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.748951+0000', 'last_undegraded': '2024-09-17T05:08:06.748951+0000', 'last_fullsized': '2024-09-17T05:08:06.748951+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:PG 18.35 is not active+clean 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742789+0000', 'last_change': '2024-09-17T05:08:06.742789+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742789+0000', 'last_undegraded': '2024-09-17T05:08:06.742789+0000', 'last_fullsized': '2024-09-17T05:08:06.742789+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:PG 18.34 is not active+clean 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.749238+0000', 'last_change': '2024-09-17T05:08:06.749238+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.749238+0000', 'last_undegraded': '2024-09-17T05:08:06.749238+0000', 'last_fullsized': '2024-09-17T05:08:06.749238+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:PG 18.32 is not active+clean 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.745703+0000', 'last_change': '2024-09-17T05:08:06.745703+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.745703+0000', 'last_undegraded': '2024-09-17T05:08:06.745703+0000', 'last_fullsized': '2024-09-17T05:08:06.745703+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:PG 18.31 is not active+clean 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742487+0000', 'last_change': '2024-09-17T05:08:06.742487+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742487+0000', 'last_undegraded': '2024-09-17T05:08:06.742487+0000', 'last_fullsized': '2024-09-17T05:08:06.742487+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:08:12.799 INFO:tasks.ceph.ceph_manager.ceph:PG 18.30 is not active+clean 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.746658+0000', 'last_change': '2024-09-17T05:08:06.746658+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.746658+0000', 'last_undegraded': '2024-09-17T05:08:06.746658+0000', 'last_fullsized': '2024-09-17T05:08:06.746658+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2f is not active+clean 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.750232+0000', 'last_change': '2024-09-17T05:08:06.750232+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.750232+0000', 'last_undegraded': '2024-09-17T05:08:06.750232+0000', 'last_fullsized': '2024-09-17T05:08:06.750232+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2e is not active+clean 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.750393+0000', 'last_change': '2024-09-17T05:08:06.750393+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.750393+0000', 'last_undegraded': '2024-09-17T05:08:06.750393+0000', 'last_fullsized': '2024-09-17T05:08:06.750393+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2d is not active+clean 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.737941+0000', 'last_change': '2024-09-17T05:08:06.737941+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.737941+0000', 'last_undegraded': '2024-09-17T05:08:06.737941+0000', 'last_fullsized': '2024-09-17T05:08:06.737941+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2b is not active+clean 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.738056+0000', 'last_change': '2024-09-17T05:08:06.738056+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.738056+0000', 'last_undegraded': '2024-09-17T05:08:06.738056+0000', 'last_fullsized': '2024-09-17T05:08:06.738056+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.800 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2a is not active+clean 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.744125+0000', 'last_change': '2024-09-17T05:08:06.744125+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.744125+0000', 'last_undegraded': '2024-09-17T05:08:06.744125+0000', 'last_fullsized': '2024-09-17T05:08:06.744125+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:PG 18.29 is not active+clean 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741899+0000', 'last_change': '2024-09-17T05:08:06.741899+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741899+0000', 'last_undegraded': '2024-09-17T05:08:06.741899+0000', 'last_fullsized': '2024-09-17T05:08:06.741899+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:PG 18.28 is not active+clean 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.749357+0000', 'last_change': '2024-09-17T05:08:06.749357+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.749357+0000', 'last_undegraded': '2024-09-17T05:08:06.749357+0000', 'last_fullsized': '2024-09-17T05:08:06.749357+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:PG 18.27 is not active+clean 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.743023+0000', 'last_change': '2024-09-17T05:08:06.743023+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.743023+0000', 'last_undegraded': '2024-09-17T05:08:06.743023+0000', 'last_fullsized': '2024-09-17T05:08:06.743023+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:PG 18.26 is not active+clean 2024-09-17T05:08:12.801 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742252+0000', 'last_change': '2024-09-17T05:08:06.742252+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742252+0000', 'last_undegraded': '2024-09-17T05:08:06.742252+0000', 'last_fullsized': '2024-09-17T05:08:06.742252+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:PG 18.25 is not active+clean 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.737421+0000', 'last_change': '2024-09-17T05:08:06.737421+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.737421+0000', 'last_undegraded': '2024-09-17T05:08:06.737421+0000', 'last_fullsized': '2024-09-17T05:08:06.737421+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:PG 18.24 is not active+clean 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.748659+0000', 'last_change': '2024-09-17T05:08:06.748659+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.748659+0000', 'last_undegraded': '2024-09-17T05:08:06.748659+0000', 'last_fullsized': '2024-09-17T05:08:06.748659+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:PG 18.23 is not active+clean 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741360+0000', 'last_change': '2024-09-17T05:08:06.741360+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741360+0000', 'last_undegraded': '2024-09-17T05:08:06.741360+0000', 'last_fullsized': '2024-09-17T05:08:06.741360+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:PG 18.22 is not active+clean 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742132+0000', 'last_change': '2024-09-17T05:08:06.742132+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742132+0000', 'last_undegraded': '2024-09-17T05:08:06.742132+0000', 'last_fullsized': '2024-09-17T05:08:06.742132+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:PG 18.21 is not active+clean 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742237+0000', 'last_change': '2024-09-17T05:08:06.742237+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742237+0000', 'last_undegraded': '2024-09-17T05:08:06.742237+0000', 'last_fullsized': '2024-09-17T05:08:06.742237+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.802 INFO:tasks.ceph.ceph_manager.ceph:PG 18.20 is not active+clean 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742081+0000', 'last_change': '2024-09-17T05:08:06.742081+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742081+0000', 'last_undegraded': '2024-09-17T05:08:06.742081+0000', 'last_fullsized': '2024-09-17T05:08:06.742081+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1c is not active+clean 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.740105+0000', 'last_change': '2024-09-17T05:08:06.740105+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.740105+0000', 'last_undegraded': '2024-09-17T05:08:06.740105+0000', 'last_fullsized': '2024-09-17T05:08:06.740105+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1e is not active+clean 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741598+0000', 'last_change': '2024-09-17T05:08:06.741598+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741598+0000', 'last_undegraded': '2024-09-17T05:08:06.741598+0000', 'last_fullsized': '2024-09-17T05:08:06.741598+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1f is not active+clean 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.748461+0000', 'last_change': '2024-09-17T05:08:06.748461+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.748461+0000', 'last_undegraded': '2024-09-17T05:08:06.748461+0000', 'last_fullsized': '2024-09-17T05:08:06.748461+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:PG 18.18 is not active+clean 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.740276+0000', 'last_change': '2024-09-17T05:08:06.740276+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.740276+0000', 'last_undegraded': '2024-09-17T05:08:06.740276+0000', 'last_fullsized': '2024-09-17T05:08:06.740276+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:08:12.803 INFO:tasks.ceph.ceph_manager.ceph:PG 18.19 is not active+clean 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741257+0000', 'last_change': '2024-09-17T05:08:06.741257+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741257+0000', 'last_undegraded': '2024-09-17T05:08:06.741257+0000', 'last_fullsized': '2024-09-17T05:08:06.741257+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1a is not active+clean 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741592+0000', 'last_change': '2024-09-17T05:08:06.741592+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741592+0000', 'last_undegraded': '2024-09-17T05:08:06.741592+0000', 'last_fullsized': '2024-09-17T05:08:06.741592+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1b is not active+clean 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741671+0000', 'last_change': '2024-09-17T05:08:06.741671+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741671+0000', 'last_undegraded': '2024-09-17T05:08:06.741671+0000', 'last_fullsized': '2024-09-17T05:08:06.741671+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:PG 18.14 is not active+clean 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.744703+0000', 'last_change': '2024-09-17T05:08:06.744703+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.744703+0000', 'last_undegraded': '2024-09-17T05:08:06.744703+0000', 'last_fullsized': '2024-09-17T05:08:06.744703+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:PG 18.15 is not active+clean 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.747209+0000', 'last_change': '2024-09-17T05:08:06.747209+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.747209+0000', 'last_undegraded': '2024-09-17T05:08:06.747209+0000', 'last_fullsized': '2024-09-17T05:08:06.747209+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:08:12.804 INFO:tasks.ceph.ceph_manager.ceph:PG 18.16 is not active+clean 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.744234+0000', 'last_change': '2024-09-17T05:08:06.744234+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.744234+0000', 'last_undegraded': '2024-09-17T05:08:06.744234+0000', 'last_fullsized': '2024-09-17T05:08:06.744234+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:PG 18.17 is not active+clean 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.738887+0000', 'last_change': '2024-09-17T05:08:06.738887+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.738887+0000', 'last_undegraded': '2024-09-17T05:08:06.738887+0000', 'last_fullsized': '2024-09-17T05:08:06.738887+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:PG 18.10 is not active+clean 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.737240+0000', 'last_change': '2024-09-17T05:08:06.737240+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.737240+0000', 'last_undegraded': '2024-09-17T05:08:06.737240+0000', 'last_fullsized': '2024-09-17T05:08:06.737240+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:PG 18.12 is not active+clean 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.744709+0000', 'last_change': '2024-09-17T05:08:06.744709+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.744709+0000', 'last_undegraded': '2024-09-17T05:08:06.744709+0000', 'last_fullsized': '2024-09-17T05:08:06.744709+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:PG 18.c is not active+clean 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.746007+0000', 'last_change': '2024-09-17T05:08:06.746007+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.746007+0000', 'last_undegraded': '2024-09-17T05:08:06.746007+0000', 'last_fullsized': '2024-09-17T05:08:06.746007+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:PG 18.d is not active+clean 2024-09-17T05:08:12.805 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741247+0000', 'last_change': '2024-09-17T05:08:06.741247+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741247+0000', 'last_undegraded': '2024-09-17T05:08:06.741247+0000', 'last_fullsized': '2024-09-17T05:08:06.741247+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:PG 18.f is not active+clean 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.738350+0000', 'last_change': '2024-09-17T05:08:06.738350+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.738350+0000', 'last_undegraded': '2024-09-17T05:08:06.738350+0000', 'last_fullsized': '2024-09-17T05:08:06.738350+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:PG 18.9 is not active+clean 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.746537+0000', 'last_change': '2024-09-17T05:08:06.746537+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.746537+0000', 'last_undegraded': '2024-09-17T05:08:06.746537+0000', 'last_fullsized': '2024-09-17T05:08:06.746537+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:PG 18.a is not active+clean 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.740954+0000', 'last_change': '2024-09-17T05:08:06.740954+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.740954+0000', 'last_undegraded': '2024-09-17T05:08:06.740954+0000', 'last_fullsized': '2024-09-17T05:08:06.740954+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:PG 18.b is not active+clean 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741382+0000', 'last_change': '2024-09-17T05:08:06.741382+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741382+0000', 'last_undegraded': '2024-09-17T05:08:06.741382+0000', 'last_fullsized': '2024-09-17T05:08:06.741382+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:PG 18.4 is not active+clean 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.747995+0000', 'last_change': '2024-09-17T05:08:06.747995+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.747995+0000', 'last_undegraded': '2024-09-17T05:08:06.747995+0000', 'last_fullsized': '2024-09-17T05:08:06.747995+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.806 INFO:tasks.ceph.ceph_manager.ceph:PG 18.5 is not active+clean 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742980+0000', 'last_change': '2024-09-17T05:08:06.742980+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742980+0000', 'last_undegraded': '2024-09-17T05:08:06.742980+0000', 'last_fullsized': '2024-09-17T05:08:06.742980+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:PG 18.6 is not active+clean 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.746304+0000', 'last_change': '2024-09-17T05:08:06.746304+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.746304+0000', 'last_undegraded': '2024-09-17T05:08:06.746304+0000', 'last_fullsized': '2024-09-17T05:08:06.746304+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:PG 18.7 is not active+clean 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.742504+0000', 'last_change': '2024-09-17T05:08:06.742504+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.742504+0000', 'last_undegraded': '2024-09-17T05:08:06.742504+0000', 'last_fullsized': '2024-09-17T05:08:06.742504+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:PG 18.0 is not active+clean 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.745993+0000', 'last_change': '2024-09-17T05:08:06.745993+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.745993+0000', 'last_undegraded': '2024-09-17T05:08:06.745993+0000', 'last_fullsized': '2024-09-17T05:08:06.745993+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2 is not active+clean 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.737087+0000', 'last_change': '2024-09-17T05:08:06.737087+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.737087+0000', 'last_undegraded': '2024-09-17T05:08:06.737087+0000', 'last_fullsized': '2024-09-17T05:08:06.737087+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.807 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3 is not active+clean 2024-09-17T05:08:12.808 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.741219+0000', 'last_change': '2024-09-17T05:08:06.741219+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.741219+0000', 'last_undegraded': '2024-09-17T05:08:06.741219+0000', 'last_fullsized': '2024-09-17T05:08:06.741219+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T05:08:12.808 INFO:tasks.ceph.ceph_manager.ceph:PG 18.13 is not active+clean 2024-09-17T05:08:12.808 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 111, 'state': 'creating+peering', 'last_fresh': '2024-09-17T05:08:06.739124+0000', 'last_change': '2024-09-17T05:08:06.739124+0000', 'last_active': '2024-09-17T05:08:06.727230+0000', 'last_peered': '2024-09-17T05:08:06.727230+0000', 'last_clean': '2024-09-17T05:08:06.727230+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T05:08:06.739124+0000', 'last_undegraded': '2024-09-17T05:08:06.739124+0000', 'last_fullsized': '2024-09-17T05:08:06.739124+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'last_clean_scrub_stamp': '2024-09-17T05:08:06.727230+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T05:08:12.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T05:08:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 -- 172.21.15.16:0/3790336 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 msgr2=0x7f03480b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 --2- 172.21.15.16:0/3790336 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 0x7f03480b2050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0334009840 tx=0x7f033401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 -- 172.21.15.16:0/3790336 shutdown_connections 2024-09-17T05:08:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 --2- 172.21.15.16:0/3790336 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 0x7f03480b2050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 --2- 172.21.15.16:0/3790336 >> v2:172.21.15.104:3300/0 conn(0x7f034815a050 0x7f03480cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 -- 172.21.15.16:0/3790336 >> 172.21.15.16:0/3790336 conn(0x7f03480d2240 msgr2=0x7f03480d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 -- 172.21.15.16:0/3790336 shutdown_connections 2024-09-17T05:08:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 -- 172.21.15.16:0/3790336 wait complete. 2024-09-17T05:08:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 Processor -- start 2024-09-17T05:08:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.994+0000 7f034fa00640 1 -- start start 2024-09-17T05:08:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 0x7f03481fab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 0x7f0348159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f034815a050 0x7f0348159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0348146170 con 0x7f03480b6a90 2024-09-17T05:08:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0348145ff0 con 0x7f034815a050 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f034816a250 con 0x7f03480b1c40 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 0x7f0348159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 0x7f0348159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34122/0 (socket says 172.21.15.16:34122) 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 -- 172.21.15.16:0/1118571625 learned_addr learned my addr 172.21.15.16:0/1118571625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034d000640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 0x7f03481fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034da00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f034815a050 0x7f0348159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 -- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 msgr2=0x7f03481fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 0x7f03481fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 -- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f034815a050 msgr2=0x7f0348159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f034815a050 0x7f0348159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 -- 172.21.15.16:0/1118571625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03481fb0d0 con 0x7f03480b6a90 2024-09-17T05:08:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0347e00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 0x7f0348159150 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f033401ccd0 tx=0x7f0334005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340b2040 con 0x7f03480b6a90 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0334004040 con 0x7f03480b6a90 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334004410 con 0x7f03480b6a90 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0348140310 con 0x7f03480b6a90 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034813b4c0 con 0x7f03480b6a90 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:12.998+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034813af50 con 0x7f03480b6a90 2024-09-17T05:08:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.002+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f03340bd030 con 0x7f03480b6a90 2024-09-17T05:08:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.002+0000 7f0344c00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0328073200 0x7f0328075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.002+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f033412ddb0 con 0x7f03480b6a90 2024-09-17T05:08:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.002+0000 7f034d000640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0328073200 0x7f0328075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.002+0000 7f034d000640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0328073200 0x7f0328075710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f033c009b40 tx=0x7f033c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:13.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.006+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0334132070 con 0x7f03480b6a90 2024-09-17T05:08:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.242+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f03480b2810 con 0x7f03480b6a90 2024-09-17T05:08:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.242+0000 7f0344c00640 1 -- 172.21.15.16:0/1118571625 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7f03340c4060 con 0x7f03480b6a90 2024-09-17T05:08:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:13.244 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":451,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1726549257,"num_in_osds":8,"osd_in_since":1726549246,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":53},{"state_name":"active+clean","count":44}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":591980,"bytes_used":276254720,"bytes_avail":772817858560,"bytes_total":773094113280,"inactive_pgs_ratio":0.54639172554016113,"write_bytes_sec":524,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":75,"btime":"2024-09-17T05:08:10:776523+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":7332}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-17T05:05:59.539048+0000","services":{}},"progress_events":{}} 2024-09-17T05:08:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0328073200 msgr2=0x7f0328075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0328073200 0x7f0328075710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f033c009b40 tx=0x7f033c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 msgr2=0x7f0348159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 0x7f0348159150 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f033401ccd0 tx=0x7f0334005ca0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 shutdown_connections 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0328073200 0x7f0328075710 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f034815a050 0x7f0348159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03480b6a90 0x7f0348159150 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 --2- 172.21.15.16:0/1118571625 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03480b1c40 0x7f03481fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 >> 172.21.15.16:0/1118571625 conn(0x7f03480d2240 msgr2=0x7f034813e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 shutdown_connections 2024-09-17T05:08:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.246+0000 7f034fa00640 1 -- 172.21.15.16:0/1118571625 wait complete. 2024-09-17T05:08:13.266 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T05:08:13.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 -- 172.21.15.16:0/4058556382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815c0a0 msgr2=0x7f442815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 --2- 172.21.15.16:0/4058556382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815c0a0 0x7f442815c4b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4414009900 tx=0x7f441401c850 comp rx=0 tx=0).stop 2024-09-17T05:08:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 -- 172.21.15.16:0/4058556382 shutdown_connections 2024-09-17T05:08:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 --2- 172.21.15.16:0/4058556382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815c0a0 0x7f442815c4b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 --2- 172.21.15.16:0/4058556382 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44280b2f50 0x7f442815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 -- 172.21.15.16:0/4058556382 >> 172.21.15.16:0/4058556382 conn(0x7f442806f900 msgr2=0x7f44280b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 -- 172.21.15.16:0/4058556382 shutdown_connections 2024-09-17T05:08:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 -- 172.21.15.16:0/4058556382 wait complete. 2024-09-17T05:08:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 Processor -- start 2024-09-17T05:08:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.426+0000 7f4430600640 1 -- start start 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44280b2f50 0x7f4428165af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f442815c0a0 0x7f44281419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 0x7f44280cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4428169120 con 0x7f442815f150 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4428168fa0 con 0x7f44280b2f50 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f44281692a0 con 0x7f442815c0a0 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 0x7f44280cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:13.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 0x7f44280cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34140/0 (socket says 172.21.15.16:34140) 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 -- 172.21.15.16:0/2960480379 learned_addr learned my addr 172.21.15.16:0/2960480379 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442dc00640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44280b2f50 0x7f4428165af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442d200640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f442815c0a0 0x7f44281419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 -- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f442815c0a0 msgr2=0x7f44281419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f442815c0a0 0x7f44281419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 -- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44280b2f50 msgr2=0x7f4428165af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44280b2f50 0x7f4428165af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 -- 172.21.15.16:0/2960480379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4428141f10 con 0x7f442815f150 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442d200640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f442815c0a0 0x7f44281419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f442e600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 0x7f44280cf520 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f442000c760 tx=0x7f442000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420003db0 con 0x7f442815f150 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442000ce10 con 0x7f442815f150 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4428164a50 con 0x7f442815f150 2024-09-17T05:08:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44200052d0 con 0x7f442815f150 2024-09-17T05:08:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4428165010 con 0x7f442815f150 2024-09-17T05:08:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.430+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44281647c0 con 0x7f442815f150 2024-09-17T05:08:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.434+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4420005470 con 0x7f442815f150 2024-09-17T05:08:13.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.434+0000 7f4425600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4408073200 0x7f4408075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:13.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.434+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f44200941d0 con 0x7f442815f150 2024-09-17T05:08:13.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.434+0000 7f442dc00640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4408073200 0x7f4408075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:13.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.434+0000 7f442dc00640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4408073200 0x7f4408075710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f441c005fd0 tx=0x7f441c005d10 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.438+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f442005efc0 con 0x7f442815f150 2024-09-17T05:08:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.610+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4428166030 con 0x7f4408073200 2024-09-17T05:08:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.614+0000 7f4425600640 1 -- 172.21.15.16:0/2960480379 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f4428166030 con 0x7f4408073200 2024-09-17T05:08:13.618 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:13.620 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:08:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4408073200 msgr2=0x7f4408075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4408073200 0x7f4408075710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f441c005fd0 tx=0x7f441c005d10 comp rx=0 tx=0).stop 2024-09-17T05:08:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 msgr2=0x7f44280cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 0x7f44280cf520 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f442000c760 tx=0x7f442000cc20 comp rx=0 tx=0).stop 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 shutdown_connections 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4408073200 0x7f4408075710 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f442815f150 0x7f44280cf520 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f442815c0a0 0x7f44281419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 --2- 172.21.15.16:0/2960480379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44280b2f50 0x7f4428165af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 >> 172.21.15.16:0/2960480379 conn(0x7f442806f900 msgr2=0x7f442806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 shutdown_connections 2024-09-17T05:08:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:13.618+0000 7f4430600640 1 -- 172.21.15.16:0/2960480379 wait complete. 2024-09-17T05:08:13.641 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-17T05:08:13.578837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270008,"kb_used_data":12168,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704712,"statfs":{"total":773094113280,"available":772817625088,"internally_reserved":0,"allocated":12460032,"data_stored":6353100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.849032"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318641+0000","last_change":"2024-09-17T05:08:07.795883+0000","last_active":"2024-09-17T05:08:09.318641+0000","last_peered":"2024-09-17T05:08:09.318641+0000","last_clean":"2024-09-17T05:08:09.318641+0000","last_became_active":"2024-09-17T05:08:07.794893+0000","last_became_peered":"2024-09-17T05:08:07.794893+0000","last_unstale":"2024-09-17T05:08:09.318641+0000","last_undegraded":"2024-09-17T05:08:09.318641+0000","last_fullsized":"2024-09-17T05:08:09.318641+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:36.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774172+0000","last_change":"2024-09-17T05:08:07.761565+0000","last_active":"2024-09-17T05:08:08.774172+0000","last_peered":"2024-09-17T05:08:08.774172+0000","last_clean":"2024-09-17T05:08:08.774172+0000","last_became_active":"2024-09-17T05:08:07.761261+0000","last_became_peered":"2024-09-17T05:08:07.761261+0000","last_unstale":"2024-09-17T05:08:08.774172+0000","last_undegraded":"2024-09-17T05:08:08.774172+0000","last_fullsized":"2024-09-17T05:08:08.774172+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:41.539621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317532+0000","last_change":"2024-09-17T05:08:07.776242+0000","last_active":"2024-09-17T05:08:09.317532+0000","last_peered":"2024-09-17T05:08:09.317532+0000","last_clean":"2024-09-17T05:08:09.317532+0000","last_became_active":"2024-09-17T05:08:07.775861+0000","last_became_peered":"2024-09-17T05:08:07.775861+0000","last_unstale":"2024-09-17T05:08:09.317532+0000","last_undegraded":"2024-09-17T05:08:09.317532+0000","last_fullsized":"2024-09-17T05:08:09.317532+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:35.212289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772138+0000","last_change":"2024-09-17T05:08:07.781599+0000","last_active":"2024-09-17T05:08:08.772138+0000","last_peered":"2024-09-17T05:08:08.772138+0000","last_clean":"2024-09-17T05:08:08.772138+0000","last_became_active":"2024-09-17T05:08:07.781146+0000","last_became_peered":"2024-09-17T05:08:07.781146+0000","last_unstale":"2024-09-17T05:08:08.772138+0000","last_undegraded":"2024-09-17T05:08:08.772138+0000","last_fullsized":"2024-09-17T05:08:08.772138+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:01.034082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773258+0000","last_change":"2024-09-17T05:08:07.792531+0000","last_active":"2024-09-17T05:08:08.773258+0000","last_peered":"2024-09-17T05:08:08.773258+0000","last_clean":"2024-09-17T05:08:08.773258+0000","last_became_active":"2024-09-17T05:08:07.792187+0000","last_became_peered":"2024-09-17T05:08:07.792187+0000","last_unstale":"2024-09-17T05:08:08.773258+0000","last_undegraded":"2024-09-17T05:08:08.773258+0000","last_fullsized":"2024-09-17T05:08:08.773258+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:43.484137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773432+0000","last_change":"2024-09-17T05:08:07.758709+0000","last_active":"2024-09-17T05:08:08.773432+0000","last_peered":"2024-09-17T05:08:08.773432+0000","last_clean":"2024-09-17T05:08:08.773432+0000","last_became_active":"2024-09-17T05:08:07.758382+0000","last_became_peered":"2024-09-17T05:08:07.758382+0000","last_unstale":"2024-09-17T05:08:08.773432+0000","last_undegraded":"2024-09-17T05:08:08.773432+0000","last_fullsized":"2024-09-17T05:08:08.773432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:11.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773005+0000","last_change":"2024-09-17T05:08:07.787949+0000","last_active":"2024-09-17T05:08:08.773005+0000","last_peered":"2024-09-17T05:08:08.773005+0000","last_clean":"2024-09-17T05:08:08.773005+0000","last_became_active":"2024-09-17T05:08:07.787577+0000","last_became_peered":"2024-09-17T05:08:07.787577+0000","last_unstale":"2024-09-17T05:08:08.773005+0000","last_undegraded":"2024-09-17T05:08:08.773005+0000","last_fullsized":"2024-09-17T05:08:08.773005+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:33.927797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317528+0000","last_change":"2024-09-17T05:08:07.779739+0000","last_active":"2024-09-17T05:08:09.317528+0000","last_peered":"2024-09-17T05:08:09.317528+0000","last_clean":"2024-09-17T05:08:09.317528+0000","last_became_active":"2024-09-17T05:08:07.779423+0000","last_became_peered":"2024-09-17T05:08:07.779423+0000","last_unstale":"2024-09-17T05:08:09.317528+0000","last_undegraded":"2024-09-17T05:08:09.317528+0000","last_fullsized":"2024-09-17T05:08:09.317528+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:11.762316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772836+0000","last_change":"2024-09-17T05:08:07.779961+0000","last_active":"2024-09-17T05:08:08.772836+0000","last_peered":"2024-09-17T05:08:08.772836+0000","last_clean":"2024-09-17T05:08:08.772836+0000","last_became_active":"2024-09-17T05:08:07.779640+0000","last_became_peered":"2024-09-17T05:08:07.779640+0000","last_unstale":"2024-09-17T05:08:08.772836+0000","last_undegraded":"2024-09-17T05:08:08.772836+0000","last_fullsized":"2024-09-17T05:08:08.772836+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:13.274700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774347+0000","last_change":"2024-09-17T05:08:07.777536+0000","last_active":"2024-09-17T05:08:08.774347+0000","last_peered":"2024-09-17T05:08:08.774347+0000","last_clean":"2024-09-17T05:08:08.774347+0000","last_became_active":"2024-09-17T05:08:07.777182+0000","last_became_peered":"2024-09-17T05:08:07.777182+0000","last_unstale":"2024-09-17T05:08:08.774347+0000","last_undegraded":"2024-09-17T05:08:08.774347+0000","last_fullsized":"2024-09-17T05:08:08.774347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:26.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318781+0000","last_change":"2024-09-17T05:08:07.771467+0000","last_active":"2024-09-17T05:08:09.318781+0000","last_peered":"2024-09-17T05:08:09.318781+0000","last_clean":"2024-09-17T05:08:09.318781+0000","last_became_active":"2024-09-17T05:08:07.771043+0000","last_became_peered":"2024-09-17T05:08:07.771043+0000","last_unstale":"2024-09-17T05:08:09.318781+0000","last_undegraded":"2024-09-17T05:08:09.318781+0000","last_fullsized":"2024-09-17T05:08:09.318781+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:28.962461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775918+0000","last_change":"2024-09-17T05:08:07.789334+0000","last_active":"2024-09-17T05:08:08.775918+0000","last_peered":"2024-09-17T05:08:08.775918+0000","last_clean":"2024-09-17T05:08:08.775918+0000","last_became_active":"2024-09-17T05:08:07.788735+0000","last_became_peered":"2024-09-17T05:08:07.788735+0000","last_unstale":"2024-09-17T05:08:08.775918+0000","last_undegraded":"2024-09-17T05:08:08.775918+0000","last_fullsized":"2024-09-17T05:08:08.775918+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:18.764696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773816+0000","last_change":"2024-09-17T05:08:07.776167+0000","last_active":"2024-09-17T05:08:08.773816+0000","last_peered":"2024-09-17T05:08:08.773816+0000","last_clean":"2024-09-17T05:08:08.773816+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:08.773816+0000","last_undegraded":"2024-09-17T05:08:08.773816+0000","last_fullsized":"2024-09-17T05:08:08.773816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:33.590100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775993+0000","last_change":"2024-09-17T05:08:07.771910+0000","last_active":"2024-09-17T05:08:08.775993+0000","last_peered":"2024-09-17T05:08:08.775993+0000","last_clean":"2024-09-17T05:08:08.775993+0000","last_became_active":"2024-09-17T05:08:07.771593+0000","last_became_peered":"2024-09-17T05:08:07.771593+0000","last_unstale":"2024-09-17T05:08:08.775993+0000","last_undegraded":"2024-09-17T05:08:08.775993+0000","last_fullsized":"2024-09-17T05:08:08.775993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:13.168344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317771+0000","last_change":"2024-09-17T05:08:07.785079+0000","last_active":"2024-09-17T05:08:09.317771+0000","last_peered":"2024-09-17T05:08:09.317771+0000","last_clean":"2024-09-17T05:08:09.317771+0000","last_became_active":"2024-09-17T05:08:07.784658+0000","last_became_peered":"2024-09-17T05:08:07.784658+0000","last_unstale":"2024-09-17T05:08:09.317771+0000","last_undegraded":"2024-09-17T05:08:09.317771+0000","last_fullsized":"2024-09-17T05:08:09.317771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:30.065188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775598+0000","last_change":"2024-09-17T05:08:07.774835+0000","last_active":"2024-09-17T05:08:08.775598+0000","last_peered":"2024-09-17T05:08:08.775598+0000","last_clean":"2024-09-17T05:08:08.775598+0000","last_became_active":"2024-09-17T05:08:07.772572+0000","last_became_peered":"2024-09-17T05:08:07.772572+0000","last_unstale":"2024-09-17T05:08:08.775598+0000","last_undegraded":"2024-09-17T05:08:08.775598+0000","last_fullsized":"2024-09-17T05:08:08.775598+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:21.713633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776280+0000","last_change":"2024-09-17T05:08:07.793244+0000","last_active":"2024-09-17T05:08:08.776280+0000","last_peered":"2024-09-17T05:08:08.776280+0000","last_clean":"2024-09-17T05:08:08.776280+0000","last_became_active":"2024-09-17T05:08:07.792918+0000","last_became_peered":"2024-09-17T05:08:07.792918+0000","last_unstale":"2024-09-17T05:08:08.776280+0000","last_undegraded":"2024-09-17T05:08:08.776280+0000","last_fullsized":"2024-09-17T05:08:08.776280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:07.709317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776210+0000","last_change":"2024-09-17T05:08:07.786589+0000","last_active":"2024-09-17T05:08:08.776210+0000","last_peered":"2024-09-17T05:08:08.776210+0000","last_clean":"2024-09-17T05:08:08.776210+0000","last_became_active":"2024-09-17T05:08:07.786229+0000","last_became_peered":"2024-09-17T05:08:07.786229+0000","last_unstale":"2024-09-17T05:08:08.776210+0000","last_undegraded":"2024-09-17T05:08:08.776210+0000","last_fullsized":"2024-09-17T05:08:08.776210+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:05.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773450+0000","last_change":"2024-09-17T05:08:07.778835+0000","last_active":"2024-09-17T05:08:08.773450+0000","last_peered":"2024-09-17T05:08:08.773450+0000","last_clean":"2024-09-17T05:08:08.773450+0000","last_became_active":"2024-09-17T05:08:07.777881+0000","last_became_peered":"2024-09-17T05:08:07.777881+0000","last_unstale":"2024-09-17T05:08:08.773450+0000","last_undegraded":"2024-09-17T05:08:08.773450+0000","last_fullsized":"2024-09-17T05:08:08.773450+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:36.803182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773372+0000","last_change":"2024-09-17T05:08:07.774967+0000","last_active":"2024-09-17T05:08:08.773372+0000","last_peered":"2024-09-17T05:08:08.773372+0000","last_clean":"2024-09-17T05:08:08.773372+0000","last_became_active":"2024-09-17T05:08:07.774314+0000","last_became_peered":"2024-09-17T05:08:07.774314+0000","last_unstale":"2024-09-17T05:08:08.773372+0000","last_undegraded":"2024-09-17T05:08:08.773372+0000","last_fullsized":"2024-09-17T05:08:08.773372+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:57.062162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773506+0000","last_change":"2024-09-17T05:08:07.788520+0000","last_active":"2024-09-17T05:08:08.773506+0000","last_peered":"2024-09-17T05:08:08.773506+0000","last_clean":"2024-09-17T05:08:08.773506+0000","last_became_active":"2024-09-17T05:08:07.788178+0000","last_became_peered":"2024-09-17T05:08:07.788178+0000","last_unstale":"2024-09-17T05:08:08.773506+0000","last_undegraded":"2024-09-17T05:08:08.773506+0000","last_fullsized":"2024-09-17T05:08:08.773506+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:30.065188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318265+0000","last_change":"2024-09-17T05:08:07.771912+0000","last_active":"2024-09-17T05:08:09.318265+0000","last_peered":"2024-09-17T05:08:09.318265+0000","last_clean":"2024-09-17T05:08:09.318265+0000","last_became_active":"2024-09-17T05:08:07.771506+0000","last_became_peered":"2024-09-17T05:08:07.771506+0000","last_unstale":"2024-09-17T05:08:09.318265+0000","last_undegraded":"2024-09-17T05:08:09.318265+0000","last_fullsized":"2024-09-17T05:08:09.318265+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:19.548866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317923+0000","last_change":"2024-09-17T05:08:07.785226+0000","last_active":"2024-09-17T05:08:09.317923+0000","last_peered":"2024-09-17T05:08:09.317923+0000","last_clean":"2024-09-17T05:08:09.317923+0000","last_became_active":"2024-09-17T05:08:07.784890+0000","last_became_peered":"2024-09-17T05:08:07.784890+0000","last_unstale":"2024-09-17T05:08:09.317923+0000","last_undegraded":"2024-09-17T05:08:09.317923+0000","last_fullsized":"2024-09-17T05:08:09.317923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:50.691059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317761+0000","last_change":"2024-09-17T05:08:07.773890+0000","last_active":"2024-09-17T05:08:09.317761+0000","last_peered":"2024-09-17T05:08:09.317761+0000","last_clean":"2024-09-17T05:08:09.317761+0000","last_became_active":"2024-09-17T05:08:07.772818+0000","last_became_peered":"2024-09-17T05:08:07.772818+0000","last_unstale":"2024-09-17T05:08:09.317761+0000","last_undegraded":"2024-09-17T05:08:09.317761+0000","last_fullsized":"2024-09-17T05:08:09.317761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:52.945082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318921+0000","last_change":"2024-09-17T05:08:07.790671+0000","last_active":"2024-09-17T05:08:09.318921+0000","last_peered":"2024-09-17T05:08:09.318921+0000","last_clean":"2024-09-17T05:08:09.318921+0000","last_became_active":"2024-09-17T05:08:07.790295+0000","last_became_peered":"2024-09-17T05:08:07.790295+0000","last_unstale":"2024-09-17T05:08:09.318921+0000","last_undegraded":"2024-09-17T05:08:09.318921+0000","last_fullsized":"2024-09-17T05:08:09.318921+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:53.330797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318428+0000","last_change":"2024-09-17T05:08:07.763546+0000","last_active":"2024-09-17T05:08:09.318428+0000","last_peered":"2024-09-17T05:08:09.318428+0000","last_clean":"2024-09-17T05:08:09.318428+0000","last_became_active":"2024-09-17T05:08:07.761760+0000","last_became_peered":"2024-09-17T05:08:07.761760+0000","last_unstale":"2024-09-17T05:08:09.318428+0000","last_undegraded":"2024-09-17T05:08:09.318428+0000","last_fullsized":"2024-09-17T05:08:09.318428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:12.953196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773736+0000","last_change":"2024-09-17T05:08:07.793784+0000","last_active":"2024-09-17T05:08:08.773736+0000","last_peered":"2024-09-17T05:08:08.773736+0000","last_clean":"2024-09-17T05:08:08.773736+0000","last_became_active":"2024-09-17T05:08:07.792348+0000","last_became_peered":"2024-09-17T05:08:07.792348+0000","last_unstale":"2024-09-17T05:08:08.773736+0000","last_undegraded":"2024-09-17T05:08:08.773736+0000","last_fullsized":"2024-09-17T05:08:08.773736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:56.337697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776000+0000","last_change":"2024-09-17T05:08:07.789530+0000","last_active":"2024-09-17T05:08:08.776000+0000","last_peered":"2024-09-17T05:08:08.776000+0000","last_clean":"2024-09-17T05:08:08.776000+0000","last_became_active":"2024-09-17T05:08:07.788907+0000","last_became_peered":"2024-09-17T05:08:07.788907+0000","last_unstale":"2024-09-17T05:08:08.776000+0000","last_undegraded":"2024-09-17T05:08:08.776000+0000","last_fullsized":"2024-09-17T05:08:08.776000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:09.724198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317742+0000","last_change":"2024-09-17T05:08:07.775236+0000","last_active":"2024-09-17T05:08:09.317742+0000","last_peered":"2024-09-17T05:08:09.317742+0000","last_clean":"2024-09-17T05:08:09.317742+0000","last_became_active":"2024-09-17T05:08:07.774745+0000","last_became_peered":"2024-09-17T05:08:07.774745+0000","last_unstale":"2024-09-17T05:08:09.317742+0000","last_undegraded":"2024-09-17T05:08:09.317742+0000","last_fullsized":"2024-09-17T05:08:09.317742+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:55.455059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317109+0000","last_change":"2024-09-17T05:08:07.767001+0000","last_active":"2024-09-17T05:08:09.317109+0000","last_peered":"2024-09-17T05:08:09.317109+0000","last_clean":"2024-09-17T05:08:09.317109+0000","last_became_active":"2024-09-17T05:08:07.766612+0000","last_became_peered":"2024-09-17T05:08:07.766612+0000","last_unstale":"2024-09-17T05:08:09.317109+0000","last_undegraded":"2024-09-17T05:08:09.317109+0000","last_fullsized":"2024-09-17T05:08:09.317109+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:51.502611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317812+0000","last_change":"2024-09-17T05:08:07.778092+0000","last_active":"2024-09-17T05:08:09.317812+0000","last_peered":"2024-09-17T05:08:09.317812+0000","last_clean":"2024-09-17T05:08:09.317812+0000","last_became_active":"2024-09-17T05:08:07.777764+0000","last_became_peered":"2024-09-17T05:08:07.777764+0000","last_unstale":"2024-09-17T05:08:09.317812+0000","last_undegraded":"2024-09-17T05:08:09.317812+0000","last_fullsized":"2024-09-17T05:08:09.317812+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:26.543253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317888+0000","last_change":"2024-09-17T05:08:07.764573+0000","last_active":"2024-09-17T05:08:09.317888+0000","last_peered":"2024-09-17T05:08:09.317888+0000","last_clean":"2024-09-17T05:08:09.317888+0000","last_became_active":"2024-09-17T05:08:07.764177+0000","last_became_peered":"2024-09-17T05:08:07.764177+0000","last_unstale":"2024-09-17T05:08:09.317888+0000","last_undegraded":"2024-09-17T05:08:09.317888+0000","last_fullsized":"2024-09-17T05:08:09.317888+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:42.857705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318244+0000","last_change":"2024-09-17T05:08:07.768737+0000","last_active":"2024-09-17T05:08:09.318244+0000","last_peered":"2024-09-17T05:08:09.318244+0000","last_clean":"2024-09-17T05:08:09.318244+0000","last_became_active":"2024-09-17T05:08:07.768436+0000","last_became_peered":"2024-09-17T05:08:07.768436+0000","last_unstale":"2024-09-17T05:08:09.318244+0000","last_undegraded":"2024-09-17T05:08:09.318244+0000","last_fullsized":"2024-09-17T05:08:09.318244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:38.344394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"112'4","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774490+0000","last_change":"2024-09-17T05:08:06.757535+0000","last_active":"2024-09-17T05:08:08.774490+0000","last_peered":"2024-09-17T05:08:08.774490+0000","last_clean":"2024-09-17T05:08:08.774490+0000","last_became_active":"2024-09-17T05:08:06.757229+0000","last_became_peered":"2024-09-17T05:08:06.757229+0000","last_unstale":"2024-09-17T05:08:08.774490+0000","last_undegraded":"2024-09-17T05:08:08.774490+0000","last_fullsized":"2024-09-17T05:08:08.774490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:12.881506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773515+0000","last_change":"2024-09-17T05:08:07.767395+0000","last_active":"2024-09-17T05:08:08.773515+0000","last_peered":"2024-09-17T05:08:08.773515+0000","last_clean":"2024-09-17T05:08:08.773515+0000","last_became_active":"2024-09-17T05:08:07.767090+0000","last_became_peered":"2024-09-17T05:08:07.767090+0000","last_unstale":"2024-09-17T05:08:08.773515+0000","last_undegraded":"2024-09-17T05:08:08.773515+0000","last_fullsized":"2024-09-17T05:08:08.773515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:54.194851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776442+0000","last_change":"2024-09-17T05:08:06.758493+0000","last_active":"2024-09-17T05:08:08.776442+0000","last_peered":"2024-09-17T05:08:08.776442+0000","last_clean":"2024-09-17T05:08:08.776442+0000","last_became_active":"2024-09-17T05:08:06.758182+0000","last_became_peered":"2024-09-17T05:08:06.758182+0000","last_unstale":"2024-09-17T05:08:08.776442+0000","last_undegraded":"2024-09-17T05:08:08.776442+0000","last_fullsized":"2024-09-17T05:08:08.776442+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:27.873341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318494+0000","last_change":"2024-09-17T05:08:07.789474+0000","last_active":"2024-09-17T05:08:09.318494+0000","last_peered":"2024-09-17T05:08:09.318494+0000","last_clean":"2024-09-17T05:08:09.318494+0000","last_became_active":"2024-09-17T05:08:07.789118+0000","last_became_peered":"2024-09-17T05:08:07.789118+0000","last_unstale":"2024-09-17T05:08:09.318494+0000","last_undegraded":"2024-09-17T05:08:09.318494+0000","last_fullsized":"2024-09-17T05:08:09.318494+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:10.564947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318627+0000","last_change":"2024-09-17T05:08:06.761067+0000","last_active":"2024-09-17T05:08:09.318627+0000","last_peered":"2024-09-17T05:08:09.318627+0000","last_clean":"2024-09-17T05:08:09.318627+0000","last_became_active":"2024-09-17T05:08:06.760761+0000","last_became_peered":"2024-09-17T05:08:06.760761+0000","last_unstale":"2024-09-17T05:08:09.318627+0000","last_undegraded":"2024-09-17T05:08:09.318627+0000","last_fullsized":"2024-09-17T05:08:09.318627+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:47.048472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773108+0000","last_change":"2024-09-17T05:08:07.779226+0000","last_active":"2024-09-17T05:08:08.773108+0000","last_peered":"2024-09-17T05:08:08.773108+0000","last_clean":"2024-09-17T05:08:08.773108+0000","last_became_active":"2024-09-17T05:08:07.778909+0000","last_became_peered":"2024-09-17T05:08:07.778909+0000","last_unstale":"2024-09-17T05:08:08.773108+0000","last_undegraded":"2024-09-17T05:08:08.773108+0000","last_fullsized":"2024-09-17T05:08:08.773108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:41.342754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773564+0000","last_change":"2024-09-17T05:08:06.761480+0000","last_active":"2024-09-17T05:08:08.773564+0000","last_peered":"2024-09-17T05:08:08.773564+0000","last_clean":"2024-09-17T05:08:08.773564+0000","last_became_active":"2024-09-17T05:08:06.761119+0000","last_became_peered":"2024-09-17T05:08:06.761119+0000","last_unstale":"2024-09-17T05:08:08.773564+0000","last_undegraded":"2024-09-17T05:08:08.773564+0000","last_fullsized":"2024-09-17T05:08:08.773564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317805+0000","last_change":"2024-09-17T05:08:07.768471+0000","last_active":"2024-09-17T05:08:09.317805+0000","last_peered":"2024-09-17T05:08:09.317805+0000","last_clean":"2024-09-17T05:08:09.317805+0000","last_became_active":"2024-09-17T05:08:07.767987+0000","last_became_peered":"2024-09-17T05:08:07.767987+0000","last_unstale":"2024-09-17T05:08:09.317805+0000","last_undegraded":"2024-09-17T05:08:09.317805+0000","last_fullsized":"2024-09-17T05:08:09.317805+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:23.625287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318461+0000","last_change":"2024-09-17T05:08:06.749915+0000","last_active":"2024-09-17T05:08:09.318461+0000","last_peered":"2024-09-17T05:08:09.318461+0000","last_clean":"2024-09-17T05:08:09.318461+0000","last_became_active":"2024-09-17T05:08:06.749590+0000","last_became_peered":"2024-09-17T05:08:06.749590+0000","last_unstale":"2024-09-17T05:08:09.318461+0000","last_undegraded":"2024-09-17T05:08:09.318461+0000","last_fullsized":"2024-09-17T05:08:09.318461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:32.056062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318104+0000","last_change":"2024-09-17T05:08:07.792210+0000","last_active":"2024-09-17T05:08:09.318104+0000","last_peered":"2024-09-17T05:08:09.318104+0000","last_clean":"2024-09-17T05:08:09.318104+0000","last_became_active":"2024-09-17T05:08:07.789880+0000","last_became_peered":"2024-09-17T05:08:07.789880+0000","last_unstale":"2024-09-17T05:08:09.318104+0000","last_undegraded":"2024-09-17T05:08:09.318104+0000","last_fullsized":"2024-09-17T05:08:09.318104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:57.288839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317710+0000","last_change":"2024-09-17T05:08:06.761320+0000","last_active":"2024-09-17T05:08:09.317710+0000","last_peered":"2024-09-17T05:08:09.317710+0000","last_clean":"2024-09-17T05:08:09.317710+0000","last_became_active":"2024-09-17T05:08:06.760045+0000","last_became_peered":"2024-09-17T05:08:06.760045+0000","last_unstale":"2024-09-17T05:08:09.317710+0000","last_undegraded":"2024-09-17T05:08:09.317710+0000","last_fullsized":"2024-09-17T05:08:09.317710+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:49.710369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317004+0000","last_change":"2024-09-17T05:08:07.776123+0000","last_active":"2024-09-17T05:08:09.317004+0000","last_peered":"2024-09-17T05:08:09.317004+0000","last_clean":"2024-09-17T05:08:09.317004+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:09.317004+0000","last_undegraded":"2024-09-17T05:08:09.317004+0000","last_fullsized":"2024-09-17T05:08:09.317004+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:47.421496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318035+0000","last_change":"2024-09-17T05:08:06.762962+0000","last_active":"2024-09-17T05:08:09.318035+0000","last_peered":"2024-09-17T05:08:09.318035+0000","last_clean":"2024-09-17T05:08:09.318035+0000","last_became_active":"2024-09-17T05:08:06.762619+0000","last_became_peered":"2024-09-17T05:08:06.762619+0000","last_unstale":"2024-09-17T05:08:09.318035+0000","last_undegraded":"2024-09-17T05:08:09.318035+0000","last_fullsized":"2024-09-17T05:08:09.318035+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:29.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318035+0000","last_change":"2024-09-17T05:08:07.762347+0000","last_active":"2024-09-17T05:08:09.318035+0000","last_peered":"2024-09-17T05:08:09.318035+0000","last_clean":"2024-09-17T05:08:09.318035+0000","last_became_active":"2024-09-17T05:08:07.760607+0000","last_became_peered":"2024-09-17T05:08:07.760607+0000","last_unstale":"2024-09-17T05:08:09.318035+0000","last_undegraded":"2024-09-17T05:08:09.318035+0000","last_fullsized":"2024-09-17T05:08:09.318035+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:33.331797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.316840+0000","last_change":"2024-09-17T05:08:06.757901+0000","last_active":"2024-09-17T05:08:09.316840+0000","last_peered":"2024-09-17T05:08:09.316840+0000","last_clean":"2024-09-17T05:08:09.316840+0000","last_became_active":"2024-09-17T05:08:06.757576+0000","last_became_peered":"2024-09-17T05:08:06.757576+0000","last_unstale":"2024-09-17T05:08:09.316840+0000","last_undegraded":"2024-09-17T05:08:09.316840+0000","last_fullsized":"2024-09-17T05:08:09.316840+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:14.133127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318368+0000","last_change":"2024-09-17T05:08:07.790571+0000","last_active":"2024-09-17T05:08:09.318368+0000","last_peered":"2024-09-17T05:08:09.318368+0000","last_clean":"2024-09-17T05:08:09.318368+0000","last_became_active":"2024-09-17T05:08:07.790207+0000","last_became_peered":"2024-09-17T05:08:07.790207+0000","last_unstale":"2024-09-17T05:08:09.318368+0000","last_undegraded":"2024-09-17T05:08:09.318368+0000","last_fullsized":"2024-09-17T05:08:09.318368+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:01.034082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773202+0000","last_change":"2024-09-17T05:08:06.756508+0000","last_active":"2024-09-17T05:08:08.773202+0000","last_peered":"2024-09-17T05:08:08.773202+0000","last_clean":"2024-09-17T05:08:08.773202+0000","last_became_active":"2024-09-17T05:08:06.756117+0000","last_became_peered":"2024-09-17T05:08:06.756117+0000","last_unstale":"2024-09-17T05:08:08.773202+0000","last_undegraded":"2024-09-17T05:08:08.773202+0000","last_fullsized":"2024-09-17T05:08:08.773202+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:09.073307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772446+0000","last_change":"2024-09-17T05:08:07.781110+0000","last_active":"2024-09-17T05:08:08.772446+0000","last_peered":"2024-09-17T05:08:08.772446+0000","last_clean":"2024-09-17T05:08:08.772446+0000","last_became_active":"2024-09-17T05:08:07.780749+0000","last_became_peered":"2024-09-17T05:08:07.780749+0000","last_unstale":"2024-09-17T05:08:08.772446+0000","last_undegraded":"2024-09-17T05:08:08.772446+0000","last_fullsized":"2024-09-17T05:08:08.772446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:19.548866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773001+0000","last_change":"2024-09-17T05:08:06.762648+0000","last_active":"2024-09-17T05:08:08.773001+0000","last_peered":"2024-09-17T05:08:08.773001+0000","last_clean":"2024-09-17T05:08:08.773001+0000","last_became_active":"2024-09-17T05:08:06.762351+0000","last_became_peered":"2024-09-17T05:08:06.762351+0000","last_unstale":"2024-09-17T05:08:08.773001+0000","last_undegraded":"2024-09-17T05:08:08.773001+0000","last_fullsized":"2024-09-17T05:08:08.773001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:29.526777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317076+0000","last_change":"2024-09-17T05:08:07.771518+0000","last_active":"2024-09-17T05:08:09.317076+0000","last_peered":"2024-09-17T05:08:09.317076+0000","last_clean":"2024-09-17T05:08:09.317076+0000","last_became_active":"2024-09-17T05:08:07.771149+0000","last_became_peered":"2024-09-17T05:08:07.771149+0000","last_unstale":"2024-09-17T05:08:09.317076+0000","last_undegraded":"2024-09-17T05:08:09.317076+0000","last_fullsized":"2024-09-17T05:08:09.317076+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:07.285307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"112'3","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.319004+0000","last_change":"2024-09-17T05:08:06.756469+0000","last_active":"2024-09-17T05:08:09.319004+0000","last_peered":"2024-09-17T05:08:09.319004+0000","last_clean":"2024-09-17T05:08:09.319004+0000","last_became_active":"2024-09-17T05:08:06.756119+0000","last_became_peered":"2024-09-17T05:08:06.756119+0000","last_unstale":"2024-09-17T05:08:09.319004+0000","last_undegraded":"2024-09-17T05:08:09.319004+0000","last_fullsized":"2024-09-17T05:08:09.319004+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:27.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774274+0000","last_change":"2024-09-17T05:08:07.796237+0000","last_active":"2024-09-17T05:08:08.774274+0000","last_peered":"2024-09-17T05:08:08.774274+0000","last_clean":"2024-09-17T05:08:08.774274+0000","last_became_active":"2024-09-17T05:08:07.793673+0000","last_became_peered":"2024-09-17T05:08:07.793673+0000","last_unstale":"2024-09-17T05:08:08.774274+0000","last_undegraded":"2024-09-17T05:08:08.774274+0000","last_fullsized":"2024-09-17T05:08:08.774274+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:55.087320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":21,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773530+0000","last_change":"2024-09-17T05:08:06.751740+0000","last_active":"2024-09-17T05:08:08.773530+0000","last_peered":"2024-09-17T05:08:08.773530+0000","last_clean":"2024-09-17T05:08:08.773530+0000","last_became_active":"2024-09-17T05:08:06.751425+0000","last_became_peered":"2024-09-17T05:08:06.751425+0000","last_unstale":"2024-09-17T05:08:08.773530+0000","last_undegraded":"2024-09-17T05:08:08.773530+0000","last_fullsized":"2024-09-17T05:08:08.773530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:19.025078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772650+0000","last_change":"2024-09-17T05:08:07.775856+0000","last_active":"2024-09-17T05:08:08.772650+0000","last_peered":"2024-09-17T05:08:08.772650+0000","last_clean":"2024-09-17T05:08:08.772650+0000","last_became_active":"2024-09-17T05:08:07.775439+0000","last_became_peered":"2024-09-17T05:08:07.775439+0000","last_unstale":"2024-09-17T05:08:08.772650+0000","last_undegraded":"2024-09-17T05:08:08.772650+0000","last_fullsized":"2024-09-17T05:08:08.772650+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:21.426193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318290+0000","last_change":"2024-09-17T05:08:06.755780+0000","last_active":"2024-09-17T05:08:09.318290+0000","last_peered":"2024-09-17T05:08:09.318290+0000","last_clean":"2024-09-17T05:08:09.318290+0000","last_became_active":"2024-09-17T05:08:06.754229+0000","last_became_peered":"2024-09-17T05:08:06.754229+0000","last_unstale":"2024-09-17T05:08:09.318290+0000","last_undegraded":"2024-09-17T05:08:09.318290+0000","last_fullsized":"2024-09-17T05:08:09.318290+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:11.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773646+0000","last_change":"2024-09-17T05:08:07.765055+0000","last_active":"2024-09-17T05:08:08.773646+0000","last_peered":"2024-09-17T05:08:08.773646+0000","last_clean":"2024-09-17T05:08:08.773646+0000","last_became_active":"2024-09-17T05:08:07.764271+0000","last_became_peered":"2024-09-17T05:08:07.764271+0000","last_unstale":"2024-09-17T05:08:08.773646+0000","last_undegraded":"2024-09-17T05:08:08.773646+0000","last_fullsized":"2024-09-17T05:08:08.773646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.819206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317451+0000","last_change":"2024-09-17T05:08:06.754758+0000","last_active":"2024-09-17T05:08:09.317451+0000","last_peered":"2024-09-17T05:08:09.317451+0000","last_clean":"2024-09-17T05:08:09.317451+0000","last_became_active":"2024-09-17T05:08:06.754461+0000","last_became_peered":"2024-09-17T05:08:06.754461+0000","last_unstale":"2024-09-17T05:08:09.317451+0000","last_undegraded":"2024-09-17T05:08:09.317451+0000","last_fullsized":"2024-09-17T05:08:09.317451+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:42.829403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776079+0000","last_change":"2024-09-17T05:08:07.791789+0000","last_active":"2024-09-17T05:08:08.776079+0000","last_peered":"2024-09-17T05:08:08.776079+0000","last_clean":"2024-09-17T05:08:08.776079+0000","last_became_active":"2024-09-17T05:08:07.791512+0000","last_became_peered":"2024-09-17T05:08:07.791512+0000","last_unstale":"2024-09-17T05:08:08.776079+0000","last_undegraded":"2024-09-17T05:08:08.776079+0000","last_fullsized":"2024-09-17T05:08:08.776079+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:38.297384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317179+0000","last_change":"2024-09-17T05:08:06.756328+0000","last_active":"2024-09-17T05:08:09.317179+0000","last_peered":"2024-09-17T05:08:09.317179+0000","last_clean":"2024-09-17T05:08:09.317179+0000","last_became_active":"2024-09-17T05:08:06.756035+0000","last_became_peered":"2024-09-17T05:08:06.756035+0000","last_unstale":"2024-09-17T05:08:09.317179+0000","last_undegraded":"2024-09-17T05:08:09.317179+0000","last_fullsized":"2024-09-17T05:08:09.317179+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:42.859744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772307+0000","last_change":"2024-09-17T05:08:07.781526+0000","last_active":"2024-09-17T05:08:08.772307+0000","last_peered":"2024-09-17T05:08:08.772307+0000","last_clean":"2024-09-17T05:08:08.772307+0000","last_became_active":"2024-09-17T05:08:07.780689+0000","last_became_peered":"2024-09-17T05:08:07.780689+0000","last_unstale":"2024-09-17T05:08:08.772307+0000","last_undegraded":"2024-09-17T05:08:08.772307+0000","last_fullsized":"2024-09-17T05:08:08.772307+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:50.691059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776292+0000","last_change":"2024-09-17T05:08:06.760008+0000","last_active":"2024-09-17T05:08:08.776292+0000","last_peered":"2024-09-17T05:08:08.776292+0000","last_clean":"2024-09-17T05:08:08.776292+0000","last_became_active":"2024-09-17T05:08:06.759701+0000","last_became_peered":"2024-09-17T05:08:06.759701+0000","last_unstale":"2024-09-17T05:08:08.776292+0000","last_undegraded":"2024-09-17T05:08:08.776292+0000","last_fullsized":"2024-09-17T05:08:08.776292+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:00.123242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318783+0000","last_change":"2024-09-17T05:08:07.763579+0000","last_active":"2024-09-17T05:08:09.318783+0000","last_peered":"2024-09-17T05:08:09.318783+0000","last_clean":"2024-09-17T05:08:09.318783+0000","last_became_active":"2024-09-17T05:08:07.763140+0000","last_became_peered":"2024-09-17T05:08:07.763140+0000","last_unstale":"2024-09-17T05:08:09.318783+0000","last_undegraded":"2024-09-17T05:08:09.318783+0000","last_fullsized":"2024-09-17T05:08:09.318783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:38.344394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776486+0000","last_change":"2024-09-17T05:08:06.763335+0000","last_active":"2024-09-17T05:08:08.776486+0000","last_peered":"2024-09-17T05:08:08.776486+0000","last_clean":"2024-09-17T05:08:08.776486+0000","last_became_active":"2024-09-17T05:08:06.762302+0000","last_became_peered":"2024-09-17T05:08:06.762302+0000","last_unstale":"2024-09-17T05:08:08.776486+0000","last_undegraded":"2024-09-17T05:08:08.776486+0000","last_fullsized":"2024-09-17T05:08:08.776486+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:25.339809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773938+0000","last_change":"2024-09-17T05:08:07.775183+0000","last_active":"2024-09-17T05:08:08.773938+0000","last_peered":"2024-09-17T05:08:08.773938+0000","last_clean":"2024-09-17T05:08:08.773938+0000","last_became_active":"2024-09-17T05:08:07.774107+0000","last_became_peered":"2024-09-17T05:08:07.774107+0000","last_unstale":"2024-09-17T05:08:08.773938+0000","last_undegraded":"2024-09-17T05:08:08.773938+0000","last_fullsized":"2024-09-17T05:08:08.773938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:50.882298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773728+0000","last_change":"2024-09-17T05:08:06.756451+0000","last_active":"2024-09-17T05:08:08.773728+0000","last_peered":"2024-09-17T05:08:08.773728+0000","last_clean":"2024-09-17T05:08:08.773728+0000","last_became_active":"2024-09-17T05:08:06.756108+0000","last_became_peered":"2024-09-17T05:08:06.756108+0000","last_unstale":"2024-09-17T05:08:08.773728+0000","last_undegraded":"2024-09-17T05:08:08.773728+0000","last_fullsized":"2024-09-17T05:08:08.773728+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:07.818594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774345+0000","last_change":"2024-09-17T05:08:07.801223+0000","last_active":"2024-09-17T05:08:08.774345+0000","last_peered":"2024-09-17T05:08:08.774345+0000","last_clean":"2024-09-17T05:08:08.774345+0000","last_became_active":"2024-09-17T05:08:07.800861+0000","last_became_peered":"2024-09-17T05:08:07.800861+0000","last_unstale":"2024-09-17T05:08:08.774345+0000","last_undegraded":"2024-09-17T05:08:08.774345+0000","last_fullsized":"2024-09-17T05:08:08.774345+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:06.112808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318721+0000","last_change":"2024-09-17T05:08:06.759565+0000","last_active":"2024-09-17T05:08:09.318721+0000","last_peered":"2024-09-17T05:08:09.318721+0000","last_clean":"2024-09-17T05:08:09.318721+0000","last_became_active":"2024-09-17T05:08:06.759251+0000","last_became_peered":"2024-09-17T05:08:06.759251+0000","last_unstale":"2024-09-17T05:08:09.318721+0000","last_undegraded":"2024-09-17T05:08:09.318721+0000","last_fullsized":"2024-09-17T05:08:09.318721+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:34.624245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773295+0000","last_change":"2024-09-17T05:08:07.766627+0000","last_active":"2024-09-17T05:08:08.773295+0000","last_peered":"2024-09-17T05:08:08.773295+0000","last_clean":"2024-09-17T05:08:08.773295+0000","last_became_active":"2024-09-17T05:08:07.766265+0000","last_became_peered":"2024-09-17T05:08:07.766265+0000","last_unstale":"2024-09-17T05:08:08.773295+0000","last_undegraded":"2024-09-17T05:08:08.773295+0000","last_fullsized":"2024-09-17T05:08:08.773295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:16.381763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775691+0000","last_change":"2024-09-17T05:08:06.758648+0000","last_active":"2024-09-17T05:08:08.775691+0000","last_peered":"2024-09-17T05:08:08.775691+0000","last_clean":"2024-09-17T05:08:08.775691+0000","last_became_active":"2024-09-17T05:08:06.756522+0000","last_became_peered":"2024-09-17T05:08:06.756522+0000","last_unstale":"2024-09-17T05:08:08.775691+0000","last_undegraded":"2024-09-17T05:08:08.775691+0000","last_fullsized":"2024-09-17T05:08:08.775691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:54.079704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775621+0000","last_change":"2024-09-17T05:08:07.793164+0000","last_active":"2024-09-17T05:08:08.775621+0000","last_peered":"2024-09-17T05:08:08.775621+0000","last_clean":"2024-09-17T05:08:08.775621+0000","last_became_active":"2024-09-17T05:08:07.792897+0000","last_became_peered":"2024-09-17T05:08:07.792897+0000","last_unstale":"2024-09-17T05:08:08.775621+0000","last_undegraded":"2024-09-17T05:08:08.775621+0000","last_fullsized":"2024-09-17T05:08:08.775621+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:12.831669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318459+0000","last_change":"2024-09-17T05:08:06.759894+0000","last_active":"2024-09-17T05:08:09.318459+0000","last_peered":"2024-09-17T05:08:09.318459+0000","last_clean":"2024-09-17T05:08:09.318459+0000","last_became_active":"2024-09-17T05:08:06.759190+0000","last_became_peered":"2024-09-17T05:08:06.759190+0000","last_unstale":"2024-09-17T05:08:09.318459+0000","last_undegraded":"2024-09-17T05:08:09.318459+0000","last_fullsized":"2024-09-17T05:08:09.318459+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:20.714922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317310+0000","last_change":"2024-09-17T05:08:07.782682+0000","last_active":"2024-09-17T05:08:09.317310+0000","last_peered":"2024-09-17T05:08:09.317310+0000","last_clean":"2024-09-17T05:08:09.317310+0000","last_became_active":"2024-09-17T05:08:07.782259+0000","last_became_peered":"2024-09-17T05:08:07.782259+0000","last_unstale":"2024-09-17T05:08:09.317310+0000","last_undegraded":"2024-09-17T05:08:09.317310+0000","last_fullsized":"2024-09-17T05:08:09.317310+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:49.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.316859+0000","last_change":"2024-09-17T05:08:06.757737+0000","last_active":"2024-09-17T05:08:09.316859+0000","last_peered":"2024-09-17T05:08:09.316859+0000","last_clean":"2024-09-17T05:08:09.316859+0000","last_became_active":"2024-09-17T05:08:06.757394+0000","last_became_peered":"2024-09-17T05:08:06.757394+0000","last_unstale":"2024-09-17T05:08:09.316859+0000","last_undegraded":"2024-09-17T05:08:09.316859+0000","last_fullsized":"2024-09-17T05:08:09.316859+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:02.184148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318285+0000","last_change":"2024-09-17T05:08:07.765739+0000","last_active":"2024-09-17T05:08:09.318285+0000","last_peered":"2024-09-17T05:08:09.318285+0000","last_clean":"2024-09-17T05:08:09.318285+0000","last_became_active":"2024-09-17T05:08:07.765426+0000","last_became_peered":"2024-09-17T05:08:07.765426+0000","last_unstale":"2024-09-17T05:08:09.318285+0000","last_undegraded":"2024-09-17T05:08:09.318285+0000","last_fullsized":"2024-09-17T05:08:09.318285+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:09.876043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772589+0000","last_change":"2024-09-17T05:08:06.752439+0000","last_active":"2024-09-17T05:08:08.772589+0000","last_peered":"2024-09-17T05:08:08.772589+0000","last_clean":"2024-09-17T05:08:08.772589+0000","last_became_active":"2024-09-17T05:08:06.752134+0000","last_became_peered":"2024-09-17T05:08:06.752134+0000","last_unstale":"2024-09-17T05:08:08.772589+0000","last_undegraded":"2024-09-17T05:08:08.772589+0000","last_fullsized":"2024-09-17T05:08:08.772589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.197920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776212+0000","last_change":"2024-09-17T05:08:07.786689+0000","last_active":"2024-09-17T05:08:08.776212+0000","last_peered":"2024-09-17T05:08:08.776212+0000","last_clean":"2024-09-17T05:08:08.776212+0000","last_became_active":"2024-09-17T05:08:07.786275+0000","last_became_peered":"2024-09-17T05:08:07.786275+0000","last_unstale":"2024-09-17T05:08:08.776212+0000","last_undegraded":"2024-09-17T05:08:08.776212+0000","last_fullsized":"2024-09-17T05:08:08.776212+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:51.289187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773884+0000","last_change":"2024-09-17T05:08:06.758852+0000","last_active":"2024-09-17T05:08:08.773884+0000","last_peered":"2024-09-17T05:08:08.773884+0000","last_clean":"2024-09-17T05:08:08.773884+0000","last_became_active":"2024-09-17T05:08:06.757319+0000","last_became_peered":"2024-09-17T05:08:06.757319+0000","last_unstale":"2024-09-17T05:08:08.773884+0000","last_undegraded":"2024-09-17T05:08:08.773884+0000","last_fullsized":"2024-09-17T05:08:08.773884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:35.769440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317597+0000","last_change":"2024-09-17T05:08:07.784239+0000","last_active":"2024-09-17T05:08:09.317597+0000","last_peered":"2024-09-17T05:08:09.317597+0000","last_clean":"2024-09-17T05:08:09.317597+0000","last_became_active":"2024-09-17T05:08:07.783925+0000","last_became_peered":"2024-09-17T05:08:07.783925+0000","last_unstale":"2024-09-17T05:08:09.317597+0000","last_undegraded":"2024-09-17T05:08:09.317597+0000","last_fullsized":"2024-09-17T05:08:09.317597+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:22.931083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"113'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774112+0000","last_change":"2024-09-17T05:08:06.763961+0000","last_active":"2024-09-17T05:08:08.774112+0000","last_peered":"2024-09-17T05:08:08.774112+0000","last_clean":"2024-09-17T05:08:08.774112+0000","last_became_active":"2024-09-17T05:08:06.763384+0000","last_became_peered":"2024-09-17T05:08:06.763384+0000","last_unstale":"2024-09-17T05:08:08.774112+0000","last_undegraded":"2024-09-17T05:08:08.774112+0000","last_fullsized":"2024-09-17T05:08:08.774112+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:28.930783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772913+0000","last_change":"2024-09-17T05:08:07.785279+0000","last_active":"2024-09-17T05:08:08.772913+0000","last_peered":"2024-09-17T05:08:08.772913+0000","last_clean":"2024-09-17T05:08:08.772913+0000","last_became_active":"2024-09-17T05:08:07.784850+0000","last_became_peered":"2024-09-17T05:08:07.784850+0000","last_unstale":"2024-09-17T05:08:08.772913+0000","last_undegraded":"2024-09-17T05:08:08.772913+0000","last_fullsized":"2024-09-17T05:08:08.772913+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:32.584088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773844+0000","last_change":"2024-09-17T05:08:06.758429+0000","last_active":"2024-09-17T05:08:08.773844+0000","last_peered":"2024-09-17T05:08:08.773844+0000","last_clean":"2024-09-17T05:08:08.773844+0000","last_became_active":"2024-09-17T05:08:06.758102+0000","last_became_peered":"2024-09-17T05:08:06.758102+0000","last_unstale":"2024-09-17T05:08:08.773844+0000","last_undegraded":"2024-09-17T05:08:08.773844+0000","last_fullsized":"2024-09-17T05:08:08.773844+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:26.345504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317209+0000","last_change":"2024-09-17T05:08:07.780337+0000","last_active":"2024-09-17T05:08:09.317209+0000","last_peered":"2024-09-17T05:08:09.317209+0000","last_clean":"2024-09-17T05:08:09.317209+0000","last_became_active":"2024-09-17T05:08:07.779384+0000","last_became_peered":"2024-09-17T05:08:07.779384+0000","last_unstale":"2024-09-17T05:08:09.317209+0000","last_undegraded":"2024-09-17T05:08:09.317209+0000","last_fullsized":"2024-09-17T05:08:09.317209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:39.355491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776504+0000","last_change":"2024-09-17T05:08:06.754339+0000","last_active":"2024-09-17T05:08:08.776504+0000","last_peered":"2024-09-17T05:08:08.776504+0000","last_clean":"2024-09-17T05:08:08.776504+0000","last_became_active":"2024-09-17T05:08:06.753263+0000","last_became_peered":"2024-09-17T05:08:06.753263+0000","last_unstale":"2024-09-17T05:08:08.776504+0000","last_undegraded":"2024-09-17T05:08:08.776504+0000","last_fullsized":"2024-09-17T05:08:08.776504+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:05.105191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775741+0000","last_change":"2024-09-17T05:08:07.784770+0000","last_active":"2024-09-17T05:08:08.775741+0000","last_peered":"2024-09-17T05:08:08.775741+0000","last_clean":"2024-09-17T05:08:08.775741+0000","last_became_active":"2024-09-17T05:08:07.784404+0000","last_became_peered":"2024-09-17T05:08:07.784404+0000","last_unstale":"2024-09-17T05:08:08.775741+0000","last_undegraded":"2024-09-17T05:08:08.775741+0000","last_fullsized":"2024-09-17T05:08:08.775741+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.024816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773667+0000","last_change":"2024-09-17T05:08:06.750182+0000","last_active":"2024-09-17T05:08:08.773667+0000","last_peered":"2024-09-17T05:08:08.773667+0000","last_clean":"2024-09-17T05:08:08.773667+0000","last_became_active":"2024-09-17T05:08:06.749803+0000","last_became_peered":"2024-09-17T05:08:06.749803+0000","last_unstale":"2024-09-17T05:08:08.773667+0000","last_undegraded":"2024-09-17T05:08:08.773667+0000","last_fullsized":"2024-09-17T05:08:08.773667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:11.666395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773077+0000","last_change":"2024-09-17T05:08:07.765395+0000","last_active":"2024-09-17T05:08:08.773077+0000","last_peered":"2024-09-17T05:08:08.773077+0000","last_clean":"2024-09-17T05:08:08.773077+0000","last_became_active":"2024-09-17T05:08:07.764733+0000","last_became_peered":"2024-09-17T05:08:07.764733+0000","last_unstale":"2024-09-17T05:08:08.773077+0000","last_undegraded":"2024-09-17T05:08:08.773077+0000","last_fullsized":"2024-09-17T05:08:08.773077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:38.846186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318638+0000","last_change":"2024-09-17T05:08:06.760241+0000","last_active":"2024-09-17T05:08:09.318638+0000","last_peered":"2024-09-17T05:08:09.318638+0000","last_clean":"2024-09-17T05:08:09.318638+0000","last_became_active":"2024-09-17T05:08:06.759577+0000","last_became_peered":"2024-09-17T05:08:06.759577+0000","last_unstale":"2024-09-17T05:08:09.318638+0000","last_undegraded":"2024-09-17T05:08:09.318638+0000","last_fullsized":"2024-09-17T05:08:09.318638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:36.445025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772493+0000","last_change":"2024-09-17T05:08:07.779539+0000","last_active":"2024-09-17T05:08:08.772493+0000","last_peered":"2024-09-17T05:08:08.772493+0000","last_clean":"2024-09-17T05:08:08.772493+0000","last_became_active":"2024-09-17T05:08:07.779173+0000","last_became_peered":"2024-09-17T05:08:07.779173+0000","last_unstale":"2024-09-17T05:08:08.772493+0000","last_undegraded":"2024-09-17T05:08:08.772493+0000","last_fullsized":"2024-09-17T05:08:08.772493+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:39.173719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773378+0000","last_change":"2024-09-17T05:08:06.756365+0000","last_active":"2024-09-17T05:08:08.773378+0000","last_peered":"2024-09-17T05:08:08.773378+0000","last_clean":"2024-09-17T05:08:08.773378+0000","last_became_active":"2024-09-17T05:08:06.756054+0000","last_became_peered":"2024-09-17T05:08:06.756054+0000","last_unstale":"2024-09-17T05:08:08.773378+0000","last_undegraded":"2024-09-17T05:08:08.773378+0000","last_fullsized":"2024-09-17T05:08:08.773378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.455026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318035+0000","last_change":"2024-09-17T05:08:07.776281+0000","last_active":"2024-09-17T05:08:09.318035+0000","last_peered":"2024-09-17T05:08:09.318035+0000","last_clean":"2024-09-17T05:08:09.318035+0000","last_became_active":"2024-09-17T05:08:07.775971+0000","last_became_peered":"2024-09-17T05:08:07.775971+0000","last_unstale":"2024-09-17T05:08:09.318035+0000","last_undegraded":"2024-09-17T05:08:09.318035+0000","last_fullsized":"2024-09-17T05:08:09.318035+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:36.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772699+0000","last_change":"2024-09-17T05:08:06.756541+0000","last_active":"2024-09-17T05:08:08.772699+0000","last_peered":"2024-09-17T05:08:08.772699+0000","last_clean":"2024-09-17T05:08:08.772699+0000","last_became_active":"2024-09-17T05:08:06.756230+0000","last_became_peered":"2024-09-17T05:08:06.756230+0000","last_unstale":"2024-09-17T05:08:08.772699+0000","last_undegraded":"2024-09-17T05:08:08.772699+0000","last_fullsized":"2024-09-17T05:08:08.772699+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:34.204673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772926+0000","last_change":"2024-09-17T05:08:07.778659+0000","last_active":"2024-09-17T05:08:08.772926+0000","last_peered":"2024-09-17T05:08:08.772926+0000","last_clean":"2024-09-17T05:08:08.772926+0000","last_became_active":"2024-09-17T05:08:07.778270+0000","last_became_peered":"2024-09-17T05:08:07.778270+0000","last_unstale":"2024-09-17T05:08:08.772926+0000","last_undegraded":"2024-09-17T05:08:08.772926+0000","last_fullsized":"2024-09-17T05:08:08.772926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:58.124955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"112'8","reported_seq":23,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317787+0000","last_change":"2024-09-17T05:08:06.756186+0000","last_active":"2024-09-17T05:08:09.317787+0000","last_peered":"2024-09-17T05:08:09.317787+0000","last_clean":"2024-09-17T05:08:09.317787+0000","last_became_active":"2024-09-17T05:08:06.755300+0000","last_became_peered":"2024-09-17T05:08:06.755300+0000","last_unstale":"2024-09-17T05:08:09.317787+0000","last_undegraded":"2024-09-17T05:08:09.317787+0000","last_fullsized":"2024-09-17T05:08:09.317787+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:03.637415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":288,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317684+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:09.317684+0000","last_peered":"2024-09-17T05:08:09.317684+0000","last_clean":"2024-09-17T05:08:09.317684+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:09.317684+0000","last_undegraded":"2024-09-17T05:08:09.317684+0000","last_fullsized":"2024-09-17T05:08:09.317684+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":645943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":645965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":645999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1404928,"data_stored":646033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509529,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":648137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37312,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334528,"statfs":{"total":96636764160,"available":96598556672,"internally_reserved":0,"allocated":2002944,"data_stored":1236471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":646629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1237923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:08:16.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e38a00640 1 -- 172.21.15.16:0/3250215581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e18004da0 msgr2=0x7f1e18005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e38a00640 1 --2- 172.21.15.16:0/3250215581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e18004da0 0x7f1e18005230 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1e34072d60 tx=0x7f1e2c01d4f0 comp rx=0 tx=0).stop 2024-09-17T05:08:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e32800640 1 -- 172.21.15.16:0/3250215581 reap_dead start 2024-09-17T05:08:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e38a00640 1 -- 172.21.15.16:0/3250215581 shutdown_connections 2024-09-17T05:08:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e38a00640 1 -- 172.21.15.16:0/3250215581 >> 172.21.15.16:0/3250215581 conn(0x7f1e3406f900 msgr2=0x7f1e340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e38a00640 1 -- 172.21.15.16:0/3250215581 shutdown_connections 2024-09-17T05:08:16.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.818+0000 7f1e38a00640 1 -- 172.21.15.16:0/3250215581 wait complete. 2024-09-17T05:08:16.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 Processor -- start 2024-09-17T05:08:16.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 -- start start 2024-09-17T05:08:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e18004da0 0x7f1e341556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e34145940 0x7f1e34145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 0x7f1e340aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34073ce0 con 0x7f1e340ae690 2024-09-17T05:08:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34073b60 con 0x7f1e34145940 2024-09-17T05:08:16.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e38a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1e34073e60 con 0x7f1e18004da0 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 0x7f1e340aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 0x7f1e340aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34156/0 (socket says 172.21.15.16:34156) 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 -- 172.21.15.16:0/515352149 learned_addr learned my addr 172.21.15.16:0/515352149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e31e00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e18004da0 0x7f1e341556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 -- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e18004da0 msgr2=0x7f1e341556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e18004da0 0x7f1e341556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 -- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e34145940 msgr2=0x7f1e34145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e29400640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e34145940 0x7f1e34145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e34145940 0x7f1e34145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 -- 172.21.15.16:0/515352149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e34154600 con 0x7f1e340ae690 2024-09-17T05:08:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e31e00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e18004da0 0x7f1e341556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e29400640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e34145940 0x7f1e34145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e32800640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 0x7f1e340aeaa0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c01cf60 tx=0x7f1e2c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24001480 con 0x7f1e340ae690 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e2c0aa4d0 con 0x7f1e340ae690 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.822+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c009200 con 0x7f1e340ae690 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e341548e0 con 0x7f1e340ae690 2024-09-17T05:08:16.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e34154ea0 con 0x7f1e340ae690 2024-09-17T05:08:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e340743c0 con 0x7f1e340ae690 2024-09-17T05:08:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e2c003b80 con 0x7f1e340ae690 2024-09-17T05:08:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e2aa00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e10073200 0x7f1e10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f1e2c12f460 con 0x7f1e340ae690 2024-09-17T05:08:16.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.826+0000 7f1e31e00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e10073200 0x7f1e10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:16.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.830+0000 7f1e31e00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e10073200 0x7f1e10075710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006fa0 tx=0x7f1e24006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:16.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:16.830+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e2c0ae020 con 0x7f1e340ae690 2024-09-17T05:08:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.002+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e3406e4b0 con 0x7f1e10073200 2024-09-17T05:08:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.010+0000 7f1e2aa00640 1 -- 172.21.15.16:0/515352149 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f1e3406e4b0 con 0x7f1e10073200 2024-09-17T05:08:17.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:08:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e10073200 msgr2=0x7f1e10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e10073200 0x7f1e10075710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006fa0 tx=0x7f1e24006b20 comp rx=0 tx=0).stop 2024-09-17T05:08:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 msgr2=0x7f1e340aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 0x7f1e340aeaa0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c01cf60 tx=0x7f1e2c009480 comp rx=0 tx=0).stop 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 shutdown_connections 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e10073200 0x7f1e10075710 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e340ae690 0x7f1e340aeaa0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e34145940 0x7f1e34145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 --2- 172.21.15.16:0/515352149 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e18004da0 0x7f1e341556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 >> 172.21.15.16:0/515352149 conn(0x7f1e3406f900 msgr2=0x7f1e340cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 shutdown_connections 2024-09-17T05:08:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.014+0000 7f1e38a00640 1 -- 172.21.15.16:0/515352149 wait complete. 2024-09-17T05:08:17.034 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-17T05:08:15.579603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270008,"kb_used_data":12168,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704712,"statfs":{"total":773094113280,"available":772817625088,"internally_reserved":0,"allocated":12460032,"data_stored":6353100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002268"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318641+0000","last_change":"2024-09-17T05:08:07.795883+0000","last_active":"2024-09-17T05:08:09.318641+0000","last_peered":"2024-09-17T05:08:09.318641+0000","last_clean":"2024-09-17T05:08:09.318641+0000","last_became_active":"2024-09-17T05:08:07.794893+0000","last_became_peered":"2024-09-17T05:08:07.794893+0000","last_unstale":"2024-09-17T05:08:09.318641+0000","last_undegraded":"2024-09-17T05:08:09.318641+0000","last_fullsized":"2024-09-17T05:08:09.318641+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:36.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774172+0000","last_change":"2024-09-17T05:08:07.761565+0000","last_active":"2024-09-17T05:08:08.774172+0000","last_peered":"2024-09-17T05:08:08.774172+0000","last_clean":"2024-09-17T05:08:08.774172+0000","last_became_active":"2024-09-17T05:08:07.761261+0000","last_became_peered":"2024-09-17T05:08:07.761261+0000","last_unstale":"2024-09-17T05:08:08.774172+0000","last_undegraded":"2024-09-17T05:08:08.774172+0000","last_fullsized":"2024-09-17T05:08:08.774172+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:41.539621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317532+0000","last_change":"2024-09-17T05:08:07.776242+0000","last_active":"2024-09-17T05:08:09.317532+0000","last_peered":"2024-09-17T05:08:09.317532+0000","last_clean":"2024-09-17T05:08:09.317532+0000","last_became_active":"2024-09-17T05:08:07.775861+0000","last_became_peered":"2024-09-17T05:08:07.775861+0000","last_unstale":"2024-09-17T05:08:09.317532+0000","last_undegraded":"2024-09-17T05:08:09.317532+0000","last_fullsized":"2024-09-17T05:08:09.317532+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:35.212289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772138+0000","last_change":"2024-09-17T05:08:07.781599+0000","last_active":"2024-09-17T05:08:08.772138+0000","last_peered":"2024-09-17T05:08:08.772138+0000","last_clean":"2024-09-17T05:08:08.772138+0000","last_became_active":"2024-09-17T05:08:07.781146+0000","last_became_peered":"2024-09-17T05:08:07.781146+0000","last_unstale":"2024-09-17T05:08:08.772138+0000","last_undegraded":"2024-09-17T05:08:08.772138+0000","last_fullsized":"2024-09-17T05:08:08.772138+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:01.034082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773258+0000","last_change":"2024-09-17T05:08:07.792531+0000","last_active":"2024-09-17T05:08:08.773258+0000","last_peered":"2024-09-17T05:08:08.773258+0000","last_clean":"2024-09-17T05:08:08.773258+0000","last_became_active":"2024-09-17T05:08:07.792187+0000","last_became_peered":"2024-09-17T05:08:07.792187+0000","last_unstale":"2024-09-17T05:08:08.773258+0000","last_undegraded":"2024-09-17T05:08:08.773258+0000","last_fullsized":"2024-09-17T05:08:08.773258+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:43.484137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773432+0000","last_change":"2024-09-17T05:08:07.758709+0000","last_active":"2024-09-17T05:08:08.773432+0000","last_peered":"2024-09-17T05:08:08.773432+0000","last_clean":"2024-09-17T05:08:08.773432+0000","last_became_active":"2024-09-17T05:08:07.758382+0000","last_became_peered":"2024-09-17T05:08:07.758382+0000","last_unstale":"2024-09-17T05:08:08.773432+0000","last_undegraded":"2024-09-17T05:08:08.773432+0000","last_fullsized":"2024-09-17T05:08:08.773432+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:43:11.382948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773005+0000","last_change":"2024-09-17T05:08:07.787949+0000","last_active":"2024-09-17T05:08:08.773005+0000","last_peered":"2024-09-17T05:08:08.773005+0000","last_clean":"2024-09-17T05:08:08.773005+0000","last_became_active":"2024-09-17T05:08:07.787577+0000","last_became_peered":"2024-09-17T05:08:07.787577+0000","last_unstale":"2024-09-17T05:08:08.773005+0000","last_undegraded":"2024-09-17T05:08:08.773005+0000","last_fullsized":"2024-09-17T05:08:08.773005+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:33.927797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317528+0000","last_change":"2024-09-17T05:08:07.779739+0000","last_active":"2024-09-17T05:08:09.317528+0000","last_peered":"2024-09-17T05:08:09.317528+0000","last_clean":"2024-09-17T05:08:09.317528+0000","last_became_active":"2024-09-17T05:08:07.779423+0000","last_became_peered":"2024-09-17T05:08:07.779423+0000","last_unstale":"2024-09-17T05:08:09.317528+0000","last_undegraded":"2024-09-17T05:08:09.317528+0000","last_fullsized":"2024-09-17T05:08:09.317528+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:11.762316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772836+0000","last_change":"2024-09-17T05:08:07.779961+0000","last_active":"2024-09-17T05:08:08.772836+0000","last_peered":"2024-09-17T05:08:08.772836+0000","last_clean":"2024-09-17T05:08:08.772836+0000","last_became_active":"2024-09-17T05:08:07.779640+0000","last_became_peered":"2024-09-17T05:08:07.779640+0000","last_unstale":"2024-09-17T05:08:08.772836+0000","last_undegraded":"2024-09-17T05:08:08.772836+0000","last_fullsized":"2024-09-17T05:08:08.772836+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:13.274700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774347+0000","last_change":"2024-09-17T05:08:07.777536+0000","last_active":"2024-09-17T05:08:08.774347+0000","last_peered":"2024-09-17T05:08:08.774347+0000","last_clean":"2024-09-17T05:08:08.774347+0000","last_became_active":"2024-09-17T05:08:07.777182+0000","last_became_peered":"2024-09-17T05:08:07.777182+0000","last_unstale":"2024-09-17T05:08:08.774347+0000","last_undegraded":"2024-09-17T05:08:08.774347+0000","last_fullsized":"2024-09-17T05:08:08.774347+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:26.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318781+0000","last_change":"2024-09-17T05:08:07.771467+0000","last_active":"2024-09-17T05:08:09.318781+0000","last_peered":"2024-09-17T05:08:09.318781+0000","last_clean":"2024-09-17T05:08:09.318781+0000","last_became_active":"2024-09-17T05:08:07.771043+0000","last_became_peered":"2024-09-17T05:08:07.771043+0000","last_unstale":"2024-09-17T05:08:09.318781+0000","last_undegraded":"2024-09-17T05:08:09.318781+0000","last_fullsized":"2024-09-17T05:08:09.318781+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:58:28.962461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775918+0000","last_change":"2024-09-17T05:08:07.789334+0000","last_active":"2024-09-17T05:08:08.775918+0000","last_peered":"2024-09-17T05:08:08.775918+0000","last_clean":"2024-09-17T05:08:08.775918+0000","last_became_active":"2024-09-17T05:08:07.788735+0000","last_became_peered":"2024-09-17T05:08:07.788735+0000","last_unstale":"2024-09-17T05:08:08.775918+0000","last_undegraded":"2024-09-17T05:08:08.775918+0000","last_fullsized":"2024-09-17T05:08:08.775918+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:18.764696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773816+0000","last_change":"2024-09-17T05:08:07.776167+0000","last_active":"2024-09-17T05:08:08.773816+0000","last_peered":"2024-09-17T05:08:08.773816+0000","last_clean":"2024-09-17T05:08:08.773816+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:08.773816+0000","last_undegraded":"2024-09-17T05:08:08.773816+0000","last_fullsized":"2024-09-17T05:08:08.773816+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:33.590100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775993+0000","last_change":"2024-09-17T05:08:07.771910+0000","last_active":"2024-09-17T05:08:08.775993+0000","last_peered":"2024-09-17T05:08:08.775993+0000","last_clean":"2024-09-17T05:08:08.775993+0000","last_became_active":"2024-09-17T05:08:07.771593+0000","last_became_peered":"2024-09-17T05:08:07.771593+0000","last_unstale":"2024-09-17T05:08:08.775993+0000","last_undegraded":"2024-09-17T05:08:08.775993+0000","last_fullsized":"2024-09-17T05:08:08.775993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:13.168344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317771+0000","last_change":"2024-09-17T05:08:07.785079+0000","last_active":"2024-09-17T05:08:09.317771+0000","last_peered":"2024-09-17T05:08:09.317771+0000","last_clean":"2024-09-17T05:08:09.317771+0000","last_became_active":"2024-09-17T05:08:07.784658+0000","last_became_peered":"2024-09-17T05:08:07.784658+0000","last_unstale":"2024-09-17T05:08:09.317771+0000","last_undegraded":"2024-09-17T05:08:09.317771+0000","last_fullsized":"2024-09-17T05:08:09.317771+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:30.065188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775598+0000","last_change":"2024-09-17T05:08:07.774835+0000","last_active":"2024-09-17T05:08:08.775598+0000","last_peered":"2024-09-17T05:08:08.775598+0000","last_clean":"2024-09-17T05:08:08.775598+0000","last_became_active":"2024-09-17T05:08:07.772572+0000","last_became_peered":"2024-09-17T05:08:07.772572+0000","last_unstale":"2024-09-17T05:08:08.775598+0000","last_undegraded":"2024-09-17T05:08:08.775598+0000","last_fullsized":"2024-09-17T05:08:08.775598+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:21.713633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776280+0000","last_change":"2024-09-17T05:08:07.793244+0000","last_active":"2024-09-17T05:08:08.776280+0000","last_peered":"2024-09-17T05:08:08.776280+0000","last_clean":"2024-09-17T05:08:08.776280+0000","last_became_active":"2024-09-17T05:08:07.792918+0000","last_became_peered":"2024-09-17T05:08:07.792918+0000","last_unstale":"2024-09-17T05:08:08.776280+0000","last_undegraded":"2024-09-17T05:08:08.776280+0000","last_fullsized":"2024-09-17T05:08:08.776280+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:07.709317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776210+0000","last_change":"2024-09-17T05:08:07.786589+0000","last_active":"2024-09-17T05:08:08.776210+0000","last_peered":"2024-09-17T05:08:08.776210+0000","last_clean":"2024-09-17T05:08:08.776210+0000","last_became_active":"2024-09-17T05:08:07.786229+0000","last_became_peered":"2024-09-17T05:08:07.786229+0000","last_unstale":"2024-09-17T05:08:08.776210+0000","last_undegraded":"2024-09-17T05:08:08.776210+0000","last_fullsized":"2024-09-17T05:08:08.776210+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:05.350778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773450+0000","last_change":"2024-09-17T05:08:07.778835+0000","last_active":"2024-09-17T05:08:08.773450+0000","last_peered":"2024-09-17T05:08:08.773450+0000","last_clean":"2024-09-17T05:08:08.773450+0000","last_became_active":"2024-09-17T05:08:07.777881+0000","last_became_peered":"2024-09-17T05:08:07.777881+0000","last_unstale":"2024-09-17T05:08:08.773450+0000","last_undegraded":"2024-09-17T05:08:08.773450+0000","last_fullsized":"2024-09-17T05:08:08.773450+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:36.803182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773372+0000","last_change":"2024-09-17T05:08:07.774967+0000","last_active":"2024-09-17T05:08:08.773372+0000","last_peered":"2024-09-17T05:08:08.773372+0000","last_clean":"2024-09-17T05:08:08.773372+0000","last_became_active":"2024-09-17T05:08:07.774314+0000","last_became_peered":"2024-09-17T05:08:07.774314+0000","last_unstale":"2024-09-17T05:08:08.773372+0000","last_undegraded":"2024-09-17T05:08:08.773372+0000","last_fullsized":"2024-09-17T05:08:08.773372+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:57.062162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773506+0000","last_change":"2024-09-17T05:08:07.788520+0000","last_active":"2024-09-17T05:08:08.773506+0000","last_peered":"2024-09-17T05:08:08.773506+0000","last_clean":"2024-09-17T05:08:08.773506+0000","last_became_active":"2024-09-17T05:08:07.788178+0000","last_became_peered":"2024-09-17T05:08:07.788178+0000","last_unstale":"2024-09-17T05:08:08.773506+0000","last_undegraded":"2024-09-17T05:08:08.773506+0000","last_fullsized":"2024-09-17T05:08:08.773506+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:30:30.065188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318265+0000","last_change":"2024-09-17T05:08:07.771912+0000","last_active":"2024-09-17T05:08:09.318265+0000","last_peered":"2024-09-17T05:08:09.318265+0000","last_clean":"2024-09-17T05:08:09.318265+0000","last_became_active":"2024-09-17T05:08:07.771506+0000","last_became_peered":"2024-09-17T05:08:07.771506+0000","last_unstale":"2024-09-17T05:08:09.318265+0000","last_undegraded":"2024-09-17T05:08:09.318265+0000","last_fullsized":"2024-09-17T05:08:09.318265+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:19.548866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317923+0000","last_change":"2024-09-17T05:08:07.785226+0000","last_active":"2024-09-17T05:08:09.317923+0000","last_peered":"2024-09-17T05:08:09.317923+0000","last_clean":"2024-09-17T05:08:09.317923+0000","last_became_active":"2024-09-17T05:08:07.784890+0000","last_became_peered":"2024-09-17T05:08:07.784890+0000","last_unstale":"2024-09-17T05:08:09.317923+0000","last_undegraded":"2024-09-17T05:08:09.317923+0000","last_fullsized":"2024-09-17T05:08:09.317923+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:50.691059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317761+0000","last_change":"2024-09-17T05:08:07.773890+0000","last_active":"2024-09-17T05:08:09.317761+0000","last_peered":"2024-09-17T05:08:09.317761+0000","last_clean":"2024-09-17T05:08:09.317761+0000","last_became_active":"2024-09-17T05:08:07.772818+0000","last_became_peered":"2024-09-17T05:08:07.772818+0000","last_unstale":"2024-09-17T05:08:09.317761+0000","last_undegraded":"2024-09-17T05:08:09.317761+0000","last_fullsized":"2024-09-17T05:08:09.317761+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:52.945082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318921+0000","last_change":"2024-09-17T05:08:07.790671+0000","last_active":"2024-09-17T05:08:09.318921+0000","last_peered":"2024-09-17T05:08:09.318921+0000","last_clean":"2024-09-17T05:08:09.318921+0000","last_became_active":"2024-09-17T05:08:07.790295+0000","last_became_peered":"2024-09-17T05:08:07.790295+0000","last_unstale":"2024-09-17T05:08:09.318921+0000","last_undegraded":"2024-09-17T05:08:09.318921+0000","last_fullsized":"2024-09-17T05:08:09.318921+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:29:53.330797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318428+0000","last_change":"2024-09-17T05:08:07.763546+0000","last_active":"2024-09-17T05:08:09.318428+0000","last_peered":"2024-09-17T05:08:09.318428+0000","last_clean":"2024-09-17T05:08:09.318428+0000","last_became_active":"2024-09-17T05:08:07.761760+0000","last_became_peered":"2024-09-17T05:08:07.761760+0000","last_unstale":"2024-09-17T05:08:09.318428+0000","last_undegraded":"2024-09-17T05:08:09.318428+0000","last_fullsized":"2024-09-17T05:08:09.318428+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:12.953196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773736+0000","last_change":"2024-09-17T05:08:07.793784+0000","last_active":"2024-09-17T05:08:08.773736+0000","last_peered":"2024-09-17T05:08:08.773736+0000","last_clean":"2024-09-17T05:08:08.773736+0000","last_became_active":"2024-09-17T05:08:07.792348+0000","last_became_peered":"2024-09-17T05:08:07.792348+0000","last_unstale":"2024-09-17T05:08:08.773736+0000","last_undegraded":"2024-09-17T05:08:08.773736+0000","last_fullsized":"2024-09-17T05:08:08.773736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:56.337697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776000+0000","last_change":"2024-09-17T05:08:07.789530+0000","last_active":"2024-09-17T05:08:08.776000+0000","last_peered":"2024-09-17T05:08:08.776000+0000","last_clean":"2024-09-17T05:08:08.776000+0000","last_became_active":"2024-09-17T05:08:07.788907+0000","last_became_peered":"2024-09-17T05:08:07.788907+0000","last_unstale":"2024-09-17T05:08:08.776000+0000","last_undegraded":"2024-09-17T05:08:08.776000+0000","last_fullsized":"2024-09-17T05:08:08.776000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:09.724198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317742+0000","last_change":"2024-09-17T05:08:07.775236+0000","last_active":"2024-09-17T05:08:09.317742+0000","last_peered":"2024-09-17T05:08:09.317742+0000","last_clean":"2024-09-17T05:08:09.317742+0000","last_became_active":"2024-09-17T05:08:07.774745+0000","last_became_peered":"2024-09-17T05:08:07.774745+0000","last_unstale":"2024-09-17T05:08:09.317742+0000","last_undegraded":"2024-09-17T05:08:09.317742+0000","last_fullsized":"2024-09-17T05:08:09.317742+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:22:55.455059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317109+0000","last_change":"2024-09-17T05:08:07.767001+0000","last_active":"2024-09-17T05:08:09.317109+0000","last_peered":"2024-09-17T05:08:09.317109+0000","last_clean":"2024-09-17T05:08:09.317109+0000","last_became_active":"2024-09-17T05:08:07.766612+0000","last_became_peered":"2024-09-17T05:08:07.766612+0000","last_unstale":"2024-09-17T05:08:09.317109+0000","last_undegraded":"2024-09-17T05:08:09.317109+0000","last_fullsized":"2024-09-17T05:08:09.317109+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:51.502611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317812+0000","last_change":"2024-09-17T05:08:07.778092+0000","last_active":"2024-09-17T05:08:09.317812+0000","last_peered":"2024-09-17T05:08:09.317812+0000","last_clean":"2024-09-17T05:08:09.317812+0000","last_became_active":"2024-09-17T05:08:07.777764+0000","last_became_peered":"2024-09-17T05:08:07.777764+0000","last_unstale":"2024-09-17T05:08:09.317812+0000","last_undegraded":"2024-09-17T05:08:09.317812+0000","last_fullsized":"2024-09-17T05:08:09.317812+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:26.543253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317888+0000","last_change":"2024-09-17T05:08:07.764573+0000","last_active":"2024-09-17T05:08:09.317888+0000","last_peered":"2024-09-17T05:08:09.317888+0000","last_clean":"2024-09-17T05:08:09.317888+0000","last_became_active":"2024-09-17T05:08:07.764177+0000","last_became_peered":"2024-09-17T05:08:07.764177+0000","last_unstale":"2024-09-17T05:08:09.317888+0000","last_undegraded":"2024-09-17T05:08:09.317888+0000","last_fullsized":"2024-09-17T05:08:09.317888+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:19:42.857705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318244+0000","last_change":"2024-09-17T05:08:07.768737+0000","last_active":"2024-09-17T05:08:09.318244+0000","last_peered":"2024-09-17T05:08:09.318244+0000","last_clean":"2024-09-17T05:08:09.318244+0000","last_became_active":"2024-09-17T05:08:07.768436+0000","last_became_peered":"2024-09-17T05:08:07.768436+0000","last_unstale":"2024-09-17T05:08:09.318244+0000","last_undegraded":"2024-09-17T05:08:09.318244+0000","last_fullsized":"2024-09-17T05:08:09.318244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:38.344394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"112'4","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774490+0000","last_change":"2024-09-17T05:08:06.757535+0000","last_active":"2024-09-17T05:08:08.774490+0000","last_peered":"2024-09-17T05:08:08.774490+0000","last_clean":"2024-09-17T05:08:08.774490+0000","last_became_active":"2024-09-17T05:08:06.757229+0000","last_became_peered":"2024-09-17T05:08:06.757229+0000","last_unstale":"2024-09-17T05:08:08.774490+0000","last_undegraded":"2024-09-17T05:08:08.774490+0000","last_fullsized":"2024-09-17T05:08:08.774490+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:12.881506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773515+0000","last_change":"2024-09-17T05:08:07.767395+0000","last_active":"2024-09-17T05:08:08.773515+0000","last_peered":"2024-09-17T05:08:08.773515+0000","last_clean":"2024-09-17T05:08:08.773515+0000","last_became_active":"2024-09-17T05:08:07.767090+0000","last_became_peered":"2024-09-17T05:08:07.767090+0000","last_unstale":"2024-09-17T05:08:08.773515+0000","last_undegraded":"2024-09-17T05:08:08.773515+0000","last_fullsized":"2024-09-17T05:08:08.773515+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:54.194851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776442+0000","last_change":"2024-09-17T05:08:06.758493+0000","last_active":"2024-09-17T05:08:08.776442+0000","last_peered":"2024-09-17T05:08:08.776442+0000","last_clean":"2024-09-17T05:08:08.776442+0000","last_became_active":"2024-09-17T05:08:06.758182+0000","last_became_peered":"2024-09-17T05:08:06.758182+0000","last_unstale":"2024-09-17T05:08:08.776442+0000","last_undegraded":"2024-09-17T05:08:08.776442+0000","last_fullsized":"2024-09-17T05:08:08.776442+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:41:27.873341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318494+0000","last_change":"2024-09-17T05:08:07.789474+0000","last_active":"2024-09-17T05:08:09.318494+0000","last_peered":"2024-09-17T05:08:09.318494+0000","last_clean":"2024-09-17T05:08:09.318494+0000","last_became_active":"2024-09-17T05:08:07.789118+0000","last_became_peered":"2024-09-17T05:08:07.789118+0000","last_unstale":"2024-09-17T05:08:09.318494+0000","last_undegraded":"2024-09-17T05:08:09.318494+0000","last_fullsized":"2024-09-17T05:08:09.318494+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:10.564947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318627+0000","last_change":"2024-09-17T05:08:06.761067+0000","last_active":"2024-09-17T05:08:09.318627+0000","last_peered":"2024-09-17T05:08:09.318627+0000","last_clean":"2024-09-17T05:08:09.318627+0000","last_became_active":"2024-09-17T05:08:06.760761+0000","last_became_peered":"2024-09-17T05:08:06.760761+0000","last_unstale":"2024-09-17T05:08:09.318627+0000","last_undegraded":"2024-09-17T05:08:09.318627+0000","last_fullsized":"2024-09-17T05:08:09.318627+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:47.048472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773108+0000","last_change":"2024-09-17T05:08:07.779226+0000","last_active":"2024-09-17T05:08:08.773108+0000","last_peered":"2024-09-17T05:08:08.773108+0000","last_clean":"2024-09-17T05:08:08.773108+0000","last_became_active":"2024-09-17T05:08:07.778909+0000","last_became_peered":"2024-09-17T05:08:07.778909+0000","last_unstale":"2024-09-17T05:08:08.773108+0000","last_undegraded":"2024-09-17T05:08:08.773108+0000","last_fullsized":"2024-09-17T05:08:08.773108+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:40:41.342754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773564+0000","last_change":"2024-09-17T05:08:06.761480+0000","last_active":"2024-09-17T05:08:08.773564+0000","last_peered":"2024-09-17T05:08:08.773564+0000","last_clean":"2024-09-17T05:08:08.773564+0000","last_became_active":"2024-09-17T05:08:06.761119+0000","last_became_peered":"2024-09-17T05:08:06.761119+0000","last_unstale":"2024-09-17T05:08:08.773564+0000","last_undegraded":"2024-09-17T05:08:08.773564+0000","last_fullsized":"2024-09-17T05:08:08.773564+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:38.181798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317805+0000","last_change":"2024-09-17T05:08:07.768471+0000","last_active":"2024-09-17T05:08:09.317805+0000","last_peered":"2024-09-17T05:08:09.317805+0000","last_clean":"2024-09-17T05:08:09.317805+0000","last_became_active":"2024-09-17T05:08:07.767987+0000","last_became_peered":"2024-09-17T05:08:07.767987+0000","last_unstale":"2024-09-17T05:08:09.317805+0000","last_undegraded":"2024-09-17T05:08:09.317805+0000","last_fullsized":"2024-09-17T05:08:09.317805+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:46:23.625287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318461+0000","last_change":"2024-09-17T05:08:06.749915+0000","last_active":"2024-09-17T05:08:09.318461+0000","last_peered":"2024-09-17T05:08:09.318461+0000","last_clean":"2024-09-17T05:08:09.318461+0000","last_became_active":"2024-09-17T05:08:06.749590+0000","last_became_peered":"2024-09-17T05:08:06.749590+0000","last_unstale":"2024-09-17T05:08:09.318461+0000","last_undegraded":"2024-09-17T05:08:09.318461+0000","last_fullsized":"2024-09-17T05:08:09.318461+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:32.056062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318104+0000","last_change":"2024-09-17T05:08:07.792210+0000","last_active":"2024-09-17T05:08:09.318104+0000","last_peered":"2024-09-17T05:08:09.318104+0000","last_clean":"2024-09-17T05:08:09.318104+0000","last_became_active":"2024-09-17T05:08:07.789880+0000","last_became_peered":"2024-09-17T05:08:07.789880+0000","last_unstale":"2024-09-17T05:08:09.318104+0000","last_undegraded":"2024-09-17T05:08:09.318104+0000","last_fullsized":"2024-09-17T05:08:09.318104+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:57.288839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317710+0000","last_change":"2024-09-17T05:08:06.761320+0000","last_active":"2024-09-17T05:08:09.317710+0000","last_peered":"2024-09-17T05:08:09.317710+0000","last_clean":"2024-09-17T05:08:09.317710+0000","last_became_active":"2024-09-17T05:08:06.760045+0000","last_became_peered":"2024-09-17T05:08:06.760045+0000","last_unstale":"2024-09-17T05:08:09.317710+0000","last_undegraded":"2024-09-17T05:08:09.317710+0000","last_fullsized":"2024-09-17T05:08:09.317710+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:21:49.710369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317004+0000","last_change":"2024-09-17T05:08:07.776123+0000","last_active":"2024-09-17T05:08:09.317004+0000","last_peered":"2024-09-17T05:08:09.317004+0000","last_clean":"2024-09-17T05:08:09.317004+0000","last_became_active":"2024-09-17T05:08:07.775800+0000","last_became_peered":"2024-09-17T05:08:07.775800+0000","last_unstale":"2024-09-17T05:08:09.317004+0000","last_undegraded":"2024-09-17T05:08:09.317004+0000","last_fullsized":"2024-09-17T05:08:09.317004+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:47.421496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318035+0000","last_change":"2024-09-17T05:08:06.762962+0000","last_active":"2024-09-17T05:08:09.318035+0000","last_peered":"2024-09-17T05:08:09.318035+0000","last_clean":"2024-09-17T05:08:09.318035+0000","last_became_active":"2024-09-17T05:08:06.762619+0000","last_became_peered":"2024-09-17T05:08:06.762619+0000","last_unstale":"2024-09-17T05:08:09.318035+0000","last_undegraded":"2024-09-17T05:08:09.318035+0000","last_fullsized":"2024-09-17T05:08:09.318035+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:29.175367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318035+0000","last_change":"2024-09-17T05:08:07.762347+0000","last_active":"2024-09-17T05:08:09.318035+0000","last_peered":"2024-09-17T05:08:09.318035+0000","last_clean":"2024-09-17T05:08:09.318035+0000","last_became_active":"2024-09-17T05:08:07.760607+0000","last_became_peered":"2024-09-17T05:08:07.760607+0000","last_unstale":"2024-09-17T05:08:09.318035+0000","last_undegraded":"2024-09-17T05:08:09.318035+0000","last_fullsized":"2024-09-17T05:08:09.318035+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:16:33.331797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.316840+0000","last_change":"2024-09-17T05:08:06.757901+0000","last_active":"2024-09-17T05:08:09.316840+0000","last_peered":"2024-09-17T05:08:09.316840+0000","last_clean":"2024-09-17T05:08:09.316840+0000","last_became_active":"2024-09-17T05:08:06.757576+0000","last_became_peered":"2024-09-17T05:08:06.757576+0000","last_unstale":"2024-09-17T05:08:09.316840+0000","last_undegraded":"2024-09-17T05:08:09.316840+0000","last_fullsized":"2024-09-17T05:08:09.316840+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:16:14.133127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318368+0000","last_change":"2024-09-17T05:08:07.790571+0000","last_active":"2024-09-17T05:08:09.318368+0000","last_peered":"2024-09-17T05:08:09.318368+0000","last_clean":"2024-09-17T05:08:09.318368+0000","last_became_active":"2024-09-17T05:08:07.790207+0000","last_became_peered":"2024-09-17T05:08:07.790207+0000","last_unstale":"2024-09-17T05:08:09.318368+0000","last_undegraded":"2024-09-17T05:08:09.318368+0000","last_fullsized":"2024-09-17T05:08:09.318368+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:20:01.034082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773202+0000","last_change":"2024-09-17T05:08:06.756508+0000","last_active":"2024-09-17T05:08:08.773202+0000","last_peered":"2024-09-17T05:08:08.773202+0000","last_clean":"2024-09-17T05:08:08.773202+0000","last_became_active":"2024-09-17T05:08:06.756117+0000","last_became_peered":"2024-09-17T05:08:06.756117+0000","last_unstale":"2024-09-17T05:08:08.773202+0000","last_undegraded":"2024-09-17T05:08:08.773202+0000","last_fullsized":"2024-09-17T05:08:08.773202+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:09.073307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772446+0000","last_change":"2024-09-17T05:08:07.781110+0000","last_active":"2024-09-17T05:08:08.772446+0000","last_peered":"2024-09-17T05:08:08.772446+0000","last_clean":"2024-09-17T05:08:08.772446+0000","last_became_active":"2024-09-17T05:08:07.780749+0000","last_became_peered":"2024-09-17T05:08:07.780749+0000","last_unstale":"2024-09-17T05:08:08.772446+0000","last_undegraded":"2024-09-17T05:08:08.772446+0000","last_fullsized":"2024-09-17T05:08:08.772446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:19.548866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773001+0000","last_change":"2024-09-17T05:08:06.762648+0000","last_active":"2024-09-17T05:08:08.773001+0000","last_peered":"2024-09-17T05:08:08.773001+0000","last_clean":"2024-09-17T05:08:08.773001+0000","last_became_active":"2024-09-17T05:08:06.762351+0000","last_became_peered":"2024-09-17T05:08:06.762351+0000","last_unstale":"2024-09-17T05:08:08.773001+0000","last_undegraded":"2024-09-17T05:08:08.773001+0000","last_fullsized":"2024-09-17T05:08:08.773001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:14:29.526777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317076+0000","last_change":"2024-09-17T05:08:07.771518+0000","last_active":"2024-09-17T05:08:09.317076+0000","last_peered":"2024-09-17T05:08:09.317076+0000","last_clean":"2024-09-17T05:08:09.317076+0000","last_became_active":"2024-09-17T05:08:07.771149+0000","last_became_peered":"2024-09-17T05:08:07.771149+0000","last_unstale":"2024-09-17T05:08:09.317076+0000","last_undegraded":"2024-09-17T05:08:09.317076+0000","last_fullsized":"2024-09-17T05:08:09.317076+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:07.285307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"112'3","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.319004+0000","last_change":"2024-09-17T05:08:06.756469+0000","last_active":"2024-09-17T05:08:09.319004+0000","last_peered":"2024-09-17T05:08:09.319004+0000","last_clean":"2024-09-17T05:08:09.319004+0000","last_became_active":"2024-09-17T05:08:06.756119+0000","last_became_peered":"2024-09-17T05:08:06.756119+0000","last_unstale":"2024-09-17T05:08:09.319004+0000","last_undegraded":"2024-09-17T05:08:09.319004+0000","last_fullsized":"2024-09-17T05:08:09.319004+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:27.860204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774274+0000","last_change":"2024-09-17T05:08:07.796237+0000","last_active":"2024-09-17T05:08:08.774274+0000","last_peered":"2024-09-17T05:08:08.774274+0000","last_clean":"2024-09-17T05:08:08.774274+0000","last_became_active":"2024-09-17T05:08:07.793673+0000","last_became_peered":"2024-09-17T05:08:07.793673+0000","last_unstale":"2024-09-17T05:08:08.774274+0000","last_undegraded":"2024-09-17T05:08:08.774274+0000","last_fullsized":"2024-09-17T05:08:08.774274+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:55.087320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"112'6","reported_seq":21,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773530+0000","last_change":"2024-09-17T05:08:06.751740+0000","last_active":"2024-09-17T05:08:08.773530+0000","last_peered":"2024-09-17T05:08:08.773530+0000","last_clean":"2024-09-17T05:08:08.773530+0000","last_became_active":"2024-09-17T05:08:06.751425+0000","last_became_peered":"2024-09-17T05:08:06.751425+0000","last_unstale":"2024-09-17T05:08:08.773530+0000","last_undegraded":"2024-09-17T05:08:08.773530+0000","last_fullsized":"2024-09-17T05:08:08.773530+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:19.025078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772650+0000","last_change":"2024-09-17T05:08:07.775856+0000","last_active":"2024-09-17T05:08:08.772650+0000","last_peered":"2024-09-17T05:08:08.772650+0000","last_clean":"2024-09-17T05:08:08.772650+0000","last_became_active":"2024-09-17T05:08:07.775439+0000","last_became_peered":"2024-09-17T05:08:07.775439+0000","last_unstale":"2024-09-17T05:08:08.772650+0000","last_undegraded":"2024-09-17T05:08:08.772650+0000","last_fullsized":"2024-09-17T05:08:08.772650+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:21.426193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318290+0000","last_change":"2024-09-17T05:08:06.755780+0000","last_active":"2024-09-17T05:08:09.318290+0000","last_peered":"2024-09-17T05:08:09.318290+0000","last_clean":"2024-09-17T05:08:09.318290+0000","last_became_active":"2024-09-17T05:08:06.754229+0000","last_became_peered":"2024-09-17T05:08:06.754229+0000","last_unstale":"2024-09-17T05:08:09.318290+0000","last_undegraded":"2024-09-17T05:08:09.318290+0000","last_fullsized":"2024-09-17T05:08:09.318290+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:11.209610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773646+0000","last_change":"2024-09-17T05:08:07.765055+0000","last_active":"2024-09-17T05:08:08.773646+0000","last_peered":"2024-09-17T05:08:08.773646+0000","last_clean":"2024-09-17T05:08:08.773646+0000","last_became_active":"2024-09-17T05:08:07.764271+0000","last_became_peered":"2024-09-17T05:08:07.764271+0000","last_unstale":"2024-09-17T05:08:08.773646+0000","last_undegraded":"2024-09-17T05:08:08.773646+0000","last_fullsized":"2024-09-17T05:08:08.773646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.819206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317451+0000","last_change":"2024-09-17T05:08:06.754758+0000","last_active":"2024-09-17T05:08:09.317451+0000","last_peered":"2024-09-17T05:08:09.317451+0000","last_clean":"2024-09-17T05:08:09.317451+0000","last_became_active":"2024-09-17T05:08:06.754461+0000","last_became_peered":"2024-09-17T05:08:06.754461+0000","last_unstale":"2024-09-17T05:08:09.317451+0000","last_undegraded":"2024-09-17T05:08:09.317451+0000","last_fullsized":"2024-09-17T05:08:09.317451+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:42.829403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776079+0000","last_change":"2024-09-17T05:08:07.791789+0000","last_active":"2024-09-17T05:08:08.776079+0000","last_peered":"2024-09-17T05:08:08.776079+0000","last_clean":"2024-09-17T05:08:08.776079+0000","last_became_active":"2024-09-17T05:08:07.791512+0000","last_became_peered":"2024-09-17T05:08:07.791512+0000","last_unstale":"2024-09-17T05:08:08.776079+0000","last_undegraded":"2024-09-17T05:08:08.776079+0000","last_fullsized":"2024-09-17T05:08:08.776079+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:38.297384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317179+0000","last_change":"2024-09-17T05:08:06.756328+0000","last_active":"2024-09-17T05:08:09.317179+0000","last_peered":"2024-09-17T05:08:09.317179+0000","last_clean":"2024-09-17T05:08:09.317179+0000","last_became_active":"2024-09-17T05:08:06.756035+0000","last_became_peered":"2024-09-17T05:08:06.756035+0000","last_unstale":"2024-09-17T05:08:09.317179+0000","last_undegraded":"2024-09-17T05:08:09.317179+0000","last_fullsized":"2024-09-17T05:08:09.317179+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T17:03:42.859744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772307+0000","last_change":"2024-09-17T05:08:07.781526+0000","last_active":"2024-09-17T05:08:08.772307+0000","last_peered":"2024-09-17T05:08:08.772307+0000","last_clean":"2024-09-17T05:08:08.772307+0000","last_became_active":"2024-09-17T05:08:07.780689+0000","last_became_peered":"2024-09-17T05:08:07.780689+0000","last_unstale":"2024-09-17T05:08:08.772307+0000","last_undegraded":"2024-09-17T05:08:08.772307+0000","last_fullsized":"2024-09-17T05:08:08.772307+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:50.691059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776292+0000","last_change":"2024-09-17T05:08:06.760008+0000","last_active":"2024-09-17T05:08:08.776292+0000","last_peered":"2024-09-17T05:08:08.776292+0000","last_clean":"2024-09-17T05:08:08.776292+0000","last_became_active":"2024-09-17T05:08:06.759701+0000","last_became_peered":"2024-09-17T05:08:06.759701+0000","last_unstale":"2024-09-17T05:08:08.776292+0000","last_undegraded":"2024-09-17T05:08:08.776292+0000","last_fullsized":"2024-09-17T05:08:08.776292+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:00.123242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318783+0000","last_change":"2024-09-17T05:08:07.763579+0000","last_active":"2024-09-17T05:08:09.318783+0000","last_peered":"2024-09-17T05:08:09.318783+0000","last_clean":"2024-09-17T05:08:09.318783+0000","last_became_active":"2024-09-17T05:08:07.763140+0000","last_became_peered":"2024-09-17T05:08:07.763140+0000","last_unstale":"2024-09-17T05:08:09.318783+0000","last_undegraded":"2024-09-17T05:08:09.318783+0000","last_fullsized":"2024-09-17T05:08:09.318783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:07:38.344394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776486+0000","last_change":"2024-09-17T05:08:06.763335+0000","last_active":"2024-09-17T05:08:08.776486+0000","last_peered":"2024-09-17T05:08:08.776486+0000","last_clean":"2024-09-17T05:08:08.776486+0000","last_became_active":"2024-09-17T05:08:06.762302+0000","last_became_peered":"2024-09-17T05:08:06.762302+0000","last_unstale":"2024-09-17T05:08:08.776486+0000","last_undegraded":"2024-09-17T05:08:08.776486+0000","last_fullsized":"2024-09-17T05:08:08.776486+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:25.339809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773938+0000","last_change":"2024-09-17T05:08:07.775183+0000","last_active":"2024-09-17T05:08:08.773938+0000","last_peered":"2024-09-17T05:08:08.773938+0000","last_clean":"2024-09-17T05:08:08.773938+0000","last_became_active":"2024-09-17T05:08:07.774107+0000","last_became_peered":"2024-09-17T05:08:07.774107+0000","last_unstale":"2024-09-17T05:08:08.773938+0000","last_undegraded":"2024-09-17T05:08:08.773938+0000","last_fullsized":"2024-09-17T05:08:08.773938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:50.882298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"112'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773728+0000","last_change":"2024-09-17T05:08:06.756451+0000","last_active":"2024-09-17T05:08:08.773728+0000","last_peered":"2024-09-17T05:08:08.773728+0000","last_clean":"2024-09-17T05:08:08.773728+0000","last_became_active":"2024-09-17T05:08:06.756108+0000","last_became_peered":"2024-09-17T05:08:06.756108+0000","last_unstale":"2024-09-17T05:08:08.773728+0000","last_undegraded":"2024-09-17T05:08:08.773728+0000","last_fullsized":"2024-09-17T05:08:08.773728+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:07.818594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774345+0000","last_change":"2024-09-17T05:08:07.801223+0000","last_active":"2024-09-17T05:08:08.774345+0000","last_peered":"2024-09-17T05:08:08.774345+0000","last_clean":"2024-09-17T05:08:08.774345+0000","last_became_active":"2024-09-17T05:08:07.800861+0000","last_became_peered":"2024-09-17T05:08:07.800861+0000","last_unstale":"2024-09-17T05:08:08.774345+0000","last_undegraded":"2024-09-17T05:08:08.774345+0000","last_fullsized":"2024-09-17T05:08:08.774345+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:06.112808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318721+0000","last_change":"2024-09-17T05:08:06.759565+0000","last_active":"2024-09-17T05:08:09.318721+0000","last_peered":"2024-09-17T05:08:09.318721+0000","last_clean":"2024-09-17T05:08:09.318721+0000","last_became_active":"2024-09-17T05:08:06.759251+0000","last_became_peered":"2024-09-17T05:08:06.759251+0000","last_unstale":"2024-09-17T05:08:09.318721+0000","last_undegraded":"2024-09-17T05:08:09.318721+0000","last_fullsized":"2024-09-17T05:08:09.318721+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:09:34.624245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773295+0000","last_change":"2024-09-17T05:08:07.766627+0000","last_active":"2024-09-17T05:08:08.773295+0000","last_peered":"2024-09-17T05:08:08.773295+0000","last_clean":"2024-09-17T05:08:08.773295+0000","last_became_active":"2024-09-17T05:08:07.766265+0000","last_became_peered":"2024-09-17T05:08:07.766265+0000","last_unstale":"2024-09-17T05:08:08.773295+0000","last_undegraded":"2024-09-17T05:08:08.773295+0000","last_fullsized":"2024-09-17T05:08:08.773295+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:48:16.381763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775691+0000","last_change":"2024-09-17T05:08:06.758648+0000","last_active":"2024-09-17T05:08:08.775691+0000","last_peered":"2024-09-17T05:08:08.775691+0000","last_clean":"2024-09-17T05:08:08.775691+0000","last_became_active":"2024-09-17T05:08:06.756522+0000","last_became_peered":"2024-09-17T05:08:06.756522+0000","last_unstale":"2024-09-17T05:08:08.775691+0000","last_undegraded":"2024-09-17T05:08:08.775691+0000","last_fullsized":"2024-09-17T05:08:08.775691+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:54.079704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775621+0000","last_change":"2024-09-17T05:08:07.793164+0000","last_active":"2024-09-17T05:08:08.775621+0000","last_peered":"2024-09-17T05:08:08.775621+0000","last_clean":"2024-09-17T05:08:08.775621+0000","last_became_active":"2024-09-17T05:08:07.792897+0000","last_became_peered":"2024-09-17T05:08:07.792897+0000","last_unstale":"2024-09-17T05:08:08.775621+0000","last_undegraded":"2024-09-17T05:08:08.775621+0000","last_fullsized":"2024-09-17T05:08:08.775621+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:12.831669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318459+0000","last_change":"2024-09-17T05:08:06.759894+0000","last_active":"2024-09-17T05:08:09.318459+0000","last_peered":"2024-09-17T05:08:09.318459+0000","last_clean":"2024-09-17T05:08:09.318459+0000","last_became_active":"2024-09-17T05:08:06.759190+0000","last_became_peered":"2024-09-17T05:08:06.759190+0000","last_unstale":"2024-09-17T05:08:09.318459+0000","last_undegraded":"2024-09-17T05:08:09.318459+0000","last_fullsized":"2024-09-17T05:08:09.318459+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:20.714922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317310+0000","last_change":"2024-09-17T05:08:07.782682+0000","last_active":"2024-09-17T05:08:09.317310+0000","last_peered":"2024-09-17T05:08:09.317310+0000","last_clean":"2024-09-17T05:08:09.317310+0000","last_became_active":"2024-09-17T05:08:07.782259+0000","last_became_peered":"2024-09-17T05:08:07.782259+0000","last_unstale":"2024-09-17T05:08:09.317310+0000","last_undegraded":"2024-09-17T05:08:09.317310+0000","last_fullsized":"2024-09-17T05:08:09.317310+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:49.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.316859+0000","last_change":"2024-09-17T05:08:06.757737+0000","last_active":"2024-09-17T05:08:09.316859+0000","last_peered":"2024-09-17T05:08:09.316859+0000","last_clean":"2024-09-17T05:08:09.316859+0000","last_became_active":"2024-09-17T05:08:06.757394+0000","last_became_peered":"2024-09-17T05:08:06.757394+0000","last_unstale":"2024-09-17T05:08:09.316859+0000","last_undegraded":"2024-09-17T05:08:09.316859+0000","last_fullsized":"2024-09-17T05:08:09.316859+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:02.184148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318285+0000","last_change":"2024-09-17T05:08:07.765739+0000","last_active":"2024-09-17T05:08:09.318285+0000","last_peered":"2024-09-17T05:08:09.318285+0000","last_clean":"2024-09-17T05:08:09.318285+0000","last_became_active":"2024-09-17T05:08:07.765426+0000","last_became_peered":"2024-09-17T05:08:07.765426+0000","last_unstale":"2024-09-17T05:08:09.318285+0000","last_undegraded":"2024-09-17T05:08:09.318285+0000","last_fullsized":"2024-09-17T05:08:09.318285+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:09.876043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772589+0000","last_change":"2024-09-17T05:08:06.752439+0000","last_active":"2024-09-17T05:08:08.772589+0000","last_peered":"2024-09-17T05:08:08.772589+0000","last_clean":"2024-09-17T05:08:08.772589+0000","last_became_active":"2024-09-17T05:08:06.752134+0000","last_became_peered":"2024-09-17T05:08:06.752134+0000","last_unstale":"2024-09-17T05:08:08.772589+0000","last_undegraded":"2024-09-17T05:08:08.772589+0000","last_fullsized":"2024-09-17T05:08:08.772589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:48.197920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776212+0000","last_change":"2024-09-17T05:08:07.786689+0000","last_active":"2024-09-17T05:08:08.776212+0000","last_peered":"2024-09-17T05:08:08.776212+0000","last_clean":"2024-09-17T05:08:08.776212+0000","last_became_active":"2024-09-17T05:08:07.786275+0000","last_became_peered":"2024-09-17T05:08:07.786275+0000","last_unstale":"2024-09-17T05:08:08.776212+0000","last_undegraded":"2024-09-17T05:08:08.776212+0000","last_fullsized":"2024-09-17T05:08:08.776212+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:38:51.289187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"112'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773884+0000","last_change":"2024-09-17T05:08:06.758852+0000","last_active":"2024-09-17T05:08:08.773884+0000","last_peered":"2024-09-17T05:08:08.773884+0000","last_clean":"2024-09-17T05:08:08.773884+0000","last_became_active":"2024-09-17T05:08:06.757319+0000","last_became_peered":"2024-09-17T05:08:06.757319+0000","last_unstale":"2024-09-17T05:08:08.773884+0000","last_undegraded":"2024-09-17T05:08:08.773884+0000","last_fullsized":"2024-09-17T05:08:08.773884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:26:35.769440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317597+0000","last_change":"2024-09-17T05:08:07.784239+0000","last_active":"2024-09-17T05:08:09.317597+0000","last_peered":"2024-09-17T05:08:09.317597+0000","last_clean":"2024-09-17T05:08:09.317597+0000","last_became_active":"2024-09-17T05:08:07.783925+0000","last_became_peered":"2024-09-17T05:08:07.783925+0000","last_unstale":"2024-09-17T05:08:09.317597+0000","last_undegraded":"2024-09-17T05:08:09.317597+0000","last_fullsized":"2024-09-17T05:08:09.317597+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:22.931083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"113'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.774112+0000","last_change":"2024-09-17T05:08:06.763961+0000","last_active":"2024-09-17T05:08:08.774112+0000","last_peered":"2024-09-17T05:08:08.774112+0000","last_clean":"2024-09-17T05:08:08.774112+0000","last_became_active":"2024-09-17T05:08:06.763384+0000","last_became_peered":"2024-09-17T05:08:06.763384+0000","last_unstale":"2024-09-17T05:08:08.774112+0000","last_undegraded":"2024-09-17T05:08:08.774112+0000","last_fullsized":"2024-09-17T05:08:08.774112+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:09:28.930783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772913+0000","last_change":"2024-09-17T05:08:07.785279+0000","last_active":"2024-09-17T05:08:08.772913+0000","last_peered":"2024-09-17T05:08:08.772913+0000","last_clean":"2024-09-17T05:08:08.772913+0000","last_became_active":"2024-09-17T05:08:07.784850+0000","last_became_peered":"2024-09-17T05:08:07.784850+0000","last_unstale":"2024-09-17T05:08:08.772913+0000","last_undegraded":"2024-09-17T05:08:08.772913+0000","last_fullsized":"2024-09-17T05:08:08.772913+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:32.584088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773844+0000","last_change":"2024-09-17T05:08:06.758429+0000","last_active":"2024-09-17T05:08:08.773844+0000","last_peered":"2024-09-17T05:08:08.773844+0000","last_clean":"2024-09-17T05:08:08.773844+0000","last_became_active":"2024-09-17T05:08:06.758102+0000","last_became_peered":"2024-09-17T05:08:06.758102+0000","last_unstale":"2024-09-17T05:08:08.773844+0000","last_undegraded":"2024-09-17T05:08:08.773844+0000","last_fullsized":"2024-09-17T05:08:08.773844+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:26.345504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317209+0000","last_change":"2024-09-17T05:08:07.780337+0000","last_active":"2024-09-17T05:08:09.317209+0000","last_peered":"2024-09-17T05:08:09.317209+0000","last_clean":"2024-09-17T05:08:09.317209+0000","last_became_active":"2024-09-17T05:08:07.779384+0000","last_became_peered":"2024-09-17T05:08:07.779384+0000","last_unstale":"2024-09-17T05:08:09.317209+0000","last_undegraded":"2024-09-17T05:08:09.317209+0000","last_fullsized":"2024-09-17T05:08:09.317209+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:39.355491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.776504+0000","last_change":"2024-09-17T05:08:06.754339+0000","last_active":"2024-09-17T05:08:08.776504+0000","last_peered":"2024-09-17T05:08:08.776504+0000","last_clean":"2024-09-17T05:08:08.776504+0000","last_became_active":"2024-09-17T05:08:06.753263+0000","last_became_peered":"2024-09-17T05:08:06.753263+0000","last_unstale":"2024-09-17T05:08:08.776504+0000","last_undegraded":"2024-09-17T05:08:08.776504+0000","last_fullsized":"2024-09-17T05:08:08.776504+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:05.105191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.775741+0000","last_change":"2024-09-17T05:08:07.784770+0000","last_active":"2024-09-17T05:08:08.775741+0000","last_peered":"2024-09-17T05:08:08.775741+0000","last_clean":"2024-09-17T05:08:08.775741+0000","last_became_active":"2024-09-17T05:08:07.784404+0000","last_became_peered":"2024-09-17T05:08:07.784404+0000","last_unstale":"2024-09-17T05:08:08.775741+0000","last_undegraded":"2024-09-17T05:08:08.775741+0000","last_fullsized":"2024-09-17T05:08:08.775741+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:45.024816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773667+0000","last_change":"2024-09-17T05:08:06.750182+0000","last_active":"2024-09-17T05:08:08.773667+0000","last_peered":"2024-09-17T05:08:08.773667+0000","last_clean":"2024-09-17T05:08:08.773667+0000","last_became_active":"2024-09-17T05:08:06.749803+0000","last_became_peered":"2024-09-17T05:08:06.749803+0000","last_unstale":"2024-09-17T05:08:08.773667+0000","last_undegraded":"2024-09-17T05:08:08.773667+0000","last_fullsized":"2024-09-17T05:08:08.773667+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:11.666395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773077+0000","last_change":"2024-09-17T05:08:07.765395+0000","last_active":"2024-09-17T05:08:08.773077+0000","last_peered":"2024-09-17T05:08:08.773077+0000","last_clean":"2024-09-17T05:08:08.773077+0000","last_became_active":"2024-09-17T05:08:07.764733+0000","last_became_peered":"2024-09-17T05:08:07.764733+0000","last_unstale":"2024-09-17T05:08:08.773077+0000","last_undegraded":"2024-09-17T05:08:08.773077+0000","last_fullsized":"2024-09-17T05:08:08.773077+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:48:38.846186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318638+0000","last_change":"2024-09-17T05:08:06.760241+0000","last_active":"2024-09-17T05:08:09.318638+0000","last_peered":"2024-09-17T05:08:09.318638+0000","last_clean":"2024-09-17T05:08:09.318638+0000","last_became_active":"2024-09-17T05:08:06.759577+0000","last_became_peered":"2024-09-17T05:08:06.759577+0000","last_unstale":"2024-09-17T05:08:09.318638+0000","last_undegraded":"2024-09-17T05:08:09.318638+0000","last_fullsized":"2024-09-17T05:08:09.318638+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:36.445025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772493+0000","last_change":"2024-09-17T05:08:07.779539+0000","last_active":"2024-09-17T05:08:08.772493+0000","last_peered":"2024-09-17T05:08:08.772493+0000","last_clean":"2024-09-17T05:08:08.772493+0000","last_became_active":"2024-09-17T05:08:07.779173+0000","last_became_peered":"2024-09-17T05:08:07.779173+0000","last_unstale":"2024-09-17T05:08:08.772493+0000","last_undegraded":"2024-09-17T05:08:08.772493+0000","last_fullsized":"2024-09-17T05:08:08.772493+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:23:39.173719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.773378+0000","last_change":"2024-09-17T05:08:06.756365+0000","last_active":"2024-09-17T05:08:08.773378+0000","last_peered":"2024-09-17T05:08:08.773378+0000","last_clean":"2024-09-17T05:08:08.773378+0000","last_became_active":"2024-09-17T05:08:06.756054+0000","last_became_peered":"2024-09-17T05:08:06.756054+0000","last_unstale":"2024-09-17T05:08:08.773378+0000","last_undegraded":"2024-09-17T05:08:08.773378+0000","last_fullsized":"2024-09-17T05:08:08.773378+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.455026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.318035+0000","last_change":"2024-09-17T05:08:07.776281+0000","last_active":"2024-09-17T05:08:09.318035+0000","last_peered":"2024-09-17T05:08:09.318035+0000","last_clean":"2024-09-17T05:08:09.318035+0000","last_became_active":"2024-09-17T05:08:07.775971+0000","last_became_peered":"2024-09-17T05:08:07.775971+0000","last_unstale":"2024-09-17T05:08:09.318035+0000","last_undegraded":"2024-09-17T05:08:09.318035+0000","last_fullsized":"2024-09-17T05:08:09.318035+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:36.179607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772699+0000","last_change":"2024-09-17T05:08:06.756541+0000","last_active":"2024-09-17T05:08:08.772699+0000","last_peered":"2024-09-17T05:08:08.772699+0000","last_clean":"2024-09-17T05:08:08.772699+0000","last_became_active":"2024-09-17T05:08:06.756230+0000","last_became_peered":"2024-09-17T05:08:06.756230+0000","last_unstale":"2024-09-17T05:08:08.772699+0000","last_undegraded":"2024-09-17T05:08:08.772699+0000","last_fullsized":"2024-09-17T05:08:08.772699+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:28:34.204673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":13,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:08.772926+0000","last_change":"2024-09-17T05:08:07.778659+0000","last_active":"2024-09-17T05:08:08.772926+0000","last_peered":"2024-09-17T05:08:08.772926+0000","last_clean":"2024-09-17T05:08:08.772926+0000","last_became_active":"2024-09-17T05:08:07.778270+0000","last_became_peered":"2024-09-17T05:08:07.778270+0000","last_unstale":"2024-09-17T05:08:08.772926+0000","last_undegraded":"2024-09-17T05:08:08.772926+0000","last_fullsized":"2024-09-17T05:08:08.772926+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:06.727230+0000","last_clean_scrub_stamp":"2024-09-17T05:08:06.727230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:58.124955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"112'8","reported_seq":23,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317787+0000","last_change":"2024-09-17T05:08:06.756186+0000","last_active":"2024-09-17T05:08:09.317787+0000","last_peered":"2024-09-17T05:08:09.317787+0000","last_clean":"2024-09-17T05:08:09.317787+0000","last_became_active":"2024-09-17T05:08:06.755300+0000","last_became_peered":"2024-09-17T05:08:06.755300+0000","last_unstale":"2024-09-17T05:08:09.317787+0000","last_undegraded":"2024-09-17T05:08:09.317787+0000","last_fullsized":"2024-09-17T05:08:09.317787+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:05.719614+0000","last_clean_scrub_stamp":"2024-09-17T05:08:05.719614+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:15:03.637415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":288,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-17T05:08:09.317684+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:09.317684+0000","last_peered":"2024-09-17T05:08:09.317684+0000","last_clean":"2024-09-17T05:08:09.317684+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:09.317684+0000","last_undegraded":"2024-09-17T05:08:09.317684+0000","last_fullsized":"2024-09-17T05:08:09.317684+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":645943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":645965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":645999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36724,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335116,"statfs":{"total":96636764160,"available":96599158784,"internally_reserved":0,"allocated":1404928,"data_stored":646033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509529,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1417216,"data_stored":648137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37312,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334528,"statfs":{"total":96636764160,"available":96598556672,"internally_reserved":0,"allocated":2002944,"data_stored":1236471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509531,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1413120,"data_stored":646629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33148,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338692,"statfs":{"total":96636764160,"available":96602820608,"internally_reserved":0,"allocated":2007040,"data_stored":1237923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:08:17.038 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:08:17.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:08:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a7e00640 1 -- 172.21.15.16:0/357756969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6188004810 msgr2=0x7f6188004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a7e00640 1 --2- 172.21.15.16:0/357756969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6188004810 0x7f6188004c20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f61a00b2970 tx=0x7f618c01d1f0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a5e00640 1 -- 172.21.15.16:0/357756969 reap_dead start 2024-09-17T05:08:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a7e00640 1 -- 172.21.15.16:0/357756969 shutdown_connections 2024-09-17T05:08:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a7e00640 1 -- 172.21.15.16:0/357756969 >> 172.21.15.16:0/357756969 conn(0x7f61a006f900 msgr2=0x7f61a00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a7e00640 1 -- 172.21.15.16:0/357756969 shutdown_connections 2024-09-17T05:08:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.242+0000 7f61a7e00640 1 -- 172.21.15.16:0/357756969 wait complete. 2024-09-17T05:08:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 Processor -- start 2024-09-17T05:08:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 -- start start 2024-09-17T05:08:17.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6188004810 0x7f61a006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a006de10 0x7f61a00cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 0x7f61a01516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0166c60 con 0x7f61a00d0250 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61a0166ae0 con 0x7f61a006de10 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a7e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f61a0166de0 con 0x7f6188004810 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 0x7f61a01516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 0x7f61a01516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36670/0 (socket says 172.21.15.16:36670) 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 -- 172.21.15.16:0/2781273902 learned_addr learned my addr 172.21.15.16:0/2781273902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a4a00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a006de10 0x7f61a00cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5400640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6188004810 0x7f61a006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 -- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6188004810 msgr2=0x7f61a006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6188004810 0x7f61a006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 -- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a006de10 msgr2=0x7f61a00cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a006de10 0x7f61a00cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 -- 172.21.15.16:0/2781273902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a0152c80 con 0x7f61a00d0250 2024-09-17T05:08:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.246+0000 7f61a5e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 0x7f61a01516b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f61a0072d40 tx=0x7f619800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198015070 con 0x7f61a00d0250 2024-09-17T05:08:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f619800cab0 con 0x7f61a00d0250 2024-09-17T05:08:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198011460 con 0x7f61a00d0250 2024-09-17T05:08:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a01f9090 con 0x7f61a00d0250 2024-09-17T05:08:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a01f9650 con 0x7f61a00d0250 2024-09-17T05:08:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a00d07b0 con 0x7f61a00d0250 2024-09-17T05:08:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.250+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6198018070 con 0x7f61a00d0250 2024-09-17T05:08:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.254+0000 7f619d600640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6180073200 0x7f6180075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.254+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f61980947f0 con 0x7f61a00d0250 2024-09-17T05:08:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.254+0000 7f61a5400640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6180073200 0x7f6180075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.254+0000 7f61a5400640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6180073200 0x7f6180075710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6194009c20 tx=0x7f61940091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.258+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f619805f700 con 0x7f61a00d0250 2024-09-17T05:08:17.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.494+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f61a015fa30 con 0x7f61a00d0250 2024-09-17T05:08:17.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f619d600640 1 -- 172.21.15.16:0/2781273902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v37) ==== 190+0+147 (secure 0 0 0) 0x7f619805f000 con 0x7f61a00d0250 2024-09-17T05:08:17.497 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:08:17.497 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:08:17.497 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:08:17.497 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:08:17.497 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:08:17.497 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:08:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6180073200 msgr2=0x7f6180075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6180073200 0x7f6180075710 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6194009c20 tx=0x7f61940091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 msgr2=0x7f61a01516b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 0x7f61a01516b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f61a0072d40 tx=0x7f619800ed10 comp rx=0 tx=0).stop 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 shutdown_connections 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6180073200 0x7f6180075710 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00d0250 0x7f61a01516b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a006de10 0x7f61a00cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 --2- 172.21.15.16:0/2781273902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6188004810 0x7f61a006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 >> 172.21.15.16:0/2781273902 conn(0x7f61a006f900 msgr2=0x7f61a015c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:17.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 shutdown_connections 2024-09-17T05:08:17.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.498+0000 7f61a7e00640 1 -- 172.21.15.16:0/2781273902 wait complete. 2024-09-17T05:08:17.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:17.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 -- 172.21.15.16:0/4124050343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47015d600 msgr2=0x7fa4700cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 --2- 172.21.15.16:0/4124050343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47015d600 0x7fa4700cee50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa470072d40 tx=0x7fa46001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 -- 172.21.15.16:0/4124050343 shutdown_connections 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 --2- 172.21.15.16:0/4124050343 >> v2:172.21.15.104:3300/0 conn(0x7fa4701606b0 0x7fa470160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 --2- 172.21.15.16:0/4124050343 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa4700cf390 0x7fa470160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 --2- 172.21.15.16:0/4124050343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa47015d600 0x7fa4700cee50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 -- 172.21.15.16:0/4124050343 >> 172.21.15.16:0/4124050343 conn(0x7fa47006f900 msgr2=0x7fa4700b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 -- 172.21.15.16:0/4124050343 shutdown_connections 2024-09-17T05:08:17.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.678+0000 7fa475800640 1 -- 172.21.15.16:0/4124050343 wait complete. 2024-09-17T05:08:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 Processor -- start 2024-09-17T05:08:17.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 -- start start 2024-09-17T05:08:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4700cf390 0x7fa4701581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa47015d600 0x7fa470155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 0x7fa470128f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700b4320 con 0x7fa4701606b0 2024-09-17T05:08:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4700b41a0 con 0x7fa4700cf390 2024-09-17T05:08:17.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa475800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa4700b44a0 con 0x7fa47015d600 2024-09-17T05:08:17.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 0x7fa470128f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 0x7fa470128f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36690/0 (socket says 172.21.15.16:36690) 2024-09-17T05:08:17.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 -- 172.21.15.16:0/2370367600 learned_addr learned my addr 172.21.15.16:0/2370367600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:17.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46ea00640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4700cf390 0x7fa4701581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa467e00640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa47015d600 0x7fa470155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 -- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa47015d600 msgr2=0x7fa470155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa47015d600 0x7fa470155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 -- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4700cf390 msgr2=0x7fa4701581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4700cf390 0x7fa4701581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.682+0000 7fa46f400640 1 -- 172.21.15.16:0/2370367600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4701562f0 con 0x7fa4701606b0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa46f400640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 0x7fa470128f70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa46800cae0 tx=0x7fa46800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800ae30 con 0x7fa4701606b0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa46800f560 con 0x7fa4701606b0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800f9b0 con 0x7fa4701606b0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4701573e0 con 0x7fa4701606b0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4701579a0 con 0x7fa4701606b0 2024-09-17T05:08:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47013bd40 con 0x7fa4701606b0 2024-09-17T05:08:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa46800fb50 con 0x7fa4701606b0 2024-09-17T05:08:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa466a00640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa44c073200 0x7fa44c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:17.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fa468094dd0 con 0x7fa4701606b0 2024-09-17T05:08:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.686+0000 7fa46ea00640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa44c073200 0x7fa44c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:17.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.690+0000 7fa46ea00640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa44c073200 0x7fa44c075710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4600af000 tx=0x7fa46001b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:17.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.694+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa468014030 con 0x7fa4701606b0 2024-09-17T05:08:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4700cee50 con 0x7fa4701606b0 2024-09-17T05:08:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa466a00640 1 -- 172.21.15.16:0/2370367600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7fa468096020 con 0x7fa4701606b0 2024-09-17T05:08:17.905 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:17.906 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"btime":"2024-09-17T05:08:10:776523+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:10.202627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 75 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa44c073200 msgr2=0x7fa44c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa44c073200 0x7fa44c075710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa4600af000 tx=0x7fa46001b100 comp rx=0 tx=0).stop 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 msgr2=0x7fa470128f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 0x7fa470128f70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa46800cae0 tx=0x7fa46800a250 comp rx=0 tx=0).stop 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 shutdown_connections 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa44c073200 0x7fa44c075710 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4701606b0 0x7fa470128f70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa47015d600 0x7fa470155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 --2- 172.21.15.16:0/2370367600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4700cf390 0x7fa4701581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 >> 172.21.15.16:0/2370367600 conn(0x7fa47006f900 msgr2=0x7fa4700b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 shutdown_connections 2024-09-17T05:08:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:17.906+0000 7fa475800640 1 -- 172.21.15.16:0/2370367600 wait complete. 2024-09-17T05:08:18.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:08:07.160604+0000', 'modified': '2024-09-17T05:08:10.202627+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7332}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7332': {'gid': 7332, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/759404348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 759404348}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 759404348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7332, 'qdb_cluster': [7332]} 2024-09-17T05:08:18.330 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:08:18.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T05:08:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 -- 172.21.15.16:0/3399293729 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 msgr2=0x7f888c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 --2- 172.21.15.16:0/3399293729 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 0x7f888c15efe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f887c009900 tx=0x7f887c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 -- 172.21.15.16:0/3399293729 shutdown_connections 2024-09-17T05:08:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 --2- 172.21.15.16:0/3399293729 >> v2:172.21.15.104:3300/0 conn(0x7f888c15f520 0x7f888c15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 --2- 172.21.15.16:0/3399293729 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 0x7f888c15efe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 -- 172.21.15.16:0/3399293729 >> 172.21.15.16:0/3399293729 conn(0x7f888c0b2250 msgr2=0x7f888c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 -- 172.21.15.16:0/3399293729 shutdown_connections 2024-09-17T05:08:18.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.498+0000 7f8892e00640 1 -- 172.21.15.16:0/3399293729 wait complete. 2024-09-17T05:08:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 Processor -- start 2024-09-17T05:08:18.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 -- start start 2024-09-17T05:08:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 0x7f888c1f9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 0x7f888c154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f888c15f520 0x7f888c165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f888c168f30 con 0x7f888c074e30 2024-09-17T05:08:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f888c168db0 con 0x7f888c15f520 2024-09-17T05:08:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8892e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f888c1690b0 con 0x7f888c074a20 2024-09-17T05:08:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 0x7f888c154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 0x7f888c154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36692/0 (socket says 172.21.15.16:36692) 2024-09-17T05:08:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 -- 172.21.15.16:0/3731381532 learned_addr learned my addr 172.21.15.16:0/3731381532 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8890e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f888c15f520 0x7f888c165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f888be00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 0x7f888c1f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 -- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 msgr2=0x7f888c1f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 0x7f888c1f9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 -- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f888c15f520 msgr2=0x7f888c165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f888c15f520 0x7f888c165820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.502+0000 7f8883e00640 1 -- 172.21.15.16:0/3731381532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c154fa0 con 0x7f888c074e30 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8883e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 0x7f888c154a60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f887400c3d0 tx=0x7f887400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8888c00640 1 -- 172.21.15.16:0/3731381532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874017070 con 0x7f888c074e30 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8888c00640 1 -- 172.21.15.16:0/3731381532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88740053e0 con 0x7f888c074e30 2024-09-17T05:08:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8888c00640 1 -- 172.21.15.16:0/3731381532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874003880 con 0x7f888c074e30 2024-09-17T05:08:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c164780 con 0x7f888c074e30 2024-09-17T05:08:18.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c164d40 con 0x7f888c074e30 2024-09-17T05:08:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.506+0000 7f8888c00640 1 -- 172.21.15.16:0/3731381532 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f887401a070 con 0x7f888c074e30 2024-09-17T05:08:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8888c00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8868073200 0x7f8868075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8888c00640 1 -- 172.21.15.16:0/3731381532 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f8874093770 con 0x7f888c074e30 2024-09-17T05:08:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f888be00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8868073200 0x7f8868075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 Processor -- start 2024-09-17T05:08:18.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 -- start start 2024-09-17T05:08:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f888be00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8868073200 0x7f8868075710 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f887c0af000 tx=0x7f887c01b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 0x5617780bfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f57ca0 0x5617780c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561777f94410 con 0x561777f6e950 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561777ed1b40 con 0x561777f57ca0 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x561777ec0f70 con 0x561777b7e460 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 0x5617780bfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 0x5617780bfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36708/0 (socket says 172.21.15.16:36708) 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 -- 172.21.15.16:0/3376902571 learned_addr learned my addr 172.21.15.16:0/3376902571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8890e00640 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f57ca0 0x5617780c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f888be00640 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 -- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 msgr2=0x5617780bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 -- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f57ca0 msgr2=0x5617780c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f57ca0 0x5617780c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.510+0000 7f8883e00640 1 -- 172.21.15.16:0/3376902571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561777e885b0 con 0x561777f6e950 2024-09-17T05:08:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f8883e00640 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 0x5617780bfbe0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f887405b210 tx=0x7f887405d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f568875f6d98e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f8880c00640 1 -- 172.21.15.16:0/3376902571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874067880 con 0x561777f6e950 2024-09-17T05:08:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f8880c00640 1 -- 172.21.15.16:0/3376902571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f887405d200 con 0x561777f6e950 2024-09-17T05:08:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f8880c00640 1 -- 172.21.15.16:0/3376902571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88740598f0 con 0x561777f6e950 2024-09-17T05:08:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 -- 172.21.15.16:0/3376902571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 msgr2=0x5617780bfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 0x5617780bfbe0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f887405b210 tx=0x7f887405d7d0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 -- 172.21.15.16:0/3376902571 shutdown_connections 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f57ca0 0x5617780c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f6e950 0x5617780bfbe0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 --2- 172.21.15.16:0/3376902571 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 -- 172.21.15.16:0/3376902571 >> 172.21.15.16:0/3376902571 conn(0x561778044d90 msgr2=0x561778074690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 -- 172.21.15.16:0/3376902571 shutdown_connections 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 -- 172.21.15.16:0/3376902571 wait complete. 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 -- start start 2024-09-17T05:08:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.514+0000 7f889a657000 1 Processor -- start 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x5617780c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x5617780c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561777e5b5f0 con 0x561777f57ca0 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561777ec4d80 con 0x561777f6e950 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x561777e67520 con 0x561777b7e460 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 10 client.0 _ll_register_callbacks cb 0x561777e6b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x5617780c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x5617780c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36718/0 (socket says 172.21.15.16:36718) 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 -- 172.21.15.16:0/4015864281 learned_addr learned my addr 172.21.15.16:0/4015864281 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f888be00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8890e00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x5617780c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 -- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 msgr2=0x5617780c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 -- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 msgr2=0x5617780c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x5617780c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 -- 172.21.15.16:0/4015864281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561777e5f190 con 0x561777f57ca0 2024-09-17T05:08:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8883e00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x5617780c6e60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8874000fc0 tx=0x7f887405a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25051f59528643ff server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:08:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 1 -- 172.21.15.16:0/4015864281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887405d400 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 1 -- 172.21.15.16:0/4015864281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f887407ac40 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561777e59110 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 1 -- 172.21.15.16:0/4015864281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8874067880 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 10 client.7467 fetch_fsmap learned FSMap version 75 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x561777f45f10 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 1 -- 172.21.15.16:0/4015864281 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x561777e59110 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 1 -- 172.21.15.16:0/4015864281 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f88740598f0 con 0x561777f57ca0 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 10 client.7467 fetch_fsmap finished waiting for FSMap version 75 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 10 client.7467 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 20 client.7467 populate_metadata read hostname 'smithi016' 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b637d0 0x5617780d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 4 client.7467 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:08:18.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 --> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561777ed3a90 con 0x561777b637d0 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f888be00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b637d0 0x5617780d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f888be00640 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b637d0 0x5617780d0d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b1ded4d4732a91 server_cookie=a97e0608fb8714a6 in_seq=0 out_seq=0 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 10 client.7467 ms_handle_connect on v2:172.21.15.16:6834/759404348 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 1 -- 172.21.15.16:0/4015864281 <== mds.0 v2:172.21.15.16:6834/759404348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561777ed3a90 con 0x561777b637d0 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f8879600640 10 client.7467 handle_command_reply: tid=0 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.518+0000 7f889a657000 1 client.7467 shutdown 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 msgr2=0x5617780c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x5617780c6e60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f8874000fc0 tx=0x7f887405a440 comp rx=0 tx=0).stop 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 shutdown_connections 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b637d0 0x5617780d0d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x5617780c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x5617780c6e60 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 --2- 172.21.15.16:0/4015864281 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x5617780c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f8879600640 0 client.7467 ms_handle_reset on v2:172.21.15.16:6834/759404348 2024-09-17T05:08:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 >> 172.21.15.16:0/4015864281 conn(0x561778044d90 msgr2=0x56177807b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 shutdown_connections 2024-09-17T05:08:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 1 -- 172.21.15.16:0/4015864281 wait complete. 2024-09-17T05:08:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.522+0000 7f889a657000 20 client.7467 trim_cache size 0 max 16384 2024-09-17T05:08:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.538+0000 7f889a657000 1 Processor -- start 2024-09-17T05:08:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.538+0000 7f889a657000 1 -- start start 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777b7e460 0x561778139e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777f6e950 0x56177813a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f888c205b40 con 0x561777f57ca0 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f888c204b20 con 0x561777b7e460 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f888c151da0 con 0x561777f6e950 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36728/0 (socket says 172.21.15.16:36728) 2024-09-17T05:08:18.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 -- 172.21.15.16:0/3966070087 learned_addr learned my addr 172.21.15.16:0/3966070087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 -- 172.21.15.16:0/3966070087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777f6e950 msgr2=0x56177813a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777f6e950 0x56177813a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 -- 172.21.15.16:0/3966070087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777b7e460 msgr2=0x561778139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777b7e460 0x561778139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 -- 172.21.15.16:0/3966070087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561777f1cdb0 con 0x561777f57ca0 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f8883e00640 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813a370 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8874067a40 tx=0x7f8874059a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ea3ca888c37f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f887be00640 1 -- 172.21.15.16:0/3966070087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887405d400 con 0x561777f57ca0 2024-09-17T05:08:18.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f887be00640 1 -- 172.21.15.16:0/3966070087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8874063b60 con 0x561777f57ca0 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f887be00640 1 -- 172.21.15.16:0/3966070087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874067ad0 con 0x561777f57ca0 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- 172.21.15.16:0/3966070087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 msgr2=0x56177813a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813a370 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8874067a40 tx=0x7f8874059a80 comp rx=0 tx=0).stop 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- 172.21.15.16:0/3966070087 shutdown_connections 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777f6e950 0x56177813a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813a370 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 --2- 172.21.15.16:0/3966070087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777b7e460 0x561778139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- 172.21.15.16:0/3966070087 >> 172.21.15.16:0/3966070087 conn(0x561778044d90 msgr2=0x5617780bd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- 172.21.15.16:0/3966070087 shutdown_connections 2024-09-17T05:08:18.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- 172.21.15.16:0/3966070087 wait complete. 2024-09-17T05:08:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.542+0000 7f889a657000 1 -- start start 2024-09-17T05:08:18.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 Processor -- start 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x56177813d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x56177813e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561777e67520 con 0x561777f57ca0 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f888c1fd210 con 0x561777f6e950 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x561777e5b5f0 con 0x561777b7e460 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 10 client.0 _ll_register_callbacks cb 0x7f8884009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36742/0 (socket says 172.21.15.16:36742) 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 -- 172.21.15.16:0/2664650945 learned_addr learned my addr 172.21.15.16:0/2664650945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f888be00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x56177813d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8890e00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x56177813e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 -- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 msgr2=0x56177813d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x56177813d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 -- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 msgr2=0x56177813e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x56177813e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 -- 172.21.15.16:0/2664650945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561777e87ab0 con 0x561777f57ca0 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8883e00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813ded0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8874000fc0 tx=0x7f887405d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3975053d7daf9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8879600640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8879600640 1 -- 172.21.15.16:0/2664650945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887405cc70 con 0x561777f57ca0 2024-09-17T05:08:18.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8879600640 1 -- 172.21.15.16:0/2664650945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f8874073cc0 con 0x561777f57ca0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8874063b60 con 0x561777f57ca0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8879600640 1 -- 172.21.15.16:0/2664650945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8874089050 con 0x561777f57ca0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 10 client.7473 fetch_fsmap learned FSMap version 75 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x561777e66070 con 0x561777f57ca0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8879600640 1 -- 172.21.15.16:0/2664650945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8874063b60 con 0x561777f57ca0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f8879600640 1 -- 172.21.15.16:0/2664650945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f887408c020 con 0x561777f57ca0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 10 client.7473 fetch_fsmap finished waiting for FSMap version 75 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 10 client.7473 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 20 client.7473 populate_metadata read hostname 'smithi016' 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b51830 0x5617780f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 4 client.7473 mds_command: new command op to 7332 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 --> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] -- command(tid 0: {"prefix": "status"}) -- 0x561777ecf540 con 0x561777b51830 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.546+0000 7f888be00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b51830 0x5617780f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f888be00640 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b51830 0x5617780f9df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943ae85dfdf1a8c3 server_cookie=4b40989acec47ff2 in_seq=0 out_seq=0 2024-09-17T05:08:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8879600640 10 client.7473 ms_handle_connect on v2:172.21.15.16:6834/759404348 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8879600640 1 -- 172.21.15.16:0/2664650945 <== mds.0 v2:172.21.15.16:6834/759404348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x561777ecf540 con 0x561777b51830 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8879600640 10 client.7473 handle_command_reply: tid=0 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 client.7473 shutdown 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 msgr2=0x56177813ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813ded0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8874000fc0 tx=0x7f887405d430 comp rx=0 tx=0).stop 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 shutdown_connections 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:6834/759404348,v1:172.21.15.16:6835/759404348] conn(0x561777b51830 0x5617780f9df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561777f6e950 0x56177813e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561777f57ca0 0x56177813ded0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 --2- 172.21.15.16:0/2664650945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x561777b7e460 0x56177813d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 >> 172.21.15.16:0/2664650945 conn(0x561778044d90 msgr2=0x5617780d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8879600640 0 client.7473 ms_handle_reset on v2:172.21.15.16:6834/759404348 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 shutdown_connections 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 1 -- 172.21.15.16:0/2664650945 wait complete. 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f889a657000 20 client.7473 trim_cache size 0 max 16384 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7332, 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:08:18.553 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 10.788787085999999, 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 75, 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 113, 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 113, 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 15.198775660000001 2024-09-17T05:08:18.554 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8868073200 msgr2=0x7f8868075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8868073200 0x7f8868075710 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f887c0af000 tx=0x7f887c01b120 comp rx=0 tx=0).stop 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 msgr2=0x7f888c154a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 0x7f888c154a60 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f887400c3d0 tx=0x7f887400c890 comp rx=0 tx=0).stop 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 shutdown_connections 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8868073200 0x7f8868075710 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f888c15f520 0x7f888c165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f888c074e30 0x7f888c154a60 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 --2- 172.21.15.16:0/3731381532 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f888c074a20 0x7f888c1f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 >> 172.21.15.16:0/3731381532 conn(0x7f888c0b2250 msgr2=0x7f888c155800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 shutdown_connections 2024-09-17T05:08:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.550+0000 7f8892e00640 1 -- 172.21.15.16:0/3731381532 wait complete. 2024-09-17T05:08:18.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 -- 172.21.15.16:0/754184277 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780cc6f0 msgr2=0x7f3e780ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/754184277 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780cc6f0 0x7f3e780ccb00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009c80 tx=0x7f3e7001cd10 comp rx=0 tx=0).stop 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 -- 172.21.15.16:0/754184277 shutdown_connections 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/754184277 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780cc6f0 0x7f3e780ccb00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/754184277 >> v2:172.21.15.104:3300/0 conn(0x7f3e780749d0 0x7f3e78152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 -- 172.21.15.16:0/754184277 >> 172.21.15.16:0/754184277 conn(0x7f3e780cdb80 msgr2=0x7f3e780cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 -- 172.21.15.16:0/754184277 shutdown_connections 2024-09-17T05:08:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 -- 172.21.15.16:0/754184277 wait complete. 2024-09-17T05:08:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 Processor -- start 2024-09-17T05:08:18.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.730+0000 7f3e7ec00640 1 -- start start 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 0x7f3e781461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780749d0 0x7f3e781517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e780cc6f0 0x7f3e78150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7815c870 con 0x7f3e7806dfa0 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7815c6f0 con 0x7f3e780cc6f0 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3e78169110 con 0x7f3e780749d0 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 0x7f3e781461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 0x7f3e781461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36744/0 (socket says 172.21.15.16:36744) 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 -- 172.21.15.16:0/1801709964 learned_addr learned my addr 172.21.15.16:0/1801709964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e6f400640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780749d0 0x7f3e781517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7cc00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e780cc6f0 0x7f3e78150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 -- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780749d0 msgr2=0x7f3e781517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780749d0 0x7f3e781517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 -- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e780cc6f0 msgr2=0x7f3e78150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e780cc6f0 0x7f3e78150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 -- 172.21.15.16:0/1801709964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e78154320 con 0x7f3e7806dfa0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e77e00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 0x7f3e781461c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800e7c0 tx=0x7f3e6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800ccc0 con 0x7f3e7806dfa0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e6800ce80 con 0x7f3e7806dfa0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68005280 con 0x7f3e7806dfa0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7814e190 con 0x7f3e7806dfa0 2024-09-17T05:08:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e780b4ed0 con 0x7f3e7806dfa0 2024-09-17T05:08:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.734+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e78152250 con 0x7f3e7806dfa0 2024-09-17T05:08:18.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.738+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3e68005420 con 0x7f3e7806dfa0 2024-09-17T05:08:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.738+0000 7f3e74c00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e54073260 0x7f3e54075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.738+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3e68094c70 con 0x7f3e7806dfa0 2024-09-17T05:08:18.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.738+0000 7f3e6f400640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e54073260 0x7f3e54075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:18.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.738+0000 7f3e6f400640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e54073260 0x7f3e54075770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3e780705c0 tx=0x7f3e60009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:18.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.742+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e6805f530 con 0x7f3e7806dfa0 2024-09-17T05:08:18.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.954+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e780b6920 con 0x7f3e7806dfa0 2024-09-17T05:08:18.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.954+0000 7f3e74c00640 1 -- 172.21.15.16:0/1801709964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7f3e68063f60 con 0x7f3e7806dfa0 2024-09-17T05:08:18.956 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e54073260 msgr2=0x7f3e54075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e54073260 0x7f3e54075770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3e780705c0 tx=0x7f3e60009040 comp rx=0 tx=0).stop 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 msgr2=0x7f3e781461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 0x7f3e781461c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800e7c0 tx=0x7f3e6800ec80 comp rx=0 tx=0).stop 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 shutdown_connections 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3e54073260 0x7f3e54075770 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e780cc6f0 0x7f3e78150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3e780749d0 0x7f3e781517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 --2- 172.21.15.16:0/1801709964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3e7806dfa0 0x7f3e781461c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:18.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 >> 172.21.15.16:0/1801709964 conn(0x7f3e780cdb80 msgr2=0x7f3e7806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 shutdown_connections 2024-09-17T05:08:18.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:18.958+0000 7f3e7ec00640 1 -- 172.21.15.16:0/1801709964 wait complete. 2024-09-17T05:08:18.978 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:08:18.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:08:18.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:08:18.978 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:08:18.978 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:08:18.978 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:08:18.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:08:18.979 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:08:18.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:08:18.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:18.979 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:08:18.989 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:08:18.989 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:08:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41994sec preferred_lft 41994sec 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:08:18.991 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.992 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:08:18.993 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:08:18.993 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:08:18.993 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:18.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:18.993 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:08:19.049 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:08:19.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:19.052 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:08:19.105 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:08:19.106 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:08:19.106 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:08:19.106 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:08:19.106 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:08:19.106 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:08:19.106 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:08:19.106 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:08:19.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:08:19.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:08:19.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:08:19.107 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:08:19.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:08:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:08:19.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:19.244 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:08:19.297 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:08:19.297 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:08:19.297 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:08:19.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:19.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:08:19.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:08:19.364 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:08:19.416 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:08:19.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:08:19.416 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:08:19.471 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:08:19.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:19.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:08:19.573 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:08:19.573 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:08:19.573 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:08:19.591 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:08:19.591 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:08:19.591 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:08:19.591 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:08:19.591 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:08:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:08:19.602 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:08:19.605 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:08:19.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:19.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:08:19.625 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:08:19.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:08:19.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:08:19.732 INFO:teuthology.orchestra.run.smithi016.stdout:client7479 2024-09-17T05:08:19.733 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:19.733 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:19.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 -- 172.21.15.16:0/365045019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 msgr2=0x7f68fc0b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 --2- 172.21.15.16:0/365045019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0b47b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009840 tx=0x7f68e401c820 comp rx=0 tx=0).stop 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 -- 172.21.15.16:0/365045019 shutdown_connections 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 --2- 172.21.15.16:0/365045019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0b47b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 --2- 172.21.15.16:0/365045019 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc06da50 0x7f68fc0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 -- 172.21.15.16:0/365045019 >> 172.21.15.16:0/365045019 conn(0x7f68fc06f900 msgr2=0x7f68fc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 -- 172.21.15.16:0/365045019 shutdown_connections 2024-09-17T05:08:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 -- 172.21.15.16:0/365045019 wait complete. 2024-09-17T05:08:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.946+0000 7f6902c00640 1 Processor -- start 2024-09-17T05:08:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 -- start start 2024-09-17T05:08:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc06da50 0x7f68fc164b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b2f50 0x7f68fc0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc165ed0 con 0x7f68fc06de60 2024-09-17T05:08:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc165d50 con 0x7f68fc06da50 2024-09-17T05:08:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc166050 con 0x7f68fc0b2f50 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36768/0 (socket says 172.21.15.16:36768) 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 -- 172.21.15.16:0/1997956639 learned_addr learned my addr 172.21.15.16:0/1997956639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 -- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b2f50 msgr2=0x7f68fc0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b2f50 0x7f68fc0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 -- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc06da50 msgr2=0x7f68fc164b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc06da50 0x7f68fc164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 -- 172.21.15.16:0/1997956639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc0cb8c0 con 0x7f68fc06de60 2024-09-17T05:08:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f3400640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0c8d40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009790 tx=0x7f68e4005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e40b2040 con 0x7f68fc06de60 2024-09-17T05:08:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e4004320 con 0x7f68fc06de60 2024-09-17T05:08:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e40044f0 con 0x7f68fc06de60 2024-09-17T05:08:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 -- 172.21.15.16:0/1997956639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc0cbba0 con 0x7f68fc06de60 2024-09-17T05:08:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f6902c00640 1 -- 172.21.15.16:0/1997956639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc0c6790 con 0x7f68fc06de60 2024-09-17T05:08:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.950+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc0b3360 con 0x7f68fc06de60 2024-09-17T05:08:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.954+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68e40052d0 con 0x7f68fc06de60 2024-09-17T05:08:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.954+0000 7f68f8c00640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.954+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f68e412de70 con 0x7f68fc06de60 2024-09-17T05:08:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.954+0000 7f68fbe00640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.958+0000 7f68fbe00640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0058d0 tx=0x7f68ec005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:19.958+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f68e4016410 con 0x7f68fc06de60 2024-09-17T05:08:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.170+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f68fc06d7b0 con 0x7f68fc06de60 2024-09-17T05:08:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.170+0000 7f68f8c00640 1 -- 172.21.15.16:0/1997956639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7f68e40f8cf0 con 0x7f68fc06de60 2024-09-17T05:08:20.172 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 msgr2=0x7f68d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f68ec0058d0 tx=0x7f68ec005860 comp rx=0 tx=0).stop 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 msgr2=0x7f68fc0c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0c8d40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009790 tx=0x7f68e4005d20 comp rx=0 tx=0).stop 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 shutdown_connections 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f68d8073200 0x7f68d8075710 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f68fc0b2f50 0x7f68fc0afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:20.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68fc06de60 0x7f68fc0c8d40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:20.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 --2- 172.21.15.16:0/1997956639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f68fc06da50 0x7f68fc164b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:20.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 >> 172.21.15.16:0/1997956639 conn(0x7f68fc06f900 msgr2=0x7f68fc1562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:20.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 shutdown_connections 2024-09-17T05:08:20.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:20.174+0000 7f68f1600640 1 -- 172.21.15.16:0/1997956639 wait complete. 2024-09-17T05:08:20.194 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:08:20.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:08:20.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:08:20.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.196 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:20.206 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 41987sec preferred_lft 41987sec 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.207 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:08:20.208 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:08:20.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.209 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:08:20.274 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:08:20.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.277 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:08:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:08:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:08:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:08:20.336 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:08:20.337 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:08:20.337 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:08:20.337 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:08:20.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:08:20.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:08:20.338 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:08:20.338 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:08:20.430 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:08:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:08:20.472 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:08:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:08:20.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.479 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:08:20.488 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:08:20.488 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:08:20.488 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:08:20.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.489 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:08:20.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:08:20.556 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:08:20.612 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:08:20.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:08:20.612 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:08:20.667 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:08:20.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:08:20.771 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:08:20.772 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:08:20.772 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:08:20.789 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:08:20.789 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:08:20.789 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:08:20.790 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:08:20.790 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:08:20.800 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:08:20.804 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:08:20.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:20.804 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:08:20.825 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:08:20.915 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:08:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:08:20.918 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:08:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:08:20.924 INFO:teuthology.orchestra.run.smithi104.stdout:client7108 2024-09-17T05:08:20.925 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:20.925 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:20.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 -- 172.21.15.16:0/903497596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998415f6f0 msgr2=0x7f99840cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 --2- 172.21.15.16:0/903497596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998415f6f0 0x7f99840cece0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f997c009bc0 tx=0x7f997c01cba0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 -- 172.21.15.16:0/903497596 shutdown_connections 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 --2- 172.21.15.16:0/903497596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f998415f6f0 0x7f99840cece0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 -- 172.21.15.16:0/903497596 >> 172.21.15.16:0/903497596 conn(0x7f99840b2250 msgr2=0x7f99840d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 -- 172.21.15.16:0/903497596 shutdown_connections 2024-09-17T05:08:21.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 -- 172.21.15.16:0/903497596 wait complete. 2024-09-17T05:08:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 Processor -- start 2024-09-17T05:08:21.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.102+0000 7f9989a00640 1 -- start start 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 0x7f99840b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9984074e30 0x7f998415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998415f6f0 0x7f998415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f998416a060 con 0x7f9984074a20 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9984169ee0 con 0x7f998415f6f0 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f998416a1e0 con 0x7f9984074e30 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 0x7f99840b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 0x7f99840b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36792/0 (socket says 172.21.15.16:36792) 2024-09-17T05:08:21.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 -- 172.21.15.16:0/1936565926 learned_addr learned my addr 172.21.15.16:0/1936565926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9983400640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998415f6f0 0x7f998415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f997be00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9984074e30 0x7f998415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 -- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9984074e30 msgr2=0x7f998415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9984074e30 0x7f998415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 -- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998415f6f0 msgr2=0x7f998415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998415f6f0 0x7f998415a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 -- 172.21.15.16:0/1936565926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99841fcfd0 con 0x7f9984074a20 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9983400640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998415f6f0 0x7f998415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9982a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 0x7f99840b4760 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f997400c420 tx=0x7f997400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974017070 con 0x7f9984074a20 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99740053c0 con 0x7f9984074a20 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974003860 con 0x7f9984074a20 2024-09-17T05:08:21.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99841fc3c0 con 0x7f9984074a20 2024-09-17T05:08:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.106+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99841fc980 con 0x7f9984074a20 2024-09-17T05:08:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.110+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998415efe0 con 0x7f9984074a20 2024-09-17T05:08:21.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.110+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f997401a070 con 0x7f9984074a20 2024-09-17T05:08:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.110+0000 7f997aa00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9960073200 0x7f9960075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.110+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f9974093720 con 0x7f9984074a20 2024-09-17T05:08:21.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.110+0000 7f997be00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9960073200 0x7f9960075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.110+0000 7f997be00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9960073200 0x7f9960075710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f996c001180 tx=0x7f996c001110 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:21.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.114+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f997405dec0 con 0x7f9984074a20 2024-09-17T05:08:21.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.302+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f99840c9f50 con 0x7f9984074a20 2024-09-17T05:08:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.306+0000 7f997aa00640 1 -- 172.21.15.16:0/1936565926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v52)=0 v52) ==== 121+0+0 (secure 0 0 0) 0x7f99740628f0 con 0x7f9984074a20 2024-09-17T05:08:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9960073200 msgr2=0x7f9960075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9960073200 0x7f9960075710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f996c001180 tx=0x7f996c001110 comp rx=0 tx=0).stop 2024-09-17T05:08:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 msgr2=0x7f99840b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 0x7f99840b4760 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f997400c420 tx=0x7f997400c8e0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 shutdown_connections 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9960073200 0x7f9960075710 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f998415f6f0 0x7f998415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9984074e30 0x7f998415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 --2- 172.21.15.16:0/1936565926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9984074a20 0x7f99840b4760 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 >> 172.21.15.16:0/1936565926 conn(0x7f99840b2250 msgr2=0x7f99840b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 shutdown_connections 2024-09-17T05:08:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.310+0000 7f9989a00640 1 -- 172.21.15.16:0/1936565926 wait complete. 2024-09-17T05:08:21.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2024-09-17T05:08:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 -- 172.21.15.16:0/4046176785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0073f30 msgr2=0x7f7dc0074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 --2- 172.21.15.16:0/4046176785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0073f30 0x7f7dc0074340 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00b2d10 tx=0x7f7db801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 -- 172.21.15.16:0/4046176785 shutdown_connections 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 --2- 172.21.15.16:0/4046176785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc0073f30 0x7f7dc0074340 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 --2- 172.21.15.16:0/4046176785 >> v2:172.21.15.104:3300/0 conn(0x7f7dc00cadc0 0x7f7dc01590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 -- 172.21.15.16:0/4046176785 >> 172.21.15.16:0/4046176785 conn(0x7f7dc00b2250 msgr2=0x7f7dc00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 -- 172.21.15.16:0/4046176785 shutdown_connections 2024-09-17T05:08:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.490+0000 7f7dc6400640 1 -- 172.21.15.16:0/4046176785 wait complete. 2024-09-17T05:08:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 Processor -- start 2024-09-17T05:08:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 -- start start 2024-09-17T05:08:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dc0073f30 0x7f7dc01583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 0x7f7dc006d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc006d5a0 0x7f7dc0156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0168a20 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc01688a0 con 0x7f7dc006d5a0 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dc6400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc0168ba0 con 0x7f7dc0073f30 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 0x7f7dc006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 0x7f7dc006d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36810/0 (socket says 172.21.15.16:36810) 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 -- 172.21.15.16:0/2192951731 learned_addr learned my addr 172.21.15.16:0/2192951731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc006d5a0 0x7f7dc0156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dc0073f30 0x7f7dc01583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 -- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dc0073f30 msgr2=0x7f7dc01583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dc0073f30 0x7f7dc01583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 -- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc006d5a0 msgr2=0x7f7dc0156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc006d5a0 0x7f7dc0156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7db6a00640 1 -- 172.21.15.16:0/2192951731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc01f9020 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.494+0000 7f7dbfe00640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc006d5a0 0x7f7dc0156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7db6a00640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 0x7f7dc006d060 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7da800d6a0 tx=0x7f7da800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017070 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7da8005310 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8009820 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc01f9300 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc01f98c0 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc0074340 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7da80099c0 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.498+0000 7f7db7e00640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d9c073200 0x7f7d9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.502+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f7da8094990 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.502+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d9c073200 0x7f7d9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.502+0000 7f7dbf400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d9c073200 0x7f7d9c075710 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b810 tx=0x7f7db0008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.502+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7da805f800 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.678+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f7dc00b14e0 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.682+0000 7f7db7e00640 1 -- 172.21.15.16:0/2192951731 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v53)=0 v53) ==== 119+0+0 (secure 0 0 0) 0x7f7da8092e20 con 0x7f7dc00cb8d0 2024-09-17T05:08:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.682+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d9c073200 msgr2=0x7f7d9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.682+0000 7f7dc6400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d9c073200 0x7f7d9c075710 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7db000b810 tx=0x7f7db0008040 comp rx=0 tx=0).stop 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.682+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 msgr2=0x7f7dc006d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.682+0000 7f7dc6400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 0x7f7dc006d060 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7da800d6a0 tx=0x7f7da800db60 comp rx=0 tx=0).stop 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 shutdown_connections 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d9c073200 0x7f7d9c075710 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dc006d5a0 0x7f7dc0156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dc00cb8d0 0x7f7dc006d060 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 --2- 172.21.15.16:0/2192951731 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dc0073f30 0x7f7dc01583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 >> 172.21.15.16:0/2192951731 conn(0x7f7dc00b2250 msgr2=0x7f7dc00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 shutdown_connections 2024-09-17T05:08:21.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.686+0000 7f7dc6400640 1 -- 172.21.15.16:0/2192951731 wait complete. 2024-09-17T05:08:21.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2024-09-17T05:08:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 -- 172.21.15.16:0/97301589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 msgr2=0x7fe324074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 --2- 172.21.15.16:0/97301589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe324074920 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe310009900 tx=0x7fe31001d130 comp rx=0 tx=0).stop 2024-09-17T05:08:21.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 -- 172.21.15.16:0/97301589 shutdown_connections 2024-09-17T05:08:21.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 --2- 172.21.15.16:0/97301589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe324074920 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 --2- 172.21.15.16:0/97301589 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3240ceea0 0x7fe32415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 -- 172.21.15.16:0/97301589 >> 172.21.15.16:0/97301589 conn(0x7fe3240b2250 msgr2=0x7fe3240d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:21.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 -- 172.21.15.16:0/97301589 shutdown_connections 2024-09-17T05:08:21.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.866+0000 7fe32b800640 1 -- 172.21.15.16:0/97301589 wait complete. 2024-09-17T05:08:21.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 Processor -- start 2024-09-17T05:08:21.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 -- start start 2024-09-17T05:08:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32406ce90 0x7fe3241602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe3240c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3240ceea0 0x7fe3240c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe32406d720 con 0x7fe324074510 2024-09-17T05:08:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe32406d5a0 con 0x7fe32406ce90 2024-09-17T05:08:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe32b800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe324167990 con 0x7fe3240ceea0 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe3240c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe3240c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36816/0 (socket says 172.21.15.16:36816) 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 -- 172.21.15.16:0/1516092968 learned_addr learned my addr 172.21.15.16:0/1516092968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe328e00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32406ce90 0x7fe3241602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe329800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3240ceea0 0x7fe3240c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 -- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3240ceea0 msgr2=0x7fe3240c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3240ceea0 0x7fe3240c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 -- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32406ce90 msgr2=0x7fe3241602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32406ce90 0x7fe3241602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 -- 172.21.15.16:0/1516092968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3241607e0 con 0x7fe324074510 2024-09-17T05:08:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.870+0000 7fe323e00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe3240c90e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe3100b0040 tx=0x7fe310005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3100b2040 con 0x7fe324074510 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe310004320 con 0x7fe324074510 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3241fc450 con 0x7fe324074510 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3100044f0 con 0x7fe324074510 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3241fca10 con 0x7fe324074510 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3241fc1c0 con 0x7fe324074510 2024-09-17T05:08:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3100be060 con 0x7fe324074510 2024-09-17T05:08:21.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe320c00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe304073200 0x7fe304075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:21.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fe31012d770 con 0x7fe324074510 2024-09-17T05:08:21.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.874+0000 7fe328e00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe304073200 0x7fe304075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:21.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.878+0000 7fe328e00640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe304073200 0x7fe304075710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe31800bc50 tx=0x7fe31800b3e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:21.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:21.878+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3240cab40 con 0x7fe324074510 2024-09-17T05:08:22.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.054+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fe324074920 con 0x7fe324074510 2024-09-17T05:08:22.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.058+0000 7fe320c00640 1 -- 172.21.15.16:0/1516092968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v54)=0 v54) ==== 128+0+0 (secure 0 0 0) 0x7fe3100f7f10 con 0x7fe324074510 2024-09-17T05:08:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe304073200 msgr2=0x7fe304075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe304073200 0x7fe304075710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe31800bc50 tx=0x7fe31800b3e0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 msgr2=0x7fe3240c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe3240c90e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe3100b0040 tx=0x7fe310005e70 comp rx=0 tx=0).stop 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 shutdown_connections 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe304073200 0x7fe304075710 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe3240ceea0 0x7fe3240c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe324074510 0x7fe3240c90e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 --2- 172.21.15.16:0/1516092968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32406ce90 0x7fe3241602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 >> 172.21.15.16:0/1516092968 conn(0x7fe3240b2250 msgr2=0x7fe324164fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 shutdown_connections 2024-09-17T05:08:22.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.062+0000 7fe32b800640 1 -- 172.21.15.16:0/1516092968 wait complete. 2024-09-17T05:08:22.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 -- 172.21.15.16:0/2168472199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 msgr2=0x7f16b806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 --2- 172.21.15.16:0/2168472199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b806e3b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f16b80ce810 tx=0x7f16a801c8c0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 -- 172.21.15.16:0/2168472199 shutdown_connections 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 --2- 172.21.15.16:0/2168472199 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 0x7f16b80cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 --2- 172.21.15.16:0/2168472199 >> v2:172.21.15.104:3300/0 conn(0x7f16b8073310 0x7f16b8073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 --2- 172.21.15.16:0/2168472199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b806e3b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 -- 172.21.15.16:0/2168472199 >> 172.21.15.16:0/2168472199 conn(0x7f16b806f9a0 msgr2=0x7f16b80b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 -- 172.21.15.16:0/2168472199 shutdown_connections 2024-09-17T05:08:22.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.242+0000 7f16bca00640 1 -- 172.21.15.16:0/2168472199 wait complete. 2024-09-17T05:08:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 Processor -- start 2024-09-17T05:08:22.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 -- start start 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b81456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8073310 0x7f16b8150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 0x7f16b81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b815fac0 con 0x7f16b806dfa0 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16b815f940 con 0x7f16b8073310 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f16b815fc40 con 0x7f16b80ce8c0 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b81456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b81456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36828/0 (socket says 172.21.15.16:36828) 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 -- 172.21.15.16:0/12631602 learned_addr learned my addr 172.21.15.16:0/12631602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:22.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b7200640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 0x7f16b81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16afe00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8073310 0x7f16b8150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 -- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 msgr2=0x7f16b81505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 0x7f16b81505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 -- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8073310 msgr2=0x7f16b8150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8073310 0x7f16b8150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 -- 172.21.15.16:0/12631602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b814f2a0 con 0x7f16b806dfa0 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b7200640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 0x7f16b81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:22.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16b6800640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b81456e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f16a80b0040 tx=0x7f16a8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80b2040 con 0x7f16b806dfa0 2024-09-17T05:08:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16a8004330 con 0x7f16b806dfa0 2024-09-17T05:08:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8005280 con 0x7f16b806dfa0 2024-09-17T05:08:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b814f580 con 0x7f16b806dfa0 2024-09-17T05:08:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.246+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b8145c20 con 0x7f16b806dfa0 2024-09-17T05:08:22.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.250+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b8073720 con 0x7f16b806dfa0 2024-09-17T05:08:22.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.250+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f16a8005420 con 0x7f16b806dfa0 2024-09-17T05:08:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.250+0000 7f16aea00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1694073200 0x7f1694075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.250+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f16a812e4e0 con 0x7f16b806dfa0 2024-09-17T05:08:22.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.250+0000 7f16afe00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1694073200 0x7f1694075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.254+0000 7f16afe00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1694073200 0x7f1694075710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f16a00058d0 tx=0x7f16a0005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:22.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.254+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f16a80164b0 con 0x7f16b806dfa0 2024-09-17T05:08:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f16b8151ed0 con 0x7f16b806dfa0 2024-09-17T05:08:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16aea00640 1 -- 172.21.15.16:0/12631602 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v54) ==== 106+0+4 (secure 0 0 0) 0x7f16a80f9460 con 0x7f16b806dfa0 2024-09-17T05:08:22.430 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1694073200 msgr2=0x7f1694075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1694073200 0x7f1694075710 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f16a00058d0 tx=0x7f16a0005860 comp rx=0 tx=0).stop 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 msgr2=0x7f16b81456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b81456e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f16a80b0040 tx=0x7f16a8005e70 comp rx=0 tx=0).stop 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 shutdown_connections 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1694073200 0x7f1694075710 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f16b80ce8c0 0x7f16b81505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16b8073310 0x7f16b8150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 --2- 172.21.15.16:0/12631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b806dfa0 0x7f16b81456e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 >> 172.21.15.16:0/12631602 conn(0x7f16b806f9a0 msgr2=0x7f16b8146a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 shutdown_connections 2024-09-17T05:08:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.430+0000 7f16bca00640 1 -- 172.21.15.16:0/12631602 wait complete. 2024-09-17T05:08:22.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-17T05:08:22.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.614+0000 7fdc64000640 1 -- 172.21.15.16:0/661228472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc44003a10 msgr2=0x7fdc44003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.614+0000 7fdc64000640 1 --2- 172.21.15.16:0/661228472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc44003a10 0x7fdc44003e80 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c470 tx=0x7fdc5001cf70 comp rx=0 tx=0).stop 2024-09-17T05:08:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.614+0000 7fdc62000640 1 -- 172.21.15.16:0/661228472 reap_dead start 2024-09-17T05:08:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.614+0000 7fdc64000640 1 -- 172.21.15.16:0/661228472 shutdown_connections 2024-09-17T05:08:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.614+0000 7fdc64000640 1 -- 172.21.15.16:0/661228472 >> 172.21.15.16:0/661228472 conn(0x7fdc5c0b2250 msgr2=0x7fdc5c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 -- 172.21.15.16:0/661228472 shutdown_connections 2024-09-17T05:08:22.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 -- 172.21.15.16:0/661228472 wait complete. 2024-09-17T05:08:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 Processor -- start 2024-09-17T05:08:22.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 -- start start 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc44003a10 0x7fdc5c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc5c0c8f10 0x7fdc5c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 0x7fdc5c1f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0cbc70 con 0x7fdc5c158690 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0cbaf0 con 0x7fdc44003a10 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.618+0000 7fdc64000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5c165980 con 0x7fdc5c0c8f10 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 0x7fdc5c1f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 0x7fdc5c1f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36844/0 (socket says 172.21.15.16:36844) 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 -- 172.21.15.16:0/1877940496 learned_addr learned my addr 172.21.15.16:0/1877940496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc61600640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc44003a10 0x7fdc5c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc60c00640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc5c0c8f10 0x7fdc5c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 -- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc5c0c8f10 msgr2=0x7fdc5c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc5c0c8f10 0x7fdc5c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 -- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc44003a10 msgr2=0x7fdc5c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc44003a10 0x7fdc5c0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 -- 172.21.15.16:0/1877940496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c15e2e0 con 0x7fdc5c158690 2024-09-17T05:08:22.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc62000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 0x7fdc5c1f8e50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c075100 tx=0x7fdc5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54003d90 con 0x7fdc5c158690 2024-09-17T05:08:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc54004b30 con 0x7fdc5c158690 2024-09-17T05:08:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004030 con 0x7fdc5c158690 2024-09-17T05:08:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c0c9400 con 0x7fdc5c158690 2024-09-17T05:08:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c0c99c0 con 0x7fdc5c158690 2024-09-17T05:08:22.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c15e050 con 0x7fdc5c158690 2024-09-17T05:08:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.622+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc540041d0 con 0x7fdc5c158690 2024-09-17T05:08:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.626+0000 7fdc59600640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc3c073230 0x7fdc3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:22.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.626+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fdc54095100 con 0x7fdc5c158690 2024-09-17T05:08:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.626+0000 7fdc61600640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc3c073230 0x7fdc3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:22.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.626+0000 7fdc61600640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc3c073230 0x7fdc3c075740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ac000 tx=0x7fdc50019db0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.630+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc54060090 con 0x7fdc5c158690 2024-09-17T05:08:22.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.806+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7fdc5c06d2a0 con 0x7fdc5c158690 2024-09-17T05:08:22.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.810+0000 7fdc59600640 1 -- 172.21.15.16:0/1877940496 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v55)=0 v55) ==== 124+0+0 (secure 0 0 0) 0x7fdc5405f990 con 0x7fdc5c158690 2024-09-17T05:08:22.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.810+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc3c073230 msgr2=0x7fdc3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.810+0000 7fdc64000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc3c073230 0x7fdc3c075740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ac000 tx=0x7fdc50019db0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 msgr2=0x7fdc5c1f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 0x7fdc5c1f8e50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c075100 tx=0x7fdc5400f790 comp rx=0 tx=0).stop 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 shutdown_connections 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc3c073230 0x7fdc3c075740 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc5c158690 0x7fdc5c1f8e50 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc5c0c8f10 0x7fdc5c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 --2- 172.21.15.16:0/1877940496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc44003a10 0x7fdc5c0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 >> 172.21.15.16:0/1877940496 conn(0x7fdc5c0b2250 msgr2=0x7fdc5c165ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 shutdown_connections 2024-09-17T05:08:22.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.814+0000 7fdc64000640 1 -- 172.21.15.16:0/1877940496 wait complete. 2024-09-17T05:08:22.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 -- 172.21.15.16:0/2334456501 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5660074a20 msgr2=0x7f566015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 --2- 172.21.15.16:0/2334456501 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5660074a20 0x7f566015efe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5650009840 tx=0x7f565001c820 comp rx=0 tx=0).stop 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 -- 172.21.15.16:0/2334456501 shutdown_connections 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 --2- 172.21.15.16:0/2334456501 >> v2:172.21.15.104:3300/0 conn(0x7f566015f520 0x7f566015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 --2- 172.21.15.16:0/2334456501 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5660074a20 0x7f566015efe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 -- 172.21.15.16:0/2334456501 >> 172.21.15.16:0/2334456501 conn(0x7f56600b2250 msgr2=0x7f56600d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 -- 172.21.15.16:0/2334456501 shutdown_connections 2024-09-17T05:08:22.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.994+0000 7f5665800640 1 -- 172.21.15.16:0/2334456501 wait complete. 2024-09-17T05:08:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 Processor -- start 2024-09-17T05:08:22.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 -- start start 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 0x7f566015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5660074e30 0x7f56601f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f566015f520 0x7f5660153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5660168ed0 con 0x7f5660074a20 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5660168d50 con 0x7f5660074e30 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5660169050 con 0x7f566015f520 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 0x7f566015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 0x7f566015afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36854/0 (socket says 172.21.15.16:36854) 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 -- 172.21.15.16:0/2741770468 learned_addr learned my addr 172.21.15.16:0/2741770468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:23.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5657e00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5660074e30 0x7f56601f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565f400640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f566015f520 0x7f5660153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 -- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f566015f520 msgr2=0x7f5660153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f566015f520 0x7f5660153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 -- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5660074e30 msgr2=0x7f56601f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5660074e30 0x7f56601f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 -- 172.21.15.16:0/2741770468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5660159780 con 0x7f5660074a20 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f565ea00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 0x7f566015afe0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f5650009790 tx=0x7f56500a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650004140 con 0x7f5660074a20 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56500043d0 con 0x7f5660074a20 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650003860 con 0x7f5660074a20 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:22.998+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5660159a60 con 0x7f5660074a20 2024-09-17T05:08:23.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.002+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566015a020 con 0x7f5660074a20 2024-09-17T05:08:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.002+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566015efe0 con 0x7f5660074a20 2024-09-17T05:08:23.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.002+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f56500bf020 con 0x7f5660074a20 2024-09-17T05:08:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.002+0000 7f5656a00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f563c073230 0x7f563c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.002+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f565012eb70 con 0x7f5660074a20 2024-09-17T05:08:23.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.002+0000 7f5657e00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f563c073230 0x7f563c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.006+0000 7f5657e00640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f563c073230 0x7f563c075740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56600743b0 tx=0x7f5648009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:23.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.006+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f56500c6020 con 0x7f5660074a20 2024-09-17T05:08:23.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f5660150a20 con 0x7f5660074a20 2024-09-17T05:08:23.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5656a00640 1 -- 172.21.15.16:0/2741770468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v55) ==== 106+0+8 (secure 0 0 0) 0x7f5650016410 con 0x7f5660074a20 2024-09-17T05:08:23.198 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f563c073230 msgr2=0x7f563c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f563c073230 0x7f563c075740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f56600743b0 tx=0x7f5648009040 comp rx=0 tx=0).stop 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 msgr2=0x7f566015afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 0x7f566015afe0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f5650009790 tx=0x7f56500a6760 comp rx=0 tx=0).stop 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 shutdown_connections 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f563c073230 0x7f563c075740 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f566015f520 0x7f5660153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5660074e30 0x7f56601f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 --2- 172.21.15.16:0/2741770468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5660074a20 0x7f566015afe0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 >> 172.21.15.16:0/2741770468 conn(0x7f56600b2250 msgr2=0x7f56600b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:23.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 shutdown_connections 2024-09-17T05:08:23.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.198+0000 7f5665800640 1 -- 172.21.15.16:0/2741770468 wait complete. 2024-09-17T05:08:23.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-17T05:08:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 -- 172.21.15.16:0/2940300910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074a20 msgr2=0x7fac2c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 --2- 172.21.15.16:0/2940300910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074a20 0x7fac2c15efe0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0cca90 tx=0x7fac1c01c800 comp rx=0 tx=0).stop 2024-09-17T05:08:23.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 -- 172.21.15.16:0/2940300910 shutdown_connections 2024-09-17T05:08:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 --2- 172.21.15.16:0/2940300910 >> v2:172.21.15.104:3300/0 conn(0x7fac2c074e30 0x7fac2c15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 --2- 172.21.15.16:0/2940300910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074a20 0x7fac2c15efe0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 -- 172.21.15.16:0/2940300910 >> 172.21.15.16:0/2940300910 conn(0x7fac2c0b2250 msgr2=0x7fac2c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 -- 172.21.15.16:0/2940300910 shutdown_connections 2024-09-17T05:08:23.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.382+0000 7fac33800640 1 -- 172.21.15.16:0/2940300910 wait complete. 2024-09-17T05:08:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 Processor -- start 2024-09-17T05:08:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 -- start start 2024-09-17T05:08:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac2c074a20 0x7fac2c151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 0x7fac2c14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac2c15ba50 0x7fac2c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c168e20 con 0x7fac2c074e30 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac2c168ca0 con 0x7fac2c074a20 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fac2c168fa0 con 0x7fac2c15ba50 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 0x7fac2c14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 0x7fac2c14e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36868/0 (socket says 172.21.15.16:36868) 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 -- 172.21.15.16:0/250868114 learned_addr learned my addr 172.21.15.16:0/250868114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac30e00640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac2c074a20 0x7fac2c151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac31800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac2c15ba50 0x7fac2c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 -- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac2c15ba50 msgr2=0x7fac2c1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac2c15ba50 0x7fac2c1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 -- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac2c074a20 msgr2=0x7fac2c151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac2c074a20 0x7fac2c151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 -- 172.21.15.16:0/250868114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2c0c6a10 con 0x7fac2c074e30 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac23e00640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 0x7fac2c14e440 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fac140058d0 tx=0x7fac1400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14017070 con 0x7fac2c074e30 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac14009d20 con 0x7fac2c074e30 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140106c0 con 0x7fac2c074e30 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2c151bc0 con 0x7fac2c074e30 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.386+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2c1508c0 con 0x7fac2c074e30 2024-09-17T05:08:23.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.390+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2c15be60 con 0x7fac2c074e30 2024-09-17T05:08:23.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.390+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac1401a070 con 0x7fac2c074e30 2024-09-17T05:08:23.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.390+0000 7fac29600640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac08073200 0x7fac08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:23.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.390+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fac14094570 con 0x7fac2c074e30 2024-09-17T05:08:23.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.390+0000 7fac30e00640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac08073200 0x7fac08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:23.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.394+0000 7fac30e00640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac08073200 0x7fac08075710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0af000 tx=0x7fac1c01b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:23.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.394+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac2c0c6a10 con 0x7fac2c074e30 2024-09-17T05:08:23.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7fac2c14e980 con 0x7fac2c074e30 2024-09-17T05:08:23.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac29600640 1 -- 172.21.15.16:0/250868114 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v55) ==== 112+0+10 (secure 0 0 0) 0x7fac1405ed10 con 0x7fac2c074e30 2024-09-17T05:08:23.570 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-17T05:08:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac08073200 msgr2=0x7fac08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac08073200 0x7fac08075710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0af000 tx=0x7fac1c01b040 comp rx=0 tx=0).stop 2024-09-17T05:08:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 msgr2=0x7fac2c14e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 0x7fac2c14e440 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fac140058d0 tx=0x7fac1400dde0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 shutdown_connections 2024-09-17T05:08:23.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fac08073200 0x7fac08075710 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fac2c15ba50 0x7fac2c1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2c074e30 0x7fac2c14e440 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 --2- 172.21.15.16:0/250868114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac2c074a20 0x7fac2c151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 >> 172.21.15.16:0/250868114 conn(0x7fac2c0b2250 msgr2=0x7fac2c14f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.570+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 shutdown_connections 2024-09-17T05:08:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.574+0000 7fac33800640 1 -- 172.21.15.16:0/250868114 wait complete. 2024-09-17T05:08:23.593 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:23.593 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:23.602 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:08:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:08:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:08:23.705 INFO:teuthology.orchestra.run.smithi016.stdout:client7479 2024-09-17T05:08:23.705 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:23.705 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:23.756 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> n = 800 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:08:23.757 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-17T05:08:23.758 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-17T05:08:23.759 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-17T05:08:23.759 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:08:23.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 -- 172.21.15.16:0/3422569223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 msgr2=0x7fbc5c06e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 --2- 172.21.15.16:0/3422569223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c06e3b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c073890 tx=0x7fbc4801c6e0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 -- 172.21.15.16:0/3422569223 shutdown_connections 2024-09-17T05:08:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 --2- 172.21.15.16:0/3422569223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c06e3b0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 --2- 172.21.15.16:0/3422569223 >> v2:172.21.15.104:3300/0 conn(0x7fbc5c0749d0 0x7fbc5c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 -- 172.21.15.16:0/3422569223 >> 172.21.15.16:0/3422569223 conn(0x7fbc5c0cdb80 msgr2=0x7fbc5c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 -- 172.21.15.16:0/3422569223 shutdown_connections 2024-09-17T05:08:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 -- 172.21.15.16:0/3422569223 wait complete. 2024-09-17T05:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 Processor -- start 2024-09-17T05:08:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:23.998+0000 7fbc63e00640 1 -- start start 2024-09-17T05:08:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5c0749d0 0x7fbc5c14e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5c0cc6f0 0x7fbc5c14e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c15c370 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c15c1f0 con 0x7fbc5c0cc6f0 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5c15c4f0 con 0x7fbc5c0749d0 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c0c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36878/0 (socket says 172.21.15.16:36878) 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 -- 172.21.15.16:0/2031707639 learned_addr learned my addr 172.21.15.16:0/2031707639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc60a00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5c0749d0 0x7fbc5c14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5c0cc6f0 0x7fbc5c14e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 -- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5c0749d0 msgr2=0x7fbc5c14e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5c0749d0 0x7fbc5c14e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:24.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 -- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5c0cc6f0 msgr2=0x7fbc5c14e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5c0cc6f0 0x7fbc5c14e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 -- 172.21.15.16:0/2031707639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c0ca2e0 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5c0cc6f0 0x7fbc5c14e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc61400640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c0c7150 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d950 tx=0x7fbc5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50017070 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc50009d20 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c0c9a40 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500106c0 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c0ca000 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.002+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c06e3b0 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.006+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc5001a070 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.006+0000 7fbc59600640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc3c073200 0x7fbc3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.006+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fbc500944f0 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.006+0000 7fbc60a00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc3c073200 0x7fbc3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.006+0000 7fbc60a00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc3c073200 0x7fbc3c075710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbc480af000 tx=0x7fbc4801af20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.010+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc5005ed20 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.242+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc5c069770 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc59600640 1 -- 172.21.15.16:0/2031707639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5225 (secure 0 0 0) 0x7fbc50063750 con 0x7fbc5c06dfa0 2024-09-17T05:08:24.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:24.244 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"btime":"2024-09-17T05:08:10:776523+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:10.202627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:24.245 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 75 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc3c073200 msgr2=0x7fbc3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc3c073200 0x7fbc3c075710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbc480af000 tx=0x7fbc4801af20 comp rx=0 tx=0).stop 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 msgr2=0x7fbc5c0c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c0c7150 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d950 tx=0x7fbc5000de10 comp rx=0 tx=0).stop 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 shutdown_connections 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc3c073200 0x7fbc3c075710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc5c0cc6f0 0x7fbc5c14e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbc5c0749d0 0x7fbc5c14e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 --2- 172.21.15.16:0/2031707639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc5c06dfa0 0x7fbc5c0c7150 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 >> 172.21.15.16:0/2031707639 conn(0x7fbc5c0cdb80 msgr2=0x7fbc5c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 shutdown_connections 2024-09-17T05:08:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:24.246+0000 7fbc63e00640 1 -- 172.21.15.16:0/2031707639 wait complete. 2024-09-17T05:08:24.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-17T05:08:24.460 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":7108,"entity":{"name":{"type":"client","num":7108},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":202079391}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.6479325509999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9889654344551704,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7108 v1:192.168.0.1:0/202079391","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7479,"entity":{"name":{"type":"client","num":7479},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":3460568592}},"state":"open","num_leases":1,"num_caps":466,"request_load_avg":489,"uptime":4.8439103350000003,"requests_in_flight":464,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":932.38576894437142,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7479 v1:192.168.0.1:0/3460568592","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:08:24.461 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 202079391, "type": "v1" }, "name": { "num": 7108, "type": "client" } }, "id": 7108, "inst": "client.7108 v1:192.168.0.1:0/202079391", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9889654344551704 }, "state": "open", "uptime": 3.647932551 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3460568592, "type": "v1" }, "name": { "num": 7479, "type": "client" } }, "id": 7479, "inst": "client.7479 v1:192.168.0.1:0/3460568592", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 466, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 489, "requests_in_flight": 464, "session_cache_liveness": { "halflife": 300, "value": 932.3857689443714 }, "state": "open", "uptime": 4.843910335 } ] 2024-09-17T05:08:24.461 DEBUG:tasks.ceph_test_case:wait_until_equal: 466 != 802, waiting (timeout=600)... 2024-09-17T05:08:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-17T05:08:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:29.094 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:29.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:29.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 -- 172.21.15.16:0/1960069713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580b2f50 msgr2=0x7fd15815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 --2- 172.21.15.16:0/1960069713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580b2f50 0x7fd15815bb60 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd148009900 tx=0x7fd14801c850 comp rx=0 tx=0).stop 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 -- 172.21.15.16:0/1960069713 shutdown_connections 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 --2- 172.21.15.16:0/1960069713 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd158073240 0x7fd158073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 --2- 172.21.15.16:0/1960069713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1580b2f50 0x7fd15815bb60 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 -- 172.21.15.16:0/1960069713 >> 172.21.15.16:0/1960069713 conn(0x7fd15806f900 msgr2=0x7fd1580b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 -- 172.21.15.16:0/1960069713 shutdown_connections 2024-09-17T05:08:29.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 -- 172.21.15.16:0/1960069713 wait complete. 2024-09-17T05:08:29.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 Processor -- start 2024-09-17T05:08:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.634+0000 7fd15e400640 1 -- start start 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd158073240 0x7fd158143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1580b2f50 0x7fd158143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 0x7fd1580c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd158168d60 con 0x7fd15815c0a0 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd158168be0 con 0x7fd1580b2f50 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd158168ee0 con 0x7fd158073240 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 0x7fd1580c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:29.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 0x7fd1580c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39390/0 (socket says 172.21.15.16:39390) 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd158073240 0x7fd158143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 -- 172.21.15.16:0/2726947270 learned_addr learned my addr 172.21.15.16:0/2726947270 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd14ea00640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1580b2f50 0x7fd158143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 -- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd158073240 msgr2=0x7fd158143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd158073240 0x7fd158143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 -- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1580b2f50 msgr2=0x7fd158143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1580b2f50 0x7fd158143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 -- 172.21.15.16:0/2726947270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1580c6b80 con 0x7fd15815c0a0 2024-09-17T05:08:29.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd157e00640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 0x7fd1580c9f00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e910 tx=0x7fd15000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150004d30 con 0x7fd15815c0a0 2024-09-17T05:08:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd15000cb20 con 0x7fd15815c0a0 2024-09-17T05:08:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1580c6e60 con 0x7fd15815c0a0 2024-09-17T05:08:29.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.638+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150005020 con 0x7fd15815c0a0 2024-09-17T05:08:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.642+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1580c86a0 con 0x7fd15815c0a0 2024-09-17T05:08:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.642+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd158073650 con 0x7fd15815c0a0 2024-09-17T05:08:29.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.642+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1500051c0 con 0x7fd15815c0a0 2024-09-17T05:08:29.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.642+0000 7fd14fe00640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd134073230 0x7fd134075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.642+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fd150094b30 con 0x7fd15815c0a0 2024-09-17T05:08:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.642+0000 7fd157400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd134073230 0x7fd134075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:29.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.646+0000 7fd157400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd134073230 0x7fd134075740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd1480af000 tx=0x7fd14801b090 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:29.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.646+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd150016030 con 0x7fd15815c0a0 2024-09-17T05:08:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.854+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1581660b0 con 0x7fd15815c0a0 2024-09-17T05:08:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd14fe00640 1 -- 172.21.15.16:0/2726947270 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5225 (secure 0 0 0) 0x7fd15005f360 con 0x7fd15815c0a0 2024-09-17T05:08:29.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:29.856 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":76,"btime":"2024-09-17T05:08:24:316992+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:23.800426+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":7,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 76 2024-09-17T05:08:29.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd134073230 msgr2=0x7fd134075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd134073230 0x7fd134075740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd1480af000 tx=0x7fd14801b090 comp rx=0 tx=0).stop 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 msgr2=0x7fd1580c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 0x7fd1580c9f00 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd15000e910 tx=0x7fd15000edd0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 shutdown_connections 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd134073230 0x7fd134075740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15815c0a0 0x7fd1580c9f00 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1580b2f50 0x7fd158143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 --2- 172.21.15.16:0/2726947270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd158073240 0x7fd158143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 >> 172.21.15.16:0/2726947270 conn(0x7fd15806f900 msgr2=0x7fd158151c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 shutdown_connections 2024-09-17T05:08:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:29.858+0000 7fd15e400640 1 -- 172.21.15.16:0/2726947270 wait complete. 2024-09-17T05:08:29.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-17T05:08:30.052 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":7108,"entity":{"name":{"type":"client","num":7108},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":202079391}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":9.2398300379999991,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9505963202876786,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7108 v1:192.168.0.1:0/202079391","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7479,"entity":{"name":{"type":"client","num":7479},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":3460568592}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":774,"uptime":10.435807821999999,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2325.4248232719701,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":903.51910265381014,"halflife":1.5},"recall_caps_throttle2o":{"value":378.07624094267572,"halflife":0.5},"session_cache_liveness":{"value":1584.3721884833285,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7479 v1:192.168.0.1:0/3460568592","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:08:30.055 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 202079391, "type": "v1" }, "name": { "num": 7108, "type": "client" } }, "id": 7108, "inst": "client.7108 v1:192.168.0.1:0/202079391", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9505963202876786 }, "state": "open", "uptime": 9.239830038 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3460568592, "type": "v1" }, "name": { "num": 7479, "type": "client" } }, "id": 7479, "inst": "client.7479 v1:192.168.0.1:0/3460568592", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2325.42482327197 }, "recall_caps_throttle": { "halflife": 1.5, "value": 903.5191026538101 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 378.0762409426757 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 774, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1584.3721884833285 }, "state": "open", "uptime": 10.435807822 } ] 2024-09-17T05:08:30.055 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-17T05:08:30.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 -- 172.21.15.16:0/310834949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a44073240 msgr2=0x7f5a44073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 --2- 172.21.15.16:0/310834949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a44073240 0x7f5a44073650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009c80 tx=0x7f5a3c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 -- 172.21.15.16:0/310834949 shutdown_connections 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 --2- 172.21.15.16:0/310834949 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a44073240 0x7f5a44073650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 --2- 172.21.15.16:0/310834949 >> v2:172.21.15.104:3300/0 conn(0x7f5a440b2f50 0x7f5a4415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 -- 172.21.15.16:0/310834949 >> 172.21.15.16:0/310834949 conn(0x7f5a4406f900 msgr2=0x7f5a440b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 -- 172.21.15.16:0/310834949 shutdown_connections 2024-09-17T05:08:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 -- 172.21.15.16:0/310834949 wait complete. 2024-09-17T05:08:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 Processor -- start 2024-09-17T05:08:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.214+0000 7f5a4b200640 1 -- start start 2024-09-17T05:08:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a44073240 0x7f5a44142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a440b2f50 0x7f5a440cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 0x7f5a440c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44168c90 con 0x7f5a4415c0a0 2024-09-17T05:08:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44168b10 con 0x7f5a44073240 2024-09-17T05:08:30.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5a44168e10 con 0x7f5a440b2f50 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 0x7f5a440c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 0x7f5a440c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39392/0 (socket says 172.21.15.16:39392) 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 -- 172.21.15.16:0/70902187 learned_addr learned my addr 172.21.15.16:0/70902187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a43e00640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a44073240 0x7f5a44142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a3be00640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a440b2f50 0x7f5a440cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 -- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a440b2f50 msgr2=0x7f5a440cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a440b2f50 0x7f5a440cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 -- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a44073240 msgr2=0x7f5a44142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a44073240 0x7f5a44142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 -- 172.21.15.16:0/70902187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a441430b0 con 0x7f5a4415c0a0 2024-09-17T05:08:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a49200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 0x7f5a440c81c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c01c780 tx=0x7f5a3c003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0b2040 con 0x7f5a4415c0a0 2024-09-17T05:08:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a3c004040 con 0x7f5a4415c0a0 2024-09-17T05:08:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0043f0 con 0x7f5a4415c0a0 2024-09-17T05:08:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a440c7120 con 0x7f5a4415c0a0 2024-09-17T05:08:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a440c76e0 con 0x7f5a4415c0a0 2024-09-17T05:08:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.218+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a44073650 con 0x7f5a4415c0a0 2024-09-17T05:08:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.222+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a3c0bd030 con 0x7f5a4415c0a0 2024-09-17T05:08:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.222+0000 7f5a40c00640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a20073200 0x7f5a20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.222+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f5a3c12e350 con 0x7f5a4415c0a0 2024-09-17T05:08:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.222+0000 7f5a43e00640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a20073200 0x7f5a20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.222+0000 7f5a43e00640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a20073200 0x7f5a20075710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009ad0 tx=0x7f5a340091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.226+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a3c0c4020 con 0x7f5a4415c0a0 2024-09-17T05:08:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5a440d0cb0 con 0x7f5a4415c0a0 2024-09-17T05:08:30.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a40c00640 1 -- 172.21.15.16:0/70902187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f5a3c016990 con 0x7f5a4415c0a0 2024-09-17T05:08:30.437 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:30.437 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:08:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a20073200 msgr2=0x7f5a20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a20073200 0x7f5a20075710 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009ad0 tx=0x7f5a340091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 msgr2=0x7f5a440c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 0x7f5a440c81c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c01c780 tx=0x7f5a3c003f40 comp rx=0 tx=0).stop 2024-09-17T05:08:30.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 shutdown_connections 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a20073200 0x7f5a20075710 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4415c0a0 0x7f5a440c81c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a440b2f50 0x7f5a440cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 --2- 172.21.15.16:0/70902187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a44073240 0x7f5a44142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 >> 172.21.15.16:0/70902187 conn(0x7f5a4406f900 msgr2=0x7f5a44151510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 shutdown_connections 2024-09-17T05:08:30.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:30.438+0000 7f5a4b200640 1 -- 172.21.15.16:0/70902187 wait complete. 2024-09-17T05:08:30.458 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-09-17T05:08:30.458 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:08:31.096 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:31.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:08:31.342 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:08:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:35.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 -- 172.21.15.16:0/1889048270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c1606b0 msgr2=0x7f2c2c160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 --2- 172.21.15.16:0/1889048270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c1606b0 0x7f2c2c160ac0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c070f40 tx=0x7f2c2401cd80 comp rx=0 tx=0).stop 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 -- 172.21.15.16:0/1889048270 shutdown_connections 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 --2- 172.21.15.16:0/1889048270 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c1606b0 0x7f2c2c160ac0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 --2- 172.21.15.16:0/1889048270 >> v2:172.21.15.104:3300/0 conn(0x7f2c2c0cf390 0x7f2c2c160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 --2- 172.21.15.16:0/1889048270 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c2c15d600 0x7f2c2c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 -- 172.21.15.16:0/1889048270 >> 172.21.15.16:0/1889048270 conn(0x7f2c2c06f900 msgr2=0x7f2c2c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 -- 172.21.15.16:0/1889048270 shutdown_connections 2024-09-17T05:08:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 -- 172.21.15.16:0/1889048270 wait complete. 2024-09-17T05:08:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 Processor -- start 2024-09-17T05:08:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.630+0000 7f2c32000640 1 -- start start 2024-09-17T05:08:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c2c0cf390 0x7f2c2c166ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 0x7f2c2c141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c2c1606b0 0x7f2c2c13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b4320 con 0x7f2c2c15d600 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b41a0 con 0x7f2c2c0cf390 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b44a0 con 0x7f2c2c1606b0 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 0x7f2c2c141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 0x7f2c2c141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39424/0 (socket says 172.21.15.16:39424) 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 -- 172.21.15.16:0/920205669 learned_addr learned my addr 172.21.15.16:0/920205669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c2b400640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c2c0cf390 0x7f2c2c166ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c2be00640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c2c1606b0 0x7f2c2c13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 -- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c2c1606b0 msgr2=0x7f2c2c13f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c2c1606b0 0x7f2c2c13f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 -- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c2c0cf390 msgr2=0x7f2c2c166ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c2c0cf390 0x7f2c2c166ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 -- 172.21.15.16:0/920205669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1f6d70 con 0x7f2c2c15d600 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c22a00640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 0x7f2c2c141ac0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c970 tx=0x7f2c1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14007b50 con 0x7f2c2c15d600 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c14007de0 con 0x7f2c2c15d600 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1f7050 con 0x7f2c2c15d600 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400f6b0 con 0x7f2c2c15d600 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c142170 con 0x7f2c2c15d600 2024-09-17T05:08:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.634+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c0cee50 con 0x7f2c2c15d600 2024-09-17T05:08:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.638+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c1400f850 con 0x7f2c2c15d600 2024-09-17T05:08:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.638+0000 7f2c23e00640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c08073230 0x7f2c08075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.638+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f2c14094960 con 0x7f2c2c15d600 2024-09-17T05:08:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.638+0000 7f2c2b400640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c08073230 0x7f2c08075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:35.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.638+0000 7f2c2b400640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c08073230 0x7f2c08075740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0058d0 tx=0x7f2c1c005f90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:35.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.642+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2c14014030 con 0x7f2c2c15d600 2024-09-17T05:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.854+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2c2c0b3530 con 0x7f2c2c15d600 2024-09-17T05:08:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.854+0000 7f2c23e00640 1 -- 172.21.15.16:0/920205669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f2c1405f190 con 0x7f2c2c15d600 2024-09-17T05:08:35.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:08:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c08073230 msgr2=0x7f2c08075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c08073230 0x7f2c08075740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0058d0 tx=0x7f2c1c005f90 comp rx=0 tx=0).stop 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 msgr2=0x7f2c2c141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 0x7f2c2c141ac0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c970 tx=0x7f2c1400ce30 comp rx=0 tx=0).stop 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 shutdown_connections 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c08073230 0x7f2c08075740 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c2c1606b0 0x7f2c2c13f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c2c15d600 0x7f2c2c141ac0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 --2- 172.21.15.16:0/920205669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c2c0cf390 0x7f2c2c166ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 >> 172.21.15.16:0/920205669 conn(0x7f2c2c06f900 msgr2=0x7f2c2c0afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 shutdown_connections 2024-09-17T05:08:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:35.858+0000 7f2c32000640 1 -- 172.21.15.16:0/920205669 wait complete. 2024-09-17T05:08:35.876 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:08:35.876 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-17T05:08:35.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 -- 172.21.15.16:0/356877904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe828073240 msgr2=0x7fe828073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 --2- 172.21.15.16:0/356877904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe828073240 0x7fe828073650 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe82806e960 tx=0x7fe82001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 -- 172.21.15.16:0/356877904 shutdown_connections 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 --2- 172.21.15.16:0/356877904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe828073240 0x7fe828073650 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 --2- 172.21.15.16:0/356877904 >> v2:172.21.15.104:3300/0 conn(0x7fe82815c270 0x7fe82815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 -- 172.21.15.16:0/356877904 >> 172.21.15.16:0/356877904 conn(0x7fe82806f900 msgr2=0x7fe8280b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 -- 172.21.15.16:0/356877904 shutdown_connections 2024-09-17T05:08:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 -- 172.21.15.16:0/356877904 wait complete. 2024-09-17T05:08:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 Processor -- start 2024-09-17T05:08:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.034+0000 7fe82d400640 1 -- start start 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe828073240 0x7fe8281501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8280b2f50 0x7fe8281506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 0x7fe8281f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe828168d10 con 0x7fe82815c270 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe828168b90 con 0x7fe828073240 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe828168e90 con 0x7fe8280b2f50 2024-09-17T05:08:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 0x7fe8281f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 0x7fe8281f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39450/0 (socket says 172.21.15.16:39450) 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 -- 172.21.15.16:0/2844448593 learned_addr learned my addr 172.21.15.16:0/2844448593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe826800640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe828073240 0x7fe8281501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe81de00640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8280b2f50 0x7fe8281506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 -- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8280b2f50 msgr2=0x7fe8281506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8280b2f50 0x7fe8281506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 -- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe828073240 msgr2=0x7fe8281501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe828073240 0x7fe8281501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 -- 172.21.15.16:0/2844448593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8281f96c0 con 0x7fe82815c270 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe826800640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe828073240 0x7fe8281501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe827200640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 0x7fe8281f8530 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe82001b3a0 tx=0x7fe820003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8200bc070 con 0x7fe82815c270 2024-09-17T05:08:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8200ad8e0 con 0x7fe82815c270 2024-09-17T05:08:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8200b2040 con 0x7fe82815c270 2024-09-17T05:08:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8281f99a0 con 0x7fe82815c270 2024-09-17T05:08:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8281fa2b0 con 0x7fe82815c270 2024-09-17T05:08:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.038+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8281f9430 con 0x7fe82815c270 2024-09-17T05:08:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.042+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe820004000 con 0x7fe82815c270 2024-09-17T05:08:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.042+0000 7fe81f400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe804073230 0x7fe804075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.042+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fe82012e470 con 0x7fe82815c270 2024-09-17T05:08:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.042+0000 7fe826800640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe804073230 0x7fe804075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.042+0000 7fe826800640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe804073230 0x7fe804075740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe818009890 tx=0x7fe818008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:36.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.046+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8200f2db0 con 0x7fe82815c270 2024-09-17T05:08:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe828073650 con 0x7fe82815c270 2024-09-17T05:08:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe81f400640 1 -- 172.21.15.16:0/2844448593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fe820016970 con 0x7fe82815c270 2024-09-17T05:08:36.257 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:36.257 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:08:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe804073230 msgr2=0x7fe804075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe804073230 0x7fe804075740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe818009890 tx=0x7fe818008040 comp rx=0 tx=0).stop 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 msgr2=0x7fe8281f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 0x7fe8281f8530 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe82001b3a0 tx=0x7fe820003f80 comp rx=0 tx=0).stop 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 shutdown_connections 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe804073230 0x7fe804075740 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe82815c270 0x7fe8281f8530 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8280b2f50 0x7fe8281506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 --2- 172.21.15.16:0/2844448593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe828073240 0x7fe8281501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 >> 172.21.15.16:0/2844448593 conn(0x7fe82806f900 msgr2=0x7fe828151a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 shutdown_connections 2024-09-17T05:08:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36.258+0000 7fe82d400640 1 -- 172.21.15.16:0/2844448593 wait complete. 2024-09-17T05:08:36.280 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:08:36.280 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-17T05:08:36.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:36 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-17T05:08:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:08:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-17T05:08:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:08:37.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 -- 172.21.15.16:0/1401313887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 msgr2=0x7fd9e8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 --2- 172.21.15.16:0/1401313887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fd9e8005160 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fda040b27b0 tx=0x7fd9fc01d2c0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda09a00640 1 -- 172.21.15.16:0/1401313887 reap_dead start 2024-09-17T05:08:37.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 -- 172.21.15.16:0/1401313887 shutdown_connections 2024-09-17T05:08:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 -- 172.21.15.16:0/1401313887 >> 172.21.15.16:0/1401313887 conn(0x7fda0406f900 msgr2=0x7fda040b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 -- 172.21.15.16:0/1401313887 shutdown_connections 2024-09-17T05:08:37.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 -- 172.21.15.16:0/1401313887 wait complete. 2024-09-17T05:08:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 Processor -- start 2024-09-17T05:08:37.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.346+0000 7fda0ba00640 1 -- start start 2024-09-17T05:08:37.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fda04153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:37.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda04152dc0 0x7fda04152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:37.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda040b0c10 0x7fda040b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda04160440 con 0x7fd9e8004d50 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda0406e770 con 0x7fda04152dc0 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fda041605c0 con 0x7fda040b0c10 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fda04153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fda04153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55780/0 (socket says 172.21.15.16:55780) 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 -- 172.21.15.16:0/3900245670 learned_addr learned my addr 172.21.15.16:0/3900245670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09a00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda040b0c10 0x7fda040b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fd9fbe00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda04152dc0 0x7fda04152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 -- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda040b0c10 msgr2=0x7fda040b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:37.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda040b0c10 0x7fda040b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 -- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda04152dc0 msgr2=0x7fda04152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:37.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda04152dc0 0x7fda04152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 -- 172.21.15.16:0/3900245670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0415d310 con 0x7fd9e8004d50 2024-09-17T05:08:37.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda09000640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fda04153260 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4011ae0 tx=0x7fd9f4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400fc70 con 0x7fd9e8004d50 2024-09-17T05:08:37.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f400fe10 con 0x7fd9e8004d50 2024-09-17T05:08:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4004280 con 0x7fd9e8004d50 2024-09-17T05:08:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda041580b0 con 0x7fd9e8004d50 2024-09-17T05:08:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda04154d90 con 0x7fd9e8004d50 2024-09-17T05:08:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.350+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0415d640 con 0x7fd9e8004d50 2024-09-17T05:08:37.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.354+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd9f4003600 con 0x7fd9e8004d50 2024-09-17T05:08:37.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.354+0000 7fda01600640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd9e0073200 0x7fd9e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:37.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.354+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fd9f4094f40 con 0x7fd9e8004d50 2024-09-17T05:08:37.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.354+0000 7fd9fbe00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd9e0073200 0x7fd9e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:37.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.354+0000 7fd9fbe00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd9e0073200 0x7fd9e0075710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda040742b0 tx=0x7fd9ec0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:37.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.358+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda041580b0 con 0x7fd9e8004d50 2024-09-17T05:08:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.566+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda04153920 con 0x7fd9e8004d50 2024-09-17T05:08:37.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.566+0000 7fda01600640 1 -- 172.21.15.16:0/3900245670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5225 (secure 0 0 0) 0x7fd9f405fdc0 con 0x7fd9e8004d50 2024-09-17T05:08:37.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:37.568 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"btime":"2024-09-17T05:08:32:644988+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:31.802627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:37.568 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 77 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd9e0073200 msgr2=0x7fd9e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd9e0073200 0x7fd9e0075710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fda040742b0 tx=0x7fd9ec0091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 msgr2=0x7fda04153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fda04153260 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4011ae0 tx=0x7fd9f4009e90 comp rx=0 tx=0).stop 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 shutdown_connections 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd9e0073200 0x7fd9e0075710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda040b0c10 0x7fda040b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda04152dc0 0x7fda04152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 --2- 172.21.15.16:0/3900245670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9e8004d50 0x7fda04153260 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 >> 172.21.15.16:0/3900245670 conn(0x7fda0406f900 msgr2=0x7fda040aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 shutdown_connections 2024-09-17T05:08:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:37.570+0000 7fda0ba00640 1 -- 172.21.15.16:0/3900245670 wait complete. 2024-09-17T05:08:37.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-17T05:08:37.766 INFO:teuthology.orchestra.run.smithi016.stdout:[{"id":7108,"entity":{"name":{"type":"client","num":7108},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":202079391}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":16.955690311000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.8984608935579317,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7108 v1:192.168.0.1:0/202079391","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7479,"entity":{"name":{"type":"client","num":7479},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":3460568592}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":708,"uptime":18.151668095000002,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":4802.6406397264454,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":692.67028966668272,"halflife":1.5},"recall_caps_throttle2o":{"value":141.6748090165172,"halflife":0.5},"session_cache_liveness":{"value":1556.3771965634344,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7479 v1:192.168.0.1:0/3460568592","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:08:37.767 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 202079391, "type": "v1" }, "name": { "num": 7108, "type": "client" } }, "id": 7108, "inst": "client.7108 v1:192.168.0.1:0/202079391", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.8984608935579317 }, "state": "open", "uptime": 16.955690311 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3460568592, "type": "v1" }, "name": { "num": 7479, "type": "client" } }, "id": 7479, "inst": "client.7479 v1:192.168.0.1:0/3460568592", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 4802.640639726445 }, "recall_caps_throttle": { "halflife": 1.5, "value": 692.6702896666827 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 141.6748090165172 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 708, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1556.3771965634344 }, "state": "open", "uptime": 18.151668095 } ] 2024-09-17T05:08:37.767 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:08:37.767 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:08:37.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:08:37.847 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:08:37.939 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:08:37.940 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:08:37.940 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 800\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-17T05:08:37.940 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:37.940 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:37.950 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:37.950 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:38.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:08:38.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 -- 172.21.15.16:0/303995255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 msgr2=0x7f1ea80ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 --2- 172.21.15.16:0/303995255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80ca6b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009840 tx=0x7f1e9801c760 comp rx=0 tx=0).stop 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 -- 172.21.15.16:0/303995255 shutdown_connections 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 --2- 172.21.15.16:0/303995255 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ea80cac90 0x7f1ea80cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 --2- 172.21.15.16:0/303995255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80ca6b0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 -- 172.21.15.16:0/303995255 >> 172.21.15.16:0/303995255 conn(0x7f1ea80b2250 msgr2=0x7f1ea80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 -- 172.21.15.16:0/303995255 shutdown_connections 2024-09-17T05:08:38.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.202+0000 7f1eaee00640 1 -- 172.21.15.16:0/303995255 wait complete. 2024-09-17T05:08:38.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 Processor -- start 2024-09-17T05:08:38.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 -- start start 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ea8074be0 0x7f1ea8145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ea80cac90 0x7f1ea815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea81688f0 con 0x7f1ea806ce90 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea815f2c0 con 0x7f1ea8074be0 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1eaee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea8168a70 con 0x7f1ea80cac90 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55808/0 (socket says 172.21.15.16:55808) 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 -- 172.21.15.16:0/190751621 learned_addr learned my addr 172.21.15.16:0/190751621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 -- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ea80cac90 msgr2=0x7f1ea815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ea80cac90 0x7f1ea815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1e9fe00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ea8074be0 0x7f1ea8145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 -- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ea8074be0 msgr2=0x7f1ea8145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ea8074be0 0x7f1ea8145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1ea7e00640 1 -- 172.21.15.16:0/190751621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea81f97e0 con 0x7f1ea806ce90 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.206+0000 7f1e9fe00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ea8074be0 0x7f1ea8145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:38.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1ea7e00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80b2f50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009790 tx=0x7f1e980a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980bb070 con 0x7f1ea806ce90 2024-09-17T05:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e98005030 con 0x7f1ea806ce90 2024-09-17T05:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea80b5be0 con 0x7f1ea806ce90 2024-09-17T05:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980b2040 con 0x7f1ea806ce90 2024-09-17T05:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea815b490 con 0x7f1ea806ce90 2024-09-17T05:08:38.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea80ca6b0 con 0x7f1ea806ce90 2024-09-17T05:08:38.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e980be070 con 0x7f1ea806ce90 2024-09-17T05:08:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.210+0000 7f1ea4c00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e84073230 0x7f1e84075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.214+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f1e9812df30 con 0x7f1ea806ce90 2024-09-17T05:08:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.214+0000 7f1e9fe00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e84073230 0x7f1e84075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.214+0000 7f1e9fe00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e84073230 0x7f1e84075740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80b1930 tx=0x7f1e90009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.214+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e9815cdb0 con 0x7f1ea806ce90 2024-09-17T05:08:38.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1ea80b13a0 con 0x7f1ea806ce90 2024-09-17T05:08:38.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1ea4c00640 1 -- 172.21.15.16:0/190751621 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f1e98016410 con 0x7f1ea806ce90 2024-09-17T05:08:38.406 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:08:38.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e84073230 msgr2=0x7f1e84075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e84073230 0x7f1e84075740 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80b1930 tx=0x7f1e90009040 comp rx=0 tx=0).stop 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 msgr2=0x7f1ea80b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80b2f50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009790 tx=0x7f1e980a5710 comp rx=0 tx=0).stop 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 shutdown_connections 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e84073230 0x7f1e84075740 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1ea80cac90 0x7f1ea815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ea8074be0 0x7f1ea8145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 --2- 172.21.15.16:0/190751621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea806ce90 0x7f1ea80b2f50 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 >> 172.21.15.16:0/190751621 conn(0x7f1ea80b2250 msgr2=0x7f1ea80afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 shutdown_connections 2024-09-17T05:08:38.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.406+0000 7f1eaee00640 1 -- 172.21.15.16:0/190751621 wait complete. 2024-09-17T05:08:38.428 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:08:38.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:38.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:08:38.606 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:08:38.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:38.606 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:08:38.615 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:08:38.616 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:38.616 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:38.625 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:38.625 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:38.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:08:38.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 -- 172.21.15.16:0/4111005758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc15f6f0 msgr2=0x7f04cc0ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 --2- 172.21.15.16:0/4111005758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc15f6f0 0x7f04cc0ceb10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0ce2c0 tx=0x7f04c401cda0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 -- 172.21.15.16:0/4111005758 shutdown_connections 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 --2- 172.21.15.16:0/4111005758 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc15f6f0 0x7f04cc0ceb10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 --2- 172.21.15.16:0/4111005758 >> v2:172.21.15.104:3300/0 conn(0x7f04cc074e30 0x7f04cc0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 -- 172.21.15.16:0/4111005758 >> 172.21.15.16:0/4111005758 conn(0x7f04cc0b2250 msgr2=0x7f04cc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 -- 172.21.15.16:0/4111005758 shutdown_connections 2024-09-17T05:08:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.842+0000 7f04d4600640 1 -- 172.21.15.16:0/4111005758 wait complete. 2024-09-17T05:08:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 Processor -- start 2024-09-17T05:08:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 -- start start 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04cc074a20 0x7f04cc15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc074e30 0x7f04cc15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 0x7f04cc1fab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc168f30 con 0x7f04cc15f6f0 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc168db0 con 0x7f04cc074a20 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f04cc1690b0 con 0x7f04cc074e30 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 0x7f04cc1fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 0x7f04cc1fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55812/0 (socket says 172.21.15.16:55812) 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 -- 172.21.15.16:0/1735201474 learned_addr learned my addr 172.21.15.16:0/1735201474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d1c00640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04cc074a20 0x7f04cc15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d1200640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc074e30 0x7f04cc15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 -- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc074e30 msgr2=0x7f04cc15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc074e30 0x7f04cc15df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 -- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04cc074a20 msgr2=0x7f04cc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04cc074a20 0x7f04cc15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 -- 172.21.15.16:0/1735201474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc15e920 con 0x7f04cc15f6f0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d2600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 0x7f04cc1fab60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009bd0 tx=0x7f04c40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40bb070 con 0x7f04cc15f6f0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04c4004c50 con 0x7f04cc15f6f0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc15cc00 con 0x7f04cc15f6f0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b2040 con 0x7f04cc15f6f0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.846+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc15d1c0 con 0x7f04cc15f6f0 2024-09-17T05:08:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.850+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04cc15e690 con 0x7f04cc15f6f0 2024-09-17T05:08:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.850+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04c40be070 con 0x7f04cc15f6f0 2024-09-17T05:08:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.850+0000 7f04c9600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04ac073200 0x7f04ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.850+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f04c412de60 con 0x7f04cc15f6f0 2024-09-17T05:08:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.850+0000 7f04d1c00640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04ac073200 0x7f04ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.850+0000 7f04d1c00640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04ac073200 0x7f04ac075710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f04c0007fd0 tx=0x7f04c0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:38.854+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f04c4016990 con 0x7f04cc15f6f0 2024-09-17T05:08:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.042+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04cc0c6b10 con 0x7f04cc15f6f0 2024-09-17T05:08:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.042+0000 7f04c9600640 1 -- 172.21.15.16:0/1735201474 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) ==== 80+0+0 (secure 0 0 0) 0x7f04c4004e20 con 0x7f04cc15f6f0 2024-09-17T05:08:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:08:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04ac073200 msgr2=0x7f04ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04ac073200 0x7f04ac075710 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f04c0007fd0 tx=0x7f04c0009040 comp rx=0 tx=0).stop 2024-09-17T05:08:39.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 msgr2=0x7f04cc1fab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 0x7f04cc1fab60 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009bd0 tx=0x7f04c40092c0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 shutdown_connections 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04ac073200 0x7f04ac075710 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04cc15f6f0 0x7f04cc1fab60 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04cc074e30 0x7f04cc15df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 --2- 172.21.15.16:0/1735201474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04cc074a20 0x7f04cc15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 >> 172.21.15.16:0/1735201474 conn(0x7f04cc0b2250 msgr2=0x7f04cc0cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 shutdown_connections 2024-09-17T05:08:39.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.046+0000 7f04d4600640 1 -- 172.21.15.16:0/1735201474 wait complete. 2024-09-17T05:08:39.065 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:08:39.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:39.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:08:39.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:08:39.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:08:39.092 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:08:39.144 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:08:39.145 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:39.145 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:39.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:08:39.154 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:08:39.154 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:39.154 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:39.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:08:39.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:39.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 -- 172.21.15.16:0/125391702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100d1580 msgr2=0x7f51100d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 --2- 172.21.15.16:0/125391702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100d1580 0x7f51100d1990 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009900 tx=0x7f50fc01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 -- 172.21.15.16:0/125391702 shutdown_connections 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 --2- 172.21.15.16:0/125391702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100d1580 0x7f51100d1990 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 --2- 172.21.15.16:0/125391702 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51100cc8c0 0x7f511015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 -- 172.21.15.16:0/125391702 >> 172.21.15.16:0/125391702 conn(0x7f51100d2270 msgr2=0x7f51100d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 -- 172.21.15.16:0/125391702 shutdown_connections 2024-09-17T05:08:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.354+0000 7f5118200640 1 -- 172.21.15.16:0/125391702 wait complete. 2024-09-17T05:08:39.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 Processor -- start 2024-09-17T05:08:39.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 -- start start 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 0x7f51101fd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51100d0010 0x7f511006e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51100d1580 0x7f5110151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f511016a4d0 con 0x7f51100cc8c0 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f511016a350 con 0x7f51100d1580 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f511016a650 con 0x7f51100d0010 2024-09-17T05:08:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 0x7f51101fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 0x7f51101fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55838/0 (socket says 172.21.15.16:55838) 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 -- 172.21.15.16:0/3366018286 learned_addr learned my addr 172.21.15.16:0/3366018286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5114e00640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51100d0010 0x7f511006e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5116200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51100d1580 0x7f5110151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 -- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51100d0010 msgr2=0x7f511006e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51100d0010 0x7f511006e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 -- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51100d1580 msgr2=0x7f5110151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51100d1580 0x7f5110151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 -- 172.21.15.16:0/3366018286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101fcac0 con 0x7f51100cc8c0 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5115800640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 0x7f51101fd560 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f510400daf0 tx=0x7f510400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104010460 con 0x7f51100cc8c0 2024-09-17T05:08:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51040106f0 con 0x7f51100cc8c0 2024-09-17T05:08:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104010b40 con 0x7f51100cc8c0 2024-09-17T05:08:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511006f440 con 0x7f51100cc8c0 2024-09-17T05:08:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.358+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51101fd040 con 0x7f51100cc8c0 2024-09-17T05:08:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.362+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f511015cab0 con 0x7f51100cc8c0 2024-09-17T05:08:39.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.362+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5104010ce0 con 0x7f51100cc8c0 2024-09-17T05:08:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.362+0000 7f510d600640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50f0073200 0x7f50f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.362+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f5104094cc0 con 0x7f51100cc8c0 2024-09-17T05:08:39.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.362+0000 7f5114e00640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50f0073200 0x7f50f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.362+0000 7f5114e00640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50f0073200 0x7f50f0075710 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0af000 tx=0x7f50fc01b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:39.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.366+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f510409b080 con 0x7f51100cc8c0 2024-09-17T05:08:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51101549d0 con 0x7f51100cc8c0 2024-09-17T05:08:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f510d600640 1 -- 172.21.15.16:0/3366018286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5225 (secure 0 0 0) 0x7f5104017020 con 0x7f51100cc8c0 2024-09-17T05:08:39.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"btime":"2024-09-17T05:08:32:644988+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:31.802627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 77 2024-09-17T05:08:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50f0073200 msgr2=0x7f50f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50f0073200 0x7f50f0075710 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0af000 tx=0x7f50fc01b120 comp rx=0 tx=0).stop 2024-09-17T05:08:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 msgr2=0x7f51101fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 0x7f51101fd560 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f510400daf0 tx=0x7f510400b260 comp rx=0 tx=0).stop 2024-09-17T05:08:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 shutdown_connections 2024-09-17T05:08:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f50f0073200 0x7f50f0075710 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51100d1580 0x7f5110151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51100d0010 0x7f511006e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.578+0000 7f5118200640 1 --2- 172.21.15.16:0/3366018286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51100cc8c0 0x7f51101fd560 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.582+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 >> 172.21.15.16:0/3366018286 conn(0x7f51100d2270 msgr2=0x7f5110154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.582+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 shutdown_connections 2024-09-17T05:08:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.582+0000 7f5118200640 1 -- 172.21.15.16:0/3366018286 wait complete. 2024-09-17T05:08:39.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 -- 172.21.15.16:0/2587085330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980c9440 msgr2=0x7fee980c9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 --2- 172.21.15.16:0/2587085330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980c9440 0x7fee980c9850 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fee980c92f0 tx=0x7fee9001cb80 comp rx=0 tx=0).stop 2024-09-17T05:08:39.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 -- 172.21.15.16:0/2587085330 shutdown_connections 2024-09-17T05:08:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 --2- 172.21.15.16:0/2587085330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980c9440 0x7fee980c9850 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 --2- 172.21.15.16:0/2587085330 >> v2:172.21.15.104:3300/0 conn(0x7fee980cc8c0 0x7fee9815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 -- 172.21.15.16:0/2587085330 >> 172.21.15.16:0/2587085330 conn(0x7fee980d2270 msgr2=0x7fee980d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 -- 172.21.15.16:0/2587085330 shutdown_connections 2024-09-17T05:08:39.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.758+0000 7fee9d400640 1 -- 172.21.15.16:0/2587085330 wait complete. 2024-09-17T05:08:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 Processor -- start 2024-09-17T05:08:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 -- start start 2024-09-17T05:08:39.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee980c9440 0x7fee981513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 0x7fee9814d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee980d1580 0x7fee9814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee98169eb0 con 0x7fee980cc8c0 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee98169d30 con 0x7fee980d1580 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.762+0000 7fee9d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fee9816a030 con 0x7fee980c9440 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 0x7fee9814d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 0x7fee9814d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55864/0 (socket says 172.21.15.16:55864) 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 -- 172.21.15.16:0/3789536922 learned_addr learned my addr 172.21.15.16:0/3789536922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee97200640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee980d1580 0x7fee9814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee96800640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee980c9440 0x7fee981513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 -- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee980c9440 msgr2=0x7fee981513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee980c9440 0x7fee981513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 -- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee980d1580 msgr2=0x7fee9814dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee980d1580 0x7fee9814dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 -- 172.21.15.16:0/3789536922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee98155af0 con 0x7fee980cc8c0 2024-09-17T05:08:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8de00640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 0x7fee9814d6c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c3c0 tx=0x7fee8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80011790 con 0x7fee980cc8c0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee80017070 con 0x7fee980cc8c0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80011d40 con 0x7fee980cc8c0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee98155dd0 con 0x7fee980cc8c0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee98156390 con 0x7fee980cc8c0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee980c9850 con 0x7fee980cc8c0 2024-09-17T05:08:39.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.766+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fee8001c460 con 0x7fee980cc8c0 2024-09-17T05:08:39.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.770+0000 7fee8f400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee74073200 0x7fee74075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.770+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7fee80094110 con 0x7fee980cc8c0 2024-09-17T05:08:39.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.770+0000 7fee96800640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee74073200 0x7fee74075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:39.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.770+0000 7fee96800640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee74073200 0x7fee74075710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fee980b1bc0 tx=0x7fee8800b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:39.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.774+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fee8005e8b0 con 0x7fee980cc8c0 2024-09-17T05:08:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.990+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9815b4d0 con 0x7fee980cc8c0 2024-09-17T05:08:39.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.990+0000 7fee8f400640 1 -- 172.21.15.16:0/3789536922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5225 (secure 0 0 0) 0x7fee800632e0 con 0x7fee980cc8c0 2024-09-17T05:08:39.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:39.990 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"btime":"2024-09-17T05:08:32:644988+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:31.802627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:39.991 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 77 2024-09-17T05:08:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee74073200 msgr2=0x7fee74075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee74073200 0x7fee74075710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fee980b1bc0 tx=0x7fee8800b2e0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 msgr2=0x7fee9814d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 0x7fee9814d6c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c3c0 tx=0x7fee8000c880 comp rx=0 tx=0).stop 2024-09-17T05:08:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 shutdown_connections 2024-09-17T05:08:39.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fee74073200 0x7fee74075710 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee980d1580 0x7fee9814dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee980cc8c0 0x7fee9814d6c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 --2- 172.21.15.16:0/3789536922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fee980c9440 0x7fee981513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 >> 172.21.15.16:0/3789536922 conn(0x7fee980d2270 msgr2=0x7fee9815c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:39.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 shutdown_connections 2024-09-17T05:08:39.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:39.994+0000 7fee9d400640 1 -- 172.21.15.16:0/3789536922 wait complete. 2024-09-17T05:08:40.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3558c00640 1 -- 172.21.15.16:0/104964965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538003a10 msgr2=0x7f3538003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3558c00640 1 --2- 172.21.15.16:0/104964965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3538003a10 0x7f3538003e80 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f35540ccaa0 tx=0x7f354401cf60 comp rx=0 tx=0).stop 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3552800640 1 -- 172.21.15.16:0/104964965 reap_dead start 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3558c00640 1 -- 172.21.15.16:0/104964965 shutdown_connections 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3558c00640 1 -- 172.21.15.16:0/104964965 >> 172.21.15.16:0/104964965 conn(0x7f355406f900 msgr2=0x7f35540b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3558c00640 1 -- 172.21.15.16:0/104964965 shutdown_connections 2024-09-17T05:08:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.170+0000 7f3558c00640 1 -- 172.21.15.16:0/104964965 wait complete. 2024-09-17T05:08:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 Processor -- start 2024-09-17T05:08:40.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 -- start start 2024-09-17T05:08:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3538003a10 0x7f355413d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 0x7f35541fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f355413c3a0 0x7f355413c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35541680f0 con 0x7f35541fa820 2024-09-17T05:08:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3554167f70 con 0x7f355413c3a0 2024-09-17T05:08:40.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3558c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3554168270 con 0x7f3538003a10 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3551400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 0x7f35541fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3551400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 0x7f35541fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55900/0 (socket says 172.21.15.16:55900) 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3551400640 1 -- 172.21.15.16:0/3477937549 learned_addr learned my addr 172.21.15.16:0/3477937549 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3552800640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f355413c3a0 0x7f355413c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.174+0000 7f3551e00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3538003a10 0x7f355413d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3551400640 1 -- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3538003a10 msgr2=0x7f355413d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3551400640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3538003a10 0x7f355413d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3551400640 1 -- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f355413c3a0 msgr2=0x7f355413c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3551400640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f355413c3a0 0x7f355413c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3551400640 1 -- 172.21.15.16:0/3477937549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3554140a70 con 0x7f35541fa820 2024-09-17T05:08:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3551400640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 0x7f35541fac30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3540011800 tx=0x7f3540011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354001f070 con 0x7f35541fa820 2024-09-17T05:08:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f354000fdc0 con 0x7f35541fa820 2024-09-17T05:08:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355413e490 con 0x7f35541fa820 2024-09-17T05:08:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35400126b0 con 0x7f35541fa820 2024-09-17T05:08:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355413ea50 con 0x7f35541fa820 2024-09-17T05:08:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35540b42e0 con 0x7f35541fa820 2024-09-17T05:08:40.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.178+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3540022070 con 0x7f35541fa820 2024-09-17T05:08:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.182+0000 7f354a000640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3534073200 0x7f3534075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.182+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f3540094d50 con 0x7f35541fa820 2024-09-17T05:08:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.182+0000 7f3551e00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3534073200 0x7f3534075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.182+0000 7f3551e00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3534073200 0x7f3534075710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f35440ac000 tx=0x7f3544019da0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:40.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.186+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f354005f580 con 0x7f35541fa820 2024-09-17T05:08:40.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.362+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f355413e280 con 0x7f35541fa820 2024-09-17T05:08:40.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.362+0000 7f354a000640 1 -- 172.21.15.16:0/3477937549 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17004 (secure 0 0 0) 0x7f3540063fb0 con 0x7f35541fa820 2024-09-17T05:08:40.362 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:40.362 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":113,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:08.760898+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:04.918539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:06.167190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:40.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3534073200 msgr2=0x7f3534075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3534073200 0x7f3534075710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f35440ac000 tx=0x7f3544019da0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 msgr2=0x7f35541fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 0x7f35541fac30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3540011800 tx=0x7f3540011c00 comp rx=0 tx=0).stop 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 shutdown_connections 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3534073200 0x7f3534075710 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f355413c3a0 0x7f355413c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35541fa820 0x7f35541fac30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 --2- 172.21.15.16:0/3477937549 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3538003a10 0x7f355413d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 >> 172.21.15.16:0/3477937549 conn(0x7f355406f900 msgr2=0x7f3554166b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 shutdown_connections 2024-09-17T05:08:40.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.366+0000 7f3558c00640 1 -- 172.21.15.16:0/3477937549 wait complete. 2024-09-17T05:08:40.385 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:08:40.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 -- 172.21.15.16:0/3574175384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac0b2f50 msgr2=0x7f78ac15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 --2- 172.21.15.16:0/3574175384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac0b2f50 0x7f78ac15bb60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f789c009840 tx=0x7f789c01c790 comp rx=0 tx=0).stop 2024-09-17T05:08:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 -- 172.21.15.16:0/3574175384 shutdown_connections 2024-09-17T05:08:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 --2- 172.21.15.16:0/3574175384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78ac15c0a0 0x7f78ac15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 --2- 172.21.15.16:0/3574175384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac0b2f50 0x7f78ac15bb60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 -- 172.21.15.16:0/3574175384 >> 172.21.15.16:0/3574175384 conn(0x7f78ac06f900 msgr2=0x7f78ac0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:40.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 -- 172.21.15.16:0/3574175384 shutdown_connections 2024-09-17T05:08:40.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 -- 172.21.15.16:0/3574175384 wait complete. 2024-09-17T05:08:40.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 Processor -- start 2024-09-17T05:08:40.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.594+0000 7f78b1400640 1 -- start start 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 0x7f78ac1618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78ac15c0a0 0x7f78ac141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac15f150 0x7f78ac1623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac169150 con 0x7f78ac0b2f50 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac168fd0 con 0x7f78ac15f150 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac1692d0 con 0x7f78ac15c0a0 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 0x7f78ac1618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 0x7f78ac1618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55908/0 (socket says 172.21.15.16:55908) 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 -- 172.21.15.16:0/2129116386 learned_addr learned my addr 172.21.15.16:0/2129116386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78a1e00640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78ac15c0a0 0x7f78ac141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78ab200640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac15f150 0x7f78ac1623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 -- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78ac15c0a0 msgr2=0x7f78ac141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78ac15c0a0 0x7f78ac141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 -- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac15f150 msgr2=0x7f78ac1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac15f150 0x7f78ac1623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 -- 172.21.15.16:0/2129116386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac06e5d0 con 0x7f78ac0b2f50 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78aa800640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 0x7f78ac1618c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f789c01ae40 tx=0x7f789c0a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0bb070 con 0x7f78ac0b2f50 2024-09-17T05:08:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f789c004470 con 0x7f78ac0b2f50 2024-09-17T05:08:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c0b2040 con 0x7f78ac0b2f50 2024-09-17T05:08:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac1628f0 con 0x7f78ac0b2f50 2024-09-17T05:08:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac160b00 con 0x7f78ac0b2f50 2024-09-17T05:08:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.598+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac15bb60 con 0x7f78ac0b2f50 2024-09-17T05:08:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.602+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f789c0be070 con 0x7f78ac0b2f50 2024-09-17T05:08:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.602+0000 7f78a3400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7888073200 0x7f7888075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.602+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f789c12daf0 con 0x7f78ac0b2f50 2024-09-17T05:08:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.602+0000 7f78a1e00640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7888073200 0x7f7888075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:40.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.602+0000 7f78a1e00640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7888073200 0x7f7888075710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0ce740 tx=0x7f78940091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.606+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f789c016410 con 0x7f78ac0b2f50 2024-09-17T05:08:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.814+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78ac155d40 con 0x7f78ac0b2f50 2024-09-17T05:08:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.814+0000 7f78a3400640 1 -- 172.21.15.16:0/2129116386 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5226 (secure 0 0 0) 0x7f789c0f88f0 con 0x7f78ac0b2f50 2024-09-17T05:08:40.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:40.816 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"btime":"2024-09-17T05:08:40:674867+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:39.802535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":11,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:40.816 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7888073200 msgr2=0x7f7888075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7888073200 0x7f7888075710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f78ac0ce740 tx=0x7f78940091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 msgr2=0x7f78ac1618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 0x7f78ac1618c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f789c01ae40 tx=0x7f789c0a56f0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 shutdown_connections 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7888073200 0x7f7888075710 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78ac15f150 0x7f78ac1623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f78ac15c0a0 0x7f78ac141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 --2- 172.21.15.16:0/2129116386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78ac0b2f50 0x7f78ac1618c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 >> 172.21.15.16:0/2129116386 conn(0x7f78ac06f900 msgr2=0x7f78ac0d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 shutdown_connections 2024-09-17T05:08:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.818+0000 7f78b1400640 1 -- 172.21.15.16:0/2129116386 wait complete. 2024-09-17T05:08:40.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 -- 172.21.15.16:0/3380941764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de015c270 msgr2=0x7f8de015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 --2- 172.21.15.16:0/3380941764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de015c270 0x7f8de015c680 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8de015cc70 tx=0x7f8dc801c830 comp rx=0 tx=0).stop 2024-09-17T05:08:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 -- 172.21.15.16:0/3380941764 shutdown_connections 2024-09-17T05:08:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 --2- 172.21.15.16:0/3380941764 >> v2:172.21.15.104:3300/0 conn(0x7f8de0073240 0x7f8de0073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 --2- 172.21.15.16:0/3380941764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de015c270 0x7f8de015c680 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 -- 172.21.15.16:0/3380941764 >> 172.21.15.16:0/3380941764 conn(0x7f8de006f900 msgr2=0x7f8de00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 -- 172.21.15.16:0/3380941764 shutdown_connections 2024-09-17T05:08:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 -- 172.21.15.16:0/3380941764 wait complete. 2024-09-17T05:08:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 Processor -- start 2024-09-17T05:08:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:40.998+0000 7f8de6000640 1 -- start start 2024-09-17T05:08:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de0073240 0x7f8de0165b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 0x7f8de01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de015c270 0x7f8de01f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0168cc0 con 0x7f8de00b2f50 2024-09-17T05:08:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0168b40 con 0x7f8de0073240 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8de0168e40 con 0x7f8de015c270 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 0x7f8de01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 0x7f8de01536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55924/0 (socket says 172.21.15.16:55924) 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 -- 172.21.15.16:0/1236487079 learned_addr learned my addr 172.21.15.16:0/1236487079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8ddf400640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de0073240 0x7f8de0165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8ddfe00640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de015c270 0x7f8de01f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 -- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de015c270 msgr2=0x7f8de01f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de015c270 0x7f8de01f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 -- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de0073240 msgr2=0x7f8de0165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de0073240 0x7f8de0165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 -- 172.21.15.16:0/1236487079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de01660b0 con 0x7f8de00b2f50 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd6a00640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 0x7f8de01536d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80b0040 tx=0x7f8dc80a5830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80b2040 con 0x7f8de00b2f50 2024-09-17T05:08:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dc80044b0 con 0x7f8de00b2f50 2024-09-17T05:08:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8005290 con 0x7f8de00b2f50 2024-09-17T05:08:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de0164db0 con 0x7f8de00b2f50 2024-09-17T05:08:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de0165370 con 0x7f8de00b2f50 2024-09-17T05:08:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.002+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0073650 con 0x7f8de00b2f50 2024-09-17T05:08:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.006+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8dc8005430 con 0x7f8de00b2f50 2024-09-17T05:08:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.006+0000 7f8dd7e00640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dbc073230 0x7f8dbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.006+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f8dc812da10 con 0x7f8de00b2f50 2024-09-17T05:08:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.006+0000 7f8ddf400640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dbc073230 0x7f8dbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.006+0000 7f8ddf400640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dbc073230 0x7f8dbc075740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00098e0 tx=0x7f8dd0009060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.010+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8dc80164b0 con 0x7f8de00b2f50 2024-09-17T05:08:41.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8de0142780 con 0x7f8de00b2f50 2024-09-17T05:08:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8dd7e00640 1 -- 172.21.15.16:0/1236487079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5226 (secure 0 0 0) 0x7f8dc80f8350 con 0x7f8de00b2f50 2024-09-17T05:08:41.218 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:41.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":78,"btime":"2024-09-17T05:08:40:674867+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:07.160604+0000","modified":"2024-09-17T05:08:39.802535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7332},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7332":{"gid":7332,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":11,"addr":"172.21.15.16:6835/759404348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":759404348},{"type":"v1","addr":"172.21.15.16:6835","nonce":759404348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7332,"qdb_cluster":[7332]},"id":8}]} 2024-09-17T05:08:41.218 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 78 2024-09-17T05:08:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dbc073230 msgr2=0x7f8dbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dbc073230 0x7f8dbc075740 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00098e0 tx=0x7f8dd0009060 comp rx=0 tx=0).stop 2024-09-17T05:08:41.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 msgr2=0x7f8de01536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 0x7f8de01536d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80b0040 tx=0x7f8dc80a5830 comp rx=0 tx=0).stop 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 shutdown_connections 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8dbc073230 0x7f8dbc075740 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8de015c270 0x7f8de01f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8de00b2f50 0x7f8de01536d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.218+0000 7f8de6000640 1 --2- 172.21.15.16:0/1236487079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8de0073240 0x7f8de0165b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.222+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 >> 172.21.15.16:0/1236487079 conn(0x7f8de006f900 msgr2=0x7f8de01519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.222+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 shutdown_connections 2024-09-17T05:08:41.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.222+0000 7f8de6000640 1 -- 172.21.15.16:0/1236487079 wait complete. 2024-09-17T05:08:41.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 -- 172.21.15.16:0/273306428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0b40a0 msgr2=0x7f4bac0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/273306428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0b40a0 0x7f4bac0b44b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009840 tx=0x7f4ba401c820 comp rx=0 tx=0).stop 2024-09-17T05:08:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 -- 172.21.15.16:0/273306428 shutdown_connections 2024-09-17T05:08:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/273306428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0b40a0 0x7f4bac0b44b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 -- 172.21.15.16:0/273306428 >> 172.21.15.16:0/273306428 conn(0x7f4bac06f900 msgr2=0x7f4bac0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 -- 172.21.15.16:0/273306428 shutdown_connections 2024-09-17T05:08:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.398+0000 7f4bb2c00640 1 -- 172.21.15.16:0/273306428 wait complete. 2024-09-17T05:08:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 Processor -- start 2024-09-17T05:08:41.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 -- start start 2024-09-17T05:08:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bac0b40a0 0x7f4bac13db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 0x7f4bac13faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bac15d600 0x7f4bac15ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac16b0c0 con 0x7f4bac0cf560 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac16af40 con 0x7f4bac15d600 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac16b240 con 0x7f4bac0b40a0 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 0x7f4bac13faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 0x7f4bac13faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55944/0 (socket says 172.21.15.16:55944) 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 -- 172.21.15.16:0/2477780840 learned_addr learned my addr 172.21.15.16:0/2477780840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4babe00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bac0b40a0 0x7f4bac13db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4bb0c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bac15d600 0x7f4bac15ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 -- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bac0b40a0 msgr2=0x7f4bac13db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bac0b40a0 0x7f4bac13db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 -- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bac15d600 msgr2=0x7f4bac15ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bac15d600 0x7f4bac15ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 -- 172.21.15.16:0/2477780840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac0741b0 con 0x7f4bac0cf560 2024-09-17T05:08:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4babe00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bac0b40a0 0x7f4bac13db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba3400640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 0x7f4bac13faa0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400ba80 tx=0x7f4b9400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400cc00 con 0x7f4bac0cf560 2024-09-17T05:08:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b94003980 con 0x7f4bac0cf560 2024-09-17T05:08:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.402+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94003e10 con 0x7f4bac0cf560 2024-09-17T05:08:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac140160 con 0x7f4bac0cf560 2024-09-17T05:08:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac1fc8a0 con 0x7f4bac0cf560 2024-09-17T05:08:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac1fc3b0 con 0x7f4bac0cf560 2024-09-17T05:08:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b94006d10 con 0x7f4bac0cf560 2024-09-17T05:08:41.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4ba8c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b88073200 0x7f4b88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f4b94093fc0 con 0x7f4bac0cf560 2024-09-17T05:08:41.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.406+0000 7f4babe00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b88073200 0x7f4b88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.410+0000 7f4babe00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b88073200 0x7f4b88075710 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0046c0 tx=0x7f4b9c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:41.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.410+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b9405e830 con 0x7f4bac0cf560 2024-09-17T05:08:41.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.586+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bac0b44b0 con 0x7f4bac0cf560 2024-09-17T05:08:41.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.586+0000 7f4ba8c00640 1 -- 172.21.15.16:0/2477780840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17004 (secure 0 0 0) 0x7f4b94063260 con 0x7f4bac0cf560 2024-09-17T05:08:41.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:41.587 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":113,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:08.760898+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:04.918539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:06.167190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b88073200 msgr2=0x7f4b88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b88073200 0x7f4b88075710 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0046c0 tx=0x7f4b9c009040 comp rx=0 tx=0).stop 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 msgr2=0x7f4bac13faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 0x7f4bac13faa0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400ba80 tx=0x7f4b9400bf40 comp rx=0 tx=0).stop 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 shutdown_connections 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b88073200 0x7f4b88075710 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bac15d600 0x7f4bac15ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bac0cf560 0x7f4bac13faa0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 --2- 172.21.15.16:0/2477780840 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4bac0b40a0 0x7f4bac13db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 >> 172.21.15.16:0/2477780840 conn(0x7f4bac06f900 msgr2=0x7f4bac0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 shutdown_connections 2024-09-17T05:08:41.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.590+0000 7f4bb2c00640 1 -- 172.21.15.16:0/2477780840 wait complete. 2024-09-17T05:08:41.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:08:41.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0f400640 1 -- 172.21.15.16:0/1141220441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0004810 msgr2=0x7f6df0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0f400640 1 --2- 172.21.15.16:0/1141220441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0004810 0x7f6df0004c20 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6df4001830 tx=0x7f6df401f010 comp rx=0 tx=0).stop 2024-09-17T05:08:41.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0d400640 1 -- 172.21.15.16:0/1141220441 reap_dead start 2024-09-17T05:08:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0f400640 1 -- 172.21.15.16:0/1141220441 shutdown_connections 2024-09-17T05:08:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0f400640 1 -- 172.21.15.16:0/1141220441 >> 172.21.15.16:0/1141220441 conn(0x7f6e080b2250 msgr2=0x7f6e080d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0f400640 1 -- 172.21.15.16:0/1141220441 shutdown_connections 2024-09-17T05:08:41.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.770+0000 7f6e0f400640 1 -- 172.21.15.16:0/1141220441 wait complete. 2024-09-17T05:08:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 Processor -- start 2024-09-17T05:08:41.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 -- start start 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0004810 0x7f6e08146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e081472f0 0x7f6e080adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e080ae320 0x7f6e081f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e080749d0 con 0x7f6df0004810 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08074850 con 0x7f6e081472f0 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0f400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6e08074b50 con 0x7f6e080ae320 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0004810 0x7f6e08146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0004810 0x7f6e08146db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55972/0 (socket says 172.21.15.16:55972) 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.774+0000 7f6e0ca00640 1 -- 172.21.15.16:0/1793067941 learned_addr learned my addr 172.21.15.16:0/1793067941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0ca00640 1 -- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e080ae320 msgr2=0x7f6e081f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0ca00640 1 --2- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e080ae320 0x7f6e081f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0ca00640 1 -- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e081472f0 msgr2=0x7f6e080adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0ca00640 1 --2- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e081472f0 0x7f6e080adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0ca00640 1 -- 172.21.15.16:0/1793067941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e081f3000 con 0x7f6df0004810 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0ca00640 1 --2- 172.21.15.16:0/1793067941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6df0004810 0x7f6e08146db0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc006450 tx=0x7f6dfc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc015070 con 0x7f6df0004810 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dfc011860 con 0x7f6df0004810 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011cb0 con 0x7f6df0004810 2024-09-17T05:08:41.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0f400640 1 -- 172.21.15.16:0/1793067941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e080aebf0 con 0x7f6df0004810 2024-09-17T05:08:41.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0f400640 1 -- 172.21.15.16:0/1793067941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e080af1b0 con 0x7f6df0004810 2024-09-17T05:08:41.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e0f400640 1 -- 172.21.15.16:0/1793067941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e08073f30 con 0x7f6df0004810 2024-09-17T05:08:41.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.778+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6dfc018070 con 0x7f6df0004810 2024-09-17T05:08:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.782+0000 7f6e04c00640 1 --2- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6de8073200 0x7f6de8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:41.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.782+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f6dfc094fa0 con 0x7f6df0004810 2024-09-17T05:08:41.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.782+0000 7f6e07e00640 1 --2- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6de8073200 0x7f6de8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:41.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.782+0000 7f6e07e00640 1 --2- 172.21.15.16:0/1793067941 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6de8073200 0x7f6de8075710 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e08072d60 tx=0x7f6df0005840 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:41.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.786+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6dfc05feb0 con 0x7f6df0004810 2024-09-17T05:08:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.994+0000 7f6e0f400640 1 -- 172.21.15.16:0/1793067941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6e08074420 con 0x7f6df0004810 2024-09-17T05:08:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:41.994+0000 7f6e04c00640 1 -- 172.21.15.16:0/1793067941 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v78) ==== 316+0+0 (secure 0 0 0) 0x7f6dfc05f7b0 con 0x7f6df0004810 2024-09-17T05:08:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-17T05:08:42.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:08:42.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-17T05:08:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773fe00640 1 -- 172.21.15.16:0/1310376533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 msgr2=0x7f7720003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773fe00640 1 --2- 172.21.15.16:0/1310376533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7720003ea0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c310 tx=0x7f772c01cf80 comp rx=0 tx=0).stop 2024-09-17T05:08:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773de00640 1 -- 172.21.15.16:0/1310376533 reap_dead start 2024-09-17T05:08:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773fe00640 1 -- 172.21.15.16:0/1310376533 shutdown_connections 2024-09-17T05:08:42.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773fe00640 1 -- 172.21.15.16:0/1310376533 >> 172.21.15.16:0/1310376533 conn(0x7f77380b2250 msgr2=0x7f77380d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773fe00640 1 -- 172.21.15.16:0/1310376533 shutdown_connections 2024-09-17T05:08:42.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.166+0000 7f773fe00640 1 -- 172.21.15.16:0/1310376533 wait complete. 2024-09-17T05:08:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 Processor -- start 2024-09-17T05:08:42.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 -- start start 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7738154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77381557e0 0x7f77380c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7738155190 0x7f77380c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f773806d220 con 0x7f7720003a10 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f773806d0a0 con 0x7f77381557e0 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f773806d3a0 con 0x7f7738155190 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7738154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7738154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55988/0 (socket says 172.21.15.16:55988) 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 -- 172.21.15.16:0/2423379084 learned_addr learned my addr 172.21.15.16:0/2423379084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:42.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773ca00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77381557e0 0x7f77380c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773de00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7738155190 0x7f77380c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 -- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7738155190 msgr2=0x7f77380c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7738155190 0x7f77380c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 -- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77381557e0 msgr2=0x7f77380c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77381557e0 0x7f77380c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 -- 172.21.15.16:0/2423379084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77380c9300 con 0x7f7720003a10 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773d400640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7738154b80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f772c01ca40 tx=0x7f772c006d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c006090 con 0x7f7720003a10 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772c006230 con 0x7f7720003a10 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c003620 con 0x7f7720003a10 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77380c9d40 con 0x7f7720003a10 2024-09-17T05:08:42.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.170+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77380c4940 con 0x7f7720003a10 2024-09-17T05:08:42.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.174+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77380ce5d0 con 0x7f7720003a10 2024-09-17T05:08:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.174+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f772c0b6030 con 0x7f7720003a10 2024-09-17T05:08:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.174+0000 7f7735600640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7718073200 0x7f7718075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.174+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 6101+0+0 (secure 0 0 0) 0x7f772c12dd70 con 0x7f7720003a10 2024-09-17T05:08:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.174+0000 7f773ca00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7718073200 0x7f7718075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.174+0000 7f773ca00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7718073200 0x7f7718075710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f77380cb960 tx=0x7f77240091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:42.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.178+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f772c016d30 con 0x7f7720003a10 2024-09-17T05:08:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.386+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f77380c4b70 con 0x7f7720003a10 2024-09-17T05:08:42.389 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:08:42.390+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:08:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.390+0000 7f7735600640 1 -- 172.21.15.16:0/2423379084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) ==== 191+0+0 (secure 0 0 0) 0x7f772c006640 con 0x7f7720003a10 2024-09-17T05:08:42.390 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:08:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7718073200 msgr2=0x7f7718075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7718073200 0x7f7718075710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f77380cb960 tx=0x7f77240091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 msgr2=0x7f7738154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7738154b80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f772c01ca40 tx=0x7f772c006d10 comp rx=0 tx=0).stop 2024-09-17T05:08:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 shutdown_connections 2024-09-17T05:08:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7718073200 0x7f7718075710 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7738155190 0x7f77380c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77381557e0 0x7f77380c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 --2- 172.21.15.16:0/2423379084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720003a10 0x7f7738154b80 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 >> 172.21.15.16:0/2423379084 conn(0x7f77380b2250 msgr2=0x7f77380ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 shutdown_connections 2024-09-17T05:08:42.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.394+0000 7f773fe00640 1 -- 172.21.15.16:0/2423379084 wait complete. 2024-09-17T05:08:42.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:08:42.534 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:08:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 -- 172.21.15.16:0/2055834421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00cf680 msgr2=0x7f40f00b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 --2- 172.21.15.16:0/2055834421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00cf680 0x7f40f00b2100 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009bc0 tx=0x7f40ec01ca10 comp rx=0 tx=0).stop 2024-09-17T05:08:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 -- 172.21.15.16:0/2055834421 shutdown_connections 2024-09-17T05:08:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 --2- 172.21.15.16:0/2055834421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00cf680 0x7f40f00b2100 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 --2- 172.21.15.16:0/2055834421 >> v2:172.21.15.104:3300/0 conn(0x7f40f00d02d0 0x7f40f00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 -- 172.21.15.16:0/2055834421 >> 172.21.15.16:0/2055834421 conn(0x7f40f00b42d0 msgr2=0x7f40f00b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 -- 172.21.15.16:0/2055834421 shutdown_connections 2024-09-17T05:08:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.578+0000 7f40f8800640 1 -- 172.21.15.16:0/2055834421 wait complete. 2024-09-17T05:08:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 Processor -- start 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 -- start start 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f00cf680 0x7f40f0150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 0x7f40f00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40f0159f70 0x7f40f00b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0169e20 con 0x7f40f00d02d0 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0169ca0 con 0x7f40f00cf680 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.582+0000 7f40f8800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f40f0169fa0 con 0x7f40f0159f70 2024-09-17T05:08:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 0x7f40f00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 0x7f40f00b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56018/0 (socket says 172.21.15.16:56018) 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 -- 172.21.15.16:0/4007267066 learned_addr learned my addr 172.21.15.16:0/4007267066 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f6800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40f0159f70 0x7f40f00b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5e00640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f00cf680 0x7f40f0150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 -- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40f0159f70 msgr2=0x7f40f00b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40f0159f70 0x7f40f00b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 -- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f00cf680 msgr2=0x7f40f0150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f00cf680 0x7f40f0150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 -- 172.21.15.16:0/4007267066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f0150880 con 0x7f40f00d02d0 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5e00640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f00cf680 0x7f40f0150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f5400640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 0x7f40f00b78b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005bd0 tx=0x7f40e000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0004230 con 0x7f40f00d02d0 2024-09-17T05:08:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40e00044c0 con 0x7f40f00d02d0 2024-09-17T05:08:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0009a80 con 0x7f40f00d02d0 2024-09-17T05:08:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f0158540 con 0x7f40f00d02d0 2024-09-17T05:08:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f006e280 con 0x7f40f00d02d0 2024-09-17T05:08:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f00b2100 con 0x7f40f00d02d0 2024-09-17T05:08:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.586+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f40e0003600 con 0x7f40f00d02d0 2024-09-17T05:08:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.590+0000 7f40ea000640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f40d4073200 0x7f40d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.590+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f40e00947e0 con 0x7f40f00d02d0 2024-09-17T05:08:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.590+0000 7f40f5e00640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f40d4073200 0x7f40d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.590+0000 7f40f5e00640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f40d4073200 0x7f40d4075710 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001400 tx=0x7f40e4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.594+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f40e0017020 con 0x7f40f00d02d0 2024-09-17T05:08:42.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:42.806+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f40f006e940 con 0x7f40f00d02d0 2024-09-17T05:08:43.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40ea000640 1 -- 172.21.15.16:0/4007267066 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) ==== 104+0+0 (secure 0 0 0) 0x7f40e005f060 con 0x7f40f00d02d0 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f40d4073200 msgr2=0x7f40d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f40d4073200 0x7f40d4075710 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f40e4001400 tx=0x7f40e4009040 comp rx=0 tx=0).stop 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 msgr2=0x7f40f00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 0x7f40f00b78b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005bd0 tx=0x7f40e000db90 comp rx=0 tx=0).stop 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 shutdown_connections 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f40d4073200 0x7f40d4075710 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f40f0159f70 0x7f40f00b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f00d02d0 0x7f40f00b78b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 --2- 172.21.15.16:0/4007267066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40f00cf680 0x7f40f0150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 >> 172.21.15.16:0/4007267066 conn(0x7f40f00b42d0 msgr2=0x7f40f00701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 shutdown_connections 2024-09-17T05:08:43.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.394+0000 7f40f8800640 1 -- 172.21.15.16:0/4007267066 wait complete. 2024-09-17T05:08:43.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:08:43.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.582+0000 7f5456000640 1 -- 172.21.15.16:0/2997151221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545006e6b0 msgr2=0x7f5450073be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.582+0000 7f5456000640 1 --2- 172.21.15.16:0/2997151221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545006e6b0 0x7f5450073be0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5450073800 tx=0x7f543801c850 comp rx=0 tx=0).stop 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- 172.21.15.16:0/2997151221 shutdown_connections 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 --2- 172.21.15.16:0/2997151221 >> v2:172.21.15.104:3300/0 conn(0x7f5450074120 0x7f5450074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 --2- 172.21.15.16:0/2997151221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545006e6b0 0x7f5450073be0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- 172.21.15.16:0/2997151221 >> 172.21.15.16:0/2997151221 conn(0x7f54500d0050 msgr2=0x7f54500d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- 172.21.15.16:0/2997151221 shutdown_connections 2024-09-17T05:08:43.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- 172.21.15.16:0/2997151221 wait complete. 2024-09-17T05:08:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 Processor -- start 2024-09-17T05:08:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- start start 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5450074120 0x7f54501515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54500b4620 0x7f545014d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 0x7f545014bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5450168b30 con 0x7f545014b9e0 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5450152840 con 0x7f5450074120 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.586+0000 7f5456000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5450168cb0 con 0x7f54500b4620 2024-09-17T05:08:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 0x7f545014bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 0x7f545014bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56042/0 (socket says 172.21.15.16:56042) 2024-09-17T05:08:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 -- 172.21.15.16:0/554566723 learned_addr learned my addr 172.21.15.16:0/554566723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544f400640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5450074120 0x7f54501515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5447e00640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54500b4620 0x7f545014d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 -- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54500b4620 msgr2=0x7f545014d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54500b4620 0x7f545014d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 -- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5450074120 msgr2=0x7f54501515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5450074120 0x7f54501515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:43.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 -- 172.21.15.16:0/554566723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54501fbf30 con 0x7f545014b9e0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544f400640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5450074120 0x7f54501515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5447e00640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54500b4620 0x7f545014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f544fe00640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 0x7f545014bdf0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f544800e9a0 tx=0x7f544800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448004d30 con 0x7f545014b9e0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f544800cb20 con 0x7f545014b9e0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501fc210 con 0x7f545014b9e0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448005020 con 0x7f545014b9e0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501fc7d0 con 0x7f545014b9e0 2024-09-17T05:08:43.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5450073be0 con 0x7f545014b9e0 2024-09-17T05:08:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.590+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54480051c0 con 0x7f545014b9e0 2024-09-17T05:08:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.594+0000 7f5447400640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f542c073230 0x7f542c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:43.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.594+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6187+0+0 (secure 0 0 0) 0x7f5448094da0 con 0x7f545014b9e0 2024-09-17T05:08:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.594+0000 7f544f400640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f542c073230 0x7f542c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:43.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.594+0000 7f544f400640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f542c073230 0x7f542c075740 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f544000a850 tx=0x7f5440008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:43.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.598+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5448016030 con 0x7f545014b9e0 2024-09-17T05:08:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:43.778+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5450151cb0 con 0x7f545014b9e0 2024-09-17T05:08:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.698+0000 7f5447400640 1 -- 172.21.15.16:0/554566723 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) ==== 181+0+0 (secure 0 0 0) 0x7f544805fb40 con 0x7f545014b9e0 2024-09-17T05:08:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:08:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f542c073230 msgr2=0x7f542c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f542c073230 0x7f542c075740 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f544000a850 tx=0x7f5440008040 comp rx=0 tx=0).stop 2024-09-17T05:08:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 msgr2=0x7f545014bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 0x7f545014bdf0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f544800e9a0 tx=0x7f544800ee60 comp rx=0 tx=0).stop 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 shutdown_connections 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f542c073230 0x7f542c075740 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f545014b9e0 0x7f545014bdf0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54500b4620 0x7f545014d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 --2- 172.21.15.16:0/554566723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5450074120 0x7f54501515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.702+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 >> 172.21.15.16:0/554566723 conn(0x7f54500d0050 msgr2=0x7f545006edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.706+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 shutdown_connections 2024-09-17T05:08:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.706+0000 7f5456000640 1 -- 172.21.15.16:0/554566723 wait complete. 2024-09-17T05:08:44.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:08:44.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 -- 172.21.15.16:0/1335705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be00b81c0 msgr2=0x7f4be00d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 --2- 172.21.15.16:0/1335705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be00b81c0 0x7f4be00d0610 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4be00cfef0 tx=0x7f4bd801cd10 comp rx=0 tx=0).stop 2024-09-17T05:08:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 -- 172.21.15.16:0/1335705831 shutdown_connections 2024-09-17T05:08:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 --2- 172.21.15.16:0/1335705831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be00b81c0 0x7f4be00d0610 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 -- 172.21.15.16:0/1335705831 >> 172.21.15.16:0/1335705831 conn(0x7f4be006ef00 msgr2=0x7f4be006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 -- 172.21.15.16:0/1335705831 shutdown_connections 2024-09-17T05:08:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 -- 172.21.15.16:0/1335705831 wait complete. 2024-09-17T05:08:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 Processor -- start 2024-09-17T05:08:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 -- start start 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4be00b81c0 0x7f4be014f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.902+0000 7f4be6600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be00d0200 0x7f4be015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 0x7f4be015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4be016d670 con 0x7f4be0162de0 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4be016d4f0 con 0x7f4be00d0200 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4be016d7f0 con 0x7f4be00b81c0 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 0x7f4be015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 0x7f4be015dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56066/0 (socket says 172.21.15.16:56066) 2024-09-17T05:08:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 -- 172.21.15.16:0/455406319 learned_addr learned my addr 172.21.15.16:0/455406319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdf400640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4be00b81c0 0x7f4be014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bd7e00640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be00d0200 0x7f4be015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 -- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4be00b81c0 msgr2=0x7f4be014f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4be00b81c0 0x7f4be014f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 -- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be00d0200 msgr2=0x7f4be015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be00d0200 0x7f4be015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 -- 172.21.15.16:0/455406319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be014fb90 con 0x7f4be0162de0 2024-09-17T05:08:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bdfe00640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 0x7f4be015dbc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009bd0 tx=0x7f4bd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80bc070 con 0x7f4be0162de0 2024-09-17T05:08:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bd8004460 con 0x7f4be0162de0 2024-09-17T05:08:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80b2040 con 0x7f4be0162de0 2024-09-17T05:08:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be0156310 con 0x7f4be0162de0 2024-09-17T05:08:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be01568d0 con 0x7f4be0162de0 2024-09-17T05:08:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be0156080 con 0x7f4be0162de0 2024-09-17T05:08:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.906+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4bd80ad440 con 0x7f4be0162de0 2024-09-17T05:08:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.910+0000 7f4bd7400640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bbc073200 0x7f4bbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.910+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5749+0+0 (secure 0 0 0) 0x7f4bd812e400 con 0x7f4be0162de0 2024-09-17T05:08:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.910+0000 7f4bdf400640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bbc073200 0x7f4bbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.910+0000 7f4bdf400640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bbc073200 0x7f4bbc075710 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0007fd0 tx=0x7f4bd0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:44.914+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4bd8016990 con 0x7f4be0162de0 2024-09-17T05:08:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.090+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4be00d0610 con 0x7f4be0162de0 2024-09-17T05:08:45.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.706+0000 7f4bd7400640 1 -- 172.21.15.16:0/455406319 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) ==== 169+0+0 (secure 0 0 0) 0x7f4bd80f8d00 con 0x7f4be0162de0 2024-09-17T05:08:45.705 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:08:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bbc073200 msgr2=0x7f4bbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bbc073200 0x7f4bbc075710 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0007fd0 tx=0x7f4bd0009040 comp rx=0 tx=0).stop 2024-09-17T05:08:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 msgr2=0x7f4be015dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 0x7f4be015dbc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009bd0 tx=0x7f4bd8004270 comp rx=0 tx=0).stop 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 shutdown_connections 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4bbc073200 0x7f4bbc075710 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be0162de0 0x7f4be015dbc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be00d0200 0x7f4be015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 --2- 172.21.15.16:0/455406319 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4be00b81c0 0x7f4be014f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 >> 172.21.15.16:0/455406319 conn(0x7f4be006ef00 msgr2=0x7f4be00d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 shutdown_connections 2024-09-17T05:08:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.710+0000 7f4be6600640 1 -- 172.21.15.16:0/455406319 wait complete. 2024-09-17T05:08:45.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:08:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 -- 172.21.15.16:0/825039227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc4815f520 msgr2=0x7fdc4815c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 --2- 172.21.15.16:0/825039227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc4815f520 0x7fdc4815c160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009bc0 tx=0x7fdc4001cba0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 -- 172.21.15.16:0/825039227 shutdown_connections 2024-09-17T05:08:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 --2- 172.21.15.16:0/825039227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc4815f520 0x7fdc4815c160 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 --2- 172.21.15.16:0/825039227 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc48074a20 0x7fdc4815efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 -- 172.21.15.16:0/825039227 >> 172.21.15.16:0/825039227 conn(0x7fdc480b2250 msgr2=0x7fdc480d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 -- 172.21.15.16:0/825039227 shutdown_connections 2024-09-17T05:08:45.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.898+0000 7fdc50600640 1 -- 172.21.15.16:0/825039227 wait complete. 2024-09-17T05:08:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 Processor -- start 2024-09-17T05:08:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 -- start start 2024-09-17T05:08:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc48074a20 0x7fdc4814a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 0x7fdc480b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc4815f520 0x7fdc48154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48168f30 con 0x7fdc48074e30 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48168db0 con 0x7fdc48074a20 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc50600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdc481690b0 con 0x7fdc4815f520 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 0x7fdc480b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 0x7fdc480b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56078/0 (socket says 172.21.15.16:56078) 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 -- 172.21.15.16:0/3022033961 learned_addr learned my addr 172.21.15.16:0/3022033961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4dc00640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc48074a20 0x7fdc4814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4e600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc4815f520 0x7fdc48154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 -- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc4815f520 msgr2=0x7fdc48154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc4815f520 0x7fdc48154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 -- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc48074a20 msgr2=0x7fdc4814a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc48074a20 0x7fdc4814a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 -- 172.21.15.16:0/3022033961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc480afbc0 con 0x7fdc48074e30 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc4d200640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 0x7fdc480b1370 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c3f0 tx=0x7fdc3400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.902+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc340052d0 con 0x7fdc48074e30 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc34007020 con 0x7fdc48074e30 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc4814e810 con 0x7fdc48074e30 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34007470 con 0x7fdc48074e30 2024-09-17T05:08:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc4814c560 con 0x7fdc48074e30 2024-09-17T05:08:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc4815efe0 con 0x7fdc48074e30 2024-09-17T05:08:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc340067e0 con 0x7fdc48074e30 2024-09-17T05:08:45.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc45600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc28073200 0x7fdc28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7fdc340934f0 con 0x7fdc48074e30 2024-09-17T05:08:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.906+0000 7fdc4dc00640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc28073200 0x7fdc28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.910+0000 7fdc4dc00640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc28073200 0x7fdc28075710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0096c0 tx=0x7fdc3c009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:45.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:45.910+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc34001090 con 0x7fdc48074e30 2024-09-17T05:08:46.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.086+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fdc4814acc0 con 0x7fdc48074e30 2024-09-17T05:08:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc45600640 1 -- 172.21.15.16:0/3022033961 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v56) ==== 71+0+0 (secure 0 0 0) 0x7fdc3405e0d0 con 0x7fdc48074e30 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc28073200 msgr2=0x7fdc28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc28073200 0x7fdc28075710 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0096c0 tx=0x7fdc3c009260 comp rx=0 tx=0).stop 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 msgr2=0x7fdc480b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 0x7fdc480b1370 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c3f0 tx=0x7fdc3400c8b0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 shutdown_connections 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdc28073200 0x7fdc28075710 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdc4815f520 0x7fdc48154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc48074e30 0x7fdc480b1370 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 --2- 172.21.15.16:0/3022033961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc48074a20 0x7fdc4814a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 >> 172.21.15.16:0/3022033961 conn(0x7fdc480b2250 msgr2=0x7fdc480b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 shutdown_connections 2024-09-17T05:08:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.094+0000 7fdc50600640 1 -- 172.21.15.16:0/3022033961 wait complete. 2024-09-17T05:08:46.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-09-17T05:08:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 -- 172.21.15.16:0/633150122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0240b2f50 msgr2=0x7fb02415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 --2- 172.21.15.16:0/633150122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0240b2f50 0x7fb02415bb60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb014009c80 tx=0x7fb01401cd10 comp rx=0 tx=0).stop 2024-09-17T05:08:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 -- 172.21.15.16:0/633150122 shutdown_connections 2024-09-17T05:08:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 --2- 172.21.15.16:0/633150122 >> v2:172.21.15.104:3300/0 conn(0x7fb024073240 0x7fb024073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 --2- 172.21.15.16:0/633150122 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0240b2f50 0x7fb02415bb60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 -- 172.21.15.16:0/633150122 >> 172.21.15.16:0/633150122 conn(0x7fb02406f900 msgr2=0x7fb0240b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 -- 172.21.15.16:0/633150122 shutdown_connections 2024-09-17T05:08:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.274+0000 7fb029e00640 1 -- 172.21.15.16:0/633150122 wait complete. 2024-09-17T05:08:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 Processor -- start 2024-09-17T05:08:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 -- start start 2024-09-17T05:08:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 0x7fb0240cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0240b2f50 0x7fb024158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb02415c0a0 0x7fb024164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb024168e00 con 0x7fb024073240 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb024168c80 con 0x7fb0240b2f50 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb029e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb024168f80 con 0x7fb02415c0a0 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 0x7fb0240cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 0x7fb0240cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56088/0 (socket says 172.21.15.16:56088) 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 -- 172.21.15.16:0/1283239087 learned_addr learned my addr 172.21.15.16:0/1283239087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb023400640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb02415c0a0 0x7fb024164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb01a000640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0240b2f50 0x7fb024158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 -- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb02415c0a0 msgr2=0x7fb024164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb02415c0a0 0x7fb024164690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 -- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0240b2f50 msgr2=0x7fb024158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0240b2f50 0x7fb024158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 -- 172.21.15.16:0/1283239087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0241fc580 con 0x7fb024073240 2024-09-17T05:08:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb022a00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 0x7fb0240cfb80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb014009bd0 tx=0x7fb0140092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140bb070 con 0x7fb024073240 2024-09-17T05:08:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb014004d40 con 0x7fb024073240 2024-09-17T05:08:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.278+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b2040 con 0x7fb024073240 2024-09-17T05:08:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.282+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0240ce6e0 con 0x7fb024073240 2024-09-17T05:08:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.282+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0240ceca0 con 0x7fb024073240 2024-09-17T05:08:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.282+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0241fc750 con 0x7fb024073240 2024-09-17T05:08:46.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.282+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0140ad7f0 con 0x7fb024073240 2024-09-17T05:08:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.282+0000 7fb01be00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb000073200 0x7fb000075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.282+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7fb01412dd00 con 0x7fb024073240 2024-09-17T05:08:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.286+0000 7fb01a000640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb000073200 0x7fb000075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.286+0000 7fb01a000640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb000073200 0x7fb000075710 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb0241494d0 tx=0x7fb00c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:46.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.290+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb0140f2db0 con 0x7fb024073240 2024-09-17T05:08:46.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.522+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7fb024073650 con 0x7fb024073240 2024-09-17T05:08:46.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.706+0000 7fb01be00640 1 -- 172.21.15.16:0/1283239087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v384) ==== 143+0+0 (secure 0 0 0) 0x7fb014016990 con 0x7fb024073240 2024-09-17T05:08:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb000073200 msgr2=0x7fb000075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb000073200 0x7fb000075710 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb0241494d0 tx=0x7fb00c009040 comp rx=0 tx=0).stop 2024-09-17T05:08:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 msgr2=0x7fb0240cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 0x7fb0240cfb80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb014009bd0 tx=0x7fb0140092c0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 shutdown_connections 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb000073200 0x7fb000075710 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb02415c0a0 0x7fb024164690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0240b2f50 0x7fb024158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 --2- 172.21.15.16:0/1283239087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb024073240 0x7fb0240cfb80 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 >> 172.21.15.16:0/1283239087 conn(0x7fb02406f900 msgr2=0x7fb0240b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 shutdown_connections 2024-09-17T05:08:46.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.710+0000 7fb029e00640 1 -- 172.21.15.16:0/1283239087 wait complete. 2024-09-17T05:08:46.729 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-17T05:08:46.729 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:08:46.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 -- 172.21.15.16:0/2429441955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb40d0b80 msgr2=0x7f8cb40d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 --2- 172.21.15.16:0/2429441955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb40d0b80 0x7f8cb40d0f90 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40cdb80 tx=0x7f8cac01cc60 comp rx=0 tx=0).stop 2024-09-17T05:08:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 -- 172.21.15.16:0/2429441955 shutdown_connections 2024-09-17T05:08:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 --2- 172.21.15.16:0/2429441955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb40d0b80 0x7f8cb40d0f90 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 --2- 172.21.15.16:0/2429441955 >> v2:172.21.15.104:3300/0 conn(0x7f8cb41608d0 0x7f8cb40d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 --2- 172.21.15.16:0/2429441955 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cb415c6f0 0x7f8cb4160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 -- 172.21.15.16:0/2429441955 >> 172.21.15.16:0/2429441955 conn(0x7f8cb40cf380 msgr2=0x7f8cb40d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:46.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 -- 172.21.15.16:0/2429441955 shutdown_connections 2024-09-17T05:08:46.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.886+0000 7f8cbac00640 1 -- 172.21.15.16:0/2429441955 wait complete. 2024-09-17T05:08:46.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 Processor -- start 2024-09-17T05:08:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 -- start start 2024-09-17T05:08:46.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cb40d0b80 0x7f8cb40c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 0x7f8cb4149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb41608d0 0x7f8cb4070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb40b2c40 con 0x7f8cb415c6f0 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb40b2ac0 con 0x7f8cb41608d0 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cbac00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb40b2dc0 con 0x7f8cb40d0b80 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 0x7f8cb4149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 0x7f8cb4149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56102/0 (socket says 172.21.15.16:56102) 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 -- 172.21.15.16:0/1825893401 learned_addr learned my addr 172.21.15.16:0/1825893401 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cb3e00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cb40d0b80 0x7f8cb40c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cb8c00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb41608d0 0x7f8cb4070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 -- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cb40d0b80 msgr2=0x7f8cb40c5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cb40d0b80 0x7f8cb40c5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 -- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb41608d0 msgr2=0x7f8cb4070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb41608d0 0x7f8cb4070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.890+0000 7f8cab400640 1 -- 172.21.15.16:0/1825893401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb40c6400 con 0x7f8cb415c6f0 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cab400640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 0x7f8cb4149170 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005bd0 tx=0x7f8c9c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c009c30 con 0x7f8cb415c6f0 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c9c004030 con 0x7f8cb415c6f0 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c004460 con 0x7f8cb415c6f0 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb40c66e0 con 0x7f8cb415c6f0 2024-09-17T05:08:46.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb414b280 con 0x7f8cb415c6f0 2024-09-17T05:08:46.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb40c6170 con 0x7f8cb415c6f0 2024-09-17T05:08:46.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c9c003600 con 0x7f8cb415c6f0 2024-09-17T05:08:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cb0c00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8c90073200 0x7f8c90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.894+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7f8c9c094240 con 0x7f8cb415c6f0 2024-09-17T05:08:46.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.898+0000 7f8cb3e00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8c90073200 0x7f8c90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:46.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.898+0000 7f8cb3e00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8c90073200 0x7f8c90075710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009ac0 tx=0x7f8ca40091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:46.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:46.902+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c9c017020 con 0x7f8cb415c6f0 2024-09-17T05:08:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.078+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cb4149ad0 con 0x7f8cb415c6f0 2024-09-17T05:08:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.078+0000 7f8cb0c00640 1 -- 172.21.15.16:0/1825893401 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+13301 (secure 0 0 0) 0x7f8c9c05ed80 con 0x7f8cb415c6f0 2024-09-17T05:08:47.078 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:47.078 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":116,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:45.701786+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/759404348":"2024-09-18T05:08:42.391918+0000","172.21.15.16:6834/759404348":"2024-09-18T05:08:42.391918+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.078+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8c90073200 msgr2=0x7f8c90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.078+0000 7f8cbac00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8c90073200 0x7f8c90075710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009ac0 tx=0x7f8ca40091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.078+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 msgr2=0x7f8cb4149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.078+0000 7f8cbac00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 0x7f8cb4149170 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c005bd0 tx=0x7f8c9c00dde0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 shutdown_connections 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8c90073200 0x7f8c90075710 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb41608d0 0x7f8cb4070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb415c6f0 0x7f8cb4149170 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 --2- 172.21.15.16:0/1825893401 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8cb40d0b80 0x7f8cb40c5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 >> 172.21.15.16:0/1825893401 conn(0x7f8cb40cf380 msgr2=0x7f8cb41658d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 shutdown_connections 2024-09-17T05:08:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.082+0000 7f8cbac00640 1 -- 172.21.15.16:0/1825893401 wait complete. 2024-09-17T05:08:47.101 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:08:47.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 -- 172.21.15.16:0/938802428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd80cb350 msgr2=0x7effd80cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 --2- 172.21.15.16:0/938802428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd80cb350 0x7effd80cb760 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7effd815a680 tx=0x7effc801c6b0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 -- 172.21.15.16:0/938802428 shutdown_connections 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 --2- 172.21.15.16:0/938802428 >> v2:172.21.15.104:3300/0 conn(0x7effd806ff20 0x7effd8070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 --2- 172.21.15.16:0/938802428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd80cb350 0x7effd80cb760 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 -- 172.21.15.16:0/938802428 >> 172.21.15.16:0/938802428 conn(0x7effd80cd980 msgr2=0x7effd80cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 -- 172.21.15.16:0/938802428 shutdown_connections 2024-09-17T05:08:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.262+0000 7effdfa00640 1 -- 172.21.15.16:0/938802428 wait complete. 2024-09-17T05:08:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 Processor -- start 2024-09-17T05:08:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 -- start start 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 0x7effd81511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effd80cb350 0x7effd815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effd81600a0 0x7effd815f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7effd815d5f0 con 0x7effd806ff20 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7effd815d470 con 0x7effd80cb350 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdfa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7effd815d770 con 0x7effd81600a0 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 0x7effd81511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 0x7effd81511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52256/0 (socket says 172.21.15.16:52256) 2024-09-17T05:08:47.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 -- 172.21.15.16:0/1187102230 learned_addr learned my addr 172.21.15.16:0/1187102230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effcfe00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effd80cb350 0x7effd815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdda00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effd81600a0 0x7effd815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 -- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effd81600a0 msgr2=0x7effd815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effd81600a0 0x7effd815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 -- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effd80cb350 msgr2=0x7effd815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effd80cb350 0x7effd815efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdd000640 1 -- 172.21.15.16:0/1187102230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81f9430 con 0x7effd806ff20 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.266+0000 7effdda00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effd81600a0 0x7effd815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effdd000640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 0x7effd81511e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7effc80b0040 tx=0x7effc80a5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc80b2040 con 0x7effd806ff20 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effc80043b0 con 0x7effd806ff20 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd8156f60 con 0x7effd806ff20 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc80052d0 con 0x7effd806ff20 2024-09-17T05:08:47.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd80b52f0 con 0x7effd806ff20 2024-09-17T05:08:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd8070330 con 0x7effd806ff20 2024-09-17T05:08:47.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7effc80bd060 con 0x7effd806ff20 2024-09-17T05:08:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.270+0000 7effd5600640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7effb4073200 0x7effb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.274+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7effc812d5a0 con 0x7effd806ff20 2024-09-17T05:08:47.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.274+0000 7effcfe00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7effb4073200 0x7effb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.274+0000 7effcfe00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7effb4073200 0x7effb4075710 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7effd81541d0 tx=0x7effc000b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.278+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7effd8070870 con 0x7effd806ff20 2024-09-17T05:08:47.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.450+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effd806a180 con 0x7effd806ff20 2024-09-17T05:08:47.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.450+0000 7effd5600640 1 -- 172.21.15.16:0/1187102230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7effc80f87b0 con 0x7effd806ff20 2024-09-17T05:08:47.451 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:08:47.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7effb4073200 msgr2=0x7effb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7effb4073200 0x7effb4075710 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7effd81541d0 tx=0x7effc000b2e0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 msgr2=0x7effd81511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 0x7effd81511e0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7effc80b0040 tx=0x7effc80a5530 comp rx=0 tx=0).stop 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 shutdown_connections 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7effb4073200 0x7effb4075710 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7effd81600a0 0x7effd815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effd80cb350 0x7effd815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 --2- 172.21.15.16:0/1187102230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effd806ff20 0x7effd81511e0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 >> 172.21.15.16:0/1187102230 conn(0x7effd80cd980 msgr2=0x7effd80cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 shutdown_connections 2024-09-17T05:08:47.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.454+0000 7effdfa00640 1 -- 172.21.15.16:0/1187102230 wait complete. 2024-09-17T05:08:47.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-09-17T05:08:47.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 -- 172.21.15.16:0/2096847010 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 msgr2=0x7f54b4073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 --2- 172.21.15.16:0/2096847010 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b4073950 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f54ac009870 tx=0x7f54ac01c7d0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 -- 172.21.15.16:0/2096847010 shutdown_connections 2024-09-17T05:08:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 --2- 172.21.15.16:0/2096847010 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b4073950 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 -- 172.21.15.16:0/2096847010 >> 172.21.15.16:0/2096847010 conn(0x7f54b406f900 msgr2=0x7f54b40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 -- 172.21.15.16:0/2096847010 shutdown_connections 2024-09-17T05:08:47.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 -- 172.21.15.16:0/2096847010 wait complete. 2024-09-17T05:08:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 Processor -- start 2024-09-17T05:08:47.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 -- start start 2024-09-17T05:08:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 0x7f54b413bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.634+0000 7f54ba200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54b415c270 0x7f54b40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b40b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4169c30 con 0x7f54b40b2f50 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4169ab0 con 0x7f54b415c270 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f54b4169db0 con 0x7f54b415c680 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 0x7f54b413bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 0x7f54b413bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52268/0 (socket says 172.21.15.16:52268) 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 -- 172.21.15.16:0/157907525 learned_addr learned my addr 172.21.15.16:0/157907525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3e00640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b40b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54aaa00640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54b415c270 0x7f54b40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 -- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 msgr2=0x7f54b40b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b40b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 -- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54b415c270 msgr2=0x7f54b40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:47.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54b415c270 0x7f54b40b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 -- 172.21.15.16:0/157907525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b41f9240 con 0x7f54b40b2f50 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3e00640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b40b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54b3400640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 0x7f54b413bb00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f54a400cb40 tx=0x7f54a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4016020 con 0x7f54b40b2f50 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54a400ab00 con 0x7f54b40b2f50 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b4153310 con 0x7f54b40b2f50 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a400ce20 con 0x7f54b40b2f50 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41f8760 con 0x7f54b40b2f50 2024-09-17T05:08:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b40b1c30 con 0x7f54b40b2f50 2024-09-17T05:08:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.638+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54a4019020 con 0x7f54b40b2f50 2024-09-17T05:08:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.642+0000 7f54abe00640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5490073200 0x7f5490075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.642+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7f54a40943d0 con 0x7f54b40b2f50 2024-09-17T05:08:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.642+0000 7f54aaa00640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5490073200 0x7f5490075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.642+0000 7f54aaa00640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5490073200 0x7f5490075710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f54b415bee0 tx=0x7f549c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.646+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f54a4020020 con 0x7f54b40b2f50 2024-09-17T05:08:47.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:47.882+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7f54b415bf10 con 0x7f54b40b2f50 2024-09-17T05:08:48.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.710+0000 7f54abe00640 1 -- 172.21.15.16:0/157907525 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v386) ==== 154+0+0 (secure 0 0 0) 0x7f54a405f550 con 0x7f54b40b2f50 2024-09-17T05:08:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5490073200 msgr2=0x7f5490075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5490073200 0x7f5490075710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f54b415bee0 tx=0x7f549c009040 comp rx=0 tx=0).stop 2024-09-17T05:08:48.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 msgr2=0x7f54b413bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 0x7f54b413bb00 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f54a400cb40 tx=0x7f54a400cc30 comp rx=0 tx=0).stop 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 shutdown_connections 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5490073200 0x7f5490075710 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f54b415c680 0x7f54b40b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54b415c270 0x7f54b40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 --2- 172.21.15.16:0/157907525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54b40b2f50 0x7f54b413bb00 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 >> 172.21.15.16:0/157907525 conn(0x7f54b406f900 msgr2=0x7f54b4166ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 shutdown_connections 2024-09-17T05:08:48.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.714+0000 7f54ba200640 1 -- 172.21.15.16:0/157907525 wait complete. 2024-09-17T05:08:48.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:08:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 -- 172.21.15.16:0/163244309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8074b00 msgr2=0x7f03d80b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 --2- 172.21.15.16:0/163244309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8074b00 0x7f03d80b2f50 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009840 tx=0x7f03cc01c700 comp rx=0 tx=0).stop 2024-09-17T05:08:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 -- 172.21.15.16:0/163244309 shutdown_connections 2024-09-17T05:08:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 --2- 172.21.15.16:0/163244309 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03d8073240 0x7f03d8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 --2- 172.21.15.16:0/163244309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8074b00 0x7f03d80b2f50 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 -- 172.21.15.16:0/163244309 >> 172.21.15.16:0/163244309 conn(0x7f03d806f900 msgr2=0x7f03d80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 -- 172.21.15.16:0/163244309 shutdown_connections 2024-09-17T05:08:48.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.902+0000 7f03e0800640 1 -- 172.21.15.16:0/163244309 wait complete. 2024-09-17T05:08:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 Processor -- start 2024-09-17T05:08:48.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 -- start start 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 0x7f03d813f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d8074b00 0x7f03d8160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03d815f080 0x7f03d80cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8168ba0 con 0x7f03d8073240 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8168a20 con 0x7f03d8074b00 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f03d8168d20 con 0x7f03d815f080 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 0x7f03d813f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 0x7f03d813f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52278/0 (socket says 172.21.15.16:52278) 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 -- 172.21.15.16:0/1928446035 learned_addr learned my addr 172.21.15.16:0/1928446035 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:48.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03de800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03d815f080 0x7f03d80cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dd400640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d8074b00 0x7f03d8160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 -- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03d815f080 msgr2=0x7f03d80cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03d815f080 0x7f03d80cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 -- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d8074b00 msgr2=0x7f03d8160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d8074b00 0x7f03d8160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 -- 172.21.15.16:0/1928446035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d8140310 con 0x7f03d8073240 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03dde00640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 0x7f03d813f8b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009790 tx=0x7f03cc005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0b2040 con 0x7f03d8073240 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03cc004330 con 0x7f03d8073240 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d81405f0 con 0x7f03d8073240 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc003820 con 0x7f03d8073240 2024-09-17T05:08:48.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.906+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d8140bb0 con 0x7f03d8073240 2024-09-17T05:08:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.910+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d8073650 con 0x7f03d8073240 2024-09-17T05:08:48.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.910+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f03cc0bd060 con 0x7f03d8073240 2024-09-17T05:08:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.910+0000 7f03d2000640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f03bc073230 0x7f03bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.910+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7f03cc12d860 con 0x7f03d8073240 2024-09-17T05:08:48.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.910+0000 7f03dd400640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f03bc073230 0x7f03bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:48.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.914+0000 7f03dd400640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f03bc073230 0x7f03bc075740 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f03c8000f70 tx=0x7f03c8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:48.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:48.914+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f03cc016410 con 0x7f03d8073240 2024-09-17T05:08:49.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f03d80cde60 con 0x7f03d8073240 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03d2000640 1 -- 172.21.15.16:0/1928446035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v56) ==== 59+0+2294 (secure 0 0 0) 0x7f03cc0f8950 con 0x7f03d8073240 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:08:49.090 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:08:49.091 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:08:49.092 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:08:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f03bc073230 msgr2=0x7f03bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f03bc073230 0x7f03bc075740 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f03c8000f70 tx=0x7f03c8009040 comp rx=0 tx=0).stop 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 msgr2=0x7f03d813f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 0x7f03d813f8b0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009790 tx=0x7f03cc005d20 comp rx=0 tx=0).stop 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 shutdown_connections 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f03bc073230 0x7f03bc075740 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f03d815f080 0x7f03d80cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03d8074b00 0x7f03d8160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 --2- 172.21.15.16:0/1928446035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03d8073240 0x7f03d813f8b0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 >> 172.21.15.16:0/1928446035 conn(0x7f03d806f900 msgr2=0x7f03d8165750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 shutdown_connections 2024-09-17T05:08:49.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.090+0000 7f03e0800640 1 -- 172.21.15.16:0/1928446035 wait complete. 2024-09-17T05:08:49.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 -- 172.21.15.16:0/1458631274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc0d02d0 msgr2=0x7f51bc0cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 --2- 172.21.15.16:0/1458631274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc0d02d0 0x7f51bc0cf140 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009930 tx=0x7f51a401c820 comp rx=0 tx=0).stop 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 -- 172.21.15.16:0/1458631274 shutdown_connections 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 --2- 172.21.15.16:0/1458631274 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51bc0cf680 0x7f51bc0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 --2- 172.21.15.16:0/1458631274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc0d02d0 0x7f51bc0cf140 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 -- 172.21.15.16:0/1458631274 >> 172.21.15.16:0/1458631274 conn(0x7f51bc0b42d0 msgr2=0x7f51bc0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 -- 172.21.15.16:0/1458631274 shutdown_connections 2024-09-17T05:08:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 -- 172.21.15.16:0/1458631274 wait complete. 2024-09-17T05:08:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 Processor -- start 2024-09-17T05:08:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.274+0000 7f51c2200640 1 -- start start 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51bc0cf680 0x7f51bc155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51bc0d02d0 0x7f51bc14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 0x7f51bc1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc169e20 con 0x7f51bc159f70 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f51bc169ca0 con 0x7f51bc0cf680 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f51bc169fa0 con 0x7f51bc0d02d0 2024-09-17T05:08:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 0x7f51bc1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 0x7f51bc1f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52302/0 (socket says 172.21.15.16:52302) 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 -- 172.21.15.16:0/2573691672 learned_addr learned my addr 172.21.15.16:0/2573691672 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51b3e00640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51bc0d02d0 0x7f51bc14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bb400640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51bc0cf680 0x7f51bc155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 -- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51bc0d02d0 msgr2=0x7f51bc14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51bc0d02d0 0x7f51bc14e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 -- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51bc0cf680 msgr2=0x7f51bc155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51bc0cf680 0x7f51bc155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 -- 172.21.15.16:0/2573691672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51bc15b3d0 con 0x7f51bc159f70 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51bbe00640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 0x7f51bc1f68e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ea80 tx=0x7f51b400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4004b30 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51b4004cf0 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51bc1f7590 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b4005280 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51bc154980 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.278+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51bc1f7300 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.282+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f51b4005420 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.282+0000 7f51b3400640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5198073230 0x7f5198075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.282+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7f51b4094d80 con 0x7f51bc159f70 2024-09-17T05:08:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.282+0000 7f51bb400640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5198073230 0x7f5198075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.282+0000 7f51bb400640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5198073230 0x7f5198075740 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f51ac001650 tx=0x7f51ac009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.286+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f51b405fdf0 con 0x7f51bc159f70 2024-09-17T05:08:49.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.462+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f51bc0b2100 con 0x7f51bc159f70 2024-09-17T05:08:49.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.466+0000 7f51b3400640 1 -- 172.21.15.16:0/2573691672 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v57)=0 v57) ==== 122+0+0 (secure 0 0 0) 0x7f51b405f6f0 con 0x7f51bc159f70 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5198073230 msgr2=0x7f5198075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5198073230 0x7f5198075740 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f51ac001650 tx=0x7f51ac009040 comp rx=0 tx=0).stop 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 msgr2=0x7f51bc1f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 0x7f51bc1f68e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f51b400ea80 tx=0x7f51b400ef40 comp rx=0 tx=0).stop 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 shutdown_connections 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5198073230 0x7f5198075740 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51bc159f70 0x7f51bc1f68e0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f51bc0d02d0 0x7f51bc14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 --2- 172.21.15.16:0/2573691672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51bc0cf680 0x7f51bc155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 >> 172.21.15.16:0/2573691672 conn(0x7f51bc0b42d0 msgr2=0x7f51bc152760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 shutdown_connections 2024-09-17T05:08:49.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.470+0000 7f51c2200640 1 -- 172.21.15.16:0/2573691672 wait complete. 2024-09-17T05:08:49.489 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:08:49.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:08:49.567 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:08:49.667 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:08:49.667 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:49.667 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:49.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:08:49.676 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:49.676 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:49.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:08:49.688 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:08:49.688 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:49.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:08:49.729 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:08:49.729 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:08:49.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:08:49.739 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:08:49.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 -- 172.21.15.16:0/2649944038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30071790 msgr2=0x7fea300cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 --2- 172.21.15.16:0/2649944038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30071790 0x7fea300cf8f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fea30069cd0 tx=0x7fea2801cd80 comp rx=0 tx=0).stop 2024-09-17T05:08:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 -- 172.21.15.16:0/2649944038 shutdown_connections 2024-09-17T05:08:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 --2- 172.21.15.16:0/2649944038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30071790 0x7fea300cf8f0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 -- 172.21.15.16:0/2649944038 >> 172.21.15.16:0/2649944038 conn(0x7fea30073ca0 msgr2=0x7fea300740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 -- 172.21.15.16:0/2649944038 shutdown_connections 2024-09-17T05:08:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.934+0000 7fea36800640 1 -- 172.21.15.16:0/2649944038 wait complete. 2024-09-17T05:08:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 Processor -- start 2024-09-17T05:08:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 -- start start 2024-09-17T05:08:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea30071380 0x7fea300ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea30071790 0x7fea3014c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 0x7fea3014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea30169ea0 con 0x7fea30143b40 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea30169d20 con 0x7fea30071380 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea36800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fea3016a020 con 0x7fea30071790 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 0x7fea3014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 0x7fea3014cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52320/0 (socket says 172.21.15.16:52320) 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 -- 172.21.15.16:0/2261993629 learned_addr learned my addr 172.21.15.16:0/2261993629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35400640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea30071380 0x7fea300ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea34a00640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea30071790 0x7fea3014c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 -- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea30071790 msgr2=0x7fea3014c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea30071790 0x7fea3014c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 -- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea30071380 msgr2=0x7fea300ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea30071380 0x7fea300ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:49.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.938+0000 7fea35e00640 1 -- 172.21.15.16:0/2261993629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300adda0 con 0x7fea30143b40 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea35e00640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 0x7fea3014cc30 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fea2801a9b0 tx=0x7fea28003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea280bc070 con 0x7fea30143b40 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea280042f0 con 0x7fea30143b40 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea280b2040 con 0x7fea30143b40 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea300ae070 con 0x7fea30143b40 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3014ae90 con 0x7fea30143b40 2024-09-17T05:08:49.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3006efb0 con 0x7fea30143b40 2024-09-17T05:08:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fea28004720 con 0x7fea30143b40 2024-09-17T05:08:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea2d600640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea10073230 0x7fea10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:49.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.942+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7fea2812dea0 con 0x7fea30143b40 2024-09-17T05:08:49.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.946+0000 7fea35400640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea10073230 0x7fea10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:49.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.946+0000 7fea35400640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea10073230 0x7fea10075740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fea240098e0 tx=0x7fea24009060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:49.946+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea280f2db0 con 0x7fea30143b40 2024-09-17T05:08:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea3014da10 con 0x7fea30143b40 2024-09-17T05:08:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea2d600640 1 -- 172.21.15.16:0/2261993629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+3914 (secure 0 0 0) 0x7fea28016970 con 0x7fea30143b40 2024-09-17T05:08:50.158 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:50.158 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":80,"btime":"2024-09-17T05:08:43:393872+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6257,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/939247401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":939247401},{"type":"v1","addr":"172.21.15.104:6836","nonce":939247401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1055707523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":1055707523},{"type":"v1","addr":"172.21.15.104:6837","nonce":1055707523}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/4187856940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4187856940},{"type":"v1","addr":"172.21.15.16:6833","nonce":4187856940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7647,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2793681340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2793681340},{"type":"v1","addr":"172.21.15.16:6835","nonce":2793681340}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[]} 2024-09-17T05:08:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 80 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea10073230 msgr2=0x7fea10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea10073230 0x7fea10075740 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fea240098e0 tx=0x7fea24009060 comp rx=0 tx=0).stop 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 msgr2=0x7fea3014cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 0x7fea3014cc30 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fea2801a9b0 tx=0x7fea28003f80 comp rx=0 tx=0).stop 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 shutdown_connections 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea10073230 0x7fea10075740 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea30143b40 0x7fea3014cc30 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea30071790 0x7fea3014c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 --2- 172.21.15.16:0/2261993629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea30071380 0x7fea300ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 >> 172.21.15.16:0/2261993629 conn(0x7fea30073ca0 msgr2=0x7fea300b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 shutdown_connections 2024-09-17T05:08:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.158+0000 7fea36800640 1 -- 172.21.15.16:0/2261993629 wait complete. 2024-09-17T05:08:50.180 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:08:50.180 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:08:50.181 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:08:50.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:50.181 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:08:50.181 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:08:50.181 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:08:50.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:50.182 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:08:50.182 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:08:50.182 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:08:50.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:50.182 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:08:50.182 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:08:50.182 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:08:50.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:08:50.194 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:08:50.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:08:50.198 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:08:50.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:08:50.199 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:08:50.199 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:08:50.202 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:08:50.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:08:50.243 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:08:50.243 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:50.245 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:08:50.245 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:50.250 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:08:50.250 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:50.252 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:08:50.252 DEBUG:teuthology.parallel:result is None 2024-09-17T05:08:50.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:08:50.345 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:08:50.348 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:08:50.353 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:08:50.353 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:08:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 -- 172.21.15.16:0/910457603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074e30 msgr2=0x7f7dd80ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 --2- 172.21.15.16:0/910457603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074e30 0x7f7dd80ce5d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ce2c0 tx=0x7f7dc401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 -- 172.21.15.16:0/910457603 shutdown_connections 2024-09-17T05:08:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 --2- 172.21.15.16:0/910457603 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 0x7f7dd80ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 --2- 172.21.15.16:0/910457603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074e30 0x7f7dd80ce5d0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 -- 172.21.15.16:0/910457603 >> 172.21.15.16:0/910457603 conn(0x7f7dd80b2250 msgr2=0x7f7dd80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 -- 172.21.15.16:0/910457603 shutdown_connections 2024-09-17T05:08:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 -- 172.21.15.16:0/910457603 wait complete. 2024-09-17T05:08:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 Processor -- start 2024-09-17T05:08:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.422+0000 7f7de0000640 1 -- start start 2024-09-17T05:08:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 0x7f7dd81fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dd8074e30 0x7f7dd80b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 0x7f7dd815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8168f30 con 0x7f7dd8074a20 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd8168db0 con 0x7f7dd8074e30 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd81690b0 con 0x7f7dd815f6f0 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 0x7f7dd81fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 0x7f7dd81fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52366/0 (socket says 172.21.15.16:52366) 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 -- 172.21.15.16:0/4006519524 learned_addr learned my addr 172.21.15.16:0/4006519524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddcc00640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dd8074e30 0x7f7dd80b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7dde000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 0x7f7dd815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 -- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 msgr2=0x7f7dd815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 0x7f7dd815ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 -- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dd8074e30 msgr2=0x7f7dd80b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dd8074e30 0x7f7dd80b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 -- 172.21.15.16:0/4006519524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd80b40b0 con 0x7f7dd8074a20 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7dde000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 0x7f7dd815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7ddd600640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 0x7f7dd81fbe40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00c3a0 tx=0x7f7dcc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc017070 con 0x7f7dd8074a20 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dcc0053e0 con 0x7f7dd8074a20 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd80b4390 con 0x7f7dd8074a20 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc003880 con 0x7f7dd8074a20 2024-09-17T05:08:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd8157660 con 0x7f7dd8074a20 2024-09-17T05:08:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.426+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd815efe0 con 0x7f7dd8074a20 2024-09-17T05:08:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.430+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7dcc01a070 con 0x7f7dd8074a20 2024-09-17T05:08:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.430+0000 7f7dd5600640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7db8073200 0x7f7db8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.430+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5335+0+0 (secure 0 0 0) 0x7f7dcc093400 con 0x7f7dd8074a20 2024-09-17T05:08:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.430+0000 7f7ddcc00640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7db8073200 0x7f7db8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.430+0000 7f7ddcc00640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7db8073200 0x7f7db8075710 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40af000 tx=0x7f7dc40a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.434+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7dcc05df20 con 0x7f7dd8074a20 2024-09-17T05:08:50.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:50.622+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7dd815f280 con 0x7f7dd8074a20 2024-09-17T05:08:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.482+0000 7f7dd5600640 1 -- 172.21.15.16:0/4006519524 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) ==== 97+0+0 (secure 0 0 0) 0x7f7dcc062950 con 0x7f7dd8074a20 2024-09-17T05:08:51.481 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:08:51.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7db8073200 msgr2=0x7f7db8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7db8073200 0x7f7db8075710 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40af000 tx=0x7f7dc40a7040 comp rx=0 tx=0).stop 2024-09-17T05:08:51.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 msgr2=0x7f7dd81fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 0x7f7dd81fbe40 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00c3a0 tx=0x7f7dcc00c860 comp rx=0 tx=0).stop 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 shutdown_connections 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7db8073200 0x7f7db8075710 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7dd815f6f0 0x7f7dd815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7dd8074e30 0x7f7dd80b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 --2- 172.21.15.16:0/4006519524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd8074a20 0x7f7dd81fbe40 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 >> 172.21.15.16:0/4006519524 conn(0x7f7dd80b2250 msgr2=0x7f7dd815e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 shutdown_connections 2024-09-17T05:08:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.486+0000 7f7de0000640 1 -- 172.21.15.16:0/4006519524 wait complete. 2024-09-17T05:08:51.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:08:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 -- 172.21.15.16:0/3045966745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba415d600 msgr2=0x7f7ba40cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 --2- 172.21.15.16:0/3045966745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba415d600 0x7f7ba40cee50 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009840 tx=0x7f7b9401c820 comp rx=0 tx=0).stop 2024-09-17T05:08:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 -- 172.21.15.16:0/3045966745 shutdown_connections 2024-09-17T05:08:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 --2- 172.21.15.16:0/3045966745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba415d600 0x7f7ba40cee50 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 -- 172.21.15.16:0/3045966745 >> 172.21.15.16:0/3045966745 conn(0x7f7ba406f900 msgr2=0x7f7ba40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 -- 172.21.15.16:0/3045966745 shutdown_connections 2024-09-17T05:08:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.670+0000 7f7bab600640 1 -- 172.21.15.16:0/3045966745 wait complete. 2024-09-17T05:08:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 Processor -- start 2024-09-17T05:08:51.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 -- start start 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7ba40cf390 0x7f7ba41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba415d600 0x7f7ba40c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 0x7f7ba40c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40b5280 con 0x7f7ba4160880 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40b4b90 con 0x7f7ba415d600 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7bab600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba416b440 con 0x7f7ba40cf390 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 0x7f7ba40c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 0x7f7ba40c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52388/0 (socket says 172.21.15.16:52388) 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 -- 172.21.15.16:0/943653858 learned_addr learned my addr 172.21.15.16:0/943653858 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7b9be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba415d600 0x7f7ba40c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba8c00640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7ba40cf390 0x7f7ba41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:51.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 -- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7ba40cf390 msgr2=0x7f7ba41590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7ba40cf390 0x7f7ba41590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 -- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba415d600 msgr2=0x7f7ba40c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba415d600 0x7f7ba40c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 -- 172.21.15.16:0/943653858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba40c8ff0 con 0x7f7ba4160880 2024-09-17T05:08:51.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.674+0000 7f7ba9600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 0x7f7ba40c5400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c002240 tx=0x7f7b9c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:51.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c016020 con 0x7f7ba4160880 2024-09-17T05:08:51.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b9c00a930 con 0x7f7ba4160880 2024-09-17T05:08:51.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba40c7070 con 0x7f7ba4160880 2024-09-17T05:08:51.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00ad80 con 0x7f7ba4160880 2024-09-17T05:08:51.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba40c7630 con 0x7f7ba4160880 2024-09-17T05:08:51.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba40cee50 con 0x7f7ba4160880 2024-09-17T05:08:51.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7b9c019020 con 0x7f7ba4160880 2024-09-17T05:08:51.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba1600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b80073230 0x7f7b80075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:51.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5249+0+0 (secure 0 0 0) 0x7f7b9c094170 con 0x7f7ba4160880 2024-09-17T05:08:51.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.678+0000 7f7ba8c00640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b80073230 0x7f7b80075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:51.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.682+0000 7f7ba8c00640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b80073230 0x7f7b80075740 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7b940af000 tx=0x7f7b9401b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:51.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.682+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7b9c020020 con 0x7f7ba4160880 2024-09-17T05:08:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.918+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7ba415e860 con 0x7f7ba4160880 2024-09-17T05:08:51.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.918+0000 7f7ba1600640 1 -- 172.21.15.16:0/943653858 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) ==== 80+0+5304 (secure 0 0 0) 0x7f7b9c05ecf0 con 0x7f7ba4160880 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:51.919 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.920 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.921 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.922 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:08:51.924 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.925 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:08:51.926 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:08:51.927 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:08:51.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.930 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:08:51.931 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:08:51.932 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:08:51.933 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:08:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b80073230 msgr2=0x7f7b80075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b80073230 0x7f7b80075740 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7b940af000 tx=0x7f7b9401b060 comp rx=0 tx=0).stop 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 msgr2=0x7f7ba40c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 0x7f7ba40c5400 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c002240 tx=0x7f7b9c00cba0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 shutdown_connections 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7b80073230 0x7f7b80075740 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ba4160880 0x7f7ba40c5400 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ba415d600 0x7f7ba40c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 --2- 172.21.15.16:0/943653858 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7ba40cf390 0x7f7ba41590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 >> 172.21.15.16:0/943653858 conn(0x7f7ba406f900 msgr2=0x7f7ba415f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 shutdown_connections 2024-09-17T05:08:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:51.922+0000 7f7bab600640 1 -- 172.21.15.16:0/943653858 wait complete. 2024-09-17T05:08:51.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 -- 172.21.15.16:0/714380368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb80b2f50 msgr2=0x7fcdb815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 --2- 172.21.15.16:0/714380368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb80b2f50 0x7fcdb815bb60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcda8009900 tx=0x7fcda801c870 comp rx=0 tx=0).stop 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 -- 172.21.15.16:0/714380368 shutdown_connections 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 --2- 172.21.15.16:0/714380368 >> v2:172.21.15.104:3300/0 conn(0x7fcdb8073240 0x7fcdb8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 --2- 172.21.15.16:0/714380368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb80b2f50 0x7fcdb815bb60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 -- 172.21.15.16:0/714380368 >> 172.21.15.16:0/714380368 conn(0x7fcdb806f900 msgr2=0x7fcdb80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 -- 172.21.15.16:0/714380368 shutdown_connections 2024-09-17T05:08:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 -- 172.21.15.16:0/714380368 wait complete. 2024-09-17T05:08:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 Processor -- start 2024-09-17T05:08:52.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.102+0000 7fcdbe400640 1 -- start start 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcdb8073240 0x7fcdb81508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 0x7fcdb80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb815c0a0 0x7fcdb80d0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8168d60 con 0x7fcdb80b2f50 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8168be0 con 0x7fcdb8073240 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb8168ee0 con 0x7fcdb815c0a0 2024-09-17T05:08:52.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 0x7fcdb80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 0x7fcdb80d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52398/0 (socket says 172.21.15.16:52398) 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 -- 172.21.15.16:0/1984801870 learned_addr learned my addr 172.21.15.16:0/1984801870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdb7e00640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb815c0a0 0x7fcdb80d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdb7400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcdb8073240 0x7fcdb81508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 -- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb815c0a0 msgr2=0x7fcdb80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb815c0a0 0x7fcdb80d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 -- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcdb8073240 msgr2=0x7fcdb81508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcdb8073240 0x7fcdb81508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 -- 172.21.15.16:0/1984801870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb8162640 con 0x7fcdb80b2f50 2024-09-17T05:08:52.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdafe00640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 0x7fcdb80d0100 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fcda000c340 tx=0x7fcda000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0017070 con 0x7fcdb80b2f50 2024-09-17T05:08:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcda0003600 con 0x7fcdb80b2f50 2024-09-17T05:08:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb8161ac0 con 0x7fcdb80b2f50 2024-09-17T05:08:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0003a90 con 0x7fcdb80b2f50 2024-09-17T05:08:52.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb8162080 con 0x7fcdb80b2f50 2024-09-17T05:08:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb8073650 con 0x7fcdb80b2f50 2024-09-17T05:08:52.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.106+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcda001a070 con 0x7fcdb80b2f50 2024-09-17T05:08:52.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.110+0000 7fcdaf400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd94073200 0x7fcd94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.110+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5249+0+0 (secure 0 0 0) 0x7fcda0093270 con 0x7fcdb80b2f50 2024-09-17T05:08:52.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.110+0000 7fcdb7400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd94073200 0x7fcd94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.110+0000 7fcdb7400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd94073200 0x7fcd94075710 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcda80af000 tx=0x7fcda801b0b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.114+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcda005de00 con 0x7fcdb80b2f50 2024-09-17T05:08:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.290+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcdb806e150 con 0x7fcdb80b2f50 2024-09-17T05:08:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.290+0000 7fcdaf400640 1 -- 172.21.15.16:0/1984801870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) ==== 84+0+65 (secure 0 0 0) 0x7fcda0062830 con 0x7fcdb80b2f50 2024-09-17T05:08:52.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:52.291 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:08:52.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:08:52.292 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:08:52.292 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:08:52.292 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:08:52.292 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:08:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd94073200 msgr2=0x7fcd94075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd94073200 0x7fcd94075710 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcda80af000 tx=0x7fcda801b0b0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 msgr2=0x7fcdb80d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 0x7fcdb80d0100 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fcda000c340 tx=0x7fcda000c800 comp rx=0 tx=0).stop 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 shutdown_connections 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd94073200 0x7fcd94075710 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcdb815c0a0 0x7fcdb80d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb80b2f50 0x7fcdb80d0100 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 --2- 172.21.15.16:0/1984801870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcdb8073240 0x7fcdb81508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 >> 172.21.15.16:0/1984801870 conn(0x7fcdb806f900 msgr2=0x7fcdb806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 shutdown_connections 2024-09-17T05:08:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.294+0000 7fcdbe400640 1 -- 172.21.15.16:0/1984801870 wait complete. 2024-09-17T05:08:52.314 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:08:52.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 -- 172.21.15.16:0/3070287885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5618004da0 msgr2=0x7f5618005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 --2- 172.21.15.16:0/3070287885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5618004da0 0x7f5618005230 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f562800c610 tx=0x7f562801f9c0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5636200640 1 -- 172.21.15.16:0/3070287885 reap_dead start 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 -- 172.21.15.16:0/3070287885 shutdown_connections 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 -- 172.21.15.16:0/3070287885 >> 172.21.15.16:0/3070287885 conn(0x7f56300d2240 msgr2=0x7f56300d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 -- 172.21.15.16:0/3070287885 shutdown_connections 2024-09-17T05:08:52.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 -- 172.21.15.16:0/3070287885 wait complete. 2024-09-17T05:08:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 Processor -- start 2024-09-17T05:08:52.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.478+0000 7f5638200640 1 -- start start 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5638200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618004da0 0x7f56300d1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5638200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 0x7f56301f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5638200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56301f94e0 0x7f56301f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5638200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5630160d30 con 0x7f5630154130 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5638200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5630160bb0 con 0x7f5618004da0 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5638200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5630160eb0 con 0x7f56301f94e0 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 0x7f56301f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 0x7f56301f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52408/0 (socket says 172.21.15.16:52408) 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 -- 172.21.15.16:0/92986463 learned_addr learned my addr 172.21.15.16:0/92986463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5635800640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618004da0 0x7f56300d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5636200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56301f94e0 0x7f56301f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 -- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56301f94e0 msgr2=0x7f56301f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56301f94e0 0x7f56301f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 -- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618004da0 msgr2=0x7f56300d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618004da0 0x7f56300d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 -- 172.21.15.16:0/92986463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5630154870 con 0x7f5630154130 2024-09-17T05:08:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.482+0000 7f5634e00640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 0x7f56301f8fa0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f561c003ed0 tx=0x7f561c001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:52.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c015070 con 0x7f5630154130 2024-09-17T05:08:52.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f561c011940 con 0x7f5630154130 2024-09-17T05:08:52.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c011d90 con 0x7f5630154130 2024-09-17T05:08:52.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5630154b50 con 0x7f5630154130 2024-09-17T05:08:52.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5630155110 con 0x7f5630154130 2024-09-17T05:08:52.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56301604a0 con 0x7f5630154130 2024-09-17T05:08:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f561c018070 con 0x7f5630154130 2024-09-17T05:08:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.486+0000 7f562d600640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5610073230 0x7f5610075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.490+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5249+0+0 (secure 0 0 0) 0x7f561c094e60 con 0x7f5630154130 2024-09-17T05:08:52.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.490+0000 7f5635800640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5610073230 0x7f5610075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:52.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.490+0000 7f5635800640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5610073230 0x7f5610075740 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f56300745a0 tx=0x7f56240091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:52.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.494+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f561c05ffa0 con 0x7f5630154130 2024-09-17T05:08:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:52.670+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f56300b1d40 con 0x7f5630154130 2024-09-17T05:08:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.486+0000 7f562d600640 1 -- 172.21.15.16:0/92986463 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) ==== 138+0+0 (secure 0 0 0) 0x7f561c05f8a0 con 0x7f5630154130 2024-09-17T05:08:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:08:53.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5610073230 msgr2=0x7f5610075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:53.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5610073230 0x7f5610075740 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f56300745a0 tx=0x7f56240091e0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 msgr2=0x7f56301f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:53.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 0x7f56301f8fa0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f561c003ed0 tx=0x7f561c001810 comp rx=0 tx=0).stop 2024-09-17T05:08:53.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 shutdown_connections 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5610073230 0x7f5610075740 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56301f94e0 0x7f56301f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5630154130 0x7f56301f8fa0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 --2- 172.21.15.16:0/92986463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618004da0 0x7f56300d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 >> 172.21.15.16:0/92986463 conn(0x7f56300d2240 msgr2=0x7f56301613c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 shutdown_connections 2024-09-17T05:08:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.490+0000 7f5638200640 1 -- 172.21.15.16:0/92986463 wait complete. 2024-09-17T05:08:53.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:08:53.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 -- 172.21.15.16:0/678268881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540cf0b0 msgr2=0x7fa0540cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 --2- 172.21.15.16:0/678268881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540cf0b0 0x7fa0540cf4c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009840 tx=0x7fa03c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 -- 172.21.15.16:0/678268881 shutdown_connections 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 --2- 172.21.15.16:0/678268881 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540cf0b0 0x7fa0540cf4c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 --2- 172.21.15.16:0/678268881 >> v2:172.21.15.104:3300/0 conn(0x7fa0540d1bf0 0x7fa05415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 -- 172.21.15.16:0/678268881 >> 172.21.15.16:0/678268881 conn(0x7fa0540b4000 msgr2=0x7fa0540b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 -- 172.21.15.16:0/678268881 shutdown_connections 2024-09-17T05:08:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.690+0000 7fa053200640 1 -- 172.21.15.16:0/678268881 wait complete. 2024-09-17T05:08:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 Processor -- start 2024-09-17T05:08:53.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 -- start start 2024-09-17T05:08:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 0x7fa0540b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0540cf0b0 0x7fa05414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540d1bf0 0x7fa05414e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa054153480 con 0x7fa0540cc820 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa054153300 con 0x7fa0540cf0b0 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa053200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa054153600 con 0x7fa0540d1bf0 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 0x7fa0540b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 0x7fa0540b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52414/0 (socket says 172.21.15.16:52414) 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 -- 172.21.15.16:0/1304090230 learned_addr learned my addr 172.21.15.16:0/1304090230 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa04be00640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0540cf0b0 0x7fa05414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa052800640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540d1bf0 0x7fa05414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:53.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 -- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540d1bf0 msgr2=0x7fa05414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540d1bf0 0x7fa05414e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 -- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0540cf0b0 msgr2=0x7fa05414de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0540cf0b0 0x7fa05414de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.694+0000 7fa051e00640 1 -- 172.21.15.16:0/1304090230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05415bc60 con 0x7fa0540cc820 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa051e00640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 0x7fa0540b6ea0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa04400e7c0 tx=0x7fa04400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04400ccc0 con 0x7fa0540cc820 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa04400ce80 con 0x7fa0540cc820 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05406e0d0 con 0x7fa0540cc820 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044005280 con 0x7fa0540cc820 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05406fc30 con 0x7fa0540cc820 2024-09-17T05:08:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05415c930 con 0x7fa0540cc820 2024-09-17T05:08:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.698+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa044005420 con 0x7fa0540cc820 2024-09-17T05:08:53.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.702+0000 7fa04a000640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa030073200 0x7fa030075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.702+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5619+0+0 (secure 0 0 0) 0x7fa044094ed0 con 0x7fa0540cc820 2024-09-17T05:08:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.702+0000 7fa04be00640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa030073200 0x7fa030075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.702+0000 7fa04be00640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa030073200 0x7fa030075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0af000 tx=0x7fa03c01b030 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.706+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa04405f970 con 0x7fa0540cc820 2024-09-17T05:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:53.878+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa054071580 con 0x7fa0540cc820 2024-09-17T05:08:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.494+0000 7fa04a000640 1 -- 172.21.15.16:0/1304090230 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) ==== 170+0+0 (secure 0 0 0) 0x7fa0440643a0 con 0x7fa0540cc820 2024-09-17T05:08:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa030073200 msgr2=0x7fa030075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa030073200 0x7fa030075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0af000 tx=0x7fa03c01b030 comp rx=0 tx=0).stop 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 msgr2=0x7fa0540b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 0x7fa0540b6ea0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa04400e7c0 tx=0x7fa04400ec80 comp rx=0 tx=0).stop 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 shutdown_connections 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa030073200 0x7fa030075710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa0540d1bf0 0x7fa05414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0540cf0b0 0x7fa05414de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 --2- 172.21.15.16:0/1304090230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0540cc820 0x7fa0540b6ea0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 >> 172.21.15.16:0/1304090230 conn(0x7fa0540b4000 msgr2=0x7fa0540b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 shutdown_connections 2024-09-17T05:08:54.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.498+0000 7fa053200640 1 -- 172.21.15.16:0/1304090230 wait complete. 2024-09-17T05:08:54.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 -- 172.21.15.16:0/4171089546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 msgr2=0x7f6d5c0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 --2- 172.21.15.16:0/4171089546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c0ce5d0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0ce2c0 tx=0x7f6d4401c820 comp rx=0 tx=0).stop 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 -- 172.21.15.16:0/4171089546 shutdown_connections 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 --2- 172.21.15.16:0/4171089546 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d5c15f6f0 0x7f6d5c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 --2- 172.21.15.16:0/4171089546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c0ce5d0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 -- 172.21.15.16:0/4171089546 >> 172.21.15.16:0/4171089546 conn(0x7f6d5c0b2250 msgr2=0x7f6d5c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 -- 172.21.15.16:0/4171089546 shutdown_connections 2024-09-17T05:08:54.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 -- 172.21.15.16:0/4171089546 wait complete. 2024-09-17T05:08:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 Processor -- start 2024-09-17T05:08:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.706+0000 7f6d61400640 1 -- start start 2024-09-17T05:08:54.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d5c074a20 0x7f6d5c1f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5c15f6f0 0x7f6d5c1f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c168e80 con 0x7f6d5c074e30 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c168d00 con 0x7f6d5c15f6f0 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c169000 con 0x7f6d5c074a20 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52428/0 (socket says 172.21.15.16:52428) 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 -- 172.21.15.16:0/341581404 learned_addr learned my addr 172.21.15.16:0/341581404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d5a800640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d5c074a20 0x7f6d5c1f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d5b200640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5c15f6f0 0x7f6d5c1f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:54.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 -- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d5c074a20 msgr2=0x7f6d5c1f9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d5c074a20 0x7f6d5c1f9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 -- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5c15f6f0 msgr2=0x7f6d5c1f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5c15f6f0 0x7f6d5c1f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 -- 172.21.15.16:0/341581404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c0c7260 con 0x7f6d5c074e30 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d5b200640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5c15f6f0 0x7f6d5c1f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d53e00640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c154170 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6d44000c10 tx=0x7f6d440a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44004140 con 0x7f6d5c074e30 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d440043d0 con 0x7f6d5c074e30 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c158cf0 con 0x7f6d5c074e30 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44005280 con 0x7f6d5c074e30 2024-09-17T05:08:54.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1546b0 con 0x7f6d5c074e30 2024-09-17T05:08:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.710+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c15efe0 con 0x7f6d5c074e30 2024-09-17T05:08:54.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.714+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d440bf020 con 0x7f6d5c074e30 2024-09-17T05:08:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.714+0000 7f6d52a00640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d38073230 0x7f6d38075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.714+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6001+0+0 (secure 0 0 0) 0x7f6d4412f4b0 con 0x7f6d5c074e30 2024-09-17T05:08:54.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.714+0000 7f6d5a800640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d38073230 0x7f6d38075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:54.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.714+0000 7f6d5a800640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d38073230 0x7f6d38075740 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0046c0 tx=0x7f6d4c007ec0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:54.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.718+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d440c6020 con 0x7f6d5c074e30 2024-09-17T05:08:54.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:54.926+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6d5c164e30 con 0x7f6d5c074e30 2024-09-17T05:08:55.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.494+0000 7f6d52a00640 1 -- 172.21.15.16:0/341581404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v82) ==== 174+0+0 (secure 0 0 0) 0x7f6d44016410 con 0x7f6d5c074e30 2024-09-17T05:08:55.496 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 19 and data pool 20 2024-09-17T05:08:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d38073230 msgr2=0x7f6d38075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d38073230 0x7f6d38075740 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0046c0 tx=0x7f6d4c007ec0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 msgr2=0x7f6d5c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c154170 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6d44000c10 tx=0x7f6d440a6760 comp rx=0 tx=0).stop 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 shutdown_connections 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d38073230 0x7f6d38075740 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5c15f6f0 0x7f6d5c1f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c074e30 0x7f6d5c154170 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 --2- 172.21.15.16:0/341581404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d5c074a20 0x7f6d5c1f9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 >> 172.21.15.16:0/341581404 conn(0x7f6d5c0b2250 msgr2=0x7f6d5c164720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:55.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 shutdown_connections 2024-09-17T05:08:55.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.502+0000 7f6d61400640 1 -- 172.21.15.16:0/341581404 wait complete. 2024-09-17T05:08:55.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b23000640 1 -- 172.21.15.16:0/3699990565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b000039f0 msgr2=0x7f9b00003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b23000640 1 --2- 172.21.15.16:0/3699990565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b000039f0 0x7f9b00003e20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c260 tx=0x7f9b0c01cf60 comp rx=0 tx=0).stop 2024-09-17T05:08:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b21000640 1 -- 172.21.15.16:0/3699990565 reap_dead start 2024-09-17T05:08:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b23000640 1 -- 172.21.15.16:0/3699990565 shutdown_connections 2024-09-17T05:08:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b23000640 1 -- 172.21.15.16:0/3699990565 >> 172.21.15.16:0/3699990565 conn(0x7f9b1c06f900 msgr2=0x7f9b1c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:55.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b23000640 1 -- 172.21.15.16:0/3699990565 shutdown_connections 2024-09-17T05:08:55.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.710+0000 7f9b23000640 1 -- 172.21.15.16:0/3699990565 wait complete. 2024-09-17T05:08:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 Processor -- start 2024-09-17T05:08:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 -- start start 2024-09-17T05:08:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b000039f0 0x7f9b1c1408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b1c156500 0x7f9b1c156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 0x7f9b1c1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c167670 con 0x7f9b1c156e50 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c1674f0 con 0x7f9b1c156500 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b23000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c1677f0 con 0x7f9b000039f0 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 0x7f9b1c1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b21000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 0x7f9b1c1f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52448/0 (socket says 172.21.15.16:52448) 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b21000640 1 -- 172.21.15.16:0/4215481939 learned_addr learned my addr 172.21.15.16:0/4215481939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b1be00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b000039f0 0x7f9b1c1408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.714+0000 7f9b13e00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b1c156500 0x7f9b1c156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b21000640 1 -- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b000039f0 msgr2=0x7f9b1c1408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b21000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b000039f0 0x7f9b1c1408a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b21000640 1 -- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b1c156500 msgr2=0x7f9b1c156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b21000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b1c156500 0x7f9b1c156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b21000640 1 -- 172.21.15.16:0/4215481939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c15a100 con 0x7f9b1c156e50 2024-09-17T05:08:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b1be00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b000039f0 0x7f9b1c1408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b13e00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b1c156500 0x7f9b1c156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b21000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 0x7f9b1c1f61e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c15f0b0 tx=0x7f9b1400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14004b30 con 0x7f9b1c156e50 2024-09-17T05:08:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b14004cd0 con 0x7f9b1c156e50 2024-09-17T05:08:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14003be0 con 0x7f9b1c156e50 2024-09-17T05:08:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c15a630 con 0x7f9b1c156e50 2024-09-17T05:08:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c15abf0 con 0x7f9b1c156e50 2024-09-17T05:08:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1c15a3a0 con 0x7f9b1c156e50 2024-09-17T05:08:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.718+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b14004030 con 0x7f9b1c156e50 2024-09-17T05:08:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.722+0000 7f9b18c00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9af8073200 0x7f9af8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.722+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f9b14094800 con 0x7f9b1c156e50 2024-09-17T05:08:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.722+0000 7f9b1be00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9af8073200 0x7f9af8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.722+0000 7f9b1be00640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9af8073200 0x7f9af8075710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0ac000 tx=0x7f9b0c019da0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.726+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9b1405f680 con 0x7f9b1c156e50 2024-09-17T05:08:55.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.898+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b1c0b4350 con 0x7f9b1c156e50 2024-09-17T05:08:55.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b18c00640 1 -- 172.21.15.16:0/4215481939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16945 (secure 0 0 0) 0x7f9b1405ef80 con 0x7f9b1c156e50 2024-09-17T05:08:55.902 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:55.902 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":124,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:54.931610+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:52.684828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:53.893903+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9af8073200 msgr2=0x7f9af8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9af8073200 0x7f9af8075710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0ac000 tx=0x7f9b0c019da0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 msgr2=0x7f9b1c1f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 0x7f9b1c1f61e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c15f0b0 tx=0x7f9b1400f3c0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 shutdown_connections 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9af8073200 0x7f9af8075710 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b1c156e50 0x7f9b1c1f61e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b1c156500 0x7f9b1c156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 --2- 172.21.15.16:0/4215481939 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9b000039f0 0x7f9b1c1408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:55.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 >> 172.21.15.16:0/4215481939 conn(0x7f9b1c06f900 msgr2=0x7f9b1c155810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 shutdown_connections 2024-09-17T05:08:55.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:55.902+0000 7f9b23000640 1 -- 172.21.15.16:0/4215481939 wait complete. 2024-09-17T05:08:55.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 -- 172.21.15.16:0/2113588911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6015c270 msgr2=0x7f6d6015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 --2- 172.21.15.16:0/2113588911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6015c270 0x7f6d6015c680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6d6015cc70 tx=0x7f6d4801c830 comp rx=0 tx=0).stop 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 -- 172.21.15.16:0/2113588911 shutdown_connections 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 --2- 172.21.15.16:0/2113588911 >> v2:172.21.15.104:3300/0 conn(0x7f6d60073240 0x7f6d60073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 --2- 172.21.15.16:0/2113588911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6015c270 0x7f6d6015c680 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 -- 172.21.15.16:0/2113588911 >> 172.21.15.16:0/2113588911 conn(0x7f6d6006f900 msgr2=0x7f6d600b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 -- 172.21.15.16:0/2113588911 shutdown_connections 2024-09-17T05:08:56.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.082+0000 7f6d65400640 1 -- 172.21.15.16:0/2113588911 wait complete. 2024-09-17T05:08:56.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 Processor -- start 2024-09-17T05:08:56.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 -- start start 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 0x7f6d6015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d600b2f50 0x7f6d60160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d6015c270 0x7f6d601fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60168c70 con 0x7f6d60073240 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60168af0 con 0x7f6d600b2f50 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6d60168df0 con 0x7f6d6015c270 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 0x7f6d6015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 0x7f6d6015a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52454/0 (socket says 172.21.15.16:52454) 2024-09-17T05:08:56.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 -- 172.21.15.16:0/3902382470 learned_addr learned my addr 172.21.15.16:0/3902382470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d55e00640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d600b2f50 0x7f6d60160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5f200640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d6015c270 0x7f6d601fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 -- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d6015c270 msgr2=0x7f6d601fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d6015c270 0x7f6d601fa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 -- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d600b2f50 msgr2=0x7f6d60160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d600b2f50 0x7f6d60160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 -- 172.21.15.16:0/3902382470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d601611d0 con 0x7f6d60073240 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d5e800640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 0x7f6d6015a330 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c3f0 tx=0x7f6d5000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50017070 con 0x7f6d60073240 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d500053c0 con 0x7f6d60073240 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50003860 con 0x7f6d60073240 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.086+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d60159290 con 0x7f6d60073240 2024-09-17T05:08:56.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.090+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d60159850 con 0x7f6d60073240 2024-09-17T05:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.090+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d60073650 con 0x7f6d60073240 2024-09-17T05:08:56.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.090+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d5001a070 con 0x7f6d60073240 2024-09-17T05:08:56.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.090+0000 7f6d57400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d3c073200 0x7f6d3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.090+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7f6d50093670 con 0x7f6d60073240 2024-09-17T05:08:56.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.090+0000 7f6d55e00640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d3c073200 0x7f6d3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.094+0000 7f6d55e00640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d3c073200 0x7f6d3c075710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6d480af000 tx=0x7f6d4801b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:56.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.094+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d5005dea0 con 0x7f6d60073240 2024-09-17T05:08:56.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.270+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d6015e6c0 con 0x7f6d60073240 2024-09-17T05:08:56.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d57400640 1 -- 172.21.15.16:0/3902382470 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+16945 (secure 0 0 0) 0x7f6d500628d0 con 0x7f6d60073240 2024-09-17T05:08:56.273 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:56.273 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":124,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:54.931610+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:52.684828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:53.893903+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d3c073200 msgr2=0x7f6d3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d3c073200 0x7f6d3c075710 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6d480af000 tx=0x7f6d4801b070 comp rx=0 tx=0).stop 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 msgr2=0x7f6d6015a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 0x7f6d6015a330 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c3f0 tx=0x7f6d5000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 shutdown_connections 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d3c073200 0x7f6d3c075710 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d6015c270 0x7f6d601fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d600b2f50 0x7f6d60160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 --2- 172.21.15.16:0/3902382470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d60073240 0x7f6d6015a330 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 >> 172.21.15.16:0/3902382470 conn(0x7f6d6006f900 msgr2=0x7f6d6006dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 shutdown_connections 2024-09-17T05:08:56.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.274+0000 7f6d65400640 1 -- 172.21.15.16:0/3902382470 wait complete. 2024-09-17T05:08:56.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:08:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 -- 172.21.15.16:0/1154797794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc8c003c40 msgr2=0x7fbc8c004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 --2- 172.21.15.16:0/1154797794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc8c003c40 0x7fbc8c004090 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fbc98006fa0 tx=0x7fbc9801ceb0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 -- 172.21.15.16:0/1154797794 shutdown_connections 2024-09-17T05:08:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 --2- 172.21.15.16:0/1154797794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc8c003c40 0x7fbc8c004090 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 -- 172.21.15.16:0/1154797794 >> 172.21.15.16:0/1154797794 conn(0x7fbca406f9a0 msgr2=0x7fbca40b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 -- 172.21.15.16:0/1154797794 shutdown_connections 2024-09-17T05:08:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.458+0000 7fbcaae00640 1 -- 172.21.15.16:0/1154797794 wait complete. 2024-09-17T05:08:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 Processor -- start 2024-09-17T05:08:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 -- start start 2024-09-17T05:08:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc8c003c40 0x7fbca41f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbca4153eb0 0x7fbca415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 0x7fbca41e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbca415bfc0 con 0x7fbca4153740 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbca415be40 con 0x7fbc8c003c40 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbca415c140 con 0x7fbca4153eb0 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 0x7fbca41e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaa400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 0x7fbca41e6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52480/0 (socket says 172.21.15.16:52480) 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbcaa400640 1 -- 172.21.15.16:0/325075158 learned_addr learned my addr 172.21.15.16:0/325075158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbca9a00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc8c003c40 0x7fbca41f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.462+0000 7fbca9000640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbca4153eb0 0x7fbca415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaa400640 1 -- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbca4153eb0 msgr2=0x7fbca415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaa400640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbca4153eb0 0x7fbca415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaa400640 1 -- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc8c003c40 msgr2=0x7fbca41f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaa400640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc8c003c40 0x7fbca41f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaa400640 1 -- 172.21.15.16:0/325075158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca415ae00 con 0x7fbca4153740 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaa400640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 0x7fbca41e6f20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbca415c8f0 tx=0x7fbc9c001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c015020 con 0x7fbca4153740 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc9c011960 con 0x7fbca4153740 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c011d90 con 0x7fbca4153740 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca4159820 con 0x7fbca4153740 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca4159de0 con 0x7fbca4153740 2024-09-17T05:08:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca406abe0 con 0x7fbca4153740 2024-09-17T05:08:56.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.466+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbc9c018020 con 0x7fbca4153740 2024-09-17T05:08:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.470+0000 7fbca1600640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc84073200 0x7fbc84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.470+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6101+0+0 (secure 0 0 0) 0x7fbc9c094e40 con 0x7fbca4153740 2024-09-17T05:08:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.470+0000 7fbca9a00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc84073200 0x7fbc84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.470+0000 7fbca9a00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc84073200 0x7fbc84075710 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbc980ab000 tx=0x7fbc98019cf0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.474+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbc9c01f020 con 0x7fbca4153740 2024-09-17T05:08:56.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:56.694+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbca415c6a0 con 0x7fbca4153740 2024-09-17T05:08:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.526+0000 7fbca1600640 1 -- 172.21.15.16:0/325075158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7fbc9c05fd50 con 0x7fbca4153740 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc84073200 msgr2=0x7fbc84075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc84073200 0x7fbc84075710 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbc980ab000 tx=0x7fbc98019cf0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 msgr2=0x7fbca41e6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 0x7fbca41e6f20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fbca415c8f0 tx=0x7fbc9c001b90 comp rx=0 tx=0).stop 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 shutdown_connections 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbc84073200 0x7fbc84075710 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbca4153740 0x7fbca41e6f20 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbca4153eb0 0x7fbca415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 --2- 172.21.15.16:0/325075158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc8c003c40 0x7fbca41f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 >> 172.21.15.16:0/325075158 conn(0x7fbca406f9a0 msgr2=0x7fbca41559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 shutdown_connections 2024-09-17T05:08:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.530+0000 7fbcaae00640 1 -- 172.21.15.16:0/325075158 wait complete. 2024-09-17T05:08:57.554 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:08:57.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:08:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 -- 172.21.15.16:0/1521694075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d280b5a00 msgr2=0x7f7d2815c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 --2- 172.21.15.16:0/1521694075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d280b5a00 0x7f7d2815c570 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009840 tx=0x7f7d1801c820 comp rx=0 tx=0).stop 2024-09-17T05:08:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 -- 172.21.15.16:0/1521694075 shutdown_connections 2024-09-17T05:08:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 --2- 172.21.15.16:0/1521694075 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d28073170 0x7f7d28073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 --2- 172.21.15.16:0/1521694075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d280b5a00 0x7f7d2815c570 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 -- 172.21.15.16:0/1521694075 >> 172.21.15.16:0/1521694075 conn(0x7f7d280b2250 msgr2=0x7f7d280d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 -- 172.21.15.16:0/1521694075 shutdown_connections 2024-09-17T05:08:57.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 -- 172.21.15.16:0/1521694075 wait complete. 2024-09-17T05:08:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 Processor -- start 2024-09-17T05:08:57.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.722+0000 7f7d2de00640 1 -- start start 2024-09-17T05:08:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d28073170 0x7f7d2814c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d280b5a00 0x7f7d2815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 0x7f7d2815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2816a280 con 0x7f7d28160270 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2816a100 con 0x7f7d28073170 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2816a400 con 0x7f7d280b5a00 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 0x7f7d2815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 0x7f7d2815d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37304/0 (socket says 172.21.15.16:37304) 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 -- 172.21.15.16:0/668806755 learned_addr learned my addr 172.21.15.16:0/668806755 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d26a00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d28073170 0x7f7d2814c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d1fe00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d280b5a00 0x7f7d2815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 -- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d280b5a00 msgr2=0x7f7d2815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d280b5a00 0x7f7d2815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 -- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d28073170 msgr2=0x7f7d2814c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d28073170 0x7f7d2814c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 -- 172.21.15.16:0/668806755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2815f840 con 0x7f7d28160270 2024-09-17T05:08:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d27400640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 0x7f7d2815d760 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c730 tx=0x7f7d2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20003b80 con 0x7f7d28160270 2024-09-17T05:08:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d20003d40 con 0x7f7d28160270 2024-09-17T05:08:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2815de80 con 0x7f7d28160270 2024-09-17T05:08:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d200052d0 con 0x7f7d28160270 2024-09-17T05:08:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2814cbc0 con 0x7f7d28160270 2024-09-17T05:08:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.726+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2814cea0 con 0x7f7d28160270 2024-09-17T05:08:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.730+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7d20005680 con 0x7f7d28160270 2024-09-17T05:08:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.730+0000 7f7d1f400640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d04073230 0x7f7d04075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.730+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f7d20094c30 con 0x7f7d28160270 2024-09-17T05:08:57.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.730+0000 7f7d26a00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d04073230 0x7f7d04075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:57.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.730+0000 7f7d26a00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d04073230 0x7f7d04075740 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7d180af000 tx=0x7f7d1801b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:57.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.734+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d28073ac0 con 0x7f7d28160270 2024-09-17T05:08:57.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:57.942+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7d28073580 con 0x7f7d28160270 2024-09-17T05:08:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.534+0000 7f7d1f400640 1 -- 172.21.15.16:0/668806755 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f7d2005f5e0 con 0x7f7d28160270 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d04073230 msgr2=0x7f7d04075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d04073230 0x7f7d04075740 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7d180af000 tx=0x7f7d1801b060 comp rx=0 tx=0).stop 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 msgr2=0x7f7d2815d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 0x7f7d2815d760 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000c730 tx=0x7f7d2000cbf0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 shutdown_connections 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7d04073230 0x7f7d04075740 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d28160270 0x7f7d2815d760 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f7d280b5a00 0x7f7d2815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 --2- 172.21.15.16:0/668806755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d28073170 0x7f7d2814c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 >> 172.21.15.16:0/668806755 conn(0x7f7d280b2250 msgr2=0x7f7d28154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:58.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 shutdown_connections 2024-09-17T05:08:58.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.538+0000 7f7d2de00640 1 -- 172.21.15.16:0/668806755 wait complete. 2024-09-17T05:08:58.563 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:08:58.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:08:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 -- 172.21.15.16:0/955793821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd03806e6b0 msgr2=0x7fd038155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 --2- 172.21.15.16:0/955793821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd03806e6b0 0x7fd038155160 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd024009900 tx=0x7fd02401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 -- 172.21.15.16:0/955793821 shutdown_connections 2024-09-17T05:08:58.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 --2- 172.21.15.16:0/955793821 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd0381556a0 0x7fd038155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 --2- 172.21.15.16:0/955793821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd03806e6b0 0x7fd038155160 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 -- 172.21.15.16:0/955793821 >> 172.21.15.16:0/955793821 conn(0x7fd03806f9c0 msgr2=0x7fd0380b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 -- 172.21.15.16:0/955793821 shutdown_connections 2024-09-17T05:08:58.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.726+0000 7fd03ea00640 1 -- 172.21.15.16:0/955793821 wait complete. 2024-09-17T05:08:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 Processor -- start 2024-09-17T05:08:58.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 -- start start 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 0x7fd03814d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd03815c610 0x7fd038157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd03814f670 0x7fd03814fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd03815f800 con 0x7fd0381556a0 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd0380cfcf0 con 0x7fd03814f670 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd03815f980 con 0x7fd03815c610 2024-09-17T05:08:58.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 0x7fd03814d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 0x7fd03814d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37322/0 (socket says 172.21.15.16:37322) 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 -- 172.21.15.16:0/2771757808 learned_addr learned my addr 172.21.15.16:0/2771757808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03e000640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd03814f670 0x7fd03814fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03cc00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd03815c610 0x7fd038157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 -- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd03815c610 msgr2=0x7fd038157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd03815c610 0x7fd038157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 -- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd03814f670 msgr2=0x7fd03814fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd03814f670 0x7fd03814fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 -- 172.21.15.16:0/2771757808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03814ffc0 con 0x7fd0381556a0 2024-09-17T05:08:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03cc00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd03815c610 0x7fd038157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:08:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03d600640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 0x7fd03814d0e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c3c0 tx=0x7fd02c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c017070 con 0x7fd0381556a0 2024-09-17T05:08:58.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd02c0053c0 con 0x7fd0381556a0 2024-09-17T05:08:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0381529b0 con 0x7fd0381556a0 2024-09-17T05:08:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.730+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c003860 con 0x7fd0381556a0 2024-09-17T05:08:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.734+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd038152c70 con 0x7fd0381556a0 2024-09-17T05:08:58.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.734+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd038155160 con 0x7fd0381556a0 2024-09-17T05:08:58.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.734+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd02c01a070 con 0x7fd0381556a0 2024-09-17T05:08:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.734+0000 7fd035600640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd018073200 0x7fd018075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.734+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fd02c0936f0 con 0x7fd0381556a0 2024-09-17T05:08:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.734+0000 7fd03cc00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd018073200 0x7fd018075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.738+0000 7fd03cc00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd018073200 0x7fd018075710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0240af000 tx=0x7fd02401b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:58.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.738+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd03814ffc0 con 0x7fd0381556a0 2024-09-17T05:08:58.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0380b33d0 con 0x7fd0381556a0 2024-09-17T05:08:58.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd035600640 1 -- 172.21.15.16:0/2771757808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5225 (secure 0 0 0) 0x7fd02c05df20 con 0x7fd0381556a0 2024-09-17T05:08:58.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:58.949 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"btime":"2024-09-17T05:08:58:530125+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:08:57.947398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:08:58.949 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 86 2024-09-17T05:08:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd018073200 msgr2=0x7fd018075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd018073200 0x7fd018075710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd0240af000 tx=0x7fd02401b120 comp rx=0 tx=0).stop 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 msgr2=0x7fd03814d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 0x7fd03814d0e0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c3c0 tx=0x7fd02c00c880 comp rx=0 tx=0).stop 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 shutdown_connections 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd018073200 0x7fd018075710 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd03814f670 0x7fd03814fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd03815c610 0x7fd038157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 --2- 172.21.15.16:0/2771757808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0381556a0 0x7fd03814d0e0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 >> 172.21.15.16:0/2771757808 conn(0x7fd03806f9c0 msgr2=0x7fd0381584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 shutdown_connections 2024-09-17T05:08:58.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:58.950+0000 7fd03ea00640 1 -- 172.21.15.16:0/2771757808 wait complete. 2024-09-17T05:08:58.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:08:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 -- 172.21.15.16:0/2744355305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840d0830 msgr2=0x7f5684070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 --2- 172.21.15.16:0/2744355305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840d0830 0x7f5684070490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5670009840 tx=0x7f567001c700 comp rx=0 tx=0).stop 2024-09-17T05:08:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 -- 172.21.15.16:0/2744355305 shutdown_connections 2024-09-17T05:08:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 --2- 172.21.15.16:0/2744355305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840d0830 0x7f5684070490 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 --2- 172.21.15.16:0/2744355305 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56840d1aa0 0x7f56840d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 -- 172.21.15.16:0/2744355305 >> 172.21.15.16:0/2744355305 conn(0x7f56840b4160 msgr2=0x7f56840b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 -- 172.21.15.16:0/2744355305 shutdown_connections 2024-09-17T05:08:59.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.130+0000 7f5689e00640 1 -- 172.21.15.16:0/2744355305 wait complete. 2024-09-17T05:08:59.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 Processor -- start 2024-09-17T05:08:59.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 -- start start 2024-09-17T05:08:59.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840709d0 0x7f56841537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 0x7f568406e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56840d1aa0 0x7f568406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f568416a1c0 con 0x7f56840d0830 2024-09-17T05:08:59.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f568416a040 con 0x7f56840709d0 2024-09-17T05:08:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f568416a340 con 0x7f56840d1aa0 2024-09-17T05:08:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5683e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 0x7f568406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5683e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 0x7f568406e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37336/0 (socket says 172.21.15.16:37336) 2024-09-17T05:08:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840709d0 0x7f56841537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5683e00640 1 -- 172.21.15.16:0/2401088267 learned_addr learned my addr 172.21.15.16:0/2401088267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:59.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689400640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56840d1aa0 0x7f568406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 -- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56840d1aa0 msgr2=0x7f568406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56840d1aa0 0x7f568406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 -- 172.21.15.16:0/2401088267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 msgr2=0x7f568406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 0x7f568406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 -- 172.21.15.16:0/2401088267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5684153cf0 con 0x7f56840709d0 2024-09-17T05:08:59.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5683e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 0x7f568406e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5688a00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840709d0 0x7f56841537b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f567800ea20 tx=0x7f567800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567800ccc0 con 0x7f56840709d0 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f567800ce60 con 0x7f56840709d0 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678004280 con 0x7f56840709d0 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56841f96d0 con 0x7f56840709d0 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.134+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56841548b0 con 0x7f56840709d0 2024-09-17T05:08:59.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.138+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5684070490 con 0x7f56840709d0 2024-09-17T05:08:59.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.138+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5678003600 con 0x7f56840709d0 2024-09-17T05:08:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.138+0000 7f5680c00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5664073200 0x7f5664075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.138+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f5678094d50 con 0x7f56840709d0 2024-09-17T05:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.138+0000 7f5683e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5664073200 0x7f5664075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.142+0000 7f5683e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5664073200 0x7f5664075710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5670019370 tx=0x7f567001afa0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:59.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.142+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f567805f690 con 0x7f56840709d0 2024-09-17T05:08:59.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.322+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56840ccf60 con 0x7f56840709d0 2024-09-17T05:08:59.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.322+0000 7f5680c00640 1 -- 172.21.15.16:0/2401088267 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16932 (secure 0 0 0) 0x7f56780640c0 con 0x7f56840709d0 2024-09-17T05:08:59.323 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:59.323 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":125,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:56.498941+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:52.684828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:53.893903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5664073200 msgr2=0x7f5664075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5664073200 0x7f5664075710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5670019370 tx=0x7f567001afa0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840709d0 msgr2=0x7f56841537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840709d0 0x7f56841537b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f567800ea20 tx=0x7f567800eee0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 shutdown_connections 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5664073200 0x7f5664075710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f56840d1aa0 0x7f568406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56840d0830 0x7f568406e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 --2- 172.21.15.16:0/2401088267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56840709d0 0x7f56841537b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 >> 172.21.15.16:0/2401088267 conn(0x7f56840b4160 msgr2=0x7f568415f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:59.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 shutdown_connections 2024-09-17T05:08:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.326+0000 7f5689e00640 1 -- 172.21.15.16:0/2401088267 wait complete. 2024-09-17T05:08:59.346 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:08:59.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 -- 172.21.15.16:0/2168509135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9200b2f50 msgr2=0x7fb92015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 --2- 172.21.15.16:0/2168509135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9200b2f50 0x7fb92015bb60 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb92006e960 tx=0x7fb91001c830 comp rx=0 tx=0).stop 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 -- 172.21.15.16:0/2168509135 shutdown_connections 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 --2- 172.21.15.16:0/2168509135 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb92015f150 0x7fb92015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 --2- 172.21.15.16:0/2168509135 >> v2:172.21.15.104:3300/0 conn(0x7fb92015c0a0 0x7fb92015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 --2- 172.21.15.16:0/2168509135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9200b2f50 0x7fb92015bb60 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 -- 172.21.15.16:0/2168509135 >> 172.21.15.16:0/2168509135 conn(0x7fb92006f900 msgr2=0x7fb9200b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 -- 172.21.15.16:0/2168509135 shutdown_connections 2024-09-17T05:08:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 -- 172.21.15.16:0/2168509135 wait complete. 2024-09-17T05:08:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 Processor -- start 2024-09-17T05:08:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.506+0000 7fb925a00640 1 -- start start 2024-09-17T05:08:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9200b2f50 0x7fb92013b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb92015c0a0 0x7fb920155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 0x7fb92015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb92006e3d0 con 0x7fb92015f150 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb92006e250 con 0x7fb92015c0a0 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb92006e550 con 0x7fb9200b2f50 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 0x7fb92015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 0x7fb92015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37352/0 (socket says 172.21.15.16:37352) 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 -- 172.21.15.16:0/2840883056 learned_addr learned my addr 172.21.15.16:0/2840883056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb916000640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb92015c0a0 0x7fb920155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91ea00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9200b2f50 0x7fb92013b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 -- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9200b2f50 msgr2=0x7fb92013b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9200b2f50 0x7fb92013b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 -- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb92015c0a0 msgr2=0x7fb920155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb92015c0a0 0x7fb920155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 -- 172.21.15.16:0/2840883056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92006ff70 con 0x7fb92015f150 2024-09-17T05:08:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb91f400640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 0x7fb92015a630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb91800ca80 tx=0x7fb91800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91800ae20 con 0x7fb92015f150 2024-09-17T05:08:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb91800f560 con 0x7fb92015f150 2024-09-17T05:08:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9201fbc60 con 0x7fb92015f150 2024-09-17T05:08:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91800f9b0 con 0x7fb92015f150 2024-09-17T05:08:59.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201fc210 con 0x7fb92015f150 2024-09-17T05:08:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.510+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92015c4b0 con 0x7fb92015f150 2024-09-17T05:08:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.514+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb91800fb50 con 0x7fb92015f150 2024-09-17T05:08:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.514+0000 7fb917400640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8fc073230 0x7fb8fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.514+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fb918094e70 con 0x7fb92015f150 2024-09-17T05:08:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.514+0000 7fb91ea00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8fc073230 0x7fb8fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.514+0000 7fb91ea00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8fc073230 0x7fb8fc075740 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb9100af000 tx=0x7fb9100a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:59.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.518+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb918014030 con 0x7fb92015f150 2024-09-17T05:08:59.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.698+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb920155880 con 0x7fb8fc073230 2024-09-17T05:08:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.702+0000 7fb917400640 1 -- 172.21.15.16:0/2840883056 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7fb920155880 con 0x7fb8fc073230 2024-09-17T05:08:59.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:08:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:08:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8fc073230 msgr2=0x7fb8fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8fc073230 0x7fb8fc075740 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb9100af000 tx=0x7fb9100a7040 comp rx=0 tx=0).stop 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 msgr2=0x7fb92015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 0x7fb92015a630 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fb91800ca80 tx=0x7fb91800cf40 comp rx=0 tx=0).stop 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 shutdown_connections 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb8fc073230 0x7fb8fc075740 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb92015f150 0x7fb92015a630 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb92015c0a0 0x7fb920155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 --2- 172.21.15.16:0/2840883056 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb9200b2f50 0x7fb92013b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 >> 172.21.15.16:0/2840883056 conn(0x7fb92006f900 msgr2=0x7fb9200d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:59.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 shutdown_connections 2024-09-17T05:08:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.706+0000 7fb925a00640 1 -- 172.21.15.16:0/2840883056 wait complete. 2024-09-17T05:08:59.726 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-17T05:08:59.592055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275328,"kb_used_data":13308,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754699392,"statfs":{"total":773094113280,"available":772812177408,"internally_reserved":0,"allocated":13627392,"data_stored":6904145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001577"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.523213+0000","last_change":"2024-09-17T05:08:55.514755+0000","last_active":"2024-09-17T05:08:55.523213+0000","last_peered":"2024-09-17T05:08:55.523213+0000","last_clean":"2024-09-17T05:08:55.523213+0000","last_became_active":"2024-09-17T05:08:55.514429+0000","last_became_peered":"2024-09-17T05:08:55.514429+0000","last_unstale":"2024-09-17T05:08:55.523213+0000","last_undegraded":"2024-09-17T05:08:55.523213+0000","last_fullsized":"2024-09-17T05:08:55.523213+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:35.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511740+0000","last_change":"2024-09-17T05:08:55.545468+0000","last_active":"2024-09-17T05:08:57.511740+0000","last_peered":"2024-09-17T05:08:57.511740+0000","last_clean":"2024-09-17T05:08:57.511740+0000","last_became_active":"2024-09-17T05:08:55.545138+0000","last_became_peered":"2024-09-17T05:08:55.545138+0000","last_unstale":"2024-09-17T05:08:57.511740+0000","last_undegraded":"2024-09-17T05:08:57.511740+0000","last_fullsized":"2024-09-17T05:08:57.511740+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:39.013033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710192+0000","last_change":"2024-09-17T05:08:55.549337+0000","last_active":"2024-09-17T05:08:56.710192+0000","last_peered":"2024-09-17T05:08:56.710192+0000","last_clean":"2024-09-17T05:08:56.710192+0000","last_became_active":"2024-09-17T05:08:55.548925+0000","last_became_peered":"2024-09-17T05:08:55.548925+0000","last_unstale":"2024-09-17T05:08:56.710192+0000","last_undegraded":"2024-09-17T05:08:56.710192+0000","last_fullsized":"2024-09-17T05:08:56.710192+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:01.617056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710006+0000","last_change":"2024-09-17T05:08:55.553974+0000","last_active":"2024-09-17T05:08:56.710006+0000","last_peered":"2024-09-17T05:08:56.710006+0000","last_clean":"2024-09-17T05:08:56.710006+0000","last_became_active":"2024-09-17T05:08:55.553602+0000","last_became_peered":"2024-09-17T05:08:55.553602+0000","last_unstale":"2024-09-17T05:08:56.710006+0000","last_undegraded":"2024-09-17T05:08:56.710006+0000","last_fullsized":"2024-09-17T05:08:56.710006+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:21.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188346+0000","last_change":"2024-09-17T05:08:55.535322+0000","last_active":"2024-09-17T05:08:57.188346+0000","last_peered":"2024-09-17T05:08:57.188346+0000","last_clean":"2024-09-17T05:08:57.188346+0000","last_became_active":"2024-09-17T05:08:55.534956+0000","last_became_peered":"2024-09-17T05:08:55.534956+0000","last_unstale":"2024-09-17T05:08:57.188346+0000","last_undegraded":"2024-09-17T05:08:57.188346+0000","last_fullsized":"2024-09-17T05:08:57.188346+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:09.991327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.187958+0000","last_change":"2024-09-17T05:08:55.531482+0000","last_active":"2024-09-17T05:08:57.187958+0000","last_peered":"2024-09-17T05:08:57.187958+0000","last_clean":"2024-09-17T05:08:57.187958+0000","last_became_active":"2024-09-17T05:08:55.531168+0000","last_became_peered":"2024-09-17T05:08:55.531168+0000","last_unstale":"2024-09-17T05:08:57.187958+0000","last_undegraded":"2024-09-17T05:08:57.187958+0000","last_fullsized":"2024-09-17T05:08:57.187958+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:28.424029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511181+0000","last_change":"2024-09-17T05:08:55.548098+0000","last_active":"2024-09-17T05:08:57.511181+0000","last_peered":"2024-09-17T05:08:57.511181+0000","last_clean":"2024-09-17T05:08:57.511181+0000","last_became_active":"2024-09-17T05:08:55.547738+0000","last_became_peered":"2024-09-17T05:08:55.547738+0000","last_unstale":"2024-09-17T05:08:57.511181+0000","last_undegraded":"2024-09-17T05:08:57.511181+0000","last_fullsized":"2024-09-17T05:08:57.511181+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:16.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709833+0000","last_change":"2024-09-17T05:08:55.516230+0000","last_active":"2024-09-17T05:08:56.709833+0000","last_peered":"2024-09-17T05:08:56.709833+0000","last_clean":"2024-09-17T05:08:56.709833+0000","last_became_active":"2024-09-17T05:08:55.515831+0000","last_became_peered":"2024-09-17T05:08:55.515831+0000","last_unstale":"2024-09-17T05:08:56.709833+0000","last_undegraded":"2024-09-17T05:08:56.709833+0000","last_fullsized":"2024-09-17T05:08:56.709833+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:30.388469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188506+0000","last_change":"2024-09-17T05:08:55.540727+0000","last_active":"2024-09-17T05:08:57.188506+0000","last_peered":"2024-09-17T05:08:57.188506+0000","last_clean":"2024-09-17T05:08:57.188506+0000","last_became_active":"2024-09-17T05:08:55.540403+0000","last_became_peered":"2024-09-17T05:08:55.540403+0000","last_unstale":"2024-09-17T05:08:57.188506+0000","last_undegraded":"2024-09-17T05:08:57.188506+0000","last_fullsized":"2024-09-17T05:08:57.188506+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:27.129169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188578+0000","last_change":"2024-09-17T05:08:55.538666+0000","last_active":"2024-09-17T05:08:57.188578+0000","last_peered":"2024-09-17T05:08:57.188578+0000","last_clean":"2024-09-17T05:08:57.188578+0000","last_became_active":"2024-09-17T05:08:55.538412+0000","last_became_peered":"2024-09-17T05:08:55.538412+0000","last_unstale":"2024-09-17T05:08:57.188578+0000","last_undegraded":"2024-09-17T05:08:57.188578+0000","last_fullsized":"2024-09-17T05:08:57.188578+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:21.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922645+0000","last_change":"2024-09-17T05:08:55.525021+0000","last_active":"2024-09-17T05:08:56.922645+0000","last_peered":"2024-09-17T05:08:56.922645+0000","last_clean":"2024-09-17T05:08:56.922645+0000","last_became_active":"2024-09-17T05:08:55.524696+0000","last_became_peered":"2024-09-17T05:08:55.524696+0000","last_unstale":"2024-09-17T05:08:56.922645+0000","last_undegraded":"2024-09-17T05:08:56.922645+0000","last_fullsized":"2024-09-17T05:08:56.922645+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:59.524586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503524+0000","last_change":"2024-09-17T05:08:55.519271+0000","last_active":"2024-09-17T05:08:56.503524+0000","last_peered":"2024-09-17T05:08:56.503524+0000","last_clean":"2024-09-17T05:08:56.503524+0000","last_became_active":"2024-09-17T05:08:55.518944+0000","last_became_peered":"2024-09-17T05:08:55.518944+0000","last_unstale":"2024-09-17T05:08:56.503524+0000","last_undegraded":"2024-09-17T05:08:56.503524+0000","last_fullsized":"2024-09-17T05:08:56.503524+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:30.388469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597934+0000","last_change":"2024-09-17T05:08:55.532207+0000","last_active":"2024-09-17T05:08:57.597934+0000","last_peered":"2024-09-17T05:08:57.597934+0000","last_clean":"2024-09-17T05:08:57.597934+0000","last_became_active":"2024-09-17T05:08:55.531861+0000","last_became_peered":"2024-09-17T05:08:55.531861+0000","last_unstale":"2024-09-17T05:08:57.597934+0000","last_undegraded":"2024-09-17T05:08:57.597934+0000","last_fullsized":"2024-09-17T05:08:57.597934+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:28.875714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.523169+0000","last_change":"2024-09-17T05:08:55.514806+0000","last_active":"2024-09-17T05:08:55.523169+0000","last_peered":"2024-09-17T05:08:55.523169+0000","last_clean":"2024-09-17T05:08:55.523169+0000","last_became_active":"2024-09-17T05:08:55.514458+0000","last_became_peered":"2024-09-17T05:08:55.514458+0000","last_unstale":"2024-09-17T05:08:55.523169+0000","last_undegraded":"2024-09-17T05:08:55.523169+0000","last_fullsized":"2024-09-17T05:08:55.523169+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:30.103241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597637+0000","last_change":"2024-09-17T05:08:55.549618+0000","last_active":"2024-09-17T05:08:57.597637+0000","last_peered":"2024-09-17T05:08:57.597637+0000","last_clean":"2024-09-17T05:08:57.597637+0000","last_became_active":"2024-09-17T05:08:55.549260+0000","last_became_peered":"2024-09-17T05:08:55.549260+0000","last_unstale":"2024-09-17T05:08:57.597637+0000","last_undegraded":"2024-09-17T05:08:57.597637+0000","last_fullsized":"2024-09-17T05:08:57.597637+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:01.036971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.523121+0000","last_change":"2024-09-17T05:08:55.514474+0000","last_active":"2024-09-17T05:08:55.523121+0000","last_peered":"2024-09-17T05:08:55.523121+0000","last_clean":"2024-09-17T05:08:55.523121+0000","last_became_active":"2024-09-17T05:08:55.513109+0000","last_became_peered":"2024-09-17T05:08:55.513109+0000","last_unstale":"2024-09-17T05:08:55.523121+0000","last_undegraded":"2024-09-17T05:08:55.523121+0000","last_fullsized":"2024-09-17T05:08:55.523121+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:36.933091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.533289+0000","last_change":"2024-09-17T05:08:55.529450+0000","last_active":"2024-09-17T05:08:55.533289+0000","last_peered":"2024-09-17T05:08:55.533289+0000","last_clean":"2024-09-17T05:08:55.533289+0000","last_became_active":"2024-09-17T05:08:55.529145+0000","last_became_peered":"2024-09-17T05:08:55.529145+0000","last_unstale":"2024-09-17T05:08:55.533289+0000","last_undegraded":"2024-09-17T05:08:55.533289+0000","last_fullsized":"2024-09-17T05:08:55.533289+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:40.063642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502843+0000","last_change":"2024-09-17T05:08:55.545124+0000","last_active":"2024-09-17T05:08:56.502843+0000","last_peered":"2024-09-17T05:08:56.502843+0000","last_clean":"2024-09-17T05:08:56.502843+0000","last_became_active":"2024-09-17T05:08:55.544838+0000","last_became_peered":"2024-09-17T05:08:55.544838+0000","last_unstale":"2024-09-17T05:08:56.502843+0000","last_undegraded":"2024-09-17T05:08:56.502843+0000","last_fullsized":"2024-09-17T05:08:56.502843+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:12.180060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503567+0000","last_change":"2024-09-17T05:08:55.521067+0000","last_active":"2024-09-17T05:08:56.503567+0000","last_peered":"2024-09-17T05:08:56.503567+0000","last_clean":"2024-09-17T05:08:56.503567+0000","last_became_active":"2024-09-17T05:08:55.520646+0000","last_became_peered":"2024-09-17T05:08:55.520646+0000","last_unstale":"2024-09-17T05:08:56.503567+0000","last_undegraded":"2024-09-17T05:08:56.503567+0000","last_fullsized":"2024-09-17T05:08:56.503567+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:45.051109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503773+0000","last_change":"2024-09-17T05:08:55.551234+0000","last_active":"2024-09-17T05:08:56.503773+0000","last_peered":"2024-09-17T05:08:56.503773+0000","last_clean":"2024-09-17T05:08:56.503773+0000","last_became_active":"2024-09-17T05:08:55.550884+0000","last_became_peered":"2024-09-17T05:08:55.550884+0000","last_unstale":"2024-09-17T05:08:56.503773+0000","last_undegraded":"2024-09-17T05:08:56.503773+0000","last_fullsized":"2024-09-17T05:08:56.503773+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:51.145983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710162+0000","last_change":"2024-09-17T05:08:55.550313+0000","last_active":"2024-09-17T05:08:56.710162+0000","last_peered":"2024-09-17T05:08:56.710162+0000","last_clean":"2024-09-17T05:08:56.710162+0000","last_became_active":"2024-09-17T05:08:55.549992+0000","last_became_peered":"2024-09-17T05:08:55.549992+0000","last_unstale":"2024-09-17T05:08:56.710162+0000","last_undegraded":"2024-09-17T05:08:56.710162+0000","last_fullsized":"2024-09-17T05:08:56.710162+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:03.070280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597513+0000","last_change":"2024-09-17T05:08:55.550870+0000","last_active":"2024-09-17T05:08:57.597513+0000","last_peered":"2024-09-17T05:08:57.597513+0000","last_clean":"2024-09-17T05:08:57.597513+0000","last_became_active":"2024-09-17T05:08:55.550387+0000","last_became_peered":"2024-09-17T05:08:55.550387+0000","last_unstale":"2024-09-17T05:08:57.597513+0000","last_undegraded":"2024-09-17T05:08:57.597513+0000","last_fullsized":"2024-09-17T05:08:57.597513+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:53.875078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503048+0000","last_change":"2024-09-17T05:08:55.519391+0000","last_active":"2024-09-17T05:08:56.503048+0000","last_peered":"2024-09-17T05:08:56.503048+0000","last_clean":"2024-09-17T05:08:56.503048+0000","last_became_active":"2024-09-17T05:08:55.519087+0000","last_became_peered":"2024-09-17T05:08:55.519087+0000","last_unstale":"2024-09-17T05:08:56.503048+0000","last_undegraded":"2024-09-17T05:08:56.503048+0000","last_fullsized":"2024-09-17T05:08:56.503048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:26.059654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.187867+0000","last_change":"2024-09-17T05:08:55.538064+0000","last_active":"2024-09-17T05:08:57.187867+0000","last_peered":"2024-09-17T05:08:57.187867+0000","last_clean":"2024-09-17T05:08:57.187867+0000","last_became_active":"2024-09-17T05:08:55.537650+0000","last_became_peered":"2024-09-17T05:08:55.537650+0000","last_unstale":"2024-09-17T05:08:57.187867+0000","last_undegraded":"2024-09-17T05:08:57.187867+0000","last_fullsized":"2024-09-17T05:08:57.187867+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:04.475742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710406+0000","last_change":"2024-09-17T05:08:55.524778+0000","last_active":"2024-09-17T05:08:56.710406+0000","last_peered":"2024-09-17T05:08:56.710406+0000","last_clean":"2024-09-17T05:08:56.710406+0000","last_became_active":"2024-09-17T05:08:55.524128+0000","last_became_peered":"2024-09-17T05:08:55.524128+0000","last_unstale":"2024-09-17T05:08:56.710406+0000","last_undegraded":"2024-09-17T05:08:56.710406+0000","last_fullsized":"2024-09-17T05:08:56.710406+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:01.203297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710195+0000","last_change":"2024-09-17T05:08:55.538687+0000","last_active":"2024-09-17T05:08:56.710195+0000","last_peered":"2024-09-17T05:08:56.710195+0000","last_clean":"2024-09-17T05:08:56.710195+0000","last_became_active":"2024-09-17T05:08:55.538320+0000","last_became_peered":"2024-09-17T05:08:55.538320+0000","last_unstale":"2024-09-17T05:08:56.710195+0000","last_undegraded":"2024-09-17T05:08:56.710195+0000","last_fullsized":"2024-09-17T05:08:56.710195+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:05.276366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503589+0000","last_change":"2024-09-17T05:08:55.533040+0000","last_active":"2024-09-17T05:08:56.503589+0000","last_peered":"2024-09-17T05:08:56.503589+0000","last_clean":"2024-09-17T05:08:56.503589+0000","last_became_active":"2024-09-17T05:08:55.532731+0000","last_became_peered":"2024-09-17T05:08:55.532731+0000","last_unstale":"2024-09-17T05:08:56.503589+0000","last_undegraded":"2024-09-17T05:08:56.503589+0000","last_fullsized":"2024-09-17T05:08:56.503589+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:11.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597860+0000","last_change":"2024-09-17T05:08:55.537011+0000","last_active":"2024-09-17T05:08:57.597860+0000","last_peered":"2024-09-17T05:08:57.597860+0000","last_clean":"2024-09-17T05:08:57.597860+0000","last_became_active":"2024-09-17T05:08:55.536714+0000","last_became_peered":"2024-09-17T05:08:55.536714+0000","last_unstale":"2024-09-17T05:08:57.597860+0000","last_undegraded":"2024-09-17T05:08:57.597860+0000","last_fullsized":"2024-09-17T05:08:57.597860+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:24.607566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511793+0000","last_change":"2024-09-17T05:08:55.538652+0000","last_active":"2024-09-17T05:08:57.511793+0000","last_peered":"2024-09-17T05:08:57.511793+0000","last_clean":"2024-09-17T05:08:57.511793+0000","last_became_active":"2024-09-17T05:08:55.538306+0000","last_became_peered":"2024-09-17T05:08:55.538306+0000","last_unstale":"2024-09-17T05:08:57.511793+0000","last_undegraded":"2024-09-17T05:08:57.511793+0000","last_fullsized":"2024-09-17T05:08:57.511793+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:44.099688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503084+0000","last_change":"2024-09-17T05:08:55.519826+0000","last_active":"2024-09-17T05:08:56.503084+0000","last_peered":"2024-09-17T05:08:56.503084+0000","last_clean":"2024-09-17T05:08:56.503084+0000","last_became_active":"2024-09-17T05:08:55.518738+0000","last_became_peered":"2024-09-17T05:08:55.518738+0000","last_unstale":"2024-09-17T05:08:56.503084+0000","last_undegraded":"2024-09-17T05:08:56.503084+0000","last_fullsized":"2024-09-17T05:08:56.503084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:39.051457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503086+0000","last_change":"2024-09-17T05:08:55.514328+0000","last_active":"2024-09-17T05:08:56.503086+0000","last_peered":"2024-09-17T05:08:56.503086+0000","last_clean":"2024-09-17T05:08:56.503086+0000","last_became_active":"2024-09-17T05:08:55.513986+0000","last_became_peered":"2024-09-17T05:08:55.513986+0000","last_unstale":"2024-09-17T05:08:56.503086+0000","last_undegraded":"2024-09-17T05:08:56.503086+0000","last_fullsized":"2024-09-17T05:08:56.503086+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:54.542387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.534048+0000","last_change":"2024-09-17T05:08:55.530983+0000","last_active":"2024-09-17T05:08:55.534048+0000","last_peered":"2024-09-17T05:08:55.534048+0000","last_clean":"2024-09-17T05:08:55.534048+0000","last_became_active":"2024-09-17T05:08:55.530624+0000","last_became_peered":"2024-09-17T05:08:55.530624+0000","last_unstale":"2024-09-17T05:08:55.534048+0000","last_undegraded":"2024-09-17T05:08:55.534048+0000","last_fullsized":"2024-09-17T05:08:55.534048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:19.268795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709878+0000","last_change":"2024-09-17T05:08:55.524322+0000","last_active":"2024-09-17T05:08:56.709878+0000","last_peered":"2024-09-17T05:08:56.709878+0000","last_clean":"2024-09-17T05:08:56.709878+0000","last_became_active":"2024-09-17T05:08:55.521594+0000","last_became_peered":"2024-09-17T05:08:55.521594+0000","last_unstale":"2024-09-17T05:08:56.709878+0000","last_undegraded":"2024-09-17T05:08:56.709878+0000","last_fullsized":"2024-09-17T05:08:56.709878+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:09.991327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"124'4","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710462+0000","last_change":"2024-09-17T05:08:54.511513+0000","last_active":"2024-09-17T05:08:56.710462+0000","last_peered":"2024-09-17T05:08:56.710462+0000","last_clean":"2024-09-17T05:08:56.710462+0000","last_became_active":"2024-09-17T05:08:54.510827+0000","last_became_peered":"2024-09-17T05:08:54.510827+0000","last_unstale":"2024-09-17T05:08:56.710462+0000","last_undegraded":"2024-09-17T05:08:56.710462+0000","last_fullsized":"2024-09-17T05:08:56.710462+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:47.642807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188005+0000","last_change":"2024-09-17T05:08:55.549629+0000","last_active":"2024-09-17T05:08:57.188005+0000","last_peered":"2024-09-17T05:08:57.188005+0000","last_clean":"2024-09-17T05:08:57.188005+0000","last_became_active":"2024-09-17T05:08:55.549164+0000","last_became_peered":"2024-09-17T05:08:55.549164+0000","last_unstale":"2024-09-17T05:08:57.188005+0000","last_undegraded":"2024-09-17T05:08:57.188005+0000","last_fullsized":"2024-09-17T05:08:57.188005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:12.180060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188631+0000","last_change":"2024-09-17T05:08:54.521671+0000","last_active":"2024-09-17T05:08:57.188631+0000","last_peered":"2024-09-17T05:08:57.188631+0000","last_clean":"2024-09-17T05:08:57.188631+0000","last_became_active":"2024-09-17T05:08:54.521285+0000","last_became_peered":"2024-09-17T05:08:54.521285+0000","last_unstale":"2024-09-17T05:08:57.188631+0000","last_undegraded":"2024-09-17T05:08:57.188631+0000","last_fullsized":"2024-09-17T05:08:57.188631+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:59.007076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503310+0000","last_change":"2024-09-17T05:08:55.515179+0000","last_active":"2024-09-17T05:08:56.503310+0000","last_peered":"2024-09-17T05:08:56.503310+0000","last_clean":"2024-09-17T05:08:56.503310+0000","last_became_active":"2024-09-17T05:08:55.514833+0000","last_became_peered":"2024-09-17T05:08:55.514833+0000","last_unstale":"2024-09-17T05:08:56.503310+0000","last_undegraded":"2024-09-17T05:08:56.503310+0000","last_fullsized":"2024-09-17T05:08:56.503310+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:55.471587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511661+0000","last_change":"2024-09-17T05:08:54.526627+0000","last_active":"2024-09-17T05:08:57.511661+0000","last_peered":"2024-09-17T05:08:57.511661+0000","last_clean":"2024-09-17T05:08:57.511661+0000","last_became_active":"2024-09-17T05:08:54.526348+0000","last_became_peered":"2024-09-17T05:08:54.526348+0000","last_unstale":"2024-09-17T05:08:57.511661+0000","last_undegraded":"2024-09-17T05:08:57.511661+0000","last_fullsized":"2024-09-17T05:08:57.511661+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:38.048110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502747+0000","last_change":"2024-09-17T05:08:55.508929+0000","last_active":"2024-09-17T05:08:56.502747+0000","last_peered":"2024-09-17T05:08:56.502747+0000","last_clean":"2024-09-17T05:08:56.502747+0000","last_became_active":"2024-09-17T05:08:55.508605+0000","last_became_peered":"2024-09-17T05:08:55.508605+0000","last_unstale":"2024-09-17T05:08:56.502747+0000","last_undegraded":"2024-09-17T05:08:56.502747+0000","last_fullsized":"2024-09-17T05:08:56.502747+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:40.591794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710142+0000","last_change":"2024-09-17T05:08:54.526836+0000","last_active":"2024-09-17T05:08:56.710142+0000","last_peered":"2024-09-17T05:08:56.710142+0000","last_clean":"2024-09-17T05:08:56.710142+0000","last_became_active":"2024-09-17T05:08:54.525857+0000","last_became_peered":"2024-09-17T05:08:54.525857+0000","last_unstale":"2024-09-17T05:08:56.710142+0000","last_undegraded":"2024-09-17T05:08:56.710142+0000","last_fullsized":"2024-09-17T05:08:56.710142+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:17.275274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710220+0000","last_change":"2024-09-17T05:08:55.531510+0000","last_active":"2024-09-17T05:08:56.710220+0000","last_peered":"2024-09-17T05:08:56.710220+0000","last_clean":"2024-09-17T05:08:56.710220+0000","last_became_active":"2024-09-17T05:08:55.531179+0000","last_became_peered":"2024-09-17T05:08:55.531179+0000","last_unstale":"2024-09-17T05:08:56.710220+0000","last_undegraded":"2024-09-17T05:08:56.710220+0000","last_fullsized":"2024-09-17T05:08:56.710220+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:56.044591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511362+0000","last_change":"2024-09-17T05:08:54.527811+0000","last_active":"2024-09-17T05:08:57.511362+0000","last_peered":"2024-09-17T05:08:57.511362+0000","last_clean":"2024-09-17T05:08:57.511362+0000","last_became_active":"2024-09-17T05:08:54.527285+0000","last_became_peered":"2024-09-17T05:08:54.527285+0000","last_unstale":"2024-09-17T05:08:57.511362+0000","last_undegraded":"2024-09-17T05:08:57.511362+0000","last_fullsized":"2024-09-17T05:08:57.511362+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:56.483122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710140+0000","last_change":"2024-09-17T05:08:55.528810+0000","last_active":"2024-09-17T05:08:56.710140+0000","last_peered":"2024-09-17T05:08:56.710140+0000","last_clean":"2024-09-17T05:08:56.710140+0000","last_became_active":"2024-09-17T05:08:55.528157+0000","last_became_peered":"2024-09-17T05:08:55.528157+0000","last_unstale":"2024-09-17T05:08:56.710140+0000","last_undegraded":"2024-09-17T05:08:56.710140+0000","last_fullsized":"2024-09-17T05:08:56.710140+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:00.467816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597318+0000","last_change":"2024-09-17T05:08:54.522561+0000","last_active":"2024-09-17T05:08:57.597318+0000","last_peered":"2024-09-17T05:08:57.597318+0000","last_clean":"2024-09-17T05:08:57.597318+0000","last_became_active":"2024-09-17T05:08:54.522219+0000","last_became_peered":"2024-09-17T05:08:54.522219+0000","last_unstale":"2024-09-17T05:08:57.597318+0000","last_undegraded":"2024-09-17T05:08:57.597318+0000","last_fullsized":"2024-09-17T05:08:57.597318+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:53.256843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188481+0000","last_change":"2024-09-17T05:08:55.533386+0000","last_active":"2024-09-17T05:08:57.188481+0000","last_peered":"2024-09-17T05:08:57.188481+0000","last_clean":"2024-09-17T05:08:57.188481+0000","last_became_active":"2024-09-17T05:08:55.532703+0000","last_became_peered":"2024-09-17T05:08:55.532703+0000","last_unstale":"2024-09-17T05:08:57.188481+0000","last_undegraded":"2024-09-17T05:08:57.188481+0000","last_fullsized":"2024-09-17T05:08:57.188481+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:47.134898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.504234+0000","last_change":"2024-09-17T05:08:54.511613+0000","last_active":"2024-09-17T05:08:56.504234+0000","last_peered":"2024-09-17T05:08:56.504234+0000","last_clean":"2024-09-17T05:08:56.504234+0000","last_became_active":"2024-09-17T05:08:54.506985+0000","last_became_peered":"2024-09-17T05:08:54.506985+0000","last_unstale":"2024-09-17T05:08:56.504234+0000","last_undegraded":"2024-09-17T05:08:56.504234+0000","last_fullsized":"2024-09-17T05:08:56.504234+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:57.068108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710391+0000","last_change":"2024-09-17T05:08:55.524447+0000","last_active":"2024-09-17T05:08:56.710391+0000","last_peered":"2024-09-17T05:08:56.710391+0000","last_clean":"2024-09-17T05:08:56.710391+0000","last_became_active":"2024-09-17T05:08:55.521460+0000","last_became_peered":"2024-09-17T05:08:55.521460+0000","last_unstale":"2024-09-17T05:08:56.710391+0000","last_undegraded":"2024-09-17T05:08:56.710391+0000","last_fullsized":"2024-09-17T05:08:56.710391+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:21.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.510937+0000","last_change":"2024-09-17T05:08:54.529478+0000","last_active":"2024-09-17T05:08:57.510937+0000","last_peered":"2024-09-17T05:08:57.510937+0000","last_clean":"2024-09-17T05:08:57.510937+0000","last_became_active":"2024-09-17T05:08:54.529194+0000","last_became_peered":"2024-09-17T05:08:54.529194+0000","last_unstale":"2024-09-17T05:08:57.510937+0000","last_undegraded":"2024-09-17T05:08:57.510937+0000","last_fullsized":"2024-09-17T05:08:57.510937+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:59.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503036+0000","last_change":"2024-09-17T05:08:55.514022+0000","last_active":"2024-09-17T05:08:56.503036+0000","last_peered":"2024-09-17T05:08:56.503036+0000","last_clean":"2024-09-17T05:08:56.503036+0000","last_became_active":"2024-09-17T05:08:55.513705+0000","last_became_peered":"2024-09-17T05:08:55.513705+0000","last_unstale":"2024-09-17T05:08:56.503036+0000","last_undegraded":"2024-09-17T05:08:56.503036+0000","last_fullsized":"2024-09-17T05:08:56.503036+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:06.409373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922831+0000","last_change":"2024-09-17T05:08:54.519554+0000","last_active":"2024-09-17T05:08:56.922831+0000","last_peered":"2024-09-17T05:08:56.922831+0000","last_clean":"2024-09-17T05:08:56.922831+0000","last_became_active":"2024-09-17T05:08:54.519266+0000","last_became_peered":"2024-09-17T05:08:54.519266+0000","last_unstale":"2024-09-17T05:08:56.922831+0000","last_undegraded":"2024-09-17T05:08:56.922831+0000","last_fullsized":"2024-09-17T05:08:56.922831+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:55.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597475+0000","last_change":"2024-09-17T05:08:55.528957+0000","last_active":"2024-09-17T05:08:57.597475+0000","last_peered":"2024-09-17T05:08:57.597475+0000","last_clean":"2024-09-17T05:08:57.597475+0000","last_became_active":"2024-09-17T05:08:55.528648+0000","last_became_peered":"2024-09-17T05:08:55.528648+0000","last_unstale":"2024-09-17T05:08:57.597475+0000","last_undegraded":"2024-09-17T05:08:57.597475+0000","last_fullsized":"2024-09-17T05:08:57.597475+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:42.849591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503271+0000","last_change":"2024-09-17T05:08:54.520138+0000","last_active":"2024-09-17T05:08:56.503271+0000","last_peered":"2024-09-17T05:08:56.503271+0000","last_clean":"2024-09-17T05:08:56.503271+0000","last_became_active":"2024-09-17T05:08:54.519764+0000","last_became_peered":"2024-09-17T05:08:54.519764+0000","last_unstale":"2024-09-17T05:08:56.503271+0000","last_undegraded":"2024-09-17T05:08:56.503271+0000","last_fullsized":"2024-09-17T05:08:56.503271+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:07.759616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511280+0000","last_change":"2024-09-17T05:08:55.545937+0000","last_active":"2024-09-17T05:08:57.511280+0000","last_peered":"2024-09-17T05:08:57.511280+0000","last_clean":"2024-09-17T05:08:57.511280+0000","last_became_active":"2024-09-17T05:08:55.545613+0000","last_became_peered":"2024-09-17T05:08:55.545613+0000","last_unstale":"2024-09-17T05:08:57.511280+0000","last_undegraded":"2024-09-17T05:08:57.511280+0000","last_fullsized":"2024-09-17T05:08:57.511280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:41.093068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"124'3","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503744+0000","last_change":"2024-09-17T05:08:54.530745+0000","last_active":"2024-09-17T05:08:56.503744+0000","last_peered":"2024-09-17T05:08:56.503744+0000","last_clean":"2024-09-17T05:08:56.503744+0000","last_became_active":"2024-09-17T05:08:54.530066+0000","last_became_peered":"2024-09-17T05:08:54.530066+0000","last_unstale":"2024-09-17T05:08:56.503744+0000","last_undegraded":"2024-09-17T05:08:56.503744+0000","last_fullsized":"2024-09-17T05:08:56.503744+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:06.040454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709891+0000","last_change":"2024-09-17T05:08:55.528945+0000","last_active":"2024-09-17T05:08:56.709891+0000","last_peered":"2024-09-17T05:08:56.709891+0000","last_clean":"2024-09-17T05:08:56.709891+0000","last_became_active":"2024-09-17T05:08:55.528604+0000","last_became_peered":"2024-09-17T05:08:55.528604+0000","last_unstale":"2024-09-17T05:08:56.709891+0000","last_undegraded":"2024-09-17T05:08:56.709891+0000","last_fullsized":"2024-09-17T05:08:56.709891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:27.129169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"124'6","reported_seq":21,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511770+0000","last_change":"2024-09-17T05:08:54.529955+0000","last_active":"2024-09-17T05:08:57.511770+0000","last_peered":"2024-09-17T05:08:57.511770+0000","last_clean":"2024-09-17T05:08:57.511770+0000","last_became_active":"2024-09-17T05:08:54.529583+0000","last_became_peered":"2024-09-17T05:08:54.529583+0000","last_unstale":"2024-09-17T05:08:57.511770+0000","last_undegraded":"2024-09-17T05:08:57.511770+0000","last_fullsized":"2024-09-17T05:08:57.511770+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.109702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.521831+0000","last_change":"2024-09-17T05:08:55.512131+0000","last_active":"2024-09-17T05:08:55.521831+0000","last_peered":"2024-09-17T05:08:55.521831+0000","last_clean":"2024-09-17T05:08:55.521831+0000","last_became_active":"2024-09-17T05:08:55.511276+0000","last_became_peered":"2024-09-17T05:08:55.511276+0000","last_unstale":"2024-09-17T05:08:55.521831+0000","last_undegraded":"2024-09-17T05:08:55.521831+0000","last_fullsized":"2024-09-17T05:08:55.521831+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:16.704759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188397+0000","last_change":"2024-09-17T05:08:54.517171+0000","last_active":"2024-09-17T05:08:57.188397+0000","last_peered":"2024-09-17T05:08:57.188397+0000","last_clean":"2024-09-17T05:08:57.188397+0000","last_became_active":"2024-09-17T05:08:54.515910+0000","last_became_peered":"2024-09-17T05:08:54.515910+0000","last_unstale":"2024-09-17T05:08:57.188397+0000","last_undegraded":"2024-09-17T05:08:57.188397+0000","last_fullsized":"2024-09-17T05:08:57.188397+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:12.704149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503576+0000","last_change":"2024-09-17T05:08:55.516930+0000","last_active":"2024-09-17T05:08:56.503576+0000","last_peered":"2024-09-17T05:08:56.503576+0000","last_clean":"2024-09-17T05:08:56.503576+0000","last_became_active":"2024-09-17T05:08:55.516569+0000","last_became_peered":"2024-09-17T05:08:55.516569+0000","last_unstale":"2024-09-17T05:08:56.503576+0000","last_undegraded":"2024-09-17T05:08:56.503576+0000","last_fullsized":"2024-09-17T05:08:56.503576+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:04.258513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511357+0000","last_change":"2024-09-17T05:08:54.531118+0000","last_active":"2024-09-17T05:08:57.511357+0000","last_peered":"2024-09-17T05:08:57.511357+0000","last_clean":"2024-09-17T05:08:57.511357+0000","last_became_active":"2024-09-17T05:08:54.529275+0000","last_became_peered":"2024-09-17T05:08:54.529275+0000","last_unstale":"2024-09-17T05:08:57.511357+0000","last_undegraded":"2024-09-17T05:08:57.511357+0000","last_fullsized":"2024-09-17T05:08:57.511357+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:08.472556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188667+0000","last_change":"2024-09-17T05:08:55.528094+0000","last_active":"2024-09-17T05:08:57.188667+0000","last_peered":"2024-09-17T05:08:57.188667+0000","last_clean":"2024-09-17T05:08:57.188667+0000","last_became_active":"2024-09-17T05:08:55.526654+0000","last_became_peered":"2024-09-17T05:08:55.526654+0000","last_unstale":"2024-09-17T05:08:57.188667+0000","last_undegraded":"2024-09-17T05:08:57.188667+0000","last_fullsized":"2024-09-17T05:08:57.188667+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:48.941159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"124'8","reported_seq":23,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188028+0000","last_change":"2024-09-17T05:08:54.519669+0000","last_active":"2024-09-17T05:08:57.188028+0000","last_peered":"2024-09-17T05:08:57.188028+0000","last_clean":"2024-09-17T05:08:57.188028+0000","last_became_active":"2024-09-17T05:08:54.519223+0000","last_became_peered":"2024-09-17T05:08:54.519223+0000","last_unstale":"2024-09-17T05:08:57.188028+0000","last_undegraded":"2024-09-17T05:08:57.188028+0000","last_fullsized":"2024-09-17T05:08:57.188028+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:59.054583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922480+0000","last_change":"2024-09-17T05:08:55.518157+0000","last_active":"2024-09-17T05:08:56.922480+0000","last_peered":"2024-09-17T05:08:56.922480+0000","last_clean":"2024-09-17T05:08:56.922480+0000","last_became_active":"2024-09-17T05:08:55.517859+0000","last_became_peered":"2024-09-17T05:08:55.517859+0000","last_unstale":"2024-09-17T05:08:56.922480+0000","last_undegraded":"2024-09-17T05:08:56.922480+0000","last_fullsized":"2024-09-17T05:08:56.922480+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:55.471587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922841+0000","last_change":"2024-09-17T05:08:54.513931+0000","last_active":"2024-09-17T05:08:56.922841+0000","last_peered":"2024-09-17T05:08:56.922841+0000","last_clean":"2024-09-17T05:08:56.922841+0000","last_became_active":"2024-09-17T05:08:54.513497+0000","last_became_peered":"2024-09-17T05:08:54.513497+0000","last_unstale":"2024-09-17T05:08:56.922841+0000","last_undegraded":"2024-09-17T05:08:56.922841+0000","last_fullsized":"2024-09-17T05:08:56.922841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:37.476909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922391+0000","last_change":"2024-09-17T05:08:55.516620+0000","last_active":"2024-09-17T05:08:56.922391+0000","last_peered":"2024-09-17T05:08:56.922391+0000","last_clean":"2024-09-17T05:08:56.922391+0000","last_became_active":"2024-09-17T05:08:55.516295+0000","last_became_peered":"2024-09-17T05:08:55.516295+0000","last_unstale":"2024-09-17T05:08:56.922391+0000","last_undegraded":"2024-09-17T05:08:56.922391+0000","last_fullsized":"2024-09-17T05:08:56.922391+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:53.113049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922650+0000","last_change":"2024-09-17T05:08:54.518239+0000","last_active":"2024-09-17T05:08:56.922650+0000","last_peered":"2024-09-17T05:08:56.922650+0000","last_clean":"2024-09-17T05:08:56.922650+0000","last_became_active":"2024-09-17T05:08:54.517921+0000","last_became_peered":"2024-09-17T05:08:54.517921+0000","last_unstale":"2024-09-17T05:08:56.922650+0000","last_undegraded":"2024-09-17T05:08:56.922650+0000","last_fullsized":"2024-09-17T05:08:56.922650+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.626743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922492+0000","last_change":"2024-09-17T05:08:55.517807+0000","last_active":"2024-09-17T05:08:56.922492+0000","last_peered":"2024-09-17T05:08:56.922492+0000","last_clean":"2024-09-17T05:08:56.922492+0000","last_became_active":"2024-09-17T05:08:55.517477+0000","last_became_peered":"2024-09-17T05:08:55.517477+0000","last_unstale":"2024-09-17T05:08:56.922492+0000","last_undegraded":"2024-09-17T05:08:56.922492+0000","last_fullsized":"2024-09-17T05:08:56.922492+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:30.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502954+0000","last_change":"2024-09-17T05:08:54.508515+0000","last_active":"2024-09-17T05:08:56.502954+0000","last_peered":"2024-09-17T05:08:56.502954+0000","last_clean":"2024-09-17T05:08:56.502954+0000","last_became_active":"2024-09-17T05:08:54.506942+0000","last_became_peered":"2024-09-17T05:08:54.506942+0000","last_unstale":"2024-09-17T05:08:56.502954+0000","last_undegraded":"2024-09-17T05:08:56.502954+0000","last_fullsized":"2024-09-17T05:08:56.502954+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:50.226874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188148+0000","last_change":"2024-09-17T05:08:55.535313+0000","last_active":"2024-09-17T05:08:57.188148+0000","last_peered":"2024-09-17T05:08:57.188148+0000","last_clean":"2024-09-17T05:08:57.188148+0000","last_became_active":"2024-09-17T05:08:55.534929+0000","last_became_peered":"2024-09-17T05:08:55.534929+0000","last_unstale":"2024-09-17T05:08:57.188148+0000","last_undegraded":"2024-09-17T05:08:57.188148+0000","last_fullsized":"2024-09-17T05:08:57.188148+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:18.171281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503740+0000","last_change":"2024-09-17T05:08:54.514398+0000","last_active":"2024-09-17T05:08:56.503740+0000","last_peered":"2024-09-17T05:08:56.503740+0000","last_clean":"2024-09-17T05:08:56.503740+0000","last_became_active":"2024-09-17T05:08:54.514061+0000","last_became_peered":"2024-09-17T05:08:54.514061+0000","last_unstale":"2024-09-17T05:08:56.503740+0000","last_undegraded":"2024-09-17T05:08:56.503740+0000","last_fullsized":"2024-09-17T05:08:56.503740+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:41.018951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597921+0000","last_change":"2024-09-17T05:08:55.528262+0000","last_active":"2024-09-17T05:08:57.597921+0000","last_peered":"2024-09-17T05:08:57.597921+0000","last_clean":"2024-09-17T05:08:57.597921+0000","last_became_active":"2024-09-17T05:08:55.527577+0000","last_became_peered":"2024-09-17T05:08:55.527577+0000","last_unstale":"2024-09-17T05:08:57.597921+0000","last_undegraded":"2024-09-17T05:08:57.597921+0000","last_fullsized":"2024-09-17T05:08:57.597921+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:05.936424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503263+0000","last_change":"2024-09-17T05:08:54.517590+0000","last_active":"2024-09-17T05:08:56.503263+0000","last_peered":"2024-09-17T05:08:56.503263+0000","last_clean":"2024-09-17T05:08:56.503263+0000","last_became_active":"2024-09-17T05:08:54.517261+0000","last_became_peered":"2024-09-17T05:08:54.517261+0000","last_unstale":"2024-09-17T05:08:56.503263+0000","last_undegraded":"2024-09-17T05:08:56.503263+0000","last_fullsized":"2024-09-17T05:08:56.503263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:46.865934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.521619+0000","last_change":"2024-09-17T05:08:55.512668+0000","last_active":"2024-09-17T05:08:55.521619+0000","last_peered":"2024-09-17T05:08:55.521619+0000","last_clean":"2024-09-17T05:08:55.521619+0000","last_became_active":"2024-09-17T05:08:55.512154+0000","last_became_peered":"2024-09-17T05:08:55.512154+0000","last_unstale":"2024-09-17T05:08:55.521619+0000","last_undegraded":"2024-09-17T05:08:55.521619+0000","last_fullsized":"2024-09-17T05:08:55.521619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:50.500653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502720+0000","last_change":"2024-09-17T05:08:54.513788+0000","last_active":"2024-09-17T05:08:56.502720+0000","last_peered":"2024-09-17T05:08:56.502720+0000","last_clean":"2024-09-17T05:08:56.502720+0000","last_became_active":"2024-09-17T05:08:54.513452+0000","last_became_peered":"2024-09-17T05:08:54.513452+0000","last_unstale":"2024-09-17T05:08:56.502720+0000","last_undegraded":"2024-09-17T05:08:56.502720+0000","last_fullsized":"2024-09-17T05:08:56.502720+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:17.275274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503127+0000","last_change":"2024-09-17T05:08:55.515921+0000","last_active":"2024-09-17T05:08:56.503127+0000","last_peered":"2024-09-17T05:08:56.503127+0000","last_clean":"2024-09-17T05:08:56.503127+0000","last_became_active":"2024-09-17T05:08:55.515603+0000","last_became_peered":"2024-09-17T05:08:55.515603+0000","last_unstale":"2024-09-17T05:08:56.503127+0000","last_undegraded":"2024-09-17T05:08:56.503127+0000","last_fullsized":"2024-09-17T05:08:56.503127+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.345778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511559+0000","last_change":"2024-09-17T05:08:54.526253+0000","last_active":"2024-09-17T05:08:57.511559+0000","last_peered":"2024-09-17T05:08:57.511559+0000","last_clean":"2024-09-17T05:08:57.511559+0000","last_became_active":"2024-09-17T05:08:54.524897+0000","last_became_peered":"2024-09-17T05:08:54.524897+0000","last_unstale":"2024-09-17T05:08:57.511559+0000","last_undegraded":"2024-09-17T05:08:57.511559+0000","last_fullsized":"2024-09-17T05:08:57.511559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:31.783740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922702+0000","last_change":"2024-09-17T05:08:55.519041+0000","last_active":"2024-09-17T05:08:56.922702+0000","last_peered":"2024-09-17T05:08:56.922702+0000","last_clean":"2024-09-17T05:08:56.922702+0000","last_became_active":"2024-09-17T05:08:55.518620+0000","last_became_peered":"2024-09-17T05:08:55.518620+0000","last_unstale":"2024-09-17T05:08:56.922702+0000","last_undegraded":"2024-09-17T05:08:56.922702+0000","last_fullsized":"2024-09-17T05:08:56.922702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:11.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188151+0000","last_change":"2024-09-17T05:08:54.519417+0000","last_active":"2024-09-17T05:08:57.188151+0000","last_peered":"2024-09-17T05:08:57.188151+0000","last_clean":"2024-09-17T05:08:57.188151+0000","last_became_active":"2024-09-17T05:08:54.519030+0000","last_became_peered":"2024-09-17T05:08:54.519030+0000","last_unstale":"2024-09-17T05:08:57.188151+0000","last_undegraded":"2024-09-17T05:08:57.188151+0000","last_fullsized":"2024-09-17T05:08:57.188151+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:59.477375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188027+0000","last_change":"2024-09-17T05:08:55.534129+0000","last_active":"2024-09-17T05:08:57.188027+0000","last_peered":"2024-09-17T05:08:57.188027+0000","last_clean":"2024-09-17T05:08:57.188027+0000","last_became_active":"2024-09-17T05:08:55.533679+0000","last_became_peered":"2024-09-17T05:08:55.533679+0000","last_unstale":"2024-09-17T05:08:57.188027+0000","last_undegraded":"2024-09-17T05:08:57.188027+0000","last_fullsized":"2024-09-17T05:08:57.188027+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:34.910164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503145+0000","last_change":"2024-09-17T05:08:54.515152+0000","last_active":"2024-09-17T05:08:56.503145+0000","last_peered":"2024-09-17T05:08:56.503145+0000","last_clean":"2024-09-17T05:08:56.503145+0000","last_became_active":"2024-09-17T05:08:54.514823+0000","last_became_peered":"2024-09-17T05:08:54.514823+0000","last_unstale":"2024-09-17T05:08:56.503145+0000","last_undegraded":"2024-09-17T05:08:56.503145+0000","last_fullsized":"2024-09-17T05:08:56.503145+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:47.642807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511260+0000","last_change":"2024-09-17T05:08:55.535526+0000","last_active":"2024-09-17T05:08:57.511260+0000","last_peered":"2024-09-17T05:08:57.511260+0000","last_clean":"2024-09-17T05:08:57.511260+0000","last_became_active":"2024-09-17T05:08:55.535178+0000","last_became_peered":"2024-09-17T05:08:55.535178+0000","last_unstale":"2024-09-17T05:08:57.511260+0000","last_undegraded":"2024-09-17T05:08:57.511260+0000","last_fullsized":"2024-09-17T05:08:57.511260+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:26.572226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"125'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.510301+0000","last_change":"2024-09-17T05:08:54.531358+0000","last_active":"2024-09-17T05:08:57.510301+0000","last_peered":"2024-09-17T05:08:57.510301+0000","last_clean":"2024-09-17T05:08:57.510301+0000","last_became_active":"2024-09-17T05:08:54.530042+0000","last_became_peered":"2024-09-17T05:08:54.530042+0000","last_unstale":"2024-09-17T05:08:57.510301+0000","last_undegraded":"2024-09-17T05:08:57.510301+0000","last_fullsized":"2024-09-17T05:08:57.510301+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:52.288785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710111+0000","last_change":"2024-09-17T05:08:55.531632+0000","last_active":"2024-09-17T05:08:56.710111+0000","last_peered":"2024-09-17T05:08:56.710111+0000","last_clean":"2024-09-17T05:08:56.710111+0000","last_became_active":"2024-09-17T05:08:55.527974+0000","last_became_peered":"2024-09-17T05:08:55.527974+0000","last_unstale":"2024-09-17T05:08:56.710111+0000","last_undegraded":"2024-09-17T05:08:56.710111+0000","last_fullsized":"2024-09-17T05:08:56.710111+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:48.941159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710271+0000","last_change":"2024-09-17T05:08:54.518783+0000","last_active":"2024-09-17T05:08:56.710271+0000","last_peered":"2024-09-17T05:08:56.710271+0000","last_clean":"2024-09-17T05:08:56.710271+0000","last_became_active":"2024-09-17T05:08:54.518455+0000","last_became_peered":"2024-09-17T05:08:54.518455+0000","last_unstale":"2024-09-17T05:08:56.710271+0000","last_undegraded":"2024-09-17T05:08:56.710271+0000","last_fullsized":"2024-09-17T05:08:56.710271+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:30.054569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597626+0000","last_change":"2024-09-17T05:08:55.550026+0000","last_active":"2024-09-17T05:08:57.597626+0000","last_peered":"2024-09-17T05:08:57.597626+0000","last_clean":"2024-09-17T05:08:57.597626+0000","last_became_active":"2024-09-17T05:08:55.549337+0000","last_became_peered":"2024-09-17T05:08:55.549337+0000","last_unstale":"2024-09-17T05:08:57.597626+0000","last_undegraded":"2024-09-17T05:08:57.597626+0000","last_fullsized":"2024-09-17T05:08:57.597626+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:16.643228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188253+0000","last_change":"2024-09-17T05:08:54.518565+0000","last_active":"2024-09-17T05:08:57.188253+0000","last_peered":"2024-09-17T05:08:57.188253+0000","last_clean":"2024-09-17T05:08:57.188253+0000","last_became_active":"2024-09-17T05:08:54.518189+0000","last_became_peered":"2024-09-17T05:08:54.518189+0000","last_unstale":"2024-09-17T05:08:57.188253+0000","last_undegraded":"2024-09-17T05:08:57.188253+0000","last_fullsized":"2024-09-17T05:08:57.188253+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:10.637618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.527350+0000","last_change":"2024-09-17T05:08:55.525669+0000","last_active":"2024-09-17T05:08:55.527350+0000","last_peered":"2024-09-17T05:08:55.527350+0000","last_clean":"2024-09-17T05:08:55.527350+0000","last_became_active":"2024-09-17T05:08:55.525349+0000","last_became_peered":"2024-09-17T05:08:55.525349+0000","last_unstale":"2024-09-17T05:08:55.527350+0000","last_undegraded":"2024-09-17T05:08:55.527350+0000","last_fullsized":"2024-09-17T05:08:55.527350+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:22.539922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511790+0000","last_change":"2024-09-17T05:08:54.526743+0000","last_active":"2024-09-17T05:08:57.511790+0000","last_peered":"2024-09-17T05:08:57.511790+0000","last_clean":"2024-09-17T05:08:57.511790+0000","last_became_active":"2024-09-17T05:08:54.524326+0000","last_became_peered":"2024-09-17T05:08:54.524326+0000","last_unstale":"2024-09-17T05:08:57.511790+0000","last_undegraded":"2024-09-17T05:08:57.511790+0000","last_fullsized":"2024-09-17T05:08:57.511790+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:26.601187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.518933+0000","last_change":"2024-09-17T05:08:55.511857+0000","last_active":"2024-09-17T05:08:55.518933+0000","last_peered":"2024-09-17T05:08:55.518933+0000","last_clean":"2024-09-17T05:08:55.518933+0000","last_became_active":"2024-09-17T05:08:55.511520+0000","last_became_peered":"2024-09-17T05:08:55.511520+0000","last_unstale":"2024-09-17T05:08:55.518933+0000","last_undegraded":"2024-09-17T05:08:55.518933+0000","last_fullsized":"2024-09-17T05:08:55.518933+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:18.336237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511200+0000","last_change":"2024-09-17T05:08:54.527023+0000","last_active":"2024-09-17T05:08:57.511200+0000","last_peered":"2024-09-17T05:08:57.511200+0000","last_clean":"2024-09-17T05:08:57.511200+0000","last_became_active":"2024-09-17T05:08:54.525978+0000","last_became_peered":"2024-09-17T05:08:54.525978+0000","last_unstale":"2024-09-17T05:08:57.511200+0000","last_undegraded":"2024-09-17T05:08:57.511200+0000","last_fullsized":"2024-09-17T05:08:57.511200+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:05.523620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597754+0000","last_change":"2024-09-17T05:08:55.540236+0000","last_active":"2024-09-17T05:08:57.597754+0000","last_peered":"2024-09-17T05:08:57.597754+0000","last_clean":"2024-09-17T05:08:57.597754+0000","last_became_active":"2024-09-17T05:08:55.539907+0000","last_became_peered":"2024-09-17T05:08:55.539907+0000","last_unstale":"2024-09-17T05:08:57.597754+0000","last_undegraded":"2024-09-17T05:08:57.597754+0000","last_fullsized":"2024-09-17T05:08:57.597754+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:48.893129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.187961+0000","last_change":"2024-09-17T05:08:54.522350+0000","last_active":"2024-09-17T05:08:57.187961+0000","last_peered":"2024-09-17T05:08:57.187961+0000","last_clean":"2024-09-17T05:08:57.187961+0000","last_became_active":"2024-09-17T05:08:54.521984+0000","last_became_peered":"2024-09-17T05:08:54.521984+0000","last_unstale":"2024-09-17T05:08:57.187961+0000","last_undegraded":"2024-09-17T05:08:57.187961+0000","last_fullsized":"2024-09-17T05:08:57.187961+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:50.134274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922761+0000","last_change":"2024-09-17T05:08:55.513938+0000","last_active":"2024-09-17T05:08:56.922761+0000","last_peered":"2024-09-17T05:08:56.922761+0000","last_clean":"2024-09-17T05:08:56.922761+0000","last_became_active":"2024-09-17T05:08:55.513606+0000","last_became_peered":"2024-09-17T05:08:55.513606+0000","last_unstale":"2024-09-17T05:08:56.922761+0000","last_undegraded":"2024-09-17T05:08:56.922761+0000","last_fullsized":"2024-09-17T05:08:56.922761+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:21.094067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597121+0000","last_change":"2024-09-17T05:08:54.524802+0000","last_active":"2024-09-17T05:08:57.597121+0000","last_peered":"2024-09-17T05:08:57.597121+0000","last_clean":"2024-09-17T05:08:57.597121+0000","last_became_active":"2024-09-17T05:08:54.524467+0000","last_became_peered":"2024-09-17T05:08:54.524467+0000","last_unstale":"2024-09-17T05:08:57.597121+0000","last_undegraded":"2024-09-17T05:08:57.597121+0000","last_fullsized":"2024-09-17T05:08:57.597121+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:57.747361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511856+0000","last_change":"2024-09-17T05:08:55.546235+0000","last_active":"2024-09-17T05:08:57.511856+0000","last_peered":"2024-09-17T05:08:57.511856+0000","last_clean":"2024-09-17T05:08:57.511856+0000","last_became_active":"2024-09-17T05:08:55.545923+0000","last_became_peered":"2024-09-17T05:08:55.545923+0000","last_unstale":"2024-09-17T05:08:57.511856+0000","last_undegraded":"2024-09-17T05:08:57.511856+0000","last_fullsized":"2024-09-17T05:08:57.511856+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:21.230146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709900+0000","last_change":"2024-09-17T05:08:54.512834+0000","last_active":"2024-09-17T05:08:56.709900+0000","last_peered":"2024-09-17T05:08:56.709900+0000","last_clean":"2024-09-17T05:08:56.709900+0000","last_became_active":"2024-09-17T05:08:54.512517+0000","last_became_peered":"2024-09-17T05:08:54.512517+0000","last_unstale":"2024-09-17T05:08:56.709900+0000","last_undegraded":"2024-09-17T05:08:56.709900+0000","last_fullsized":"2024-09-17T05:08:56.709900+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:57.068108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":311,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502739+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:56.502739+0000","last_peered":"2024-09-17T05:08:56.502739+0000","last_clean":"2024-09-17T05:08:56.502739+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:56.502739+0000","last_undegraded":"2024-09-17T05:08:56.502739+0000","last_fullsized":"2024-09-17T05:08:56.502739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509541,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":712462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1548288,"data_stored":712496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1560576,"data_stored":719224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509539,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1560576,"data_stored":719736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1560576,"data_stored":719176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2166784,"data_stored":1311224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1560576,"data_stored":714164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2125824,"data_stored":1295663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:08:59.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:08:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 -- 172.21.15.16:0/3876854287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff53415d600 msgr2=0x7ff5340cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 --2- 172.21.15.16:0/3876854287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff53415d600 0x7ff5340cee50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff524009bc0 tx=0x7ff52401cba0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 -- 172.21.15.16:0/3876854287 shutdown_connections 2024-09-17T05:08:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 --2- 172.21.15.16:0/3876854287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff53415d600 0x7ff5340cee50 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 -- 172.21.15.16:0/3876854287 >> 172.21.15.16:0/3876854287 conn(0x7ff53406f900 msgr2=0x7ff5340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:08:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 -- 172.21.15.16:0/3876854287 shutdown_connections 2024-09-17T05:08:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 -- 172.21.15.16:0/3876854287 wait complete. 2024-09-17T05:08:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 Processor -- start 2024-09-17T05:08:59.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.930+0000 7ff53ae00640 1 -- start start 2024-09-17T05:08:59.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 0x7ff53413c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff53415d600 0x7ff5340c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff534160880 0x7ff5341440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff53416b440 con 0x7ff5340cf390 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5340b5280 con 0x7ff534160880 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff53416b5c0 con 0x7ff53415d600 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 0x7ff53413c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 0x7ff53413c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37368/0 (socket says 172.21.15.16:37368) 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 -- 172.21.15.16:0/1045534326 learned_addr learned my addr 172.21.15.16:0/1045534326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff538e00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff534160880 0x7ff5341440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff52b400640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff53415d600 0x7ff5340c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 -- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff53415d600 msgr2=0x7ff5340c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff53415d600 0x7ff5340c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 -- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff534160880 msgr2=0x7ff5341440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff534160880 0x7ff5341440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 -- 172.21.15.16:0/1045534326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53413dc20 con 0x7ff5340cf390 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff538e00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff534160880 0x7ff5341440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff52b400640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff53415d600 0x7ff5340c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff533e00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 0x7ff53413c530 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff52401a7d0 tx=0x7ff5240a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240bb070 con 0x7ff5340cf390 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff524004b40 con 0x7ff5340cf390 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53413df00 con 0x7ff5340cf390 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240b2040 con 0x7ff5340cf390 2024-09-17T05:08:59.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53413e4c0 con 0x7ff5340cf390 2024-09-17T05:08:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.934+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5340cee50 con 0x7ff5340cf390 2024-09-17T05:08:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.938+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff5240be070 con 0x7ff5340cf390 2024-09-17T05:08:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.938+0000 7ff530c00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff510073230 0x7ff510075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:08:59.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.938+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7ff52412e0f0 con 0x7ff5340cf390 2024-09-17T05:08:59.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.938+0000 7ff52b400640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff510073230 0x7ff510075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:08:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.938+0000 7ff52b400640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff510073230 0x7ff510075740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000f70 tx=0x7ff51c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:08:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:08:59.942+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff524016790 con 0x7ff5340cf390 2024-09-17T05:09:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.114+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5340b3530 con 0x7ff510073230 2024-09-17T05:09:00.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.118+0000 7ff530c00640 1 -- 172.21.15.16:0/1045534326 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7ff5340b3530 con 0x7ff510073230 2024-09-17T05:09:00.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:00.122 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:09:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff510073230 msgr2=0x7ff510075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff510073230 0x7ff510075740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000f70 tx=0x7ff51c009040 comp rx=0 tx=0).stop 2024-09-17T05:09:00.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 msgr2=0x7ff53413c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 0x7ff53413c530 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff52401a7d0 tx=0x7ff5240a5a00 comp rx=0 tx=0).stop 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 shutdown_connections 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff510073230 0x7ff510075740 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff534160880 0x7ff5341440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff53415d600 0x7ff5340c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 --2- 172.21.15.16:0/1045534326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340cf390 0x7ff53413c530 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 >> 172.21.15.16:0/1045534326 conn(0x7ff53406f900 msgr2=0x7ff5340cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 shutdown_connections 2024-09-17T05:09:00.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.122+0000 7ff53ae00640 1 -- 172.21.15.16:0/1045534326 wait complete. 2024-09-17T05:09:00.142 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-17T05:08:59.592055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275328,"kb_used_data":13308,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754699392,"statfs":{"total":773094113280,"available":772812177408,"internally_reserved":0,"allocated":13627392,"data_stored":6904145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001577"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.523213+0000","last_change":"2024-09-17T05:08:55.514755+0000","last_active":"2024-09-17T05:08:55.523213+0000","last_peered":"2024-09-17T05:08:55.523213+0000","last_clean":"2024-09-17T05:08:55.523213+0000","last_became_active":"2024-09-17T05:08:55.514429+0000","last_became_peered":"2024-09-17T05:08:55.514429+0000","last_unstale":"2024-09-17T05:08:55.523213+0000","last_undegraded":"2024-09-17T05:08:55.523213+0000","last_fullsized":"2024-09-17T05:08:55.523213+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:37:35.870572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511740+0000","last_change":"2024-09-17T05:08:55.545468+0000","last_active":"2024-09-17T05:08:57.511740+0000","last_peered":"2024-09-17T05:08:57.511740+0000","last_clean":"2024-09-17T05:08:57.511740+0000","last_became_active":"2024-09-17T05:08:55.545138+0000","last_became_peered":"2024-09-17T05:08:55.545138+0000","last_unstale":"2024-09-17T05:08:57.511740+0000","last_undegraded":"2024-09-17T05:08:57.511740+0000","last_fullsized":"2024-09-17T05:08:57.511740+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:39.013033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710192+0000","last_change":"2024-09-17T05:08:55.549337+0000","last_active":"2024-09-17T05:08:56.710192+0000","last_peered":"2024-09-17T05:08:56.710192+0000","last_clean":"2024-09-17T05:08:56.710192+0000","last_became_active":"2024-09-17T05:08:55.548925+0000","last_became_peered":"2024-09-17T05:08:55.548925+0000","last_unstale":"2024-09-17T05:08:56.710192+0000","last_undegraded":"2024-09-17T05:08:56.710192+0000","last_fullsized":"2024-09-17T05:08:56.710192+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:01.617056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710006+0000","last_change":"2024-09-17T05:08:55.553974+0000","last_active":"2024-09-17T05:08:56.710006+0000","last_peered":"2024-09-17T05:08:56.710006+0000","last_clean":"2024-09-17T05:08:56.710006+0000","last_became_active":"2024-09-17T05:08:55.553602+0000","last_became_peered":"2024-09-17T05:08:55.553602+0000","last_unstale":"2024-09-17T05:08:56.710006+0000","last_undegraded":"2024-09-17T05:08:56.710006+0000","last_fullsized":"2024-09-17T05:08:56.710006+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:21.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188346+0000","last_change":"2024-09-17T05:08:55.535322+0000","last_active":"2024-09-17T05:08:57.188346+0000","last_peered":"2024-09-17T05:08:57.188346+0000","last_clean":"2024-09-17T05:08:57.188346+0000","last_became_active":"2024-09-17T05:08:55.534956+0000","last_became_peered":"2024-09-17T05:08:55.534956+0000","last_unstale":"2024-09-17T05:08:57.188346+0000","last_undegraded":"2024-09-17T05:08:57.188346+0000","last_fullsized":"2024-09-17T05:08:57.188346+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:09.991327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.187958+0000","last_change":"2024-09-17T05:08:55.531482+0000","last_active":"2024-09-17T05:08:57.187958+0000","last_peered":"2024-09-17T05:08:57.187958+0000","last_clean":"2024-09-17T05:08:57.187958+0000","last_became_active":"2024-09-17T05:08:55.531168+0000","last_became_peered":"2024-09-17T05:08:55.531168+0000","last_unstale":"2024-09-17T05:08:57.187958+0000","last_undegraded":"2024-09-17T05:08:57.187958+0000","last_fullsized":"2024-09-17T05:08:57.187958+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:28.424029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511181+0000","last_change":"2024-09-17T05:08:55.548098+0000","last_active":"2024-09-17T05:08:57.511181+0000","last_peered":"2024-09-17T05:08:57.511181+0000","last_clean":"2024-09-17T05:08:57.511181+0000","last_became_active":"2024-09-17T05:08:55.547738+0000","last_became_peered":"2024-09-17T05:08:55.547738+0000","last_unstale":"2024-09-17T05:08:57.511181+0000","last_undegraded":"2024-09-17T05:08:57.511181+0000","last_fullsized":"2024-09-17T05:08:57.511181+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:16.724732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709833+0000","last_change":"2024-09-17T05:08:55.516230+0000","last_active":"2024-09-17T05:08:56.709833+0000","last_peered":"2024-09-17T05:08:56.709833+0000","last_clean":"2024-09-17T05:08:56.709833+0000","last_became_active":"2024-09-17T05:08:55.515831+0000","last_became_peered":"2024-09-17T05:08:55.515831+0000","last_unstale":"2024-09-17T05:08:56.709833+0000","last_undegraded":"2024-09-17T05:08:56.709833+0000","last_fullsized":"2024-09-17T05:08:56.709833+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:30.388469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188506+0000","last_change":"2024-09-17T05:08:55.540727+0000","last_active":"2024-09-17T05:08:57.188506+0000","last_peered":"2024-09-17T05:08:57.188506+0000","last_clean":"2024-09-17T05:08:57.188506+0000","last_became_active":"2024-09-17T05:08:55.540403+0000","last_became_peered":"2024-09-17T05:08:55.540403+0000","last_unstale":"2024-09-17T05:08:57.188506+0000","last_undegraded":"2024-09-17T05:08:57.188506+0000","last_fullsized":"2024-09-17T05:08:57.188506+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:27.129169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188578+0000","last_change":"2024-09-17T05:08:55.538666+0000","last_active":"2024-09-17T05:08:57.188578+0000","last_peered":"2024-09-17T05:08:57.188578+0000","last_clean":"2024-09-17T05:08:57.188578+0000","last_became_active":"2024-09-17T05:08:55.538412+0000","last_became_peered":"2024-09-17T05:08:55.538412+0000","last_unstale":"2024-09-17T05:08:57.188578+0000","last_undegraded":"2024-09-17T05:08:57.188578+0000","last_fullsized":"2024-09-17T05:08:57.188578+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:47:21.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922645+0000","last_change":"2024-09-17T05:08:55.525021+0000","last_active":"2024-09-17T05:08:56.922645+0000","last_peered":"2024-09-17T05:08:56.922645+0000","last_clean":"2024-09-17T05:08:56.922645+0000","last_became_active":"2024-09-17T05:08:55.524696+0000","last_became_peered":"2024-09-17T05:08:55.524696+0000","last_unstale":"2024-09-17T05:08:56.922645+0000","last_undegraded":"2024-09-17T05:08:56.922645+0000","last_fullsized":"2024-09-17T05:08:56.922645+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:59.524586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503524+0000","last_change":"2024-09-17T05:08:55.519271+0000","last_active":"2024-09-17T05:08:56.503524+0000","last_peered":"2024-09-17T05:08:56.503524+0000","last_clean":"2024-09-17T05:08:56.503524+0000","last_became_active":"2024-09-17T05:08:55.518944+0000","last_became_peered":"2024-09-17T05:08:55.518944+0000","last_unstale":"2024-09-17T05:08:56.503524+0000","last_undegraded":"2024-09-17T05:08:56.503524+0000","last_fullsized":"2024-09-17T05:08:56.503524+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:46:30.388469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597934+0000","last_change":"2024-09-17T05:08:55.532207+0000","last_active":"2024-09-17T05:08:57.597934+0000","last_peered":"2024-09-17T05:08:57.597934+0000","last_clean":"2024-09-17T05:08:57.597934+0000","last_became_active":"2024-09-17T05:08:55.531861+0000","last_became_peered":"2024-09-17T05:08:55.531861+0000","last_unstale":"2024-09-17T05:08:57.597934+0000","last_undegraded":"2024-09-17T05:08:57.597934+0000","last_fullsized":"2024-09-17T05:08:57.597934+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:28.875714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.523169+0000","last_change":"2024-09-17T05:08:55.514806+0000","last_active":"2024-09-17T05:08:55.523169+0000","last_peered":"2024-09-17T05:08:55.523169+0000","last_clean":"2024-09-17T05:08:55.523169+0000","last_became_active":"2024-09-17T05:08:55.514458+0000","last_became_peered":"2024-09-17T05:08:55.514458+0000","last_unstale":"2024-09-17T05:08:55.523169+0000","last_undegraded":"2024-09-17T05:08:55.523169+0000","last_fullsized":"2024-09-17T05:08:55.523169+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:30.103241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597637+0000","last_change":"2024-09-17T05:08:55.549618+0000","last_active":"2024-09-17T05:08:57.597637+0000","last_peered":"2024-09-17T05:08:57.597637+0000","last_clean":"2024-09-17T05:08:57.597637+0000","last_became_active":"2024-09-17T05:08:55.549260+0000","last_became_peered":"2024-09-17T05:08:55.549260+0000","last_unstale":"2024-09-17T05:08:57.597637+0000","last_undegraded":"2024-09-17T05:08:57.597637+0000","last_fullsized":"2024-09-17T05:08:57.597637+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:01.036971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.523121+0000","last_change":"2024-09-17T05:08:55.514474+0000","last_active":"2024-09-17T05:08:55.523121+0000","last_peered":"2024-09-17T05:08:55.523121+0000","last_clean":"2024-09-17T05:08:55.523121+0000","last_became_active":"2024-09-17T05:08:55.513109+0000","last_became_peered":"2024-09-17T05:08:55.513109+0000","last_unstale":"2024-09-17T05:08:55.523121+0000","last_undegraded":"2024-09-17T05:08:55.523121+0000","last_fullsized":"2024-09-17T05:08:55.523121+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:36.933091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.533289+0000","last_change":"2024-09-17T05:08:55.529450+0000","last_active":"2024-09-17T05:08:55.533289+0000","last_peered":"2024-09-17T05:08:55.533289+0000","last_clean":"2024-09-17T05:08:55.533289+0000","last_became_active":"2024-09-17T05:08:55.529145+0000","last_became_peered":"2024-09-17T05:08:55.529145+0000","last_unstale":"2024-09-17T05:08:55.533289+0000","last_undegraded":"2024-09-17T05:08:55.533289+0000","last_fullsized":"2024-09-17T05:08:55.533289+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:40.063642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502843+0000","last_change":"2024-09-17T05:08:55.545124+0000","last_active":"2024-09-17T05:08:56.502843+0000","last_peered":"2024-09-17T05:08:56.502843+0000","last_clean":"2024-09-17T05:08:56.502843+0000","last_became_active":"2024-09-17T05:08:55.544838+0000","last_became_peered":"2024-09-17T05:08:55.544838+0000","last_unstale":"2024-09-17T05:08:56.502843+0000","last_undegraded":"2024-09-17T05:08:56.502843+0000","last_fullsized":"2024-09-17T05:08:56.502843+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:12.180060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503567+0000","last_change":"2024-09-17T05:08:55.521067+0000","last_active":"2024-09-17T05:08:56.503567+0000","last_peered":"2024-09-17T05:08:56.503567+0000","last_clean":"2024-09-17T05:08:56.503567+0000","last_became_active":"2024-09-17T05:08:55.520646+0000","last_became_peered":"2024-09-17T05:08:55.520646+0000","last_unstale":"2024-09-17T05:08:56.503567+0000","last_undegraded":"2024-09-17T05:08:56.503567+0000","last_fullsized":"2024-09-17T05:08:56.503567+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:45.051109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503773+0000","last_change":"2024-09-17T05:08:55.551234+0000","last_active":"2024-09-17T05:08:56.503773+0000","last_peered":"2024-09-17T05:08:56.503773+0000","last_clean":"2024-09-17T05:08:56.503773+0000","last_became_active":"2024-09-17T05:08:55.550884+0000","last_became_peered":"2024-09-17T05:08:55.550884+0000","last_unstale":"2024-09-17T05:08:56.503773+0000","last_undegraded":"2024-09-17T05:08:56.503773+0000","last_fullsized":"2024-09-17T05:08:56.503773+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:51.145983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710162+0000","last_change":"2024-09-17T05:08:55.550313+0000","last_active":"2024-09-17T05:08:56.710162+0000","last_peered":"2024-09-17T05:08:56.710162+0000","last_clean":"2024-09-17T05:08:56.710162+0000","last_became_active":"2024-09-17T05:08:55.549992+0000","last_became_peered":"2024-09-17T05:08:55.549992+0000","last_unstale":"2024-09-17T05:08:56.710162+0000","last_undegraded":"2024-09-17T05:08:56.710162+0000","last_fullsized":"2024-09-17T05:08:56.710162+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:03.070280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597513+0000","last_change":"2024-09-17T05:08:55.550870+0000","last_active":"2024-09-17T05:08:57.597513+0000","last_peered":"2024-09-17T05:08:57.597513+0000","last_clean":"2024-09-17T05:08:57.597513+0000","last_became_active":"2024-09-17T05:08:55.550387+0000","last_became_peered":"2024-09-17T05:08:55.550387+0000","last_unstale":"2024-09-17T05:08:57.597513+0000","last_undegraded":"2024-09-17T05:08:57.597513+0000","last_fullsized":"2024-09-17T05:08:57.597513+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:53.875078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503048+0000","last_change":"2024-09-17T05:08:55.519391+0000","last_active":"2024-09-17T05:08:56.503048+0000","last_peered":"2024-09-17T05:08:56.503048+0000","last_clean":"2024-09-17T05:08:56.503048+0000","last_became_active":"2024-09-17T05:08:55.519087+0000","last_became_peered":"2024-09-17T05:08:55.519087+0000","last_unstale":"2024-09-17T05:08:56.503048+0000","last_undegraded":"2024-09-17T05:08:56.503048+0000","last_fullsized":"2024-09-17T05:08:56.503048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:26.059654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.187867+0000","last_change":"2024-09-17T05:08:55.538064+0000","last_active":"2024-09-17T05:08:57.187867+0000","last_peered":"2024-09-17T05:08:57.187867+0000","last_clean":"2024-09-17T05:08:57.187867+0000","last_became_active":"2024-09-17T05:08:55.537650+0000","last_became_peered":"2024-09-17T05:08:55.537650+0000","last_unstale":"2024-09-17T05:08:57.187867+0000","last_undegraded":"2024-09-17T05:08:57.187867+0000","last_fullsized":"2024-09-17T05:08:57.187867+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:04.475742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710406+0000","last_change":"2024-09-17T05:08:55.524778+0000","last_active":"2024-09-17T05:08:56.710406+0000","last_peered":"2024-09-17T05:08:56.710406+0000","last_clean":"2024-09-17T05:08:56.710406+0000","last_became_active":"2024-09-17T05:08:55.524128+0000","last_became_peered":"2024-09-17T05:08:55.524128+0000","last_unstale":"2024-09-17T05:08:56.710406+0000","last_undegraded":"2024-09-17T05:08:56.710406+0000","last_fullsized":"2024-09-17T05:08:56.710406+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:01.203297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710195+0000","last_change":"2024-09-17T05:08:55.538687+0000","last_active":"2024-09-17T05:08:56.710195+0000","last_peered":"2024-09-17T05:08:56.710195+0000","last_clean":"2024-09-17T05:08:56.710195+0000","last_became_active":"2024-09-17T05:08:55.538320+0000","last_became_peered":"2024-09-17T05:08:55.538320+0000","last_unstale":"2024-09-17T05:08:56.710195+0000","last_undegraded":"2024-09-17T05:08:56.710195+0000","last_fullsized":"2024-09-17T05:08:56.710195+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:18:05.276366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503589+0000","last_change":"2024-09-17T05:08:55.533040+0000","last_active":"2024-09-17T05:08:56.503589+0000","last_peered":"2024-09-17T05:08:56.503589+0000","last_clean":"2024-09-17T05:08:56.503589+0000","last_became_active":"2024-09-17T05:08:55.532731+0000","last_became_peered":"2024-09-17T05:08:55.532731+0000","last_unstale":"2024-09-17T05:08:56.503589+0000","last_undegraded":"2024-09-17T05:08:56.503589+0000","last_fullsized":"2024-09-17T05:08:56.503589+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:11.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597860+0000","last_change":"2024-09-17T05:08:55.537011+0000","last_active":"2024-09-17T05:08:57.597860+0000","last_peered":"2024-09-17T05:08:57.597860+0000","last_clean":"2024-09-17T05:08:57.597860+0000","last_became_active":"2024-09-17T05:08:55.536714+0000","last_became_peered":"2024-09-17T05:08:55.536714+0000","last_unstale":"2024-09-17T05:08:57.597860+0000","last_undegraded":"2024-09-17T05:08:57.597860+0000","last_fullsized":"2024-09-17T05:08:57.597860+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:24.607566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511793+0000","last_change":"2024-09-17T05:08:55.538652+0000","last_active":"2024-09-17T05:08:57.511793+0000","last_peered":"2024-09-17T05:08:57.511793+0000","last_clean":"2024-09-17T05:08:57.511793+0000","last_became_active":"2024-09-17T05:08:55.538306+0000","last_became_peered":"2024-09-17T05:08:55.538306+0000","last_unstale":"2024-09-17T05:08:57.511793+0000","last_undegraded":"2024-09-17T05:08:57.511793+0000","last_fullsized":"2024-09-17T05:08:57.511793+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:44.099688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503084+0000","last_change":"2024-09-17T05:08:55.519826+0000","last_active":"2024-09-17T05:08:56.503084+0000","last_peered":"2024-09-17T05:08:56.503084+0000","last_clean":"2024-09-17T05:08:56.503084+0000","last_became_active":"2024-09-17T05:08:55.518738+0000","last_became_peered":"2024-09-17T05:08:55.518738+0000","last_unstale":"2024-09-17T05:08:56.503084+0000","last_undegraded":"2024-09-17T05:08:56.503084+0000","last_fullsized":"2024-09-17T05:08:56.503084+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:39.051457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503086+0000","last_change":"2024-09-17T05:08:55.514328+0000","last_active":"2024-09-17T05:08:56.503086+0000","last_peered":"2024-09-17T05:08:56.503086+0000","last_clean":"2024-09-17T05:08:56.503086+0000","last_became_active":"2024-09-17T05:08:55.513986+0000","last_became_peered":"2024-09-17T05:08:55.513986+0000","last_unstale":"2024-09-17T05:08:56.503086+0000","last_undegraded":"2024-09-17T05:08:56.503086+0000","last_fullsized":"2024-09-17T05:08:56.503086+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:54.542387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.534048+0000","last_change":"2024-09-17T05:08:55.530983+0000","last_active":"2024-09-17T05:08:55.534048+0000","last_peered":"2024-09-17T05:08:55.534048+0000","last_clean":"2024-09-17T05:08:55.534048+0000","last_became_active":"2024-09-17T05:08:55.530624+0000","last_became_peered":"2024-09-17T05:08:55.530624+0000","last_unstale":"2024-09-17T05:08:55.534048+0000","last_undegraded":"2024-09-17T05:08:55.534048+0000","last_fullsized":"2024-09-17T05:08:55.534048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:19.268795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709878+0000","last_change":"2024-09-17T05:08:55.524322+0000","last_active":"2024-09-17T05:08:56.709878+0000","last_peered":"2024-09-17T05:08:56.709878+0000","last_clean":"2024-09-17T05:08:56.709878+0000","last_became_active":"2024-09-17T05:08:55.521594+0000","last_became_peered":"2024-09-17T05:08:55.521594+0000","last_unstale":"2024-09-17T05:08:56.709878+0000","last_undegraded":"2024-09-17T05:08:56.709878+0000","last_fullsized":"2024-09-17T05:08:56.709878+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:09.991327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"124'4","reported_seq":19,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710462+0000","last_change":"2024-09-17T05:08:54.511513+0000","last_active":"2024-09-17T05:08:56.710462+0000","last_peered":"2024-09-17T05:08:56.710462+0000","last_clean":"2024-09-17T05:08:56.710462+0000","last_became_active":"2024-09-17T05:08:54.510827+0000","last_became_peered":"2024-09-17T05:08:54.510827+0000","last_unstale":"2024-09-17T05:08:56.710462+0000","last_undegraded":"2024-09-17T05:08:56.710462+0000","last_fullsized":"2024-09-17T05:08:56.710462+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:47.642807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188005+0000","last_change":"2024-09-17T05:08:55.549629+0000","last_active":"2024-09-17T05:08:57.188005+0000","last_peered":"2024-09-17T05:08:57.188005+0000","last_clean":"2024-09-17T05:08:57.188005+0000","last_became_active":"2024-09-17T05:08:55.549164+0000","last_became_peered":"2024-09-17T05:08:55.549164+0000","last_unstale":"2024-09-17T05:08:57.188005+0000","last_undegraded":"2024-09-17T05:08:57.188005+0000","last_fullsized":"2024-09-17T05:08:57.188005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:12.180060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188631+0000","last_change":"2024-09-17T05:08:54.521671+0000","last_active":"2024-09-17T05:08:57.188631+0000","last_peered":"2024-09-17T05:08:57.188631+0000","last_clean":"2024-09-17T05:08:57.188631+0000","last_became_active":"2024-09-17T05:08:54.521285+0000","last_became_peered":"2024-09-17T05:08:54.521285+0000","last_unstale":"2024-09-17T05:08:57.188631+0000","last_undegraded":"2024-09-17T05:08:57.188631+0000","last_fullsized":"2024-09-17T05:08:57.188631+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:59.007076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503310+0000","last_change":"2024-09-17T05:08:55.515179+0000","last_active":"2024-09-17T05:08:56.503310+0000","last_peered":"2024-09-17T05:08:56.503310+0000","last_clean":"2024-09-17T05:08:56.503310+0000","last_became_active":"2024-09-17T05:08:55.514833+0000","last_became_peered":"2024-09-17T05:08:55.514833+0000","last_unstale":"2024-09-17T05:08:56.503310+0000","last_undegraded":"2024-09-17T05:08:56.503310+0000","last_fullsized":"2024-09-17T05:08:56.503310+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:55.471587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511661+0000","last_change":"2024-09-17T05:08:54.526627+0000","last_active":"2024-09-17T05:08:57.511661+0000","last_peered":"2024-09-17T05:08:57.511661+0000","last_clean":"2024-09-17T05:08:57.511661+0000","last_became_active":"2024-09-17T05:08:54.526348+0000","last_became_peered":"2024-09-17T05:08:54.526348+0000","last_unstale":"2024-09-17T05:08:57.511661+0000","last_undegraded":"2024-09-17T05:08:57.511661+0000","last_fullsized":"2024-09-17T05:08:57.511661+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:39:38.048110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502747+0000","last_change":"2024-09-17T05:08:55.508929+0000","last_active":"2024-09-17T05:08:56.502747+0000","last_peered":"2024-09-17T05:08:56.502747+0000","last_clean":"2024-09-17T05:08:56.502747+0000","last_became_active":"2024-09-17T05:08:55.508605+0000","last_became_peered":"2024-09-17T05:08:55.508605+0000","last_unstale":"2024-09-17T05:08:56.502747+0000","last_undegraded":"2024-09-17T05:08:56.502747+0000","last_fullsized":"2024-09-17T05:08:56.502747+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:16:40.591794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710142+0000","last_change":"2024-09-17T05:08:54.526836+0000","last_active":"2024-09-17T05:08:56.710142+0000","last_peered":"2024-09-17T05:08:56.710142+0000","last_clean":"2024-09-17T05:08:56.710142+0000","last_became_active":"2024-09-17T05:08:54.525857+0000","last_became_peered":"2024-09-17T05:08:54.525857+0000","last_unstale":"2024-09-17T05:08:56.710142+0000","last_undegraded":"2024-09-17T05:08:56.710142+0000","last_fullsized":"2024-09-17T05:08:56.710142+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:17.275274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710220+0000","last_change":"2024-09-17T05:08:55.531510+0000","last_active":"2024-09-17T05:08:56.710220+0000","last_peered":"2024-09-17T05:08:56.710220+0000","last_clean":"2024-09-17T05:08:56.710220+0000","last_became_active":"2024-09-17T05:08:55.531179+0000","last_became_peered":"2024-09-17T05:08:55.531179+0000","last_unstale":"2024-09-17T05:08:56.710220+0000","last_undegraded":"2024-09-17T05:08:56.710220+0000","last_fullsized":"2024-09-17T05:08:56.710220+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:18:56.044591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511362+0000","last_change":"2024-09-17T05:08:54.527811+0000","last_active":"2024-09-17T05:08:57.511362+0000","last_peered":"2024-09-17T05:08:57.511362+0000","last_clean":"2024-09-17T05:08:57.511362+0000","last_became_active":"2024-09-17T05:08:54.527285+0000","last_became_peered":"2024-09-17T05:08:54.527285+0000","last_unstale":"2024-09-17T05:08:57.511362+0000","last_undegraded":"2024-09-17T05:08:57.511362+0000","last_fullsized":"2024-09-17T05:08:57.511362+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:56.483122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710140+0000","last_change":"2024-09-17T05:08:55.528810+0000","last_active":"2024-09-17T05:08:56.710140+0000","last_peered":"2024-09-17T05:08:56.710140+0000","last_clean":"2024-09-17T05:08:56.710140+0000","last_became_active":"2024-09-17T05:08:55.528157+0000","last_became_peered":"2024-09-17T05:08:55.528157+0000","last_unstale":"2024-09-17T05:08:56.710140+0000","last_undegraded":"2024-09-17T05:08:56.710140+0000","last_fullsized":"2024-09-17T05:08:56.710140+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:00.467816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597318+0000","last_change":"2024-09-17T05:08:54.522561+0000","last_active":"2024-09-17T05:08:57.597318+0000","last_peered":"2024-09-17T05:08:57.597318+0000","last_clean":"2024-09-17T05:08:57.597318+0000","last_became_active":"2024-09-17T05:08:54.522219+0000","last_became_peered":"2024-09-17T05:08:54.522219+0000","last_unstale":"2024-09-17T05:08:57.597318+0000","last_undegraded":"2024-09-17T05:08:57.597318+0000","last_fullsized":"2024-09-17T05:08:57.597318+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:53.256843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188481+0000","last_change":"2024-09-17T05:08:55.533386+0000","last_active":"2024-09-17T05:08:57.188481+0000","last_peered":"2024-09-17T05:08:57.188481+0000","last_clean":"2024-09-17T05:08:57.188481+0000","last_became_active":"2024-09-17T05:08:55.532703+0000","last_became_peered":"2024-09-17T05:08:55.532703+0000","last_unstale":"2024-09-17T05:08:57.188481+0000","last_undegraded":"2024-09-17T05:08:57.188481+0000","last_fullsized":"2024-09-17T05:08:57.188481+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:47.134898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.504234+0000","last_change":"2024-09-17T05:08:54.511613+0000","last_active":"2024-09-17T05:08:56.504234+0000","last_peered":"2024-09-17T05:08:56.504234+0000","last_clean":"2024-09-17T05:08:56.504234+0000","last_became_active":"2024-09-17T05:08:54.506985+0000","last_became_peered":"2024-09-17T05:08:54.506985+0000","last_unstale":"2024-09-17T05:08:56.504234+0000","last_undegraded":"2024-09-17T05:08:56.504234+0000","last_fullsized":"2024-09-17T05:08:56.504234+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:57.068108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710391+0000","last_change":"2024-09-17T05:08:55.524447+0000","last_active":"2024-09-17T05:08:56.710391+0000","last_peered":"2024-09-17T05:08:56.710391+0000","last_clean":"2024-09-17T05:08:56.710391+0000","last_became_active":"2024-09-17T05:08:55.521460+0000","last_became_peered":"2024-09-17T05:08:55.521460+0000","last_unstale":"2024-09-17T05:08:56.710391+0000","last_undegraded":"2024-09-17T05:08:56.710391+0000","last_fullsized":"2024-09-17T05:08:56.710391+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:21.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.510937+0000","last_change":"2024-09-17T05:08:54.529478+0000","last_active":"2024-09-17T05:08:57.510937+0000","last_peered":"2024-09-17T05:08:57.510937+0000","last_clean":"2024-09-17T05:08:57.510937+0000","last_became_active":"2024-09-17T05:08:54.529194+0000","last_became_peered":"2024-09-17T05:08:54.529194+0000","last_unstale":"2024-09-17T05:08:57.510937+0000","last_undegraded":"2024-09-17T05:08:57.510937+0000","last_fullsized":"2024-09-17T05:08:57.510937+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:30:59.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503036+0000","last_change":"2024-09-17T05:08:55.514022+0000","last_active":"2024-09-17T05:08:56.503036+0000","last_peered":"2024-09-17T05:08:56.503036+0000","last_clean":"2024-09-17T05:08:56.503036+0000","last_became_active":"2024-09-17T05:08:55.513705+0000","last_became_peered":"2024-09-17T05:08:55.513705+0000","last_unstale":"2024-09-17T05:08:56.503036+0000","last_undegraded":"2024-09-17T05:08:56.503036+0000","last_fullsized":"2024-09-17T05:08:56.503036+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:06.409373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922831+0000","last_change":"2024-09-17T05:08:54.519554+0000","last_active":"2024-09-17T05:08:56.922831+0000","last_peered":"2024-09-17T05:08:56.922831+0000","last_clean":"2024-09-17T05:08:56.922831+0000","last_became_active":"2024-09-17T05:08:54.519266+0000","last_became_peered":"2024-09-17T05:08:54.519266+0000","last_unstale":"2024-09-17T05:08:56.922831+0000","last_undegraded":"2024-09-17T05:08:56.922831+0000","last_fullsized":"2024-09-17T05:08:56.922831+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:55.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597475+0000","last_change":"2024-09-17T05:08:55.528957+0000","last_active":"2024-09-17T05:08:57.597475+0000","last_peered":"2024-09-17T05:08:57.597475+0000","last_clean":"2024-09-17T05:08:57.597475+0000","last_became_active":"2024-09-17T05:08:55.528648+0000","last_became_peered":"2024-09-17T05:08:55.528648+0000","last_unstale":"2024-09-17T05:08:57.597475+0000","last_undegraded":"2024-09-17T05:08:57.597475+0000","last_fullsized":"2024-09-17T05:08:57.597475+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:40:42.849591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503271+0000","last_change":"2024-09-17T05:08:54.520138+0000","last_active":"2024-09-17T05:08:56.503271+0000","last_peered":"2024-09-17T05:08:56.503271+0000","last_clean":"2024-09-17T05:08:56.503271+0000","last_became_active":"2024-09-17T05:08:54.519764+0000","last_became_peered":"2024-09-17T05:08:54.519764+0000","last_unstale":"2024-09-17T05:08:56.503271+0000","last_undegraded":"2024-09-17T05:08:56.503271+0000","last_fullsized":"2024-09-17T05:08:56.503271+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:07.759616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511280+0000","last_change":"2024-09-17T05:08:55.545937+0000","last_active":"2024-09-17T05:08:57.511280+0000","last_peered":"2024-09-17T05:08:57.511280+0000","last_clean":"2024-09-17T05:08:57.511280+0000","last_became_active":"2024-09-17T05:08:55.545613+0000","last_became_peered":"2024-09-17T05:08:55.545613+0000","last_unstale":"2024-09-17T05:08:57.511280+0000","last_undegraded":"2024-09-17T05:08:57.511280+0000","last_fullsized":"2024-09-17T05:08:57.511280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:30:41.093068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"124'3","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503744+0000","last_change":"2024-09-17T05:08:54.530745+0000","last_active":"2024-09-17T05:08:56.503744+0000","last_peered":"2024-09-17T05:08:56.503744+0000","last_clean":"2024-09-17T05:08:56.503744+0000","last_became_active":"2024-09-17T05:08:54.530066+0000","last_became_peered":"2024-09-17T05:08:54.530066+0000","last_unstale":"2024-09-17T05:08:56.503744+0000","last_undegraded":"2024-09-17T05:08:56.503744+0000","last_fullsized":"2024-09-17T05:08:56.503744+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:36:06.040454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709891+0000","last_change":"2024-09-17T05:08:55.528945+0000","last_active":"2024-09-17T05:08:56.709891+0000","last_peered":"2024-09-17T05:08:56.709891+0000","last_clean":"2024-09-17T05:08:56.709891+0000","last_became_active":"2024-09-17T05:08:55.528604+0000","last_became_peered":"2024-09-17T05:08:55.528604+0000","last_unstale":"2024-09-17T05:08:56.709891+0000","last_undegraded":"2024-09-17T05:08:56.709891+0000","last_fullsized":"2024-09-17T05:08:56.709891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:47:27.129169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"124'6","reported_seq":21,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511770+0000","last_change":"2024-09-17T05:08:54.529955+0000","last_active":"2024-09-17T05:08:57.511770+0000","last_peered":"2024-09-17T05:08:57.511770+0000","last_clean":"2024-09-17T05:08:57.511770+0000","last_became_active":"2024-09-17T05:08:54.529583+0000","last_became_peered":"2024-09-17T05:08:54.529583+0000","last_unstale":"2024-09-17T05:08:57.511770+0000","last_undegraded":"2024-09-17T05:08:57.511770+0000","last_fullsized":"2024-09-17T05:08:57.511770+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:52.109702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.521831+0000","last_change":"2024-09-17T05:08:55.512131+0000","last_active":"2024-09-17T05:08:55.521831+0000","last_peered":"2024-09-17T05:08:55.521831+0000","last_clean":"2024-09-17T05:08:55.521831+0000","last_became_active":"2024-09-17T05:08:55.511276+0000","last_became_peered":"2024-09-17T05:08:55.511276+0000","last_unstale":"2024-09-17T05:08:55.521831+0000","last_undegraded":"2024-09-17T05:08:55.521831+0000","last_fullsized":"2024-09-17T05:08:55.521831+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:20:16.704759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188397+0000","last_change":"2024-09-17T05:08:54.517171+0000","last_active":"2024-09-17T05:08:57.188397+0000","last_peered":"2024-09-17T05:08:57.188397+0000","last_clean":"2024-09-17T05:08:57.188397+0000","last_became_active":"2024-09-17T05:08:54.515910+0000","last_became_peered":"2024-09-17T05:08:54.515910+0000","last_unstale":"2024-09-17T05:08:57.188397+0000","last_undegraded":"2024-09-17T05:08:57.188397+0000","last_fullsized":"2024-09-17T05:08:57.188397+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:12.704149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503576+0000","last_change":"2024-09-17T05:08:55.516930+0000","last_active":"2024-09-17T05:08:56.503576+0000","last_peered":"2024-09-17T05:08:56.503576+0000","last_clean":"2024-09-17T05:08:56.503576+0000","last_became_active":"2024-09-17T05:08:55.516569+0000","last_became_peered":"2024-09-17T05:08:55.516569+0000","last_unstale":"2024-09-17T05:08:56.503576+0000","last_undegraded":"2024-09-17T05:08:56.503576+0000","last_fullsized":"2024-09-17T05:08:56.503576+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:04.258513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511357+0000","last_change":"2024-09-17T05:08:54.531118+0000","last_active":"2024-09-17T05:08:57.511357+0000","last_peered":"2024-09-17T05:08:57.511357+0000","last_clean":"2024-09-17T05:08:57.511357+0000","last_became_active":"2024-09-17T05:08:54.529275+0000","last_became_peered":"2024-09-17T05:08:54.529275+0000","last_unstale":"2024-09-17T05:08:57.511357+0000","last_undegraded":"2024-09-17T05:08:57.511357+0000","last_fullsized":"2024-09-17T05:08:57.511357+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:08.472556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188667+0000","last_change":"2024-09-17T05:08:55.528094+0000","last_active":"2024-09-17T05:08:57.188667+0000","last_peered":"2024-09-17T05:08:57.188667+0000","last_clean":"2024-09-17T05:08:57.188667+0000","last_became_active":"2024-09-17T05:08:55.526654+0000","last_became_peered":"2024-09-17T05:08:55.526654+0000","last_unstale":"2024-09-17T05:08:57.188667+0000","last_undegraded":"2024-09-17T05:08:57.188667+0000","last_fullsized":"2024-09-17T05:08:57.188667+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:48.941159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"124'8","reported_seq":23,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188028+0000","last_change":"2024-09-17T05:08:54.519669+0000","last_active":"2024-09-17T05:08:57.188028+0000","last_peered":"2024-09-17T05:08:57.188028+0000","last_clean":"2024-09-17T05:08:57.188028+0000","last_became_active":"2024-09-17T05:08:54.519223+0000","last_became_peered":"2024-09-17T05:08:54.519223+0000","last_unstale":"2024-09-17T05:08:57.188028+0000","last_undegraded":"2024-09-17T05:08:57.188028+0000","last_fullsized":"2024-09-17T05:08:57.188028+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:59.054583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922480+0000","last_change":"2024-09-17T05:08:55.518157+0000","last_active":"2024-09-17T05:08:56.922480+0000","last_peered":"2024-09-17T05:08:56.922480+0000","last_clean":"2024-09-17T05:08:56.922480+0000","last_became_active":"2024-09-17T05:08:55.517859+0000","last_became_peered":"2024-09-17T05:08:55.517859+0000","last_unstale":"2024-09-17T05:08:56.922480+0000","last_undegraded":"2024-09-17T05:08:56.922480+0000","last_fullsized":"2024-09-17T05:08:56.922480+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:46:55.471587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922841+0000","last_change":"2024-09-17T05:08:54.513931+0000","last_active":"2024-09-17T05:08:56.922841+0000","last_peered":"2024-09-17T05:08:56.922841+0000","last_clean":"2024-09-17T05:08:56.922841+0000","last_became_active":"2024-09-17T05:08:54.513497+0000","last_became_peered":"2024-09-17T05:08:54.513497+0000","last_unstale":"2024-09-17T05:08:56.922841+0000","last_undegraded":"2024-09-17T05:08:56.922841+0000","last_fullsized":"2024-09-17T05:08:56.922841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:22:37.476909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922391+0000","last_change":"2024-09-17T05:08:55.516620+0000","last_active":"2024-09-17T05:08:56.922391+0000","last_peered":"2024-09-17T05:08:56.922391+0000","last_clean":"2024-09-17T05:08:56.922391+0000","last_became_active":"2024-09-17T05:08:55.516295+0000","last_became_peered":"2024-09-17T05:08:55.516295+0000","last_unstale":"2024-09-17T05:08:56.922391+0000","last_undegraded":"2024-09-17T05:08:56.922391+0000","last_fullsized":"2024-09-17T05:08:56.922391+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:53.113049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922650+0000","last_change":"2024-09-17T05:08:54.518239+0000","last_active":"2024-09-17T05:08:56.922650+0000","last_peered":"2024-09-17T05:08:56.922650+0000","last_clean":"2024-09-17T05:08:56.922650+0000","last_became_active":"2024-09-17T05:08:54.517921+0000","last_became_peered":"2024-09-17T05:08:54.517921+0000","last_unstale":"2024-09-17T05:08:56.922650+0000","last_undegraded":"2024-09-17T05:08:56.922650+0000","last_fullsized":"2024-09-17T05:08:56.922650+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:58:15.626743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922492+0000","last_change":"2024-09-17T05:08:55.517807+0000","last_active":"2024-09-17T05:08:56.922492+0000","last_peered":"2024-09-17T05:08:56.922492+0000","last_clean":"2024-09-17T05:08:56.922492+0000","last_became_active":"2024-09-17T05:08:55.517477+0000","last_became_peered":"2024-09-17T05:08:55.517477+0000","last_unstale":"2024-09-17T05:08:56.922492+0000","last_undegraded":"2024-09-17T05:08:56.922492+0000","last_fullsized":"2024-09-17T05:08:56.922492+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:20:30.619975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"124'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502954+0000","last_change":"2024-09-17T05:08:54.508515+0000","last_active":"2024-09-17T05:08:56.502954+0000","last_peered":"2024-09-17T05:08:56.502954+0000","last_clean":"2024-09-17T05:08:56.502954+0000","last_became_active":"2024-09-17T05:08:54.506942+0000","last_became_peered":"2024-09-17T05:08:54.506942+0000","last_unstale":"2024-09-17T05:08:56.502954+0000","last_undegraded":"2024-09-17T05:08:56.502954+0000","last_fullsized":"2024-09-17T05:08:56.502954+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:50.226874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188148+0000","last_change":"2024-09-17T05:08:55.535313+0000","last_active":"2024-09-17T05:08:57.188148+0000","last_peered":"2024-09-17T05:08:57.188148+0000","last_clean":"2024-09-17T05:08:57.188148+0000","last_became_active":"2024-09-17T05:08:55.534929+0000","last_became_peered":"2024-09-17T05:08:55.534929+0000","last_unstale":"2024-09-17T05:08:57.188148+0000","last_undegraded":"2024-09-17T05:08:57.188148+0000","last_fullsized":"2024-09-17T05:08:57.188148+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:18.171281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503740+0000","last_change":"2024-09-17T05:08:54.514398+0000","last_active":"2024-09-17T05:08:56.503740+0000","last_peered":"2024-09-17T05:08:56.503740+0000","last_clean":"2024-09-17T05:08:56.503740+0000","last_became_active":"2024-09-17T05:08:54.514061+0000","last_became_peered":"2024-09-17T05:08:54.514061+0000","last_unstale":"2024-09-17T05:08:56.503740+0000","last_undegraded":"2024-09-17T05:08:56.503740+0000","last_fullsized":"2024-09-17T05:08:56.503740+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:41.018951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597921+0000","last_change":"2024-09-17T05:08:55.528262+0000","last_active":"2024-09-17T05:08:57.597921+0000","last_peered":"2024-09-17T05:08:57.597921+0000","last_clean":"2024-09-17T05:08:57.597921+0000","last_became_active":"2024-09-17T05:08:55.527577+0000","last_became_peered":"2024-09-17T05:08:55.527577+0000","last_unstale":"2024-09-17T05:08:57.597921+0000","last_undegraded":"2024-09-17T05:08:57.597921+0000","last_fullsized":"2024-09-17T05:08:57.597921+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:29:05.936424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503263+0000","last_change":"2024-09-17T05:08:54.517590+0000","last_active":"2024-09-17T05:08:56.503263+0000","last_peered":"2024-09-17T05:08:56.503263+0000","last_clean":"2024-09-17T05:08:56.503263+0000","last_became_active":"2024-09-17T05:08:54.517261+0000","last_became_peered":"2024-09-17T05:08:54.517261+0000","last_unstale":"2024-09-17T05:08:56.503263+0000","last_undegraded":"2024-09-17T05:08:56.503263+0000","last_fullsized":"2024-09-17T05:08:56.503263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:46.865934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.521619+0000","last_change":"2024-09-17T05:08:55.512668+0000","last_active":"2024-09-17T05:08:55.521619+0000","last_peered":"2024-09-17T05:08:55.521619+0000","last_clean":"2024-09-17T05:08:55.521619+0000","last_became_active":"2024-09-17T05:08:55.512154+0000","last_became_peered":"2024-09-17T05:08:55.512154+0000","last_unstale":"2024-09-17T05:08:55.521619+0000","last_undegraded":"2024-09-17T05:08:55.521619+0000","last_fullsized":"2024-09-17T05:08:55.521619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:50.500653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502720+0000","last_change":"2024-09-17T05:08:54.513788+0000","last_active":"2024-09-17T05:08:56.502720+0000","last_peered":"2024-09-17T05:08:56.502720+0000","last_clean":"2024-09-17T05:08:56.502720+0000","last_became_active":"2024-09-17T05:08:54.513452+0000","last_became_peered":"2024-09-17T05:08:54.513452+0000","last_unstale":"2024-09-17T05:08:56.502720+0000","last_undegraded":"2024-09-17T05:08:56.502720+0000","last_fullsized":"2024-09-17T05:08:56.502720+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:17.275274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503127+0000","last_change":"2024-09-17T05:08:55.515921+0000","last_active":"2024-09-17T05:08:56.503127+0000","last_peered":"2024-09-17T05:08:56.503127+0000","last_clean":"2024-09-17T05:08:56.503127+0000","last_became_active":"2024-09-17T05:08:55.515603+0000","last_became_peered":"2024-09-17T05:08:55.515603+0000","last_unstale":"2024-09-17T05:08:56.503127+0000","last_undegraded":"2024-09-17T05:08:56.503127+0000","last_fullsized":"2024-09-17T05:08:56.503127+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:23.345778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511559+0000","last_change":"2024-09-17T05:08:54.526253+0000","last_active":"2024-09-17T05:08:57.511559+0000","last_peered":"2024-09-17T05:08:57.511559+0000","last_clean":"2024-09-17T05:08:57.511559+0000","last_became_active":"2024-09-17T05:08:54.524897+0000","last_became_peered":"2024-09-17T05:08:54.524897+0000","last_unstale":"2024-09-17T05:08:57.511559+0000","last_undegraded":"2024-09-17T05:08:57.511559+0000","last_fullsized":"2024-09-17T05:08:57.511559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:31.783740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922702+0000","last_change":"2024-09-17T05:08:55.519041+0000","last_active":"2024-09-17T05:08:56.922702+0000","last_peered":"2024-09-17T05:08:56.922702+0000","last_clean":"2024-09-17T05:08:56.922702+0000","last_became_active":"2024-09-17T05:08:55.518620+0000","last_became_peered":"2024-09-17T05:08:55.518620+0000","last_unstale":"2024-09-17T05:08:56.922702+0000","last_undegraded":"2024-09-17T05:08:56.922702+0000","last_fullsized":"2024-09-17T05:08:56.922702+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:11.387558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188151+0000","last_change":"2024-09-17T05:08:54.519417+0000","last_active":"2024-09-17T05:08:57.188151+0000","last_peered":"2024-09-17T05:08:57.188151+0000","last_clean":"2024-09-17T05:08:57.188151+0000","last_became_active":"2024-09-17T05:08:54.519030+0000","last_became_peered":"2024-09-17T05:08:54.519030+0000","last_unstale":"2024-09-17T05:08:57.188151+0000","last_undegraded":"2024-09-17T05:08:57.188151+0000","last_fullsized":"2024-09-17T05:08:57.188151+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:59.477375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188027+0000","last_change":"2024-09-17T05:08:55.534129+0000","last_active":"2024-09-17T05:08:57.188027+0000","last_peered":"2024-09-17T05:08:57.188027+0000","last_clean":"2024-09-17T05:08:57.188027+0000","last_became_active":"2024-09-17T05:08:55.533679+0000","last_became_peered":"2024-09-17T05:08:55.533679+0000","last_unstale":"2024-09-17T05:08:57.188027+0000","last_undegraded":"2024-09-17T05:08:57.188027+0000","last_fullsized":"2024-09-17T05:08:57.188027+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:34.910164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"124'2","reported_seq":17,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.503145+0000","last_change":"2024-09-17T05:08:54.515152+0000","last_active":"2024-09-17T05:08:56.503145+0000","last_peered":"2024-09-17T05:08:56.503145+0000","last_clean":"2024-09-17T05:08:56.503145+0000","last_became_active":"2024-09-17T05:08:54.514823+0000","last_became_peered":"2024-09-17T05:08:54.514823+0000","last_unstale":"2024-09-17T05:08:56.503145+0000","last_undegraded":"2024-09-17T05:08:56.503145+0000","last_fullsized":"2024-09-17T05:08:56.503145+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:02:47.642807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511260+0000","last_change":"2024-09-17T05:08:55.535526+0000","last_active":"2024-09-17T05:08:57.511260+0000","last_peered":"2024-09-17T05:08:57.511260+0000","last_clean":"2024-09-17T05:08:57.511260+0000","last_became_active":"2024-09-17T05:08:55.535178+0000","last_became_peered":"2024-09-17T05:08:55.535178+0000","last_unstale":"2024-09-17T05:08:57.511260+0000","last_undegraded":"2024-09-17T05:08:57.511260+0000","last_fullsized":"2024-09-17T05:08:57.511260+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:00:26.572226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"125'1","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.510301+0000","last_change":"2024-09-17T05:08:54.531358+0000","last_active":"2024-09-17T05:08:57.510301+0000","last_peered":"2024-09-17T05:08:57.510301+0000","last_clean":"2024-09-17T05:08:57.510301+0000","last_became_active":"2024-09-17T05:08:54.530042+0000","last_became_peered":"2024-09-17T05:08:54.530042+0000","last_unstale":"2024-09-17T05:08:57.510301+0000","last_undegraded":"2024-09-17T05:08:57.510301+0000","last_fullsized":"2024-09-17T05:08:57.510301+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:52.288785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710111+0000","last_change":"2024-09-17T05:08:55.531632+0000","last_active":"2024-09-17T05:08:56.710111+0000","last_peered":"2024-09-17T05:08:56.710111+0000","last_clean":"2024-09-17T05:08:56.710111+0000","last_became_active":"2024-09-17T05:08:55.527974+0000","last_became_peered":"2024-09-17T05:08:55.527974+0000","last_unstale":"2024-09-17T05:08:56.710111+0000","last_undegraded":"2024-09-17T05:08:56.710111+0000","last_fullsized":"2024-09-17T05:08:56.710111+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:48.941159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.710271+0000","last_change":"2024-09-17T05:08:54.518783+0000","last_active":"2024-09-17T05:08:56.710271+0000","last_peered":"2024-09-17T05:08:56.710271+0000","last_clean":"2024-09-17T05:08:56.710271+0000","last_became_active":"2024-09-17T05:08:54.518455+0000","last_became_peered":"2024-09-17T05:08:54.518455+0000","last_unstale":"2024-09-17T05:08:56.710271+0000","last_undegraded":"2024-09-17T05:08:56.710271+0000","last_fullsized":"2024-09-17T05:08:56.710271+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:30.054569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597626+0000","last_change":"2024-09-17T05:08:55.550026+0000","last_active":"2024-09-17T05:08:57.597626+0000","last_peered":"2024-09-17T05:08:57.597626+0000","last_clean":"2024-09-17T05:08:57.597626+0000","last_became_active":"2024-09-17T05:08:55.549337+0000","last_became_peered":"2024-09-17T05:08:55.549337+0000","last_unstale":"2024-09-17T05:08:57.597626+0000","last_undegraded":"2024-09-17T05:08:57.597626+0000","last_fullsized":"2024-09-17T05:08:57.597626+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:42:16.643228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.188253+0000","last_change":"2024-09-17T05:08:54.518565+0000","last_active":"2024-09-17T05:08:57.188253+0000","last_peered":"2024-09-17T05:08:57.188253+0000","last_clean":"2024-09-17T05:08:57.188253+0000","last_became_active":"2024-09-17T05:08:54.518189+0000","last_became_peered":"2024-09-17T05:08:54.518189+0000","last_unstale":"2024-09-17T05:08:57.188253+0000","last_undegraded":"2024-09-17T05:08:57.188253+0000","last_fullsized":"2024-09-17T05:08:57.188253+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:10.637618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.527350+0000","last_change":"2024-09-17T05:08:55.525669+0000","last_active":"2024-09-17T05:08:55.527350+0000","last_peered":"2024-09-17T05:08:55.527350+0000","last_clean":"2024-09-17T05:08:55.527350+0000","last_became_active":"2024-09-17T05:08:55.525349+0000","last_became_peered":"2024-09-17T05:08:55.525349+0000","last_unstale":"2024-09-17T05:08:55.527350+0000","last_undegraded":"2024-09-17T05:08:55.527350+0000","last_fullsized":"2024-09-17T05:08:55.527350+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:07:22.539922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511790+0000","last_change":"2024-09-17T05:08:54.526743+0000","last_active":"2024-09-17T05:08:57.511790+0000","last_peered":"2024-09-17T05:08:57.511790+0000","last_clean":"2024-09-17T05:08:57.511790+0000","last_became_active":"2024-09-17T05:08:54.524326+0000","last_became_peered":"2024-09-17T05:08:54.524326+0000","last_unstale":"2024-09-17T05:08:57.511790+0000","last_undegraded":"2024-09-17T05:08:57.511790+0000","last_fullsized":"2024-09-17T05:08:57.511790+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:26.601187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-17T05:08:55.518933+0000","last_change":"2024-09-17T05:08:55.511857+0000","last_active":"2024-09-17T05:08:55.518933+0000","last_peered":"2024-09-17T05:08:55.518933+0000","last_clean":"2024-09-17T05:08:55.518933+0000","last_became_active":"2024-09-17T05:08:55.511520+0000","last_became_peered":"2024-09-17T05:08:55.511520+0000","last_unstale":"2024-09-17T05:08:55.518933+0000","last_undegraded":"2024-09-17T05:08:55.518933+0000","last_fullsized":"2024-09-17T05:08:55.518933+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:13:18.336237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511200+0000","last_change":"2024-09-17T05:08:54.527023+0000","last_active":"2024-09-17T05:08:57.511200+0000","last_peered":"2024-09-17T05:08:57.511200+0000","last_clean":"2024-09-17T05:08:57.511200+0000","last_became_active":"2024-09-17T05:08:54.525978+0000","last_became_peered":"2024-09-17T05:08:54.525978+0000","last_unstale":"2024-09-17T05:08:57.511200+0000","last_undegraded":"2024-09-17T05:08:57.511200+0000","last_fullsized":"2024-09-17T05:08:57.511200+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:05.523620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597754+0000","last_change":"2024-09-17T05:08:55.540236+0000","last_active":"2024-09-17T05:08:57.597754+0000","last_peered":"2024-09-17T05:08:57.597754+0000","last_clean":"2024-09-17T05:08:57.597754+0000","last_became_active":"2024-09-17T05:08:55.539907+0000","last_became_peered":"2024-09-17T05:08:55.539907+0000","last_unstale":"2024-09-17T05:08:57.597754+0000","last_undegraded":"2024-09-17T05:08:57.597754+0000","last_fullsized":"2024-09-17T05:08:57.597754+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:48.893129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.187961+0000","last_change":"2024-09-17T05:08:54.522350+0000","last_active":"2024-09-17T05:08:57.187961+0000","last_peered":"2024-09-17T05:08:57.187961+0000","last_clean":"2024-09-17T05:08:57.187961+0000","last_became_active":"2024-09-17T05:08:54.521984+0000","last_became_peered":"2024-09-17T05:08:54.521984+0000","last_unstale":"2024-09-17T05:08:57.187961+0000","last_undegraded":"2024-09-17T05:08:57.187961+0000","last_fullsized":"2024-09-17T05:08:57.187961+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:50.134274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.922761+0000","last_change":"2024-09-17T05:08:55.513938+0000","last_active":"2024-09-17T05:08:56.922761+0000","last_peered":"2024-09-17T05:08:56.922761+0000","last_clean":"2024-09-17T05:08:56.922761+0000","last_became_active":"2024-09-17T05:08:55.513606+0000","last_became_peered":"2024-09-17T05:08:55.513606+0000","last_unstale":"2024-09-17T05:08:56.922761+0000","last_undegraded":"2024-09-17T05:08:56.922761+0000","last_fullsized":"2024-09-17T05:08:56.922761+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:21.094067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.597121+0000","last_change":"2024-09-17T05:08:54.524802+0000","last_active":"2024-09-17T05:08:57.597121+0000","last_peered":"2024-09-17T05:08:57.597121+0000","last_clean":"2024-09-17T05:08:57.597121+0000","last_became_active":"2024-09-17T05:08:54.524467+0000","last_became_peered":"2024-09-17T05:08:54.524467+0000","last_unstale":"2024-09-17T05:08:57.597121+0000","last_undegraded":"2024-09-17T05:08:57.597121+0000","last_fullsized":"2024-09-17T05:08:57.597121+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:57.747361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:57.511856+0000","last_change":"2024-09-17T05:08:55.546235+0000","last_active":"2024-09-17T05:08:57.511856+0000","last_peered":"2024-09-17T05:08:57.511856+0000","last_clean":"2024-09-17T05:08:57.511856+0000","last_became_active":"2024-09-17T05:08:55.545923+0000","last_became_peered":"2024-09-17T05:08:55.545923+0000","last_unstale":"2024-09-17T05:08:57.511856+0000","last_undegraded":"2024-09-17T05:08:57.511856+0000","last_fullsized":"2024-09-17T05:08:57.511856+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:54.489501+0000","last_clean_scrub_stamp":"2024-09-17T05:08:54.489501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:21.230146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.709900+0000","last_change":"2024-09-17T05:08:54.512834+0000","last_active":"2024-09-17T05:08:56.709900+0000","last_peered":"2024-09-17T05:08:56.709900+0000","last_clean":"2024-09-17T05:08:56.709900+0000","last_became_active":"2024-09-17T05:08:54.512517+0000","last_became_peered":"2024-09-17T05:08:54.512517+0000","last_unstale":"2024-09-17T05:08:56.709900+0000","last_undegraded":"2024-09-17T05:08:56.709900+0000","last_fullsized":"2024-09-17T05:08:56.709900+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:08:53.486154+0000","last_clean_scrub_stamp":"2024-09-17T05:08:53.486154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:57.068108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":311,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-17T05:08:56.502739+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:08:56.502739+0000","last_peered":"2024-09-17T05:08:56.502739+0000","last_clean":"2024-09-17T05:08:56.502739+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:08:56.502739+0000","last_undegraded":"2024-09-17T05:08:56.502739+0000","last_fullsized":"2024-09-17T05:08:56.502739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509541,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":712462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1548288,"data_stored":712496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1560576,"data_stored":719224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509539,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":1560576,"data_stored":719736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36872,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334968,"statfs":{"total":96636764160,"available":96599007232,"internally_reserved":0,"allocated":1560576,"data_stored":719176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509541,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37468,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334372,"statfs":{"total":96636764160,"available":96598396928,"internally_reserved":0,"allocated":2166784,"data_stored":1311224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1560576,"data_stored":714164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2125824,"data_stored":1295663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:09:00.146 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:09:00.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:09:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 -- 172.21.15.16:0/109362473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664003c40 msgr2=0x7fa664004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 --2- 172.21.15.16:0/109362473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664003c40 0x7fa664004090 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa670006fa0 tx=0x7fa67001cee0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 -- 172.21.15.16:0/109362473 shutdown_connections 2024-09-17T05:09:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 --2- 172.21.15.16:0/109362473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa664003c40 0x7fa664004090 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 -- 172.21.15.16:0/109362473 >> 172.21.15.16:0/109362473 conn(0x7fa68006f900 msgr2=0x7fa6800b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 -- 172.21.15.16:0/109362473 shutdown_connections 2024-09-17T05:09:00.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.350+0000 7fa685e00640 1 -- 172.21.15.16:0/109362473 wait complete. 2024-09-17T05:09:00.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 Processor -- start 2024-09-17T05:09:00.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 -- start start 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa664003c40 0x7fa68015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 0x7fa6801eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6801f9e90 0x7fa6801fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa680074430 con 0x7fa680156e30 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800742b0 con 0x7fa6801f9e90 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa68015f150 con 0x7fa664003c40 2024-09-17T05:09:00.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 0x7fa6801eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 0x7fa6801eaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37396/0 (socket says 172.21.15.16:37396) 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 -- 172.21.15.16:0/1866558460 learned_addr learned my addr 172.21.15.16:0/1866558460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa67ea00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa664003c40 0x7fa68015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa67f400640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6801f9e90 0x7fa6801fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 -- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa664003c40 msgr2=0x7fa68015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa664003c40 0x7fa68015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 -- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6801f9e90 msgr2=0x7fa6801fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6801f9e90 0x7fa6801fa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 -- 172.21.15.16:0/1866558460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68015aeb0 con 0x7fa680156e30 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa67f400640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6801f9e90 0x7fa6801fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 0x7fa6801eaae0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa668011a40 tx=0x7fa668011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668004460 con 0x7fa680156e30 2024-09-17T05:09:00.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa66800f9a0 con 0x7fa680156e30 2024-09-17T05:09:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800fdb0 con 0x7fa680156e30 2024-09-17T05:09:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.354+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801faae0 con 0x7fa680156e30 2024-09-17T05:09:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.358+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa680156630 con 0x7fa680156e30 2024-09-17T05:09:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.358+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68015f6b0 con 0x7fa680156e30 2024-09-17T05:09:00.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.358+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa668005020 con 0x7fa680156e30 2024-09-17T05:09:00.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.358+0000 7fa677400640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa65c073230 0x7fa65c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.358+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fa668095760 con 0x7fa680156e30 2024-09-17T05:09:00.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.358+0000 7fa67ea00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa65c073230 0x7fa65c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.362+0000 7fa67ea00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa65c073230 0x7fa65c075740 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa6700ab000 tx=0x7fa670019d20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:00.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.362+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa668060160 con 0x7fa680156e30 2024-09-17T05:09:00.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.598+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa6801fa870 con 0x7fa680156e30 2024-09-17T05:09:00.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa677400640 1 -- 172.21.15.16:0/1866558460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v39) ==== 190+0+147 (secure 0 0 0) 0x7fa668064b90 con 0x7fa680156e30 2024-09-17T05:09:00.601 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:09:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:09:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:09:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:09:00.601 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:09:00.602 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:09:00.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa65c073230 msgr2=0x7fa65c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa65c073230 0x7fa65c075740 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa6700ab000 tx=0x7fa670019d20 comp rx=0 tx=0).stop 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 msgr2=0x7fa6801eaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 0x7fa6801eaae0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa668011a40 tx=0x7fa668011f00 comp rx=0 tx=0).stop 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 shutdown_connections 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa65c073230 0x7fa65c075740 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6801f9e90 0x7fa6801fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa680156e30 0x7fa6801eaae0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 --2- 172.21.15.16:0/1866558460 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa664003c40 0x7fa68015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 >> 172.21.15.16:0/1866558460 conn(0x7fa68006f900 msgr2=0x7fa6800cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 shutdown_connections 2024-09-17T05:09:00.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.602+0000 7fa685e00640 1 -- 172.21.15.16:0/1866558460 wait complete. 2024-09-17T05:09:00.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:00.730 INFO:tasks.ceph.mds.b.smithi104.stderr:2024-09-17T05:09:00.726+0000 7f2a4a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:09:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 -- 172.21.15.16:0/591575150 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4538073240 msgr2=0x7f4538073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 --2- 172.21.15.16:0/591575150 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4538073240 0x7f4538073650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4530009c80 tx=0x7f453001cd10 comp rx=0 tx=0).stop 2024-09-17T05:09:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 -- 172.21.15.16:0/591575150 shutdown_connections 2024-09-17T05:09:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 --2- 172.21.15.16:0/591575150 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4538073240 0x7f4538073650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 --2- 172.21.15.16:0/591575150 >> v2:172.21.15.104:3300/0 conn(0x7f453815c270 0x7f453815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 -- 172.21.15.16:0/591575150 >> 172.21.15.16:0/591575150 conn(0x7f453806f900 msgr2=0x7f45380b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 -- 172.21.15.16:0/591575150 shutdown_connections 2024-09-17T05:09:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 -- 172.21.15.16:0/591575150 wait complete. 2024-09-17T05:09:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 Processor -- start 2024-09-17T05:09:00.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.786+0000 7f453d800640 1 -- start start 2024-09-17T05:09:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 0x7f45380ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45380b2f50 0x7f45380ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f453815c270 0x7f4538151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4538168d30 con 0x7f4538073240 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4538168bb0 con 0x7f45380b2f50 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4538168eb0 con 0x7f453815c270 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 0x7f45380ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 0x7f45380ab0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37402/0 (socket says 172.21.15.16:37402) 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 -- 172.21.15.16:0/3460690023 learned_addr learned my addr 172.21.15.16:0/3460690023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4537400640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f453815c270 0x7f4538151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f452fe00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45380b2f50 0x7f45380ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 -- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f453815c270 msgr2=0x7f4538151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f453815c270 0x7f4538151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 -- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45380b2f50 msgr2=0x7f45380ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45380b2f50 0x7f45380ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:00.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 -- 172.21.15.16:0/3460690023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45380ae640 con 0x7f4538073240 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f4536a00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 0x7f45380ab0f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f452800c430 tx=0x7f452800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528017070 con 0x7f4538073240 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45280053e0 con 0x7f4538073240 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45380ae920 con 0x7f4538073240 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528003860 con 0x7f4538073240 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45380aeee0 con 0x7f4538073240 2024-09-17T05:09:00.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.790+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4538073650 con 0x7f4538073240 2024-09-17T05:09:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.794+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f452801a070 con 0x7f4538073240 2024-09-17T05:09:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.794+0000 7f452ea00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4514073200 0x7f4514075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.794+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f4528093800 con 0x7f4538073240 2024-09-17T05:09:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.794+0000 7f452fe00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4514073200 0x7f4514075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.794+0000 7f452fe00640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4514073200 0x7f4514075710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f45380cc6a0 tx=0x7f45200091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:00.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:00.798+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f452805dfe0 con 0x7f4538073240 2024-09-17T05:09:01.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.010+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f453815b5a0 con 0x7f4538073240 2024-09-17T05:09:01.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.010+0000 7f452ea00640 1 -- 172.21.15.16:0/3460690023 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5225 (secure 0 0 0) 0x7f4528062a10 con 0x7f4538073240 2024-09-17T05:09:01.010 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:01.010 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"btime":"2024-09-17T05:08:58:530125+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:08:57.947398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:01.010 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 86 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4514073200 msgr2=0x7f4514075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4514073200 0x7f4514075710 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f45380cc6a0 tx=0x7f45200091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 msgr2=0x7f45380ab0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 0x7f45380ab0f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f452800c430 tx=0x7f452800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 shutdown_connections 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4514073200 0x7f4514075710 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f453815c270 0x7f4538151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45380b2f50 0x7f45380ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 --2- 172.21.15.16:0/3460690023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4538073240 0x7f45380ab0f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 >> 172.21.15.16:0/3460690023 conn(0x7f453806f900 msgr2=0x7f453806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 shutdown_connections 2024-09-17T05:09:01.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.014+0000 7f453d800640 1 -- 172.21.15.16:0/3460690023 wait complete. 2024-09-17T05:09:01.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:09:01.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:09:01.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:08:54.931567+0000', 'modified': '2024-09-17T05:08:57.947398+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7737}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7737': {'gid': 7737, 'name': 'b', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6835/892094550', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 892094550}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 892094550}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7737, 'qdb_cluster': [7737]} 2024-09-17T05:09:01.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:09:01.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T05:09:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 -- 172.21.15.16:0/653165061 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c074e30 msgr2=0x7ff70c15f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 --2- 172.21.15.16:0/653165061 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c074e30 0x7ff70c15f520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009900 tx=0x7ff6f401c8e0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 -- 172.21.15.16:0/653165061 shutdown_connections 2024-09-17T05:09:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 --2- 172.21.15.16:0/653165061 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c074e30 0x7ff70c15f520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 --2- 172.21.15.16:0/653165061 >> v2:172.21.15.104:3300/0 conn(0x7ff70c074a20 0x7ff70c15efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 -- 172.21.15.16:0/653165061 >> 172.21.15.16:0/653165061 conn(0x7ff70c0b2250 msgr2=0x7ff70c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 -- 172.21.15.16:0/653165061 shutdown_connections 2024-09-17T05:09:01.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.602+0000 7ff711400640 1 -- 172.21.15.16:0/653165061 wait complete. 2024-09-17T05:09:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 Processor -- start 2024-09-17T05:09:01.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 -- start start 2024-09-17T05:09:01.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff70c074a20 0x7ff70c165b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 0x7ff70c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c15ba50 0x7ff70c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c168f30 con 0x7ff70c074e30 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c168db0 con 0x7ff70c074a20 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff711400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff70c1690b0 con 0x7ff70c15ba50 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 0x7ff70c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 0x7ff70c0c6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37406/0 (socket says 172.21.15.16:37406) 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 -- 172.21.15.16:0/3833191787 learned_addr learned my addr 172.21.15.16:0/3833191787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff70a800640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff70c074a20 0x7ff70c165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff70b200640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c15ba50 0x7ff70c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 -- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c15ba50 msgr2=0x7ff70c154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c15ba50 0x7ff70c154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 -- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff70c074a20 msgr2=0x7ff70c165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff70c074a20 0x7ff70c165b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 -- 172.21.15.16:0/3833191787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c1f92c0 con 0x7ff70c074e30 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff70a800640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff70c074a20 0x7ff70c165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.606+0000 7ff703e00640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 0x7ff70c0c6880 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40b0040 tx=0x7ff6f4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.610+0000 7ff702a00640 1 -- 172.21.15.16:0/3833191787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40b2040 con 0x7ff70c074e30 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.610+0000 7ff702a00640 1 -- 172.21.15.16:0/3833191787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6f4004320 con 0x7ff70c074e30 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.610+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c1fcd80 con 0x7ff70c074e30 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.610+0000 7ff702a00640 1 -- 172.21.15.16:0/3833191787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4005290 con 0x7ff70c074e30 2024-09-17T05:09:01.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.610+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c1fd340 con 0x7ff70c074e30 2024-09-17T05:09:01.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.610+0000 7ff702a00640 1 -- 172.21.15.16:0/3833191787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6f4005430 con 0x7ff70c074e30 2024-09-17T05:09:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff702a00640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6e8073200 0x7ff6e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff702a00640 1 -- 172.21.15.16:0/3833191787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7ff6f412dad0 con 0x7ff70c074e30 2024-09-17T05:09:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff70a800640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6e8073200 0x7ff6e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 Processor -- start 2024-09-17T05:09:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 -- start start 2024-09-17T05:09:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff70a800640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6e8073200 0x7ff6e8075710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009b90 tx=0x7ff6fc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562723c3e460 0x56272417f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 0x56272417fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724180120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562724054410 con 0x56272402e950 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562723f91b40 con 0x562724017ca0 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x562723f80f70 con 0x562723c3e460 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 0x56272417fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 0x56272417fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37416/0 (socket says 172.21.15.16:37416) 2024-09-17T05:09:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 -- 172.21.15.16:0/1904487586 learned_addr learned my addr 172.21.15.16:0/1904487586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff70b200640 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724180120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff70a800640 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562723c3e460 0x56272417f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 -- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562723c3e460 msgr2=0x56272417f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562723c3e460 0x56272417f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 -- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 msgr2=0x562724180120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724180120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 -- 172.21.15.16:0/1904487586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562723f485b0 con 0x56272402e950 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff703e00640 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 0x56272417fbe0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40f8160 tx=0x7ff6f40f41a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4933974a8f562cb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff6fb000640 1 -- 172.21.15.16:0/1904487586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40bc070 con 0x56272402e950 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff6fb000640 1 -- 172.21.15.16:0/1904487586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6f40f7ce0 con 0x56272402e950 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.614+0000 7ff6fb000640 1 -- 172.21.15.16:0/1904487586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40f88d0 con 0x56272402e950 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- 172.21.15.16:0/1904487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 msgr2=0x56272417fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 0x56272417fbe0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40f8160 tx=0x7ff6f40f41a0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- 172.21.15.16:0/1904487586 shutdown_connections 2024-09-17T05:09:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724180120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56272402e950 0x56272417fbe0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- 172.21.15.16:0/1904487586 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562723c3e460 0x56272417f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- 172.21.15.16:0/1904487586 >> 172.21.15.16:0/1904487586 conn(0x562724104d90 msgr2=0x562724134690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- 172.21.15.16:0/1904487586 shutdown_connections 2024-09-17T05:09:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- 172.21.15.16:0/1904487586 wait complete. 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- start start 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 Processor -- start 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x562724186920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724186e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562723f1b5f0 con 0x562723c3e460 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562723f84d80 con 0x562724017ca0 2024-09-17T05:09:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x562723f27520 con 0x56272402e950 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff718b19000 10 client.0 _ll_register_callbacks cb 0x562723f2b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff70a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x562724186920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff70a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x562724186920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37424/0 (socket says 172.21.15.16:37424) 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff70b200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff703e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724186e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.618+0000 7ff70a800640 1 -- 172.21.15.16:0/1030615699 learned_addr learned my addr 172.21.15.16:0/1030615699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff70a800640 1 -- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 msgr2=0x5627241873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff70a800640 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff70a800640 1 -- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 msgr2=0x562724186e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff70a800640 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724186e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff70a800640 1 -- 172.21.15.16:0/1030615699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562723f1f190 con 0x562723c3e460 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff70a800640 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x562724186920 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0038b0 tx=0x7ff6fc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e4347438d2989d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 1 -- 172.21.15.16:0/1030615699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc012650 con 0x562723c3e460 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 1 -- 172.21.15.16:0/1030615699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7ff6fc0127f0 con 0x562723c3e460 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562723f19110 con 0x562723c3e460 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 1 -- 172.21.15.16:0/1030615699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc031050 con 0x562723c3e460 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 10 client.7851 fetch_fsmap learned FSMap version 86 2024-09-17T05:09:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x562724005f10 con 0x562723c3e460 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 1 -- 172.21.15.16:0/1030615699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562723f19110 con 0x562723c3e460 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 1 -- 172.21.15.16:0/1030615699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7ff6fc034020 con 0x562723c3e460 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 10 client.7851 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 10 client.7851 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 20 client.7851 populate_metadata read hostname 'smithi016' 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c237d0 0x562724190d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 4 client.7851 mds_command: new command op to 7737 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 --> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562723f93a90 con 0x562723c237d0 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff703e00640 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c237d0 0x562724190d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff703e00640 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c237d0 0x562724190d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada1f40d10b5f20d server_cookie=1e7da7009cc82a38 in_seq=0 out_seq=0 2024-09-17T05:09:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 10 client.7851 ms_handle_connect on v2:172.21.15.104:6834/892094550 2024-09-17T05:09:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 1 -- 172.21.15.16:0/1030615699 <== mds.0 v2:172.21.15.104:6834/892094550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562723f93a90 con 0x562723c237d0 2024-09-17T05:09:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff6ef400640 10 client.7851 handle_command_reply: tid=0 2024-09-17T05:09:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.622+0000 7ff718b19000 1 client.7851 shutdown 2024-09-17T05:09:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 msgr2=0x562724186920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x562724186920 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0038b0 tx=0x7ff6fc004740 comp rx=0 tx=0).stop 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 shutdown_connections 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x562724186e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c237d0 0x562724190d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 --2- 172.21.15.16:0/1030615699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x562724186920 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff6ef400640 0 client.7851 ms_handle_reset on v2:172.21.15.104:6834/892094550 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 >> 172.21.15.16:0/1030615699 conn(0x562724104d90 msgr2=0x56272413b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 shutdown_connections 2024-09-17T05:09:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 1 -- 172.21.15.16:0/1030615699 wait complete. 2024-09-17T05:09:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.626+0000 7ff718b19000 20 client.7851 trim_cache size 0 max 16384 2024-09-17T05:09:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.642+0000 7ff718b19000 1 Processor -- start 2024-09-17T05:09:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.642+0000 7ff718b19000 1 -- start start 2024-09-17T05:09:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x5627241f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c205c10 con 0x562723c3e460 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c204d20 con 0x562724017ca0 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff70c0b3d10 con 0x56272402e950 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37432/0 (socket says 172.21.15.16:37432) 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 -- 172.21.15.16:0/27739328 learned_addr learned my addr 172.21.15.16:0/27739328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70b200640 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff703e00640 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x5627241f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 -- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 msgr2=0x5627241f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 -- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 msgr2=0x5627241f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x5627241f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 -- 172.21.15.16:0/27739328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562723fdcdb0 con 0x562723c3e460 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff703e00640 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x5627241f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff70a800640 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f1620 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc012fd0 tx=0x7ff6fc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d0a02c3047c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff6fa600640 1 -- 172.21.15.16:0/27739328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc012c40 con 0x562723c3e460 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff6fa600640 1 -- 172.21.15.16:0/27739328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6fc025030 con 0x562723c3e460 2024-09-17T05:09:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff6fa600640 1 -- 172.21.15.16:0/27739328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0126b0 con 0x562723c3e460 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- 172.21.15.16:0/27739328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 msgr2=0x5627241f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f1620 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc012fd0 tx=0x7ff6fc0046d0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- 172.21.15.16:0/27739328 shutdown_connections 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56272402e950 0x5627241f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562724017ca0 0x5627241f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 --2- 172.21.15.16:0/27739328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f1620 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- 172.21.15.16:0/27739328 >> 172.21.15.16:0/27739328 conn(0x562724104d90 msgr2=0x56272417d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- 172.21.15.16:0/27739328 shutdown_connections 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.646+0000 7ff718b19000 1 -- 172.21.15.16:0/27739328 wait complete. 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- start start 2024-09-17T05:09:01.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 Processor -- start 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562724017ca0 0x5627241f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56272402e950 0x5627241f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562724005a80 con 0x562723c3e460 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562723f84090 con 0x56272402e950 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x562723f90910 con 0x562724017ca0 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 10 client.0 _ll_register_callbacks cb 0x7ff6fc011cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37436/0 (socket says 172.21.15.16:37436) 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 -- 172.21.15.16:0/724307094 learned_addr learned my addr 172.21.15.16:0/724307094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:01.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 -- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562724017ca0 msgr2=0x5627241f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562724017ca0 0x5627241f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 -- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56272402e950 msgr2=0x5627241f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56272402e950 0x5627241f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 -- 172.21.15.16:0/724307094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562723f47ab0 con 0x562723c3e460 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff70a800640 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f5390 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc003fd0 tx=0x7ff6fc005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d7f71e3de338d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff6ef400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff6ef400640 1 -- 172.21.15.16:0/724307094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc02f040 con 0x562723c3e460 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff6ef400640 1 -- 172.21.15.16:0/724307094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7ff6fc0127a0 con 0x562723c3e460 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6fc025030 con 0x562723c3e460 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff6ef400640 1 -- 172.21.15.16:0/724307094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc04c050 con 0x562723c3e460 2024-09-17T05:09:01.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 10 client.7857 fetch_fsmap learned FSMap version 86 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x562723f26070 con 0x562723c3e460 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff6ef400640 1 -- 172.21.15.16:0/724307094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6fc025030 con 0x562723c3e460 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff6ef400640 1 -- 172.21.15.16:0/724307094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7ff6fc04f020 con 0x562723c3e460 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 10 client.7857 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 10 client.7857 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 20 client.7857 populate_metadata read hostname 'smithi016' 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c11830 0x5627241b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 4 client.7857 mds_command: new command op to 7737 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:09:01.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.650+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 --> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] -- command(tid 0: {"prefix": "status"}) -- 0x562723f8f540 con 0x562723c11830 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff703e00640 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c11830 0x5627241b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff703e00640 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c11830 0x5627241b1670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599ced76b4e1ebeb server_cookie=8dd4105383a9cfa9 in_seq=0 out_seq=0 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff6ef400640 10 client.7857 ms_handle_connect on v2:172.21.15.104:6834/892094550 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff6ef400640 1 -- 172.21.15.16:0/724307094 <== mds.0 v2:172.21.15.104:6834/892094550 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x562723f8f540 con 0x562723c11830 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff6ef400640 10 client.7857 handle_command_reply: tid=0 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 client.7857 shutdown 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 msgr2=0x5627241f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f5390 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc003fd0 tx=0x7ff6fc005980 comp rx=0 tx=0).stop 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 shutdown_connections 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56272402e950 0x5627241f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x562724017ca0 0x5627241f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.104:6834/892094550,v1:172.21.15.104:6835/892094550] conn(0x562723c11830 0x5627241b1670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 --2- 172.21.15.16:0/724307094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562723c3e460 0x5627241f5390 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff6ef400640 0 client.7857 ms_handle_reset on v2:172.21.15.104:6834/892094550 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 >> 172.21.15.16:0/724307094 conn(0x562724104d90 msgr2=0x56272417dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 shutdown_connections 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 1 -- 172.21.15.16:0/724307094 wait complete. 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff718b19000 20 client.7857 trim_cache size 0 max 16384 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:09:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7737, 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1562780930000001, 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 86, 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 125, 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 125, 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 11.303683188999999 2024-09-17T05:09:01.659 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6e8073200 msgr2=0x7ff6e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6e8073200 0x7ff6e8075710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009b90 tx=0x7ff6fc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 msgr2=0x7ff70c0c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 0x7ff70c0c6880 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40b0040 tx=0x7ff6f4005760 comp rx=0 tx=0).stop 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 shutdown_connections 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6e8073200 0x7ff6e8075710 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff70c15ba50 0x7ff70c154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff70c074e30 0x7ff70c0c6880 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 --2- 172.21.15.16:0/3833191787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff70c074a20 0x7ff70c165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 >> 172.21.15.16:0/3833191787 conn(0x7ff70c0b2250 msgr2=0x7ff70c0cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 shutdown_connections 2024-09-17T05:09:01.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.654+0000 7ff711400640 1 -- 172.21.15.16:0/3833191787 wait complete. 2024-09-17T05:09:01.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:09:01.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 -- 172.21.15.16:0/906877320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0cc6f0 msgr2=0x7fe8cc0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 --2- 172.21.15.16:0/906877320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0cc6f0 0x7fe8cc0ccb00 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc073890 tx=0x7fe8c401ca60 comp rx=0 tx=0).stop 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 -- 172.21.15.16:0/906877320 shutdown_connections 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 --2- 172.21.15.16:0/906877320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0cc6f0 0x7fe8cc0ccb00 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 --2- 172.21.15.16:0/906877320 >> v2:172.21.15.104:3300/0 conn(0x7fe8cc0749d0 0x7fe8cc152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 -- 172.21.15.16:0/906877320 >> 172.21.15.16:0/906877320 conn(0x7fe8cc0cdb80 msgr2=0x7fe8cc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 -- 172.21.15.16:0/906877320 shutdown_connections 2024-09-17T05:09:01.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.830+0000 7fe8d1600640 1 -- 172.21.15.16:0/906877320 wait complete. 2024-09-17T05:09:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 Processor -- start 2024-09-17T05:09:01.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 -- start start 2024-09-17T05:09:01.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8cc06dfa0 0x7fe8cc0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 0x7fe8cc15a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8cc0cc6f0 0x7fe8cc14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc15c760 con 0x7fe8cc0749d0 2024-09-17T05:09:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc15c5e0 con 0x7fe8cc06dfa0 2024-09-17T05:09:01.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8d1600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc15c8e0 con 0x7fe8cc0cc6f0 2024-09-17T05:09:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 0x7fe8cc15a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8c3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 0x7fe8cc15a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37442/0 (socket says 172.21.15.16:37442) 2024-09-17T05:09:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.834+0000 7fe8c3e00640 1 -- 172.21.15.16:0/78031653 learned_addr learned my addr 172.21.15.16:0/78031653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8caa00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8cc06dfa0 0x7fe8cc0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8cb400640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8cc0cc6f0 0x7fe8cc14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c3e00640 1 -- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8cc0cc6f0 msgr2=0x7fe8cc14ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c3e00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8cc0cc6f0 0x7fe8cc14ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c3e00640 1 -- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8cc06dfa0 msgr2=0x7fe8cc0b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c3e00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8cc06dfa0 0x7fe8cc0b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c3e00640 1 -- 172.21.15.16:0/78031653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc1fc4c0 con 0x7fe8cc0749d0 2024-09-17T05:09:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8caa00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8cc06dfa0 0x7fe8cc0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:01.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8cb400640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8cc0cc6f0 0x7fe8cc14ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c3e00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 0x7fe8cc15a620 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400c400 tx=0x7fe8b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4011790 con 0x7fe8cc0749d0 2024-09-17T05:09:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8b4017070 con 0x7fe8cc0749d0 2024-09-17T05:09:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4011cc0 con 0x7fe8cc0749d0 2024-09-17T05:09:01.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8cc1fc760 con 0x7fe8cc0749d0 2024-09-17T05:09:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8cc1fcd20 con 0x7fe8cc0749d0 2024-09-17T05:09:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8cc152250 con 0x7fe8cc0749d0 2024-09-17T05:09:01.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.838+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe8b401c460 con 0x7fe8cc0749d0 2024-09-17T05:09:01.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.842+0000 7fe8c2a00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8a8073230 0x7fe8a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.842+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fe8b4094190 con 0x7fe8cc0749d0 2024-09-17T05:09:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.842+0000 7fe8caa00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8a8073230 0x7fe8a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.842+0000 7fe8caa00640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8a8073230 0x7fe8a8075740 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009ac0 tx=0x7fe8bc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:01.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:01.846+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8b405f010 con 0x7fe8cc0749d0 2024-09-17T05:09:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8cc0b0f20 con 0x7fe8cc0749d0 2024-09-17T05:09:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8c2a00640 1 -- 172.21.15.16:0/78031653 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7fe8b405e910 con 0x7fe8cc0749d0 2024-09-17T05:09:02.058 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:09:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8a8073230 msgr2=0x7fe8a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8a8073230 0x7fe8a8075740 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009ac0 tx=0x7fe8bc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 msgr2=0x7fe8cc15a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 0x7fe8cc15a620 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe8b400c400 tx=0x7fe8b400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:09:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 shutdown_connections 2024-09-17T05:09:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8a8073230 0x7fe8a8075740 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe8cc0cc6f0 0x7fe8cc14ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8cc0749d0 0x7fe8cc15a620 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 --2- 172.21.15.16:0/78031653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8cc06dfa0 0x7fe8cc0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 >> 172.21.15.16:0/78031653 conn(0x7fe8cc0cdb80 msgr2=0x7fe8cc0b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 shutdown_connections 2024-09-17T05:09:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02.058+0000 7fe8d1600640 1 -- 172.21.15.16:0/78031653 wait complete. 2024-09-17T05:09:02.080 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:09:02.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:09:02.080 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:09:02.080 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:09:02.080 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:02.080 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:09:02.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:09:02.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:09:02.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:09:02.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.081 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:09:02.093 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:09:02.093 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:09:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:09:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:09:02.094 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.095 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:02.095 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:09:02.095 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:09:02.096 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:09:02.096 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:09:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:09:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41951sec preferred_lft 41951sec 2024-09-17T05:09:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:09:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.097 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:09:02.098 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:09:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:02.098 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:09:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.099 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:09:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:09:02.099 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:09:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.100 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:09:02.100 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:09:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.101 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:09:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:09:02.101 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:02.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.102 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:09:02.154 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:09:02.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.156 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:09:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:09:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:09:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:09:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:09:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:09:02.214 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:09:02.214 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:09:02.214 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:09:02.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:09:02.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:09:02.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:09:02.215 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:09:02.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:09:02.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:02.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.356 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:09:02.366 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:09:02.366 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:09:02.367 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:09:02.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:09:02.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:02.432 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:02.488 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:09:02.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:02.489 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:02.539 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:09:02.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:02.645 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:09:02.646 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:09:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:09:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:09:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:02.664 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:02.674 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:02.674 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:09:02.674 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:09:02.674 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:02.674 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:02.674 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:09:02.675 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:09:02.675 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:09:02.678 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:09:02.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:02.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:02.700 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:09:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:09:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:02.823 INFO:teuthology.orchestra.run.smithi016.stdout:client7869 2024-09-17T05:09:02.823 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:02.824 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:02.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:09:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 -- 172.21.15.16:0/399994030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003c40 msgr2=0x7f1bd40040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 --2- 172.21.15.16:0/399994030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003c40 0x7f1bd40040b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009d00 tx=0x7f1be001d3d0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 -- 172.21.15.16:0/399994030 shutdown_connections 2024-09-17T05:09:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 --2- 172.21.15.16:0/399994030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bd4003c40 0x7f1bd40040b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 -- 172.21.15.16:0/399994030 >> 172.21.15.16:0/399994030 conn(0x7f1bec06fcf0 msgr2=0x7f1bec070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:03.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 -- 172.21.15.16:0/399994030 shutdown_connections 2024-09-17T05:09:03.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.034+0000 7f1bf4800640 1 -- 172.21.15.16:0/399994030 wait complete. 2024-09-17T05:09:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 Processor -- start 2024-09-17T05:09:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 -- start start 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4003c40 0x7f1bec1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1bec0a9460 0x7f1bec0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 0x7f1bec1ec1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec0ccfd0 con 0x7f1bec0b4d90 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec06e8e0 con 0x7f1bd4003c40 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf4800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1bec0cd150 con 0x7f1bec0a9460 2024-09-17T05:09:03.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 0x7f1bec1ec1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 0x7f1bec1ec1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37462/0 (socket says 172.21.15.16:37462) 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf2800640 1 -- 172.21.15.16:0/3046176039 learned_addr learned my addr 172.21.15.16:0/3046176039 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.038+0000 7f1bf1e00640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4003c40 0x7f1bec1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf1400640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1bec0a9460 0x7f1bec0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf2800640 1 -- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1bec0a9460 msgr2=0x7f1bec0caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1bec0a9460 0x7f1bec0caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf2800640 1 -- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4003c40 msgr2=0x7f1bec1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4003c40 0x7f1bec1603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf2800640 1 -- 172.21.15.16:0/3046176039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec0b3ff0 con 0x7f1bec0b4d90 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf2800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 0x7f1bec1ec1a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0ccaf0 tx=0x7f1be8001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8003e50 con 0x7f1bec0b4d90 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1be8016070 con 0x7f1bec0b4d90 2024-09-17T05:09:03.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be80043e0 con 0x7f1bec0b4d90 2024-09-17T05:09:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec0cb440 con 0x7f1bec0b4d90 2024-09-17T05:09:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec0b42b0 con 0x7f1bec0b4d90 2024-09-17T05:09:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec0cd670 con 0x7f1bec0b4d90 2024-09-17T05:09:03.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.042+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1be8004580 con 0x7f1bec0b4d90 2024-09-17T05:09:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.046+0000 7f1be6000640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1bd00733a0 0x7f1bd00758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:03.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.046+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f1be8094c90 con 0x7f1bec0b4d90 2024-09-17T05:09:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.046+0000 7f1bf1e00640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1bd00733a0 0x7f1bd00758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:03.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.046+0000 7f1bf1e00640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1bd00733a0 0x7f1bd00758b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1bec073010 tx=0x7f1be001a170 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:03.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.050+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1be805fa80 con 0x7f1bec0b4d90 2024-09-17T05:09:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1bec158430 con 0x7f1bec0b4d90 2024-09-17T05:09:03.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1be6000640 1 -- 172.21.15.16:0/3046176039 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7f1be805f380 con 0x7f1bec0b4d90 2024-09-17T05:09:03.262 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1bd00733a0 msgr2=0x7f1bd00758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1bd00733a0 0x7f1bd00758b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1bec073010 tx=0x7f1be001a170 comp rx=0 tx=0).stop 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 msgr2=0x7f1bec1ec1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 0x7f1bec1ec1a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0ccaf0 tx=0x7f1be8001b90 comp rx=0 tx=0).stop 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 shutdown_connections 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1bd00733a0 0x7f1bd00758b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bec0b4d90 0x7f1bec1ec1a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1bec0a9460 0x7f1bec0caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 --2- 172.21.15.16:0/3046176039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1bd4003c40 0x7f1bec1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 >> 172.21.15.16:0/3046176039 conn(0x7f1bec06fcf0 msgr2=0x7f1bec1577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 shutdown_connections 2024-09-17T05:09:03.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:03.262+0000 7f1bf4800640 1 -- 172.21.15.16:0/3046176039 wait complete. 2024-09-17T05:09:03.284 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:09:03.284 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:09:03.284 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:09:03.284 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:09:03.284 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:03.284 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:09:03.285 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:09:03.285 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:09:03.285 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:09:03.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.285 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:09:03.295 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:09:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:09:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:09:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:09:03.295 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 41944sec preferred_lft 41944sec 2024-09-17T05:09:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:03.297 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:09:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:09:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.298 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:09:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:09:03.298 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:03.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.299 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:09:03.360 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:09:03.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.363 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:09:03.416 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:09:03.417 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:09:03.418 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:09:03.418 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:09:03.418 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:09:03.418 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:09:03.418 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:09:03.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:09:03.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:09:03.418 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:09:03.419 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:09:03.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:09:03.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:03.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.568 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:09:03.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:09:03.578 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:09:03.578 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:09:03.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:09:03.644 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:03.644 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:03.692 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:09:03.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:03.692 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:03.743 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:09:03.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:03.843 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:03.843 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:09:03.844 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:09:03.861 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:03.861 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:09:03.861 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:09:03.861 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:03.861 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:03.873 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:03.873 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:09:03.873 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:09:03.873 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:03.874 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:03.874 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:09:03.874 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:09:03.874 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:09:03.875 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:09:03.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:03.875 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:03.894 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:09:03.982 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:09:03.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:03.991 INFO:teuthology.orchestra.run.smithi104.stdout:client6647 2024-09-17T05:09:03.991 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:03.991 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:04.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 -- 172.21.15.16:0/1896151817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c159f70 msgr2=0x7fa44c0d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 --2- 172.21.15.16:0/1896151817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c159f70 0x7fa44c0d1b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa440009bf0 tx=0x7fa44001cae0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 -- 172.21.15.16:0/1896151817 shutdown_connections 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 --2- 172.21.15.16:0/1896151817 >> v2:172.21.15.104:3300/0 conn(0x7fa44c0cf850 0x7fa44c0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 --2- 172.21.15.16:0/1896151817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c159f70 0x7fa44c0d1b80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 -- 172.21.15.16:0/1896151817 >> 172.21.15.16:0/1896151817 conn(0x7fa44c0b42d0 msgr2=0x7fa44c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 -- 172.21.15.16:0/1896151817 shutdown_connections 2024-09-17T05:09:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.166+0000 7fa454000640 1 -- 172.21.15.16:0/1896151817 wait complete. 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 Processor -- start 2024-09-17T05:09:04.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 -- start start 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 0x7fa44c14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa44c159f70 0x7fa44c0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c0b7f90 0x7fa44c153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c169e20 con 0x7fa44c0d02d0 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c169ca0 con 0x7fa44c159f70 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c169fa0 con 0x7fa44c0b7f90 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 0x7fa44c14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 0x7fa44c14b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37464/0 (socket says 172.21.15.16:37464) 2024-09-17T05:09:04.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 -- 172.21.15.16:0/2322183065 learned_addr learned my addr 172.21.15.16:0/2322183065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa450c00640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa44c159f70 0x7fa44c0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa452000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c0b7f90 0x7fa44c153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 -- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c0b7f90 msgr2=0x7fa44c153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c0b7f90 0x7fa44c153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 -- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa44c159f70 msgr2=0x7fa44c0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa44c159f70 0x7fa44c0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 -- 172.21.15.16:0/2322183065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c1fc250 con 0x7fa44c0d02d0 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa450c00640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa44c159f70 0x7fa44c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa452000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c0b7f90 0x7fa44c153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa451600640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 0x7fa44c14b280 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa44001c550 tx=0x7fa440005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400b2040 con 0x7fa44c0d02d0 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4400bb070 con 0x7fa44c0d02d0 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c1fc530 con 0x7fa44c0d02d0 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440004370 con 0x7fa44c0d02d0 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.170+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c1fcaf0 con 0x7fa44c0d02d0 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.174+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c0b2100 con 0x7fa44c0d02d0 2024-09-17T05:09:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.174+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa440004510 con 0x7fa44c0d02d0 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.174+0000 7fa449600640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa42c073200 0x7fa42c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.174+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fa44012ece0 con 0x7fa44c0d02d0 2024-09-17T05:09:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.174+0000 7fa450c00640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa42c073200 0x7fa42c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.178+0000 7fa450c00640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa42c073200 0x7fa42c075710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa44c1fbe40 tx=0x7fa438009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:04.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.178+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa440016790 con 0x7fa44c0d02d0 2024-09-17T05:09:04.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.358+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7fa44c0cade0 con 0x7fa44c0d02d0 2024-09-17T05:09:04.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa449600640 1 -- 172.21.15.16:0/2322183065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v58)=0 v58) ==== 121+0+0 (secure 0 0 0) 0x7fa4400f9c10 con 0x7fa44c0d02d0 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa42c073200 msgr2=0x7fa42c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa42c073200 0x7fa42c075710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa44c1fbe40 tx=0x7fa438009040 comp rx=0 tx=0).stop 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 msgr2=0x7fa44c14b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 0x7fa44c14b280 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fa44001c550 tx=0x7fa440005c40 comp rx=0 tx=0).stop 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 shutdown_connections 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa42c073200 0x7fa42c075710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa44c0b7f90 0x7fa44c153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa44c159f70 0x7fa44c0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 --2- 172.21.15.16:0/2322183065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c0d02d0 0x7fa44c14b280 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 >> 172.21.15.16:0/2322183065 conn(0x7fa44c0b42d0 msgr2=0x7fa44c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 shutdown_connections 2024-09-17T05:09:04.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.362+0000 7fa454000640 1 -- 172.21.15.16:0/2322183065 wait complete. 2024-09-17T05:09:04.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- 172.21.15.16:0/3284189337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80d1c20 msgr2=0x7fdbc80b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3284189337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80d1c20 0x7fdbc80b3ec0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009840 tx=0x7fdbb001c820 comp rx=0 tx=0).stop 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- 172.21.15.16:0/3284189337 shutdown_connections 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3284189337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80d1c20 0x7fdbc80b3ec0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- 172.21.15.16:0/3284189337 >> 172.21.15.16:0/3284189337 conn(0x7fdbc80cef40 msgr2=0x7fdbc80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- 172.21.15.16:0/3284189337 shutdown_connections 2024-09-17T05:09:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- 172.21.15.16:0/3284189337 wait complete. 2024-09-17T05:09:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 Processor -- start 2024-09-17T05:09:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- start start 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 0x7fdbc81fc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbc80d1810 0x7fdbc814dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc80d1c20 0x7fdbc80b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc816b0d0 con 0x7fdbc80b4400 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc81537b0 con 0x7fdbc80d1810 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.546+0000 7fdbc7200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc816b250 con 0x7fdbc80d1c20 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 0x7fdbc81fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 0x7fdbc81fc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37486/0 (socket says 172.21.15.16:37486) 2024-09-17T05:09:04.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 -- 172.21.15.16:0/3183908581 learned_addr learned my addr 172.21.15.16:0/3183908581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbbd400640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbc80d1810 0x7fdbc814dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc6800640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc80d1c20 0x7fdbc80b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 -- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc80d1c20 msgr2=0x7fdbc80b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc80d1c20 0x7fdbc80b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 -- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbc80d1810 msgr2=0x7fdbc814dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbc80d1810 0x7fdbc814dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 -- 172.21.15.16:0/3183908581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc80caa00 con 0x7fdbc80b4400 2024-09-17T05:09:04.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc5e00640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 0x7fdbc81fc120 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c3f0 tx=0x7fdbb800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8017070 con 0x7fdbc80b4400 2024-09-17T05:09:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbb8003600 con 0x7fdbc80b4400 2024-09-17T05:09:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8003a50 con 0x7fdbc80b4400 2024-09-17T05:09:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc81fc970 con 0x7fdbc80b4400 2024-09-17T05:09:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc81fcf30 con 0x7fdbc80b4400 2024-09-17T05:09:04.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc80b3ec0 con 0x7fdbc80b4400 2024-09-17T05:09:04.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.550+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdbb801a070 con 0x7fdbc80b4400 2024-09-17T05:09:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.554+0000 7fdbbea00640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdba4073260 0x7fdba4075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.554+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fdbb8093630 con 0x7fdbc80b4400 2024-09-17T05:09:04.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.554+0000 7fdbbd400640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdba4073260 0x7fdba4075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.554+0000 7fdbbd400640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdba4073260 0x7fdba4075770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00af000 tx=0x7fdbb001b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:04.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.558+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdbc80b4810 con 0x7fdbc80b4400 2024-09-17T05:09:04.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.738+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7fdbc815c180 con 0x7fdbc80b4400 2024-09-17T05:09:04.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbbea00640 1 -- 172.21.15.16:0/3183908581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v59)=0 v59) ==== 119+0+0 (secure 0 0 0) 0x7fdbb805ddd0 con 0x7fdbc80b4400 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdba4073260 msgr2=0x7fdba4075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdba4073260 0x7fdba4075770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdbb00af000 tx=0x7fdbb001b060 comp rx=0 tx=0).stop 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 msgr2=0x7fdbc81fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 0x7fdbc81fc120 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c3f0 tx=0x7fdbb800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 shutdown_connections 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdba4073260 0x7fdba4075770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fdbc80d1c20 0x7fdbc80b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbc80d1810 0x7fdbc814dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 --2- 172.21.15.16:0/3183908581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc80b4400 0x7fdbc81fc120 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 >> 172.21.15.16:0/3183908581 conn(0x7fdbc80cef40 msgr2=0x7fdbc80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 shutdown_connections 2024-09-17T05:09:04.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.742+0000 7fdbc7200640 1 -- 172.21.15.16:0/3183908581 wait complete. 2024-09-17T05:09:04.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2024-09-17T05:09:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 -- 172.21.15.16:0/502327362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe8159f70 msgr2=0x7fafe80d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 --2- 172.21.15.16:0/502327362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe8159f70 0x7fafe80d1b80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009900 tx=0x7fafd801c790 comp rx=0 tx=0).stop 2024-09-17T05:09:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 -- 172.21.15.16:0/502327362 shutdown_connections 2024-09-17T05:09:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 --2- 172.21.15.16:0/502327362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafe80cf850 0x7fafe80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 --2- 172.21.15.16:0/502327362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe8159f70 0x7fafe80d1b80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 -- 172.21.15.16:0/502327362 >> 172.21.15.16:0/502327362 conn(0x7fafe80b42d0 msgr2=0x7fafe80b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 -- 172.21.15.16:0/502327362 shutdown_connections 2024-09-17T05:09:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.926+0000 7fafee600640 1 -- 172.21.15.16:0/502327362 wait complete. 2024-09-17T05:09:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 Processor -- start 2024-09-17T05:09:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 -- start start 2024-09-17T05:09:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 0x7fafe8157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafe8159f70 0x7fafe8152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafe8153890 0x7fafe806f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafe8169e20 con 0x7fafe80d02d0 2024-09-17T05:09:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fafe8169ca0 con 0x7fafe8159f70 2024-09-17T05:09:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fafe8169fa0 con 0x7fafe8153890 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 0x7fafe8157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 0x7fafe8157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37500/0 (socket says 172.21.15.16:37500) 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 -- 172.21.15.16:0/1503626060 learned_addr learned my addr 172.21.15.16:0/1503626060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafdfe00640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafe8159f70 0x7fafe8152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7e00640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafe8153890 0x7fafe806f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 -- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafe8153890 msgr2=0x7fafe806f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafe8153890 0x7fafe806f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 -- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafe8159f70 msgr2=0x7fafe8152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafe8159f70 0x7fafe8152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 -- 172.21.15.16:0/1503626060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe80cc530 con 0x7fafe80d02d0 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafe7400640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 0x7fafe8157a80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009850 tx=0x7fafd80a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80b2040 con 0x7fafe80d02d0 2024-09-17T05:09:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafd80043b0 con 0x7fafe80d02d0 2024-09-17T05:09:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd8005290 con 0x7fafe80d02d0 2024-09-17T05:09:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe80cc810 con 0x7fafe80d02d0 2024-09-17T05:09:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.930+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe8149870 con 0x7fafe80d02d0 2024-09-17T05:09:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.934+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe8149b60 con 0x7fafe80d02d0 2024-09-17T05:09:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.934+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fafd80bd060 con 0x7fafe80d02d0 2024-09-17T05:09:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.934+0000 7fafdf400640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafc4073200 0x7fafc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.934+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fafd812dc90 con 0x7fafe80d02d0 2024-09-17T05:09:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.934+0000 7fafdfe00640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafc4073200 0x7fafc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.934+0000 7fafdfe00640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafc4073200 0x7fafc4075710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fafe80b7dd0 tx=0x7fafd0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:04.938+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fafd80164d0 con 0x7fafe80d02d0 2024-09-17T05:09:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.110+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fafe80b2100 con 0x7fafe80d02d0 2024-09-17T05:09:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.114+0000 7fafdf400640 1 -- 172.21.15.16:0/1503626060 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v60)=0 v60) ==== 128+0+0 (secure 0 0 0) 0x7fafd80f84d0 con 0x7fafe80d02d0 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafc4073200 msgr2=0x7fafc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafc4073200 0x7fafc4075710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fafe80b7dd0 tx=0x7fafd0009040 comp rx=0 tx=0).stop 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 msgr2=0x7fafe8157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 0x7fafe8157a80 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fafd8009850 tx=0x7fafd80a56d0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 shutdown_connections 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fafc4073200 0x7fafc4075710 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fafe8153890 0x7fafe806f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fafe8159f70 0x7fafe8152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 --2- 172.21.15.16:0/1503626060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafe80d02d0 0x7fafe8157a80 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 >> 172.21.15.16:0/1503626060 conn(0x7fafe80b42d0 msgr2=0x7fafe815b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 shutdown_connections 2024-09-17T05:09:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.118+0000 7fafee600640 1 -- 172.21.15.16:0/1503626060 wait complete. 2024-09-17T05:09:05.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-17T05:09:05.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 -- 172.21.15.16:0/1601559893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 msgr2=0x7fea10004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 --2- 172.21.15.16:0/1601559893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea10004c20 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fea1400c2e0 tx=0x7fea1401f660 comp rx=0 tx=0).stop 2024-09-17T05:09:05.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 -- 172.21.15.16:0/1601559893 shutdown_connections 2024-09-17T05:09:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 --2- 172.21.15.16:0/1601559893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea10004c20 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 -- 172.21.15.16:0/1601559893 >> 172.21.15.16:0/1601559893 conn(0x7fea280b2250 msgr2=0x7fea280d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 -- 172.21.15.16:0/1601559893 shutdown_connections 2024-09-17T05:09:05.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.298+0000 7fea30400640 1 -- 172.21.15.16:0/1601559893 wait complete. 2024-09-17T05:09:05.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 Processor -- start 2024-09-17T05:09:05.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 -- start start 2024-09-17T05:09:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea28157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea281f5480 0x7fea280b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea28158cd0 0x7fea2815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea2815f370 con 0x7fea10004810 2024-09-17T05:09:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea2815f1f0 con 0x7fea281f5480 2024-09-17T05:09:05.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea30400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fea2815f4f0 con 0x7fea28158cd0 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.302+0000 7fea2da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea28157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea28157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37526/0 (socket says 172.21.15.16:37526) 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 -- 172.21.15.16:0/2548808253 learned_addr learned my addr 172.21.15.16:0/2548808253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2e400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea28158cd0 0x7fea2815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2d000640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea281f5480 0x7fea280b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 -- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea28158cd0 msgr2=0x7fea2815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea28158cd0 0x7fea2815a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 -- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea281f5480 msgr2=0x7fea280b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea281f5480 0x7fea280b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 -- 172.21.15.16:0/2548808253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea281f5a20 con 0x7fea10004810 2024-09-17T05:09:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2e400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea28158cd0 0x7fea2815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2d000640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea281f5480 0x7fea280b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea2da00640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea28157f10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fea1c003e80 tx=0x7fea1c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c015070 con 0x7fea10004810 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea1c011840 con 0x7fea10004810 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea281f5d00 con 0x7fea10004810 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c011c70 con 0x7fea10004810 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea281f62c0 con 0x7fea10004810 2024-09-17T05:09:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2815bcd0 con 0x7fea10004810 2024-09-17T05:09:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.306+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fea1c018070 con 0x7fea10004810 2024-09-17T05:09:05.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.310+0000 7fea25600640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea08073200 0x7fea08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.310+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fea1c094fd0 con 0x7fea10004810 2024-09-17T05:09:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.310+0000 7fea2d000640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea08073200 0x7fea08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.310+0000 7fea2d000640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea08073200 0x7fea08075710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea140b0000 tx=0x7fea1401c4a0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:05.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.314+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea1c05fe50 con 0x7fea10004810 2024-09-17T05:09:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.486+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fea280cb650 con 0x7fea10004810 2024-09-17T05:09:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.486+0000 7fea25600640 1 -- 172.21.15.16:0/2548808253 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v60) ==== 106+0+4 (secure 0 0 0) 0x7fea1c05f750 con 0x7fea10004810 2024-09-17T05:09:05.488 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-17T05:09:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea08073200 msgr2=0x7fea08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea08073200 0x7fea08075710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea140b0000 tx=0x7fea1401c4a0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 msgr2=0x7fea28157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea28157f10 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fea1c003e80 tx=0x7fea1c001640 comp rx=0 tx=0).stop 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 shutdown_connections 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fea08073200 0x7fea08075710 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fea28158cd0 0x7fea2815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea281f5480 0x7fea280b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 --2- 172.21.15.16:0/2548808253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea10004810 0x7fea28157f10 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 >> 172.21.15.16:0/2548808253 conn(0x7fea280b2250 msgr2=0x7fea280ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 shutdown_connections 2024-09-17T05:09:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.490+0000 7fea30400640 1 -- 172.21.15.16:0/2548808253 wait complete. 2024-09-17T05:09:05.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-17T05:09:05.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 -- 172.21.15.16:0/1539773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56380733e0 msgr2=0x7f563815d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 --2- 172.21.15.16:0/1539773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56380733e0 0x7f563815d140 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f562801c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 -- 172.21.15.16:0/1539773889 shutdown_connections 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 --2- 172.21.15.16:0/1539773889 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 0x7f5638160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 --2- 172.21.15.16:0/1539773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56380733e0 0x7f563815d140 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 -- 172.21.15.16:0/1539773889 >> 172.21.15.16:0/1539773889 conn(0x7f56380b2250 msgr2=0x7f56380d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 -- 172.21.15.16:0/1539773889 shutdown_connections 2024-09-17T05:09:05.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 -- 172.21.15.16:0/1539773889 wait complete. 2024-09-17T05:09:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 Processor -- start 2024-09-17T05:09:05.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.674+0000 7f563d200640 1 -- start start 2024-09-17T05:09:05.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56380733e0 0x7f56381f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 0x7f5638153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 0x7f56380b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f563816a2d0 con 0x7f5638160670 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56380cae60 con 0x7f56380733e0 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f563816a450 con 0x7f563815d680 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 0x7f56380b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 0x7f56380b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37542/0 (socket says 172.21.15.16:37542) 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 -- 172.21.15.16:0/4137921428 learned_addr learned my addr 172.21.15.16:0/4137921428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5635e00640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56380733e0 0x7f56381f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f562fe00640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 0x7f5638153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 -- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 msgr2=0x7f5638153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 0x7f5638153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 -- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56380733e0 msgr2=0x7f56381f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56380733e0 0x7f56381f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 -- 172.21.15.16:0/4137921428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5638154ba0 con 0x7f5638160670 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f562fe00640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 0x7f5638153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f5636800640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 0x7f56380b4f60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f563000b9e0 tx=0x7f563000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630003b80 con 0x7f5638160670 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5630003e10 con 0x7f5638160670 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563000cb20 con 0x7f5638160670 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56380c82f0 con 0x7f5638160670 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56380c88b0 con 0x7f5638160670 2024-09-17T05:09:05.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.678+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5638154910 con 0x7f5638160670 2024-09-17T05:09:05.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.682+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5630006d10 con 0x7f5638160670 2024-09-17T05:09:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.682+0000 7f562e000640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5614073200 0x7f5614075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.682+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f5630093ee0 con 0x7f5638160670 2024-09-17T05:09:05.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.682+0000 7f5635e00640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5614073200 0x7f5614075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:05.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.682+0000 7f5635e00640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5614073200 0x7f5614075710 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f56280af000 tx=0x7f562801b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.686+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f563005ed60 con 0x7f5638160670 2024-09-17T05:09:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.862+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f563815d140 con 0x7f5638160670 2024-09-17T05:09:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.866+0000 7f562e000640 1 -- 172.21.15.16:0/4137921428 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v61)=0 v61) ==== 124+0+0 (secure 0 0 0) 0x7f563005e660 con 0x7f5638160670 2024-09-17T05:09:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5614073200 msgr2=0x7f5614075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5614073200 0x7f5614075710 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f56280af000 tx=0x7f562801b0f0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 msgr2=0x7f56380b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 0x7f56380b4f60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f563000b9e0 tx=0x7f563000bea0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 shutdown_connections 2024-09-17T05:09:05.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5614073200 0x7f5614075710 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5638160670 0x7f56380b4f60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f563815d680 0x7f5638153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 --2- 172.21.15.16:0/4137921428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56380733e0 0x7f56381f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 >> 172.21.15.16:0/4137921428 conn(0x7f56380b2250 msgr2=0x7f56380b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 shutdown_connections 2024-09-17T05:09:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:05.870+0000 7f563d200640 1 -- 172.21.15.16:0/4137921428 wait complete. 2024-09-17T05:09:05.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 -- 172.21.15.16:0/2607707188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b600733e0 msgr2=0x7f2b6015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 --2- 172.21.15.16:0/2607707188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b600733e0 0x7f2b6015d140 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009900 tx=0x7f2b5401c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 -- 172.21.15.16:0/2607707188 shutdown_connections 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 --2- 172.21.15.16:0/2607707188 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b60160840 0x7f2b601672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 --2- 172.21.15.16:0/2607707188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b600733e0 0x7f2b6015d140 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 -- 172.21.15.16:0/2607707188 >> 172.21.15.16:0/2607707188 conn(0x7f2b600b2250 msgr2=0x7f2b600d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:06.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 -- 172.21.15.16:0/2607707188 shutdown_connections 2024-09-17T05:09:06.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.050+0000 7f2b67600640 1 -- 172.21.15.16:0/2607707188 wait complete. 2024-09-17T05:09:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 Processor -- start 2024-09-17T05:09:06.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 -- start start 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b600733e0 0x7f2b60153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 0x7f2b601f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b60160840 0x7f2b601fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6016a010 con 0x7f2b6015d680 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b600cae90 con 0x7f2b600733e0 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6016a190 con 0x7f2b60160840 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 0x7f2b601f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 0x7f2b601f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37560/0 (socket says 172.21.15.16:37560) 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 -- 172.21.15.16:0/1909883760 learned_addr learned my addr 172.21.15.16:0/1909883760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b65600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b60160840 0x7f2b601fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b64c00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b600733e0 0x7f2b60153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 -- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b60160840 msgr2=0x7f2b601fa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b60160840 0x7f2b601fa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 -- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b600733e0 msgr2=0x7f2b60153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b600733e0 0x7f2b60153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 -- 172.21.15.16:0/1909883760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6015bb80 con 0x7f2b6015d680 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5fe00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 0x7f2b601f9d70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00c390 tx=0x7f2b4c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c017070 con 0x7f2b6015d680 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b4c0053c0 con 0x7f2b6015d680 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6015be60 con 0x7f2b6015d680 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c003840 con 0x7f2b6015d680 2024-09-17T05:09:06.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.054+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6015c420 con 0x7f2b6015d680 2024-09-17T05:09:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.058+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6015d140 con 0x7f2b6015d680 2024-09-17T05:09:06.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.058+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b4c01a070 con 0x7f2b6015d680 2024-09-17T05:09:06.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.058+0000 7f2b5cc00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2b40073230 0x7f2b40075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.058+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f2b4c093650 con 0x7f2b6015d680 2024-09-17T05:09:06.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.058+0000 7f2b64c00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2b40073230 0x7f2b40075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.062+0000 7f2b64c00640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2b40073230 0x7f2b40075740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2b540af000 tx=0x7f2b5401b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:06.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.062+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b4c05de70 con 0x7f2b6015d680 2024-09-17T05:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.238+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f2b6015de80 con 0x7f2b6015d680 2024-09-17T05:09:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.238+0000 7f2b5cc00640 1 -- 172.21.15.16:0/1909883760 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v61) ==== 106+0+8 (secure 0 0 0) 0x7f2b4c0628a0 con 0x7f2b6015d680 2024-09-17T05:09:06.239 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-17T05:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2b40073230 msgr2=0x7f2b40075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2b40073230 0x7f2b40075740 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2b540af000 tx=0x7f2b5401b0f0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 msgr2=0x7f2b601f9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 0x7f2b601f9d70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00c390 tx=0x7f2b4c00c850 comp rx=0 tx=0).stop 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 shutdown_connections 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2b40073230 0x7f2b40075740 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2b60160840 0x7f2b601fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b6015d680 0x7f2b601f9d70 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 --2- 172.21.15.16:0/1909883760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b600733e0 0x7f2b60153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 >> 172.21.15.16:0/1909883760 conn(0x7f2b600b2250 msgr2=0x7f2b600cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 shutdown_connections 2024-09-17T05:09:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.242+0000 7f2b67600640 1 -- 172.21.15.16:0/1909883760 wait complete. 2024-09-17T05:09:06.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-17T05:09:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 -- 172.21.15.16:0/1988940475 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 msgr2=0x7f5d5815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 --2- 172.21.15.16:0/1988940475 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d5815efe0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c009900 tx=0x7f5d4c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 -- 172.21.15.16:0/1988940475 shutdown_connections 2024-09-17T05:09:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 --2- 172.21.15.16:0/1988940475 >> v2:172.21.15.104:3300/0 conn(0x7f5d5815f520 0x7f5d5815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 --2- 172.21.15.16:0/1988940475 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d5815efe0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 -- 172.21.15.16:0/1988940475 >> 172.21.15.16:0/1988940475 conn(0x7f5d580b2250 msgr2=0x7f5d580d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 -- 172.21.15.16:0/1988940475 shutdown_connections 2024-09-17T05:09:06.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.422+0000 7f5d60000640 1 -- 172.21.15.16:0/1988940475 wait complete. 2024-09-17T05:09:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 Processor -- start 2024-09-17T05:09:06.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 -- start start 2024-09-17T05:09:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d581519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 0x7f5d580c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5815f520 0x7f5d581fc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58168ed0 con 0x7f5d58074e30 2024-09-17T05:09:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d58168d50 con 0x7f5d5815f520 2024-09-17T05:09:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d60000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5d58169050 con 0x7f5d58074a20 2024-09-17T05:09:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 0x7f5d580c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 0x7f5d580c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37570/0 (socket says 172.21.15.16:37570) 2024-09-17T05:09:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 -- 172.21.15.16:0/3995700256 learned_addr learned my addr 172.21.15.16:0/3995700256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:06.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5e000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5815f520 0x7f5d581fc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 -- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 msgr2=0x7f5d581519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5d600640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d581519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d581519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 -- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5815f520 msgr2=0x7f5d581fc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5815f520 0x7f5d581fc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 -- 172.21.15.16:0/3995700256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d58150a50 con 0x7f5d58074e30 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5e000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5815f520 0x7f5d581fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5d600640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d581519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:06.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.426+0000 7f5d5cc00640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 0x7f5d580c9db0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c430 tx=0x7f5d4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44017070 con 0x7f5d58074e30 2024-09-17T05:09:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d440053c0 con 0x7f5d58074e30 2024-09-17T05:09:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d58150d30 con 0x7f5d58074e30 2024-09-17T05:09:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003860 con 0x7f5d58074e30 2024-09-17T05:09:06.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d581599f0 con 0x7f5d58074e30 2024-09-17T05:09:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d5815efe0 con 0x7f5d58074e30 2024-09-17T05:09:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d4401a070 con 0x7f5d58074e30 2024-09-17T05:09:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d55600640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f5d440936d0 con 0x7f5d58074e30 2024-09-17T05:09:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.430+0000 7f5d5d600640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:06.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.434+0000 7f5d5d600640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af000 tx=0x7f5d4c01b120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:06.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.434+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5d4405e6a0 con 0x7f5d58074e30 2024-09-17T05:09:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.610+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f5d580cab10 con 0x7f5d58074e30 2024-09-17T05:09:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.610+0000 7f5d55600640 1 -- 172.21.15.16:0/3995700256 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v61) ==== 112+0+10 (secure 0 0 0) 0x7f5d4405dfa0 con 0x7f5d58074e30 2024-09-17T05:09:06.610 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 msgr2=0x7f5d38075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0af000 tx=0x7f5d4c01b120 comp rx=0 tx=0).stop 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 msgr2=0x7f5d580c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 0x7f5d580c9db0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c430 tx=0x7f5d4400c8f0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 shutdown_connections 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5d38073200 0x7f5d38075710 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d5815f520 0x7f5d581fc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d58074e30 0x7f5d580c9db0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 --2- 172.21.15.16:0/3995700256 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5d58074a20 0x7f5d581519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 >> 172.21.15.16:0/3995700256 conn(0x7f5d580b2250 msgr2=0x7f5d580ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:06.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 shutdown_connections 2024-09-17T05:09:06.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06.614+0000 7f5d60000640 1 -- 172.21.15.16:0/3995700256 wait complete. 2024-09-17T05:09:06.633 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:06.633 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:06.642 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:09:06.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:09:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:06.742 INFO:teuthology.orchestra.run.smithi016.stdout:client7869 2024-09-17T05:09:06.743 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:06.743 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:06.792 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> n = 200 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-17T05:09:06.793 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-17T05:09:06.794 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-17T05:09:06.795 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:09:06.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:07.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 -- 172.21.15.16:0/3970745560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541415cab0 msgr2=0x7f54140c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 --2- 172.21.15.16:0/3970745560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541415cab0 0x7f54140c9b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f540c009bc0 tx=0x7f540c01cb70 comp rx=0 tx=0).stop 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 -- 172.21.15.16:0/3970745560 shutdown_connections 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 --2- 172.21.15.16:0/3970745560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541415cab0 0x7f54140c9b50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 -- 172.21.15.16:0/3970745560 >> 172.21.15.16:0/3970745560 conn(0x7f54140d2270 msgr2=0x7f54140d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 -- 172.21.15.16:0/3970745560 shutdown_connections 2024-09-17T05:09:07.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.010+0000 7f541a000640 1 -- 172.21.15.16:0/3970745560 wait complete. 2024-09-17T05:09:07.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 Processor -- start 2024-09-17T05:09:07.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 -- start start 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54140cc8c0 0x7f54141f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 0x7f5414148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 0x7f54140b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f541416aec0 con 0x7f541415cab0 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f541416ad40 con 0x7f54140cc8c0 2024-09-17T05:09:07.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f541a000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f541416b040 con 0x7f5414156a00 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f540be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 0x7f5414148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f5413e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 0x7f54140b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f5413400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54140cc8c0 0x7f54141f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f5413e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 0x7f54140b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:37626/0 (socket says 172.21.15.16:37626) 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f540be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 0x7f5414148970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42098/0 (socket says 172.21.15.16:42098) 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f5413e00640 1 -- 172.21.15.16:0/1022907154 learned_addr learned my addr 172.21.15.16:0/1022907154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f540be00640 1 -- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 msgr2=0x7f54140b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f540be00640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 0x7f54140b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f540be00640 1 -- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54140cc8c0 msgr2=0x7f54141f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.014+0000 7f540be00640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54140cc8c0 0x7f54141f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f540be00640 1 -- 172.21.15.16:0/1022907154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54140b6240 con 0x7f541415cab0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f5413e00640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 0x7f54140b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f540be00640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 0x7f5414148970 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c3f0 tx=0x7f53fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc017070 con 0x7f541415cab0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53fc0053c0 con 0x7f541415cab0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54140b64e0 con 0x7f541415cab0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc003860 con 0x7f541415cab0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54140b6aa0 con 0x7f541415cab0 2024-09-17T05:09:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541406f750 con 0x7f541415cab0 2024-09-17T05:09:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.018+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f53fc01a070 con 0x7f541415cab0 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.022+0000 7f540b400640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53f0073200 0x7f53f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.022+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f53fc0937c0 con 0x7f541415cab0 2024-09-17T05:09:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.022+0000 7f5413400640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53f0073200 0x7f53f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.022+0000 7f5413400640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53f0073200 0x7f53f0075710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f54040046c0 tx=0x7f5404007cd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.026+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53fc05dfa0 con 0x7f541415cab0 2024-09-17T05:09:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-17T05:09:07.164 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:07.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54140d1d40 con 0x7f541415cab0 2024-09-17T05:09:07.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f540b400640 1 -- 172.21.15.16:0/1022907154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5225 (secure 0 0 0) 0x7f53fc0629d0 con 0x7f541415cab0 2024-09-17T05:09:07.254 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:07.254 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":86,"btime":"2024-09-17T05:08:58:530125+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:08:57.947398+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:07.254 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 86 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53f0073200 msgr2=0x7f53f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53f0073200 0x7f53f0075710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f54040046c0 tx=0x7f5404007cd0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 msgr2=0x7f5414148970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 0x7f5414148970 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c3f0 tx=0x7f53fc00c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 shutdown_connections 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53f0073200 0x7f53f0075710 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5414156a00 0x7f54140b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f541415cab0 0x7f5414148970 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.254+0000 7f541a000640 1 --2- 172.21.15.16:0/1022907154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54140cc8c0 0x7f54141f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.258+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 >> 172.21.15.16:0/1022907154 conn(0x7f54140d2270 msgr2=0x7f54140cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:07.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.258+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 shutdown_connections 2024-09-17T05:09:07.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.258+0000 7f541a000640 1 -- 172.21.15.16:0/1022907154 wait complete. 2024-09-17T05:09:07.277 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-17T05:09:07.446 INFO:teuthology.orchestra.run.smithi104.stdout:[{"id":6647,"entity":{"name":{"type":"client","num":6647},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2038023171}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.5639518049999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9894070644656168,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6647 v1:192.168.0.1:0/2038023171","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7869,"entity":{"name":{"type":"client","num":7869},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2629263370}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":225,"uptime":4.7599355860000001,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":404.62315529096657,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7869 v1:192.168.0.1:0/2629263370","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:09:07.447 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2038023171, "type": "v1" }, "name": { "num": 6647, "type": "client" } }, "id": 6647, "inst": "client.6647 v1:192.168.0.1:0/2038023171", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9894070644656168 }, "state": "open", "uptime": 3.563951805 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2629263370, "type": "v1" }, "name": { "num": 7869, "type": "client" } }, "id": 7869, "inst": "client.7869 v1:192.168.0.1:0/2629263370", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 225, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 404.62315529096657 }, "state": "open", "uptime": 4.759935586 } ] 2024-09-17T05:09:07.447 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-17T05:09:07.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:09:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 -- 172.21.15.16:0/2927874686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde080d0b80 msgr2=0x7fde080d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 --2- 172.21.15.16:0/2927874686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde080d0b80 0x7fde080d0f90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fde080cdb80 tx=0x7fde0001cc60 comp rx=0 tx=0).stop 2024-09-17T05:09:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 -- 172.21.15.16:0/2927874686 shutdown_connections 2024-09-17T05:09:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 --2- 172.21.15.16:0/2927874686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde080d0b80 0x7fde080d0f90 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 --2- 172.21.15.16:0/2927874686 >> v2:172.21.15.104:3300/0 conn(0x7fde081608d0 0x7fde080d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 --2- 172.21.15.16:0/2927874686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0815c6f0 0x7fde08160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 -- 172.21.15.16:0/2927874686 >> 172.21.15.16:0/2927874686 conn(0x7fde080cf380 msgr2=0x7fde080d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 -- 172.21.15.16:0/2927874686 shutdown_connections 2024-09-17T05:09:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.606+0000 7fde0e200640 1 -- 172.21.15.16:0/2927874686 wait complete. 2024-09-17T05:09:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 Processor -- start 2024-09-17T05:09:07.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 -- start start 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde080d0b80 0x7fde080cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0815c6f0 0x7fde0814d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 0x7fde0806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde080b2c40 con 0x7fde081608d0 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde080b2ac0 con 0x7fde080d0b80 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fde080b2dc0 con 0x7fde0815c6f0 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 0x7fde0806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 0x7fde0806ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42104/0 (socket says 172.21.15.16:42104) 2024-09-17T05:09:07.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 -- 172.21.15.16:0/2469720435 learned_addr learned my addr 172.21.15.16:0/2469720435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07400640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde080d0b80 0x7fde080cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fddffe00640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0815c6f0 0x7fde0814d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 -- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0815c6f0 msgr2=0x7fde0814d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0815c6f0 0x7fde0814d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 -- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde080d0b80 msgr2=0x7fde080cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde080d0b80 0x7fde080cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 -- 172.21.15.16:0/2469720435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde080ca4f0 con 0x7fde081608d0 2024-09-17T05:09:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde07e00640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 0x7fde0806ffb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fde00009bd0 tx=0x7fde00005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde000bc070 con 0x7fde081608d0 2024-09-17T05:09:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde000040b0 con 0x7fde081608d0 2024-09-17T05:09:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde000b2040 con 0x7fde081608d0 2024-09-17T05:09:07.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde080ca7d0 con 0x7fde081608d0 2024-09-17T05:09:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.610+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde080cad90 con 0x7fde081608d0 2024-09-17T05:09:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.614+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0806ad00 con 0x7fde081608d0 2024-09-17T05:09:07.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.614+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fde000bf070 con 0x7fde081608d0 2024-09-17T05:09:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.614+0000 7fddff400640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdde4073230 0x7fdde4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.614+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fde0012e360 con 0x7fde081608d0 2024-09-17T05:09:07.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.614+0000 7fde07400640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdde4073230 0x7fdde4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:07.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.614+0000 7fde07400640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdde4073230 0x7fdde4075740 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009b40 tx=0x7fddf80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.618+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fde08160ce0 con 0x7fde081608d0 2024-09-17T05:09:07.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fde080b3350 con 0x7fde081608d0 2024-09-17T05:09:07.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fddff400640 1 -- 172.21.15.16:0/2469720435 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fde00016970 con 0x7fde081608d0 2024-09-17T05:09:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:07.830 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T05:09:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdde4073230 msgr2=0x7fdde4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdde4073230 0x7fdde4075740 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009b40 tx=0x7fddf80091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 msgr2=0x7fde0806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 0x7fde0806ffb0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fde00009bd0 tx=0x7fde00005a60 comp rx=0 tx=0).stop 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 shutdown_connections 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdde4073230 0x7fdde4075740 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde081608d0 0x7fde0806ffb0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fde0815c6f0 0x7fde0814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 --2- 172.21.15.16:0/2469720435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde080d0b80 0x7fde080cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 >> 172.21.15.16:0/2469720435 conn(0x7fde080cf380 msgr2=0x7fde080b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 shutdown_connections 2024-09-17T05:09:07.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:07.830+0000 7fde0e200640 1 -- 172.21.15.16:0/2469720435 wait complete. 2024-09-17T05:09:07.851 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-17T05:09:07.851 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:09:09.164 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:11.167 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:12.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:09:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 -- 172.21.15.16:0/4203026245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c3015c680 msgr2=0x7f9c30073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 --2- 172.21.15.16:0/4203026245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c3015c680 0x7f9c30073950 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009bc0 tx=0x7f9c2801cb10 comp rx=0 tx=0).stop 2024-09-17T05:09:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 -- 172.21.15.16:0/4203026245 shutdown_connections 2024-09-17T05:09:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 --2- 172.21.15.16:0/4203026245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c3015c680 0x7f9c30073950 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 -- 172.21.15.16:0/4203026245 >> 172.21.15.16:0/4203026245 conn(0x7f9c3006f900 msgr2=0x7f9c300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 -- 172.21.15.16:0/4203026245 shutdown_connections 2024-09-17T05:09:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 -- 172.21.15.16:0/4203026245 wait complete. 2024-09-17T05:09:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 Processor -- start 2024-09-17T05:09:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.025+0000 7f9c37800640 1 -- start start 2024-09-17T05:09:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 0x7f9c301495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c3015c270 0x7f9c300cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c3015c680 0x7f9c3013f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c30169c00 con 0x7f9c300b2f50 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c30169a80 con 0x7f9c3015c680 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9c30169d80 con 0x7f9c3015c270 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 0x7f9c301495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 0x7f9c301495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42130/0 (socket says 172.21.15.16:42130) 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 -- 172.21.15.16:0/3679339216 learned_addr learned my addr 172.21.15.16:0/3679339216 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c35800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c3015c680 0x7f9c3013f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c2fe00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c3015c270 0x7f9c300cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 -- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c3015c270 msgr2=0x7f9c300cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c3015c270 0x7f9c300cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 -- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c3015c680 msgr2=0x7f9c3013f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c3015c680 0x7f9c3013f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 -- 172.21.15.16:0/3679339216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c30149b00 con 0x7f9c300b2f50 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c2fe00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c3015c270 0x7f9c300cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c35800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c3015c680 0x7f9c3013f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c34e00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 0x7f9c301495c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c3c0 tx=0x7f9c2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24017070 con 0x7f9c300b2f50 2024-09-17T05:09:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c240053c0 con 0x7f9c300b2f50 2024-09-17T05:09:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c30167000 con 0x7f9c300b2f50 2024-09-17T05:09:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24003860 con 0x7f9c300b2f50 2024-09-17T05:09:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c300ce020 con 0x7f9c300b2f50 2024-09-17T05:09:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.029+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c30074ee0 con 0x7f9c300b2f50 2024-09-17T05:09:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.033+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c2401a070 con 0x7f9c300b2f50 2024-09-17T05:09:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.033+0000 7f9c2cc00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9c10073200 0x7f9c10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.033+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f9c24093860 con 0x7f9c300b2f50 2024-09-17T05:09:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.033+0000 7f9c2fe00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9c10073200 0x7f9c10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.033+0000 7f9c2fe00640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9c10073200 0x7f9c10075710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9c30149570 tx=0x7f9c1c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.037+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c30167000 con 0x7f9c300b2f50 2024-09-17T05:09:13.168 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9c3015b5a0 con 0x7f9c300b2f50 2024-09-17T05:09:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c2cc00640 1 -- 172.21.15.16:0/3679339216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f9c2405e650 con 0x7f9c300b2f50 2024-09-17T05:09:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:13.249 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:09:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9c10073200 msgr2=0x7f9c10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9c10073200 0x7f9c10075710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9c30149570 tx=0x7f9c1c009040 comp rx=0 tx=0).stop 2024-09-17T05:09:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 msgr2=0x7f9c301495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 0x7f9c301495c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c3c0 tx=0x7f9c2400c880 comp rx=0 tx=0).stop 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 shutdown_connections 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9c10073200 0x7f9c10075710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c3015c680 0x7f9c3013f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9c3015c270 0x7f9c300cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 --2- 172.21.15.16:0/3679339216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c300b2f50 0x7f9c301495c0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 >> 172.21.15.16:0/3679339216 conn(0x7f9c3006f900 msgr2=0x7f9c300d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 shutdown_connections 2024-09-17T05:09:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.249+0000 7f9c37800640 1 -- 172.21.15.16:0/3679339216 wait complete. 2024-09-17T05:09:13.272 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:09:13.272 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-17T05:09:13.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:09:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c44400640 1 -- 172.21.15.16:0/2984504616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c24003a10 msgr2=0x7f2c24003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c44400640 1 --2- 172.21.15.16:0/2984504616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c24003a10 0x7f2c24003e80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009b60 tx=0x7f2c3001d2b0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c42400640 1 -- 172.21.15.16:0/2984504616 reap_dead start 2024-09-17T05:09:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c44400640 1 -- 172.21.15.16:0/2984504616 shutdown_connections 2024-09-17T05:09:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c44400640 1 -- 172.21.15.16:0/2984504616 >> 172.21.15.16:0/2984504616 conn(0x7f2c3c06f900 msgr2=0x7f2c3c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c44400640 1 -- 172.21.15.16:0/2984504616 shutdown_connections 2024-09-17T05:09:13.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.429+0000 7f2c44400640 1 -- 172.21.15.16:0/2984504616 wait complete. 2024-09-17T05:09:13.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 Processor -- start 2024-09-17T05:09:13.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 -- start start 2024-09-17T05:09:13.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c24003a10 0x7f2c3c15db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c3c1f8480 0x7f2c3c152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 0x7f2c3c14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06d8a0 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c06d720 con 0x7f2c3c1f8480 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c44400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c06da20 con 0x7f2c24003a10 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 0x7f2c3c14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 0x7f2c3c14cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42152/0 (socket says 172.21.15.16:42152) 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 -- 172.21.15.16:0/2978064641 learned_addr learned my addr 172.21.15.16:0/2978064641 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c41a00640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c24003a10 0x7f2c3c15db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c41000640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c3c1f8480 0x7f2c3c152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 -- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c24003a10 msgr2=0x7f2c3c15db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c24003a10 0x7f2c3c15db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 -- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c3c1f8480 msgr2=0x7f2c3c152a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c3c1f8480 0x7f2c3c152a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 -- 172.21.15.16:0/2978064641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c1f8a20 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c41000640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c3c1f8480 0x7f2c3c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c42400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 0x7f2c3c14cce0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0cd220 tx=0x7f2c3400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3401d070 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.433+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c34003ba0 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c1f8d00 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34009a90 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c15bc70 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c0b2f50 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2c34020070 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c39600640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073230 0x7f2c1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:13.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f2c340948a0 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.437+0000 7f2c41a00640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073230 0x7f2c1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:13.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.441+0000 7f2c41a00640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073230 0x7f2c1c075740 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ae000 tx=0x7f2c3001a050 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:13.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.441+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2c34027020 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.649+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2c3c152090 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c39600640 1 -- 172.21.15.16:0/2978064641 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f2c3405f7a0 con 0x7f2c3c14c8d0 2024-09-17T05:09:13.653 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:13.653 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073230 msgr2=0x7f2c1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073230 0x7f2c1c075740 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c300ae000 tx=0x7f2c3001a050 comp rx=0 tx=0).stop 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 msgr2=0x7f2c3c14cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 0x7f2c3c14cce0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0cd220 tx=0x7f2c3400ef90 comp rx=0 tx=0).stop 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 shutdown_connections 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2c1c073230 0x7f2c1c075740 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c3c14c8d0 0x7f2c3c14cce0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c3c1f8480 0x7f2c3c152a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 --2- 172.21.15.16:0/2978064641 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2c24003a10 0x7f2c3c15db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 >> 172.21.15.16:0/2978064641 conn(0x7f2c3c06f900 msgr2=0x7f2c3c150960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 shutdown_connections 2024-09-17T05:09:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13.653+0000 7f2c44400640 1 -- 172.21.15.16:0/2978064641 wait complete. 2024-09-17T05:09:13.675 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:09:13.675 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-17T05:09:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:13 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-17T05:09:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-17T05:09:15.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:15.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:15.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 -- 172.21.15.16:0/3205329853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c160aa0 msgr2=0x7fc42c0b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:15.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 --2- 172.21.15.16:0/3205329853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c160aa0 0x7fc42c0b20a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc414009840 tx=0x7fc41401c700 comp rx=0 tx=0).stop 2024-09-17T05:09:15.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 -- 172.21.15.16:0/3205329853 shutdown_connections 2024-09-17T05:09:15.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 --2- 172.21.15.16:0/3205329853 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc42c0b25e0 0x7fc42c0b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 --2- 172.21.15.16:0/3205329853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c160aa0 0x7fc42c0b20a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 -- 172.21.15.16:0/3205329853 >> 172.21.15.16:0/3205329853 conn(0x7fc42c0cf380 msgr2=0x7fc42c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 -- 172.21.15.16:0/3205329853 shutdown_connections 2024-09-17T05:09:15.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.401+0000 7fc433a00640 1 -- 172.21.15.16:0/3205329853 wait complete. 2024-09-17T05:09:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 Processor -- start 2024-09-17T05:09:15.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 -- start start 2024-09-17T05:09:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 0x7fc42c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc42c15c6f0 0x7fc42c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc42c152570 0x7fc42c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c16a170 con 0x7fc42c0b25e0 2024-09-17T05:09:15.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c169ff0 con 0x7fc42c15c6f0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc433a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c16a2f0 con 0x7fc42c152570 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc431000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 0x7fc42c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc431000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 0x7fc42c151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42160/0 (socket says 172.21.15.16:42160) 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.405+0000 7fc431000640 1 -- 172.21.15.16:0/3782513854 learned_addr learned my addr 172.21.15.16:0/3782513854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc42c152570 0x7fc42c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc423e00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc42c15c6f0 0x7fc42c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431000640 1 -- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc42c152570 msgr2=0x7fc42c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431000640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc42c152570 0x7fc42c153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431000640 1 -- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc42c15c6f0 msgr2=0x7fc42c152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431000640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc42c15c6f0 0x7fc42c152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431000640 1 -- 172.21.15.16:0/3782513854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c06dfa0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc431000640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 0x7fc42c151fd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c430 tx=0x7fc41c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c011790 con 0x7fc42c0b25e0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc41c017070 con 0x7fc42c0b25e0 2024-09-17T05:09:15.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c06e410 con 0x7fc42c0b25e0 2024-09-17T05:09:15.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c011ce0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c06e9d0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c0b29f0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.409+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc41c01c460 con 0x7fc42c0b25e0 2024-09-17T05:09:15.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.413+0000 7fc429600640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc408073230 0x7fc408075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:15.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.413+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fc41c094160 con 0x7fc42c0b25e0 2024-09-17T05:09:15.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.413+0000 7fc423e00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc408073230 0x7fc408075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:15.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.413+0000 7fc423e00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc408073230 0x7fc408075740 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc4140af000 tx=0x7fc4140b2040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.417+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc41c05f0b0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.625+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc42c155cf0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.625+0000 7fc429600640 1 -- 172.21.15.16:0/3782513854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5225 (secure 0 0 0) 0x7fc41c05e9b0 con 0x7fc42c0b25e0 2024-09-17T05:09:15.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:15.627 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":88,"btime":"2024-09-17T05:09:11:883212+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:09:11.563756+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 88 2024-09-17T05:09:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc408073230 msgr2=0x7fc408075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc408073230 0x7fc408075740 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc4140af000 tx=0x7fc4140b2040 comp rx=0 tx=0).stop 2024-09-17T05:09:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 msgr2=0x7fc42c151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 0x7fc42c151fd0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c430 tx=0x7fc41c00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 shutdown_connections 2024-09-17T05:09:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc408073230 0x7fc408075740 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc42c152570 0x7fc42c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc42c15c6f0 0x7fc42c152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 --2- 172.21.15.16:0/3782513854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc42c0b25e0 0x7fc42c151fd0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 >> 172.21.15.16:0/3782513854 conn(0x7fc42c0cf380 msgr2=0x7fc42c0b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 shutdown_connections 2024-09-17T05:09:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:15.629+0000 7fc433a00640 1 -- 172.21.15.16:0/3782513854 wait complete. 2024-09-17T05:09:15.650 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-17T05:09:15.829 INFO:teuthology.orchestra.run.smithi104.stdout:[{"id":6647,"entity":{"name":{"type":"client","num":6647},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2038023171}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":11.947839195,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9320568358826513,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6647 v1:192.168.0.1:0/2038023171","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":7869,"entity":{"name":{"type":"client","num":7869},"addr":{"type":"v1","addr":"192.168.0.1:0","nonce":2629263370}},"state":"open","num_leases":1,"num_caps":102,"request_load_avg":204,"uptime":13.143822975999999,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":757.09744672238469,"halflife":60},"release_caps":{"value":99.700090867194831,"halflife":60},"recall_caps_throttle":{"value":232.24065721624882,"halflife":1.5},"recall_caps_throttle2o":{"value":88.917686722408249,"halflife":0.5},"session_cache_liveness":{"value":396.86066930445355,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7869 v1:192.168.0.1:0/2629263370","completed_requests":[{"tid":29,"created_ino":"0x10000000001"},{"tid":30,"created_ino":"0x100000001f6"},{"tid":31,"created_ino":"0x100000001f7"},{"tid":32,"created_ino":"0x100000001f8"},{"tid":33,"created_ino":"0x100000001f9"},{"tid":34,"created_ino":"0x100000001fa"},{"tid":35,"created_ino":"0x100000001fb"},{"tid":36,"created_ino":"0x100000001fc"},{"tid":37,"created_ino":"0x100000001fd"},{"tid":38,"created_ino":"0x100000001fe"},{"tid":39,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":41,"created_ino":"0x10000000201"},{"tid":42,"created_ino":"0x10000000202"},{"tid":43,"created_ino":"0x10000000203"},{"tid":44,"created_ino":"0x10000000204"},{"tid":45,"created_ino":"0x10000000205"},{"tid":46,"created_ino":"0x10000000206"},{"tid":47,"created_ino":"0x10000000207"},{"tid":48,"created_ino":"0x10000000208"},{"tid":49,"created_ino":"0x10000000209"},{"tid":50,"created_ino":"0x1000000020a"},{"tid":51,"created_ino":"0x1000000020b"},{"tid":52,"created_ino":"0x1000000020c"},{"tid":53,"created_ino":"0x1000000020d"},{"tid":54,"created_ino":"0x1000000020e"},{"tid":55,"created_ino":"0x1000000020f"},{"tid":56,"created_ino":"0x10000000210"},{"tid":57,"created_ino":"0x10000000211"},{"tid":58,"created_ino":"0x10000000212"},{"tid":59,"created_ino":"0x10000000213"},{"tid":60,"created_ino":"0x10000000214"},{"tid":61,"created_ino":"0x10000000215"},{"tid":62,"created_ino":"0x10000000216"},{"tid":63,"created_ino":"0x10000000217"},{"tid":64,"created_ino":"0x10000000218"},{"tid":65,"created_ino":"0x10000000219"},{"tid":66,"created_ino":"0x1000000021a"},{"tid":67,"created_ino":"0x1000000021b"},{"tid":68,"created_ino":"0x1000000021c"},{"tid":69,"created_ino":"0x1000000021d"},{"tid":70,"created_ino":"0x1000000021e"},{"tid":71,"created_ino":"0x1000000021f"},{"tid":72,"created_ino":"0x10000000220"},{"tid":73,"created_ino":"0x10000000221"},{"tid":74,"created_ino":"0x10000000222"},{"tid":75,"created_ino":"0x10000000223"},{"tid":76,"created_ino":"0x10000000224"},{"tid":77,"created_ino":"0x10000000225"},{"tid":78,"created_ino":"0x10000000226"},{"tid":79,"created_ino":"0x10000000227"},{"tid":80,"created_ino":"0x10000000228"},{"tid":81,"created_ino":"0x10000000229"},{"tid":82,"created_ino":"0x1000000022a"},{"tid":83,"created_ino":"0x1000000022b"},{"tid":84,"created_ino":"0x1000000022c"},{"tid":85,"created_ino":"0x1000000022d"},{"tid":86,"created_ino":"0x1000000022e"},{"tid":87,"created_ino":"0x1000000022f"},{"tid":88,"created_ino":"0x10000000230"},{"tid":89,"created_ino":"0x10000000231"},{"tid":90,"created_ino":"0x10000000232"},{"tid":91,"created_ino":"0x10000000233"},{"tid":92,"created_ino":"0x10000000234"},{"tid":93,"created_ino":"0x10000000235"},{"tid":94,"created_ino":"0x10000000236"},{"tid":95,"created_ino":"0x10000000237"},{"tid":96,"created_ino":"0x10000000238"},{"tid":97,"created_ino":"0x10000000239"},{"tid":98,"created_ino":"0x1000000023a"},{"tid":99,"created_ino":"0x1000000023b"},{"tid":100,"created_ino":"0x1000000023c"},{"tid":101,"created_ino":"0x1000000023d"},{"tid":102,"created_ino":"0x1000000023e"},{"tid":103,"created_ino":"0x1000000023f"},{"tid":104,"created_ino":"0x10000000240"},{"tid":105,"created_ino":"0x10000000241"},{"tid":106,"created_ino":"0x10000000242"},{"tid":107,"created_ino":"0x10000000243"},{"tid":108,"created_ino":"0x10000000244"},{"tid":109,"created_ino":"0x10000000245"},{"tid":110,"created_ino":"0x10000000246"},{"tid":111,"created_ino":"0x10000000247"},{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:09:15.831 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2038023171, "type": "v1" }, "name": { "num": 6647, "type": "client" } }, "id": 6647, "inst": "client.6647 v1:192.168.0.1:0/2038023171", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9320568358826513 }, "state": "open", "uptime": 11.947839195 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 29 }, { "created_ino": "0x100000001f6", "tid": 30 }, { "created_ino": "0x100000001f7", "tid": 31 }, { "created_ino": "0x100000001f8", "tid": 32 }, { "created_ino": "0x100000001f9", "tid": 33 }, { "created_ino": "0x100000001fa", "tid": 34 }, { "created_ino": "0x100000001fb", "tid": 35 }, { "created_ino": "0x100000001fc", "tid": 36 }, { "created_ino": "0x100000001fd", "tid": 37 }, { "created_ino": "0x100000001fe", "tid": 38 }, { "created_ino": "0x100000001ff", "tid": 39 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 41 }, { "created_ino": "0x10000000202", "tid": 42 }, { "created_ino": "0x10000000203", "tid": 43 }, { "created_ino": "0x10000000204", "tid": 44 }, { "created_ino": "0x10000000205", "tid": 45 }, { "created_ino": "0x10000000206", "tid": 46 }, { "created_ino": "0x10000000207", "tid": 47 }, { "created_ino": "0x10000000208", "tid": 48 }, { "created_ino": "0x10000000209", "tid": 49 }, { "created_ino": "0x1000000020a", "tid": 50 }, { "created_ino": "0x1000000020b", "tid": 51 }, { "created_ino": "0x1000000020c", "tid": 52 }, { "created_ino": "0x1000000020d", "tid": 53 }, { "created_ino": "0x1000000020e", "tid": 54 }, { "created_ino": "0x1000000020f", "tid": 55 }, { "created_ino": "0x10000000210", "tid": 56 }, { "created_ino": "0x10000000211", "tid": 57 }, { "created_ino": "0x10000000212", "tid": 58 }, { "created_ino": "0x10000000213", "tid": 59 }, { "created_ino": "0x10000000214", "tid": 60 }, { "created_ino": "0x10000000215", "tid": 61 }, { "created_ino": "0x10000000216", "tid": 62 }, { "created_ino": "0x10000000217", "tid": 63 }, { "created_ino": "0x10000000218", "tid": 64 }, { "created_ino": "0x10000000219", "tid": 65 }, { "created_ino": "0x1000000021a", "tid": 66 }, { "created_ino": "0x1000000021b", "tid": 67 }, { "created_ino": "0x1000000021c", "tid": 68 }, { "created_ino": "0x1000000021d", "tid": 69 }, { "created_ino": "0x1000000021e", "tid": 70 }, { "created_ino": "0x1000000021f", "tid": 71 }, { "created_ino": "0x10000000220", "tid": 72 }, { "created_ino": "0x10000000221", "tid": 73 }, { "created_ino": "0x10000000222", "tid": 74 }, { "created_ino": "0x10000000223", "tid": 75 }, { "created_ino": "0x10000000224", "tid": 76 }, { "created_ino": "0x10000000225", "tid": 77 }, { "created_ino": "0x10000000226", "tid": 78 }, { "created_ino": "0x10000000227", "tid": 79 }, { "created_ino": "0x10000000228", "tid": 80 }, { "created_ino": "0x10000000229", "tid": 81 }, { "created_ino": "0x1000000022a", "tid": 82 }, { "created_ino": "0x1000000022b", "tid": 83 }, { "created_ino": "0x1000000022c", "tid": 84 }, { "created_ino": "0x1000000022d", "tid": 85 }, { "created_ino": "0x1000000022e", "tid": 86 }, { "created_ino": "0x1000000022f", "tid": 87 }, { "created_ino": "0x10000000230", "tid": 88 }, { "created_ino": "0x10000000231", "tid": 89 }, { "created_ino": "0x10000000232", "tid": 90 }, { "created_ino": "0x10000000233", "tid": 91 }, { "created_ino": "0x10000000234", "tid": 92 }, { "created_ino": "0x10000000235", "tid": 93 }, { "created_ino": "0x10000000236", "tid": 94 }, { "created_ino": "0x10000000237", "tid": 95 }, { "created_ino": "0x10000000238", "tid": 96 }, { "created_ino": "0x10000000239", "tid": 97 }, { "created_ino": "0x1000000023a", "tid": 98 }, { "created_ino": "0x1000000023b", "tid": 99 }, { "created_ino": "0x1000000023c", "tid": 100 }, { "created_ino": "0x1000000023d", "tid": 101 }, { "created_ino": "0x1000000023e", "tid": 102 }, { "created_ino": "0x1000000023f", "tid": 103 }, { "created_ino": "0x10000000240", "tid": 104 }, { "created_ino": "0x10000000241", "tid": 105 }, { "created_ino": "0x10000000242", "tid": 106 }, { "created_ino": "0x10000000243", "tid": 107 }, { "created_ino": "0x10000000244", "tid": 108 }, { "created_ino": "0x10000000245", "tid": 109 }, { "created_ino": "0x10000000246", "tid": 110 }, { "created_ino": "0x10000000247", "tid": 111 }, { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 2629263370, "type": "v1" }, "name": { "num": 7869, "type": "client" } }, "id": 7869, "inst": "client.7869 v1:192.168.0.1:0/2629263370", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 102, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 757.0974467223847 }, "recall_caps_throttle": { "halflife": 1.5, "value": 232.24065721624882 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 88.91768672240825 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 99.70009086719483 }, "request_load_avg": 204, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 396.86066930445355 }, "state": "open", "uptime": 13.143822976 } ] 2024-09-17T05:09:15.832 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:09:15.832 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:09:15.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:09:15.927 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:09:16.019 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:09:16.019 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:16.019 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 200\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-17T05:09:16.020 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:16.020 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:16.029 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:16.029 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:16.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 -- 172.21.15.16:0/1342253446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 msgr2=0x7fd1280d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 --2- 172.21.15.16:0/1342253446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd1280d1b80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd1280ce310 tx=0x7fd11c01c770 comp rx=0 tx=0).stop 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 -- 172.21.15.16:0/1342253446 shutdown_connections 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 --2- 172.21.15.16:0/1342253446 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 0x7fd1280b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 --2- 172.21.15.16:0/1342253446 >> v2:172.21.15.104:3300/0 conn(0x7fd1280d02d0 0x7fd1280cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 --2- 172.21.15.16:0/1342253446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd1280d1b80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 -- 172.21.15.16:0/1342253446 >> 172.21.15.16:0/1342253446 conn(0x7fd1280b42d0 msgr2=0x7fd1280b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 -- 172.21.15.16:0/1342253446 shutdown_connections 2024-09-17T05:09:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.285+0000 7fd130600640 1 -- 172.21.15.16:0/1342253446 wait complete. 2024-09-17T05:09:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 Processor -- start 2024-09-17T05:09:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 -- start start 2024-09-17T05:09:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 0x7fd12806fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1280d02d0 0x7fd1281574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd12815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd128160a80 con 0x7fd128159f70 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd128160900 con 0x7fd1280d02d0 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd130600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd128160c00 con 0x7fd1280cf680 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd12e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd12815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd12dc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 0x7fd12806fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd12e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd12815ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42180/0 (socket says 172.21.15.16:42180) 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd12e600640 1 -- 172.21.15.16:0/4195644306 learned_addr learned my addr 172.21.15.16:0/4195644306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd12dc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 0x7fd12806fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:37712/0 (socket says 172.21.15.16:37712) 2024-09-17T05:09:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.289+0000 7fd12d200640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1280d02d0 0x7fd1281574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd12e600640 1 -- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 msgr2=0x7fd12806fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd12e600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 0x7fd12806fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd12e600640 1 -- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1280d02d0 msgr2=0x7fd1281574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd12e600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1280d02d0 0x7fd1281574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd12e600640 1 -- 172.21.15.16:0/4195644306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1281f8db0 con 0x7fd128159f70 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd12e600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd12815ec50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e990 tx=0x7fd12000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120004d30 con 0x7fd128159f70 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd12000caa0 con 0x7fd128159f70 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1281f9090 con 0x7fd128159f70 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120005020 con 0x7fd128159f70 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd128071d70 con 0x7fd128159f70 2024-09-17T05:09:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1280cf140 con 0x7fd128159f70 2024-09-17T05:09:16.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.293+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1200051c0 con 0x7fd128159f70 2024-09-17T05:09:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.297+0000 7fd125600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd108073200 0x7fd108075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.297+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fd120094bb0 con 0x7fd128159f70 2024-09-17T05:09:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.297+0000 7fd12dc00640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd108073200 0x7fd108075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.297+0000 7fd12dc00640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd108073200 0x7fd108075710 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0af000 tx=0x7fd11c0a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.301+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd120016030 con 0x7fd128159f70 2024-09-17T05:09:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.485+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd128071b00 con 0x7fd128159f70 2024-09-17T05:09:16.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.485+0000 7fd125600640 1 -- 172.21.15.16:0/4195644306 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) ==== 80+0+0 (secure 0 0 0) 0x7fd12005f3d0 con 0x7fd128159f70 2024-09-17T05:09:16.487 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd108073200 msgr2=0x7fd108075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd108073200 0x7fd108075710 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0af000 tx=0x7fd11c0a7040 comp rx=0 tx=0).stop 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 msgr2=0x7fd12815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd12815ec50 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd12000e990 tx=0x7fd12000ee50 comp rx=0 tx=0).stop 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 shutdown_connections 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd108073200 0x7fd108075710 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd128159f70 0x7fd12815ec50 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1280d02d0 0x7fd1281574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 --2- 172.21.15.16:0/4195644306 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1280cf680 0x7fd12806fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 >> 172.21.15.16:0/4195644306 conn(0x7fd1280b42d0 msgr2=0x7fd12815b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:16.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 shutdown_connections 2024-09-17T05:09:16.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.489+0000 7fd130600640 1 -- 172.21.15.16:0/4195644306 wait complete. 2024-09-17T05:09:16.510 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:09:16.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:16.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:16.622 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:09:16.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:16.622 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:09:16.631 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:16.631 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:16.632 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:16.640 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:16.640 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:16.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 -- 172.21.15.16:0/412867551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 msgr2=0x7f617006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 --2- 172.21.15.16:0/412867551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f617006e3b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6168009bc0 tx=0x7f616801cba0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 -- 172.21.15.16:0/412867551 shutdown_connections 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 --2- 172.21.15.16:0/412867551 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f617006e3b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 --2- 172.21.15.16:0/412867551 >> v2:172.21.15.104:3300/0 conn(0x7f61700744d0 0x7f61700cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 -- 172.21.15.16:0/412867551 >> 172.21.15.16:0/412867551 conn(0x7f61700b2dd0 msgr2=0x7f61700b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 -- 172.21.15.16:0/412867551 shutdown_connections 2024-09-17T05:09:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 -- 172.21.15.16:0/412867551 wait complete. 2024-09-17T05:09:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 Processor -- start 2024-09-17T05:09:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 -- start start 2024-09-17T05:09:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.853+0000 7f6174e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f6170141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61700cce00 0x7f61700cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61700b5ec0 0x7f61700cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6170168a60 con 0x7f61700cce00 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61701688e0 con 0x7f61700b5ec0 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6170168be0 con 0x7f617006dfa0 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616de00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f6170141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616de00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f6170141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3301/0 says I am v2:172.21.15.16:37730/0 (socket says 172.21.15.16:37730) 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616de00640 1 -- 172.21.15.16:0/159687153 learned_addr learned my addr 172.21.15.16:0/159687153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616d400640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61700cce00 0x7f61700cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61700b5ec0 0x7f61700cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 -- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 msgr2=0x7f6170141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f6170141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 -- 172.21.15.16:0/159687153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61700cce00 msgr2=0x7f61700cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61700cce00 0x7f61700cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 -- 172.21.15.16:0/159687153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61700b6470 con 0x7f61700b5ec0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616de00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f6170141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f616e800640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61700b5ec0 0x7f61700cc190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f61680ac000 tx=0x7f6168009270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61680bb070 con 0x7f61700b5ec0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6168004d40 con 0x7f61700b5ec0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61680b2040 con 0x7f61700b5ec0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61700b6750 con 0x7f61700b5ec0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701f8f10 con 0x7f61700b5ec0 2024-09-17T05:09:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.857+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61701f9460 con 0x7f61700b5ec0 2024-09-17T05:09:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.861+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6168004220 con 0x7f61700b5ec0 2024-09-17T05:09:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.861+0000 7f6166000640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6150073200 0x7f6150075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.861+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f616812e590 con 0x7f61700b5ec0 2024-09-17T05:09:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.861+0000 7f616de00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6150073200 0x7f6150075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:16.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.861+0000 7f616de00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6150073200 0x7f6150075710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6160007fd0 tx=0x7f6160007f60 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:16.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:16.865+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f61680c4020 con 0x7f61700b5ec0 2024-09-17T05:09:17.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.049+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f61700b40d0 con 0x7f61700b5ec0 2024-09-17T05:09:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6166000640 1 -- 172.21.15.16:0/159687153 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) ==== 80+0+0 (secure 0 0 0) 0x7f6168016790 con 0x7f61700b5ec0 2024-09-17T05:09:17.052 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:09:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6150073200 msgr2=0x7f6150075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6150073200 0x7f6150075710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f6160007fd0 tx=0x7f6160007f60 comp rx=0 tx=0).stop 2024-09-17T05:09:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61700b5ec0 msgr2=0x7f61700cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61700b5ec0 0x7f61700cc190 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f61680ac000 tx=0x7f6168009270 comp rx=0 tx=0).stop 2024-09-17T05:09:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 shutdown_connections 2024-09-17T05:09:17.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6150073200 0x7f6150075710 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61700b5ec0 0x7f61700cc190 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61700cce00 0x7f61700cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 --2- 172.21.15.16:0/159687153 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f617006dfa0 0x7f6170141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 >> 172.21.15.16:0/159687153 conn(0x7f61700b2dd0 msgr2=0x7f6170073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 shutdown_connections 2024-09-17T05:09:17.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.053+0000 7f6174e00640 1 -- 172.21.15.16:0/159687153 wait complete. 2024-09-17T05:09:17.075 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:09:17.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:17.075 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:17.101 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:09:17.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:17.102 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:09:17.153 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:17.153 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:17.153 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:17.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:09:17.163 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:17.163 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:17.163 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:17.208 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:09:17.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:17.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 -- 172.21.15.16:0/4210774072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 msgr2=0x7f740815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 --2- 172.21.15.16:0/4210774072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f740815bb60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f740806e960 tx=0x7f73f801c830 comp rx=0 tx=0).stop 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 -- 172.21.15.16:0/4210774072 shutdown_connections 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 --2- 172.21.15.16:0/4210774072 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f740815f150 0x7f740815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 --2- 172.21.15.16:0/4210774072 >> v2:172.21.15.104:3300/0 conn(0x7f740815c0a0 0x7f740815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 --2- 172.21.15.16:0/4210774072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f740815bb60 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 -- 172.21.15.16:0/4210774072 >> 172.21.15.16:0/4210774072 conn(0x7f740806f900 msgr2=0x7f74080b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 -- 172.21.15.16:0/4210774072 shutdown_connections 2024-09-17T05:09:17.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.365+0000 7f740ca00640 1 -- 172.21.15.16:0/4210774072 wait complete. 2024-09-17T05:09:17.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 Processor -- start 2024-09-17T05:09:17.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 -- start start 2024-09-17T05:09:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f74081fcce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f740815c0a0 0x7f74080cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f740815f150 0x7f74081f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7408073ea0 con 0x7f74080b2f50 2024-09-17T05:09:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7408073d20 con 0x7f740815c0a0 2024-09-17T05:09:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f740ca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7408074020 con 0x7f740815f150 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f74081fcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f74081fcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54264/0 (socket says 172.21.15.16:54264) 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 -- 172.21.15.16:0/2259997829 learned_addr learned my addr 172.21.15.16:0/2259997829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7406800640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f740815f150 0x7f74081f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f73ffe00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f740815c0a0 0x7f74080cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 -- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f740815f150 msgr2=0x7f74081f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f740815f150 0x7f74081f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 -- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f740815c0a0 msgr2=0x7f74080cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f740815c0a0 0x7f74080cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.369+0000 7f7405e00640 1 -- 172.21.15.16:0/2259997829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74081f9790 con 0x7f74080b2f50 2024-09-17T05:09:17.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f7405e00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f74081fcce0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f73f80b0040 tx=0x7f73f80a5830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f80b2040 con 0x7f74080b2f50 2024-09-17T05:09:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73f80044b0 con 0x7f74080b2f50 2024-09-17T05:09:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74081f9a70 con 0x7f74080b2f50 2024-09-17T05:09:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8005290 con 0x7f74080b2f50 2024-09-17T05:09:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081fc4e0 con 0x7f74080b2f50 2024-09-17T05:09:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f740815c4b0 con 0x7f74080b2f50 2024-09-17T05:09:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f73f8005430 con 0x7f74080b2f50 2024-09-17T05:09:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73fe000640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f73e4073230 0x7f73e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f73f812da70 con 0x7f74080b2f50 2024-09-17T05:09:17.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.373+0000 7f73ffe00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f73e4073230 0x7f73e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.377+0000 7f73ffe00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f73e4073230 0x7f73e4075740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f74080b1d20 tx=0x7f73f000b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.377+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f73f80bc060 con 0x7f74080b2f50 2024-09-17T05:09:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74081500c0 con 0x7f74080b2f50 2024-09-17T05:09:17.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f73fe000640 1 -- 172.21.15.16:0/2259997829 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5225 (secure 0 0 0) 0x7f73f80164b0 con 0x7f74080b2f50 2024-09-17T05:09:17.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:17.590 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"btime":"2024-09-17T05:09:15:945563+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:09:15.564490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":8,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f73e4073230 msgr2=0x7f73e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f73e4073230 0x7f73e4075740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f74080b1d20 tx=0x7f73f000b260 comp rx=0 tx=0).stop 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 msgr2=0x7f74081fcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f74081fcce0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f73f80b0040 tx=0x7f73f80a5830 comp rx=0 tx=0).stop 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 shutdown_connections 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f73e4073230 0x7f73e4075740 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f740815f150 0x7f74081f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f740815c0a0 0x7f74080cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 --2- 172.21.15.16:0/2259997829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74080b2f50 0x7f74081fcce0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 >> 172.21.15.16:0/2259997829 conn(0x7f740806f900 msgr2=0x7f7408069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 shutdown_connections 2024-09-17T05:09:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.589+0000 7f740ca00640 1 -- 172.21.15.16:0/2259997829 wait complete. 2024-09-17T05:09:17.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 -- 172.21.15.16:0/2358257123 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc15d600 msgr2=0x7f00dc0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 --2- 172.21.15.16:0/2358257123 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc15d600 0x7f00dc0cee50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009840 tx=0x7f00d001c820 comp rx=0 tx=0).stop 2024-09-17T05:09:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 -- 172.21.15.16:0/2358257123 shutdown_connections 2024-09-17T05:09:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 --2- 172.21.15.16:0/2358257123 >> v2:172.21.15.104:3300/0 conn(0x7f00dc0cf390 0x7f00dc160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 --2- 172.21.15.16:0/2358257123 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc15d600 0x7f00dc0cee50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 -- 172.21.15.16:0/2358257123 >> 172.21.15.16:0/2358257123 conn(0x7f00dc06f900 msgr2=0x7f00dc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 -- 172.21.15.16:0/2358257123 shutdown_connections 2024-09-17T05:09:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.769+0000 7f00e4000640 1 -- 172.21.15.16:0/2358257123 wait complete. 2024-09-17T05:09:17.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 Processor -- start 2024-09-17T05:09:17.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 -- start start 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00dc0cf390 0x7f00dc1651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 0x7f00dc144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc1606b0 0x7f00dc167190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc0b4a30 con 0x7f00dc15d600 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc0b48b0 con 0x7f00dc0cf390 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc16a520 con 0x7f00dc1606b0 2024-09-17T05:09:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 0x7f00dc144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 0x7f00dc144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54274/0 (socket says 172.21.15.16:54274) 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 -- 172.21.15.16:0/3574857173 learned_addr learned my addr 172.21.15.16:0/3574857173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e1600640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00dc0cf390 0x7f00dc1651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e2000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc1606b0 0x7f00dc167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 -- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc1606b0 msgr2=0x7f00dc167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc1606b0 0x7f00dc167190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 -- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00dc0cf390 msgr2=0x7f00dc1651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00dc0cf390 0x7f00dc1651e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 -- 172.21.15.16:0/3574857173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc1f9e00 con 0x7f00dc15d600 2024-09-17T05:09:17.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e0c00640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 0x7f00dc144900 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f00c800ea20 tx=0x7f00c800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:17.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c800cd80 con 0x7f00dc15d600 2024-09-17T05:09:17.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00c8004020 con 0x7f00dc15d600 2024-09-17T05:09:17.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8004450 con 0x7f00dc15d600 2024-09-17T05:09:17.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc0c4ec0 con 0x7f00dc15d600 2024-09-17T05:09:17.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.773+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc0c5160 con 0x7f00dc15d600 2024-09-17T05:09:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.777+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00dc165850 con 0x7f00dc15d600 2024-09-17T05:09:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.777+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f00c80047b0 con 0x7f00dc15d600 2024-09-17T05:09:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.777+0000 7f00d9600640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f00bc073200 0x7f00bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.777+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f00c80949f0 con 0x7f00dc15d600 2024-09-17T05:09:17.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.777+0000 7f00e1600640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f00bc073200 0x7f00bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:17.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.781+0000 7f00e1600640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f00bc073200 0x7f00bc075710 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f00d00af000 tx=0x7f00d001b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:17.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.781+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f00c805f190 con 0x7f00dc15d600 2024-09-17T05:09:17.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.993+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00dc160170 con 0x7f00dc15d600 2024-09-17T05:09:17.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.993+0000 7f00d9600640 1 -- 172.21.15.16:0/3574857173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5225 (secure 0 0 0) 0x7f00c8063bc0 con 0x7f00dc15d600 2024-09-17T05:09:17.995 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:17.995 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"btime":"2024-09-17T05:09:15:945563+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:09:15.564490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":8,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:17.996 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f00bc073200 msgr2=0x7f00bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f00bc073200 0x7f00bc075710 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f00d00af000 tx=0x7f00d001b060 comp rx=0 tx=0).stop 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 msgr2=0x7f00dc144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 0x7f00dc144900 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f00c800ea20 tx=0x7f00c800eee0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 shutdown_connections 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f00bc073200 0x7f00bc075710 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f00dc1606b0 0x7f00dc167190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00dc15d600 0x7f00dc144900 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 --2- 172.21.15.16:0/3574857173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00dc0cf390 0x7f00dc1651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 >> 172.21.15.16:0/3574857173 conn(0x7f00dc06f900 msgr2=0x7f00dc06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:17.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 shutdown_connections 2024-09-17T05:09:17.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:17.997+0000 7f00e4000640 1 -- 172.21.15.16:0/3574857173 wait complete. 2024-09-17T05:09:18.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 -- 172.21.15.16:0/353268385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5698073240 msgr2=0x7f5698073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 --2- 172.21.15.16:0/353268385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5698073240 0x7f5698073650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5690009c80 tx=0x7f569001cd10 comp rx=0 tx=0).stop 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 -- 172.21.15.16:0/353268385 shutdown_connections 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 --2- 172.21.15.16:0/353268385 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5698073240 0x7f5698073650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 --2- 172.21.15.16:0/353268385 >> v2:172.21.15.104:3300/0 conn(0x7f569815c270 0x7f569815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 -- 172.21.15.16:0/353268385 >> 172.21.15.16:0/353268385 conn(0x7f569806f900 msgr2=0x7f56980b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:18.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 -- 172.21.15.16:0/353268385 shutdown_connections 2024-09-17T05:09:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 -- 172.21.15.16:0/353268385 wait complete. 2024-09-17T05:09:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 Processor -- start 2024-09-17T05:09:18.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.177+0000 7f569f000640 1 -- start start 2024-09-17T05:09:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5698073240 0x7f569814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 0x7f56980b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f569815c270 0x7f56981496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5698168d30 con 0x7f56980b2f50 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5698168bb0 con 0x7f5698073240 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5698168eb0 con 0x7f569815c270 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 0x7f56980b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 0x7f56980b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54286/0 (socket says 172.21.15.16:54286) 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 -- 172.21.15.16:0/1245555093 learned_addr learned my addr 172.21.15.16:0/1245555093 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f5697e00640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5698073240 0x7f569814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569d000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f569815c270 0x7f56981496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 -- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f569815c270 msgr2=0x7f56981496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f569815c270 0x7f56981496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 -- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5698073240 msgr2=0x7f569814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5698073240 0x7f569814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 -- 172.21.15.16:0/1245555093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56980cdfe0 con 0x7f56980b2f50 2024-09-17T05:09:18.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f568f400640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 0x7f56980b0ec0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f568000e970 tx=0x7f568000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568000cd80 con 0x7f56980b2f50 2024-09-17T05:09:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5680004020 con 0x7f56980b2f50 2024-09-17T05:09:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56980ce2c0 con 0x7f56980b2f50 2024-09-17T05:09:18.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680004470 con 0x7f56980b2f50 2024-09-17T05:09:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56980ce880 con 0x7f56980b2f50 2024-09-17T05:09:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.181+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5698073650 con 0x7f56980b2f50 2024-09-17T05:09:18.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.185+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5680003600 con 0x7f56980b2f50 2024-09-17T05:09:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.185+0000 7f5694c00640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.185+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f5680094cf0 con 0x7f56980b2f50 2024-09-17T05:09:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.185+0000 7f5697e00640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.185+0000 7f5697e00640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5688007fd0 tx=0x7f5688009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:18.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.189+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f568005f490 con 0x7f56980b2f50 2024-09-17T05:09:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.365+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f569815b3e0 con 0x7f56980b2f50 2024-09-17T05:09:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.365+0000 7f5694c00640 1 -- 172.21.15.16:0/1245555093 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16932 (secure 0 0 0) 0x7f5680063ec0 con 0x7f56980b2f50 2024-09-17T05:09:18.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:18.368 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":125,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:56.498941+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:52.684828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:53.893903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 msgr2=0x7f5674075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5688007fd0 tx=0x7f5688009040 comp rx=0 tx=0).stop 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 msgr2=0x7f56980b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 0x7f56980b0ec0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f568000e970 tx=0x7f568000ee30 comp rx=0 tx=0).stop 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 shutdown_connections 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5674073200 0x7f5674075710 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f569815c270 0x7f56981496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56980b2f50 0x7f56980b0ec0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 --2- 172.21.15.16:0/1245555093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5698073240 0x7f569814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 >> 172.21.15.16:0/1245555093 conn(0x7f569806f900 msgr2=0x7f569815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 shutdown_connections 2024-09-17T05:09:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.369+0000 7f569f000640 1 -- 172.21.15.16:0/1245555093 wait complete. 2024-09-17T05:09:18.391 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:09:18.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 -- 172.21.15.16:0/889914297 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb100b2f50 msgr2=0x7fcb1015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 --2- 172.21.15.16:0/889914297 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb100b2f50 0x7fcb1015bb60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009bc0 tx=0x7fcb0001cb10 comp rx=0 tx=0).stop 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 -- 172.21.15.16:0/889914297 shutdown_connections 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 --2- 172.21.15.16:0/889914297 >> v2:172.21.15.104:3300/0 conn(0x7fcb1015c0a0 0x7fcb1015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 --2- 172.21.15.16:0/889914297 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb100b2f50 0x7fcb1015bb60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 -- 172.21.15.16:0/889914297 >> 172.21.15.16:0/889914297 conn(0x7fcb1006f900 msgr2=0x7fcb100b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 -- 172.21.15.16:0/889914297 shutdown_connections 2024-09-17T05:09:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.549+0000 7fcb15c00640 1 -- 172.21.15.16:0/889914297 wait complete. 2024-09-17T05:09:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 Processor -- start 2024-09-17T05:09:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 -- start start 2024-09-17T05:09:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb100b2f50 0x7fcb1013ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 0x7fcb1013cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb1015f150 0x7fcb101f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10169150 con 0x7fcb1015c0a0 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10168fd0 con 0x7fcb100b2f50 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcb101692d0 con 0x7fcb1015f150 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 0x7fcb1013cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 0x7fcb1013cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54298/0 (socket says 172.21.15.16:54298) 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 -- 172.21.15.16:0/3290325026 learned_addr learned my addr 172.21.15.16:0/3290325026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb0ea00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb100b2f50 0x7fcb1013ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb0f400640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb1015f150 0x7fcb101f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 -- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb1015f150 msgr2=0x7fcb101f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb1015f150 0x7fcb101f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 -- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb100b2f50 msgr2=0x7fcb1013ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb100b2f50 0x7fcb1013ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 -- 172.21.15.16:0/3290325026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb10159280 con 0x7fcb1015c0a0 2024-09-17T05:09:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb06000640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 0x7fcb1013cf60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e710 tx=0x7fcaf800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800cab0 con 0x7fcb1015c0a0 2024-09-17T05:09:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf800cc50 con 0x7fcb1015c0a0 2024-09-17T05:09:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101fa2b0 con 0x7fcb1015c0a0 2024-09-17T05:09:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8005280 con 0x7fcb1015c0a0 2024-09-17T05:09:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.553+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101fa870 con 0x7fcb1015c0a0 2024-09-17T05:09:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.557+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb101fa020 con 0x7fcb1015c0a0 2024-09-17T05:09:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.557+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcaf80047b0 con 0x7fcb1015c0a0 2024-09-17T05:09:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.557+0000 7fcb07400640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcaec073200 0x7fcaec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.557+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fcaf8094e40 con 0x7fcb1015c0a0 2024-09-17T05:09:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.557+0000 7fcb0ea00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcaec073200 0x7fcaec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.561+0000 7fcb0ea00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcaec073200 0x7fcaec075710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcb000af000 tx=0x7fcb0001b350 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:18.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.561+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcaf805f5e0 con 0x7fcb1015c0a0 2024-09-17T05:09:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.773+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb1015bb60 con 0x7fcb1015c0a0 2024-09-17T05:09:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.773+0000 7fcb07400640 1 -- 172.21.15.16:0/3290325026 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5225 (secure 0 0 0) 0x7fcaf8064010 con 0x7fcb1015c0a0 2024-09-17T05:09:18.776 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:18.776 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"btime":"2024-09-17T05:09:15:945563+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:09:15.564490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":8,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:18.776 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2024-09-17T05:09:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcaec073200 msgr2=0x7fcaec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcaec073200 0x7fcaec075710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fcb000af000 tx=0x7fcb0001b350 comp rx=0 tx=0).stop 2024-09-17T05:09:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 msgr2=0x7fcb1013cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 0x7fcb1013cf60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e710 tx=0x7fcaf800ebd0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 shutdown_connections 2024-09-17T05:09:18.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcaec073200 0x7fcaec075710 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb1015f150 0x7fcb101f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb1015c0a0 0x7fcb1013cf60 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 --2- 172.21.15.16:0/3290325026 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb100b2f50 0x7fcb1013ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 >> 172.21.15.16:0/3290325026 conn(0x7fcb1006f900 msgr2=0x7fcb100d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 shutdown_connections 2024-09-17T05:09:18.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.777+0000 7fcb15c00640 1 -- 172.21.15.16:0/3290325026 wait complete. 2024-09-17T05:09:18.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931c200640 1 -- 172.21.15.16:0/2394017638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc004d50 msgr2=0x7f92fc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931c200640 1 --2- 172.21.15.16:0/2394017638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f92fc004d50 0x7f92fc005160 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f930c00c750 tx=0x7f930c01d2c0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931a200640 1 -- 172.21.15.16:0/2394017638 reap_dead start 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931c200640 1 -- 172.21.15.16:0/2394017638 shutdown_connections 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931c200640 1 -- 172.21.15.16:0/2394017638 >> 172.21.15.16:0/2394017638 conn(0x7f931406f900 msgr2=0x7f93140b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931c200640 1 -- 172.21.15.16:0/2394017638 shutdown_connections 2024-09-17T05:09:18.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.957+0000 7f931c200640 1 -- 172.21.15.16:0/2394017638 wait complete. 2024-09-17T05:09:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 Processor -- start 2024-09-17T05:09:18.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 -- start start 2024-09-17T05:09:18.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92fc004d50 0x7f93141f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f931415d570 0x7f9314159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 0x7f9314159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f93140cfde0 con 0x7f9314145e70 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93140cfc60 con 0x7f92fc004d50 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931c200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9314070ea0 con 0x7f931415d570 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 0x7f9314159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 0x7f9314159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54322/0 (socket says 172.21.15.16:54322) 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f931a200640 1 -- 172.21.15.16:0/3557821735 learned_addr learned my addr 172.21.15.16:0/3557821735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f9319800640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92fc004d50 0x7f93141f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.961+0000 7f9318e00640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f931415d570 0x7f9314159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931a200640 1 -- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f931415d570 msgr2=0x7f9314159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931a200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f931415d570 0x7f9314159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931a200640 1 -- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92fc004d50 msgr2=0x7f93141f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931a200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92fc004d50 0x7f93141f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931a200640 1 -- 172.21.15.16:0/3557821735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9314152a30 con 0x7f9314145e70 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931a200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 0x7f9314159cc0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f930c01cd80 tx=0x7f930c0af9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c00c380 con 0x7f9314145e70 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f930c001a50 con 0x7f9314145e70 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9314152d10 con 0x7f9314145e70 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c0afc70 con 0x7f9314145e70 2024-09-17T05:09:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93141532d0 con 0x7f9314145e70 2024-09-17T05:09:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9314153510 con 0x7f9314145e70 2024-09-17T05:09:18.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f930c0ba030 con 0x7f9314145e70 2024-09-17T05:09:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.965+0000 7f9311600640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f92f4073200 0x7f92f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.969+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f930c12eb20 con 0x7f9314145e70 2024-09-17T05:09:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.969+0000 7f9319800640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f92f4073200 0x7f92f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.969+0000 7f9319800640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f92f4073200 0x7f92f4075710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9308009c40 tx=0x7f93080091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:18.973+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9314153510 con 0x7f9314145e70 2024-09-17T05:09:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f931415e700 con 0x7f9314145e70 2024-09-17T05:09:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f9311600640 1 -- 172.21.15.16:0/3557821735 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5225 (secure 0 0 0) 0x7f931415ebf0 con 0x7f9314145e70 2024-09-17T05:09:19.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:19.181 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"btime":"2024-09-17T05:09:15:945563+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:08:54.931567+0000","modified":"2024-09-17T05:09:15.564490+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7737},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7737":{"gid":7737,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":8,"addr":"172.21.15.104:6835/892094550","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":892094550},{"type":"v1","addr":"172.21.15.104:6835","nonce":892094550}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7737,"qdb_cluster":[7737]},"id":9}]} 2024-09-17T05:09:19.181 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 89 2024-09-17T05:09:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f92f4073200 msgr2=0x7f92f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f92f4073200 0x7f92f4075710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9308009c40 tx=0x7f93080091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 msgr2=0x7f9314159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 0x7f9314159cc0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f930c01cd80 tx=0x7f930c0af9c0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 shutdown_connections 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f92f4073200 0x7f92f4075710 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9314145e70 0x7f9314159cc0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f931415d570 0x7f9314159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 --2- 172.21.15.16:0/3557821735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92fc004d50 0x7f93141f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 >> 172.21.15.16:0/3557821735 conn(0x7f931406f900 msgr2=0x7f93140cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 shutdown_connections 2024-09-17T05:09:19.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.181+0000 7f931c200640 1 -- 172.21.15.16:0/3557821735 wait complete. 2024-09-17T05:09:19.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 -- 172.21.15.16:0/398310852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8380b2f50 msgr2=0x7fe83815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 --2- 172.21.15.16:0/398310852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8380b2f50 0x7fe83815bb60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe828009bc0 tx=0x7fe82801cb10 comp rx=0 tx=0).stop 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 -- 172.21.15.16:0/398310852 shutdown_connections 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 --2- 172.21.15.16:0/398310852 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe838073240 0x7fe838073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 --2- 172.21.15.16:0/398310852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8380b2f50 0x7fe83815bb60 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 -- 172.21.15.16:0/398310852 >> 172.21.15.16:0/398310852 conn(0x7fe83806f900 msgr2=0x7fe8380b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 -- 172.21.15.16:0/398310852 shutdown_connections 2024-09-17T05:09:19.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.361+0000 7fe83d400640 1 -- 172.21.15.16:0/398310852 wait complete. 2024-09-17T05:09:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 Processor -- start 2024-09-17T05:09:19.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 -- start start 2024-09-17T05:09:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe838073240 0x7fe8380cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8380b2f50 0x7fe838164710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 0x7fe8381625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe838168cc0 con 0x7fe83815c0a0 2024-09-17T05:09:19.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe838168b40 con 0x7fe8380b2f50 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe838168e40 con 0x7fe838073240 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 0x7fe8381625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 0x7fe8381625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54340/0 (socket says 172.21.15.16:54340) 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 -- 172.21.15.16:0/3943572426 learned_addr learned my addr 172.21.15.16:0/3943572426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe82de00640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8380b2f50 0x7fe838164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe836800640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe838073240 0x7fe8380cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 -- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe838073240 msgr2=0x7fe8380cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe838073240 0x7fe8380cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 -- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8380b2f50 msgr2=0x7fe838164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8380b2f50 0x7fe838164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 -- 172.21.15.16:0/3943572426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe838164c50 con 0x7fe83815c0a0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe837200640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 0x7fe8381625d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe83000e710 tx=0x7fe83000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83000ca50 con 0x7fe83815c0a0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe83000cbf0 con 0x7fe83815c0a0 2024-09-17T05:09:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe838165160 con 0x7fe83815c0a0 2024-09-17T05:09:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe830005020 con 0x7fe83815c0a0 2024-09-17T05:09:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.365+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe838165720 con 0x7fe83815c0a0 2024-09-17T05:09:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.369+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe838073650 con 0x7fe83815c0a0 2024-09-17T05:09:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.369+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe8300051c0 con 0x7fe83815c0a0 2024-09-17T05:09:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.369+0000 7fe82f400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe814073200 0x7fe814075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.369+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7fe830094cc0 con 0x7fe83815c0a0 2024-09-17T05:09:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.369+0000 7fe836800640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe814073200 0x7fe814075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.373+0000 7fe836800640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe814073200 0x7fe814075710 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe8280af000 tx=0x7fe82801b350 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:19.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.373+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe83009b080 con 0x7fe83815c0a0 2024-09-17T05:09:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.553+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8380ccd60 con 0x7fe83815c0a0 2024-09-17T05:09:19.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.553+0000 7fe82f400640 1 -- 172.21.15.16:0/3943572426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16932 (secure 0 0 0) 0x7fe83005f4f0 con 0x7fe83815c0a0 2024-09-17T05:09:19.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:19.555 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":125,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:08:56.498941+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:08:52.684828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-09-17T05:08:53.893903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe814073200 msgr2=0x7fe814075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe814073200 0x7fe814075710 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe8280af000 tx=0x7fe82801b350 comp rx=0 tx=0).stop 2024-09-17T05:09:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 msgr2=0x7fe8381625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 0x7fe8381625d0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe83000e710 tx=0x7fe83000ebd0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 shutdown_connections 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe814073200 0x7fe814075710 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83815c0a0 0x7fe8381625d0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8380b2f50 0x7fe838164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 --2- 172.21.15.16:0/3943572426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe838073240 0x7fe8380cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 >> 172.21.15.16:0/3943572426 conn(0x7fe83806f900 msgr2=0x7fe838151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 shutdown_connections 2024-09-17T05:09:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.557+0000 7fe83d400640 1 -- 172.21.15.16:0/3943572426 wait complete. 2024-09-17T05:09:19.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f252e400640 1 -- 172.21.15.16:0/3910767216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c003fa0 msgr2=0x7f250c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f252e400640 1 --2- 172.21.15.16:0/3910767216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250c003fa0 0x7f250c004810 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f25280d08d0 tx=0x7f251001ce80 comp rx=0 tx=0).stop 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f2527e00640 1 -- 172.21.15.16:0/3910767216 reap_dead start 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f252e400640 1 -- 172.21.15.16:0/3910767216 shutdown_connections 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f252e400640 1 -- 172.21.15.16:0/3910767216 >> 172.21.15.16:0/3910767216 conn(0x7f25280b2250 msgr2=0x7f25280d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f252e400640 1 -- 172.21.15.16:0/3910767216 shutdown_connections 2024-09-17T05:09:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.737+0000 7f252e400640 1 -- 172.21.15.16:0/3910767216 wait complete. 2024-09-17T05:09:19.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 Processor -- start 2024-09-17T05:09:19.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 -- start start 2024-09-17T05:09:19.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f250c003fa0 0x7f2528157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2528158490 0x7f252815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f252815ae30 0x7f25280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25280749d0 con 0x7f2528158490 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2528074850 con 0x7f252815ae30 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2528074b50 con 0x7f250c003fa0 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2528158490 0x7f252815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2528158490 0x7f252815a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54352/0 (socket says 172.21.15.16:54352) 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 -- 172.21.15.16:0/2546417287 learned_addr learned my addr 172.21.15.16:0/2546417287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 -- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f250c003fa0 msgr2=0x7f2528157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 --2- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f250c003fa0 0x7f2528157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 -- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f252815ae30 msgr2=0x7f25280b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 --2- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f252815ae30 0x7f25280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 -- 172.21.15.16:0/2546417287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252815b240 con 0x7f2528158490 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251ea00640 1 --2- 172.21.15.16:0/2546417287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2528158490 0x7f252815a8f0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f251001c940 tx=0x7f2510001ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25100060c0 con 0x7f2528158490 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2510003600 con 0x7f2528158490 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510003a30 con 0x7f2528158490 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 -- 172.21.15.16:0/2546417287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25280b3160 con 0x7f2528158490 2024-09-17T05:09:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.741+0000 7f252e400640 1 -- 172.21.15.16:0/2546417287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25280b3720 con 0x7f2528158490 2024-09-17T05:09:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.745+0000 7f252e400640 1 -- 172.21.15.16:0/2546417287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2528073f30 con 0x7f2528158490 2024-09-17T05:09:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.745+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2510003bd0 con 0x7f2528158490 2024-09-17T05:09:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.745+0000 7f251fe00640 1 --2- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f25040733a0 0x7f25040758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.745+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7f251012ee00 con 0x7f2528158490 2024-09-17T05:09:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.745+0000 7f2527400640 1 --2- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f25040733a0 0x7f25040758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.745+0000 7f2527400640 1 --2- 172.21.15.16:0/2546417287 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f25040733a0 0x7f25040758b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2518011480 tx=0x7f2518003e10 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.749+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2510016c30 con 0x7f2528158490 2024-09-17T05:09:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.961+0000 7f252e400640 1 -- 172.21.15.16:0/2546417287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2528073f30 con 0x7f2528158490 2024-09-17T05:09:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:19.965+0000 7f251fe00640 1 -- 172.21.15.16:0/2546417287 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v89) ==== 316+0+0 (secure 0 0 0) 0x7f25100f9bf0 con 0x7f2528158490 2024-09-17T05:09:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-17T05:09:19.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:09:19.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 -- 172.21.15.16:0/2164359422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe040744d0 msgr2=0x7efe0406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 --2- 172.21.15.16:0/2164359422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe040744d0 0x7efe0406e0d0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009840 tx=0x7efdfc01c820 comp rx=0 tx=0).stop 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 -- 172.21.15.16:0/2164359422 shutdown_connections 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 --2- 172.21.15.16:0/2164359422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe040744d0 0x7efe0406e0d0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 -- 172.21.15.16:0/2164359422 >> 172.21.15.16:0/2164359422 conn(0x7efe040b2dd0 msgr2=0x7efe040b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 -- 172.21.15.16:0/2164359422 shutdown_connections 2024-09-17T05:09:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.133+0000 7efe0a200640 1 -- 172.21.15.16:0/2164359422 wait complete. 2024-09-17T05:09:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 Processor -- start 2024-09-17T05:09:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 -- start start 2024-09-17T05:09:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 0x7efe040c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe0415c490 0x7efe040c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe040cace0 0x7efe041fc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efe0416b270 con 0x7efe0406f5f0 2024-09-17T05:09:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe0416b0f0 con 0x7efe0415c490 2024-09-17T05:09:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7efe0416b3f0 con 0x7efe040cace0 2024-09-17T05:09:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 0x7efe040c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 0x7efe040c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54368/0 (socket says 172.21.15.16:54368) 2024-09-17T05:09:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 -- 172.21.15.16:0/1777453705 learned_addr learned my addr 172.21.15.16:0/1777453705 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efdfaa00640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe0415c490 0x7efe040c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03e00640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe040cace0 0x7efe041fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 -- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe040cace0 msgr2=0x7efe041fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe040cace0 0x7efe041fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 -- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe0415c490 msgr2=0x7efe040c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe0415c490 0x7efe040c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 -- 172.21.15.16:0/1777453705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe041582f0 con 0x7efe0406f5f0 2024-09-17T05:09:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe03400640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 0x7efe040c81c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c450 tx=0x7efdf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4016020 con 0x7efe0406f5f0 2024-09-17T05:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdf4012920 con 0x7efe0406f5f0 2024-09-17T05:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041f8c60 con 0x7efe0406f5f0 2024-09-17T05:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4012d50 con 0x7efe0406f5f0 2024-09-17T05:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.137+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041f9200 con 0x7efe0406f5f0 2024-09-17T05:09:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.141+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe041f9ba0 con 0x7efe0406f5f0 2024-09-17T05:09:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.141+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efdf4019020 con 0x7efe0406f5f0 2024-09-17T05:09:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.141+0000 7efdfbe00640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efde0073200 0x7efde0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.141+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6101+0+0 (secure 0 0 0) 0x7efdf4093e60 con 0x7efe0406f5f0 2024-09-17T05:09:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.141+0000 7efdfaa00640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efde0073200 0x7efde0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.145+0000 7efdfaa00640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efde0073200 0x7efde0075710 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efe041516f0 tx=0x7efdec009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.145+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efe041f9ba0 con 0x7efe0406f5f0 2024-09-17T05:09:20.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.353+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efe0406e0d0 con 0x7efe0406f5f0 2024-09-17T05:09:20.356 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:09:20.357+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:09:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.357+0000 7efdfbe00640 1 -- 172.21.15.16:0/1777453705 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 191+0+0 (secure 0 0 0) 0x7efdf4020070 con 0x7efe0406f5f0 2024-09-17T05:09:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:09:20.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efde0073200 msgr2=0x7efde0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efde0073200 0x7efde0075710 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efe041516f0 tx=0x7efdec009040 comp rx=0 tx=0).stop 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 msgr2=0x7efe040c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 0x7efe040c81c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c450 tx=0x7efdf400c910 comp rx=0 tx=0).stop 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 shutdown_connections 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7efde0073200 0x7efde0075710 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7efe040cace0 0x7efe041fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe0415c490 0x7efe040c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 --2- 172.21.15.16:0/1777453705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe0406f5f0 0x7efe040c81c0 secure :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c450 tx=0x7efdf400c910 comp rx=0 tx=0).stop 2024-09-17T05:09:20.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 >> 172.21.15.16:0/1777453705 conn(0x7efe040b2dd0 msgr2=0x7efe040d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 shutdown_connections 2024-09-17T05:09:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.361+0000 7efe0a200640 1 -- 172.21.15.16:0/1777453705 wait complete. 2024-09-17T05:09:20.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:09:20.497 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 -- 172.21.15.16:0/1766596060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c06e1d0 msgr2=0x7f268c0d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 --2- 172.21.15.16:0/1766596060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c06e1d0 0x7f268c0d0770 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f267c009840 tx=0x7f267c01c790 comp rx=0 tx=0).stop 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 -- 172.21.15.16:0/1766596060 shutdown_connections 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 --2- 172.21.15.16:0/1766596060 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 0x7f268c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 --2- 172.21.15.16:0/1766596060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c06e1d0 0x7f268c0d0770 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 -- 172.21.15.16:0/1766596060 >> 172.21.15.16:0/1766596060 conn(0x7f268c0b2250 msgr2=0x7f268c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 -- 172.21.15.16:0/1766596060 shutdown_connections 2024-09-17T05:09:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.541+0000 7f2693000640 1 -- 172.21.15.16:0/1766596060 wait complete. 2024-09-17T05:09:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 Processor -- start 2024-09-17T05:09:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 -- start start 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c06e1d0 0x7f268c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 0x7f268c1f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 0x7f268c1fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f268c169130 con 0x7f268c15b140 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f268c06e7f0 con 0x7f268c06e1d0 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2693000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f268c1692b0 con 0x7f268c15e780 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 0x7f268c1f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 0x7f268c1f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54380/0 (socket says 172.21.15.16:54380) 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 -- 172.21.15.16:0/1655057964 learned_addr learned my addr 172.21.15.16:0/1655057964 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f268be00640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c06e1d0 0x7f268c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 -- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 msgr2=0x7f268c1fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2691000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 0x7f268c1fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 0x7f268c1fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 -- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c06e1d0 msgr2=0x7f268c0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c06e1d0 0x7f268c0c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2683400640 1 -- 172.21.15.16:0/1655057964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c06dca0 con 0x7f268c15b140 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.545+0000 7f2691000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 0x7f268c1fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2683400640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 0x7f268c1f9f90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f267400d750 tx=0x7f267400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x7f268c15b140 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2674009d20 con 0x7f268c15b140 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c0c72b0 con 0x7f268c15b140 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674005280 con 0x7f268c15b140 2024-09-17T05:09:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c0c6430 con 0x7f268c15b140 2024-09-17T05:09:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c1fad30 con 0x7f268c15b140 2024-09-17T05:09:20.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2674005420 con 0x7f268c15b140 2024-09-17T05:09:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2688c00640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2668073200 0x7f2668075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.549+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6187+0+0 (secure 0 0 0) 0x7f26740945c0 con 0x7f268c15b140 2024-09-17T05:09:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.553+0000 7f268be00640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2668073200 0x7f2668075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.553+0000 7f268be00640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2668073200 0x7f2668075710 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f267c0af000 tx=0x7f267c01afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:20.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.557+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f267400de60 con 0x7f268c15b140 2024-09-17T05:09:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:20.765+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f268c0d0770 con 0x7f268c15b140 2024-09-17T05:09:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2688c00640 1 -- 172.21.15.16:0/1655057964 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7f267405f360 con 0x7f268c15b140 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2668073200 msgr2=0x7f2668075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2668073200 0x7f2668075710 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f267c0af000 tx=0x7f267c01afd0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 msgr2=0x7f268c1f9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 0x7f268c1f9f90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f267400d750 tx=0x7f267400dc10 comp rx=0 tx=0).stop 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 shutdown_connections 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2668073200 0x7f2668075710 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f268c15e780 0x7f268c1fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f268c15b140 0x7f268c1f9f90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 --2- 172.21.15.16:0/1655057964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f268c06e1d0 0x7f268c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 >> 172.21.15.16:0/1655057964 conn(0x7f268c0b2250 msgr2=0x7f268c15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 shutdown_connections 2024-09-17T05:09:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.361+0000 7f2693000640 1 -- 172.21.15.16:0/1655057964 wait complete. 2024-09-17T05:09:21.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:09:21.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 -- 172.21.15.16:0/1145235601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc06e4e0 msgr2=0x7f4fdc15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:21.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 --2- 172.21.15.16:0/1145235601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc06e4e0 0x7f4fdc15efa0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0b2c20 tx=0x7f4fc401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 -- 172.21.15.16:0/1145235601 shutdown_connections 2024-09-17T05:09:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 --2- 172.21.15.16:0/1145235601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc06e4e0 0x7f4fdc15efa0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 --2- 172.21.15.16:0/1145235601 >> v2:172.21.15.104:3300/0 conn(0x7f4fdc0b4620 0x7f4fdc06dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 -- 172.21.15.16:0/1145235601 >> 172.21.15.16:0/1145235601 conn(0x7f4fdc0d0050 msgr2=0x7f4fdc0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 -- 172.21.15.16:0/1145235601 shutdown_connections 2024-09-17T05:09:21.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.553+0000 7f4fe2600640 1 -- 172.21.15.16:0/1145235601 wait complete. 2024-09-17T05:09:21.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 Processor -- start 2024-09-17T05:09:21.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 -- start start 2024-09-17T05:09:21.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fdc06e4e0 0x7f4fdc0cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fdc0b4620 0x7f4fdc0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 0x7f4fdc09c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc074b00 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc074c80 con 0x7f4fdc0b4620 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fe2600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc074e00 con 0x7f4fdc06e4e0 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 0x7f4fdc09c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 0x7f4fdc09c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54402/0 (socket says 172.21.15.16:54402) 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 -- 172.21.15.16:0/2154275994 learned_addr learned my addr 172.21.15.16:0/2154275994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdb400640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fdc06e4e0 0x7f4fdc0cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fd3e00640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fdc0b4620 0x7f4fdc0cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 -- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fdc06e4e0 msgr2=0x7f4fdc0cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fdc06e4e0 0x7f4fdc0cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 -- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fdc0b4620 msgr2=0x7f4fdc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fdc0b4620 0x7f4fdc0cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.557+0000 7f4fdbe00640 1 -- 172.21.15.16:0/2154275994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc0cc7d0 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fdbe00640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 0x7f4fdc09c0a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400cb10 tx=0x7f4fd400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4016020 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fd400ce50 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd400ad80 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc0ccab0 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc0cd070 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.561+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc0ce150 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.565+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4fd4019020 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.565+0000 7f4fd3400640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fb8073200 0x7f4fb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.565+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 6187+0+0 (secure 0 0 0) 0x7f4fd40946b0 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.565+0000 7f4fdb400640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fb8073200 0x7f4fb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.565+0000 7f4fdb400640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fb8073200 0x7f4fb8075710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0058d0 tx=0x7f4fcc005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.569+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4fd4020020 con 0x7f4fdc15f4e0 2024-09-17T05:09:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:21.745+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4fdc15efa0 con 0x7f4fdc15f4e0 2024-09-17T05:09:22.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.365+0000 7f4fd3400640 1 -- 172.21.15.16:0/2154275994 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) ==== 181+0+0 (secure 0 0 0) 0x7f4fdc06e8f0 con 0x7f4fdc15f4e0 2024-09-17T05:09:22.367 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:09:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fb8073200 msgr2=0x7f4fb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fb8073200 0x7f4fb8075710 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc0058d0 tx=0x7f4fcc005860 comp rx=0 tx=0).stop 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 msgr2=0x7f4fdc09c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 0x7f4fdc09c0a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400cb10 tx=0x7f4fd400cc00 comp rx=0 tx=0).stop 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 shutdown_connections 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4fb8073200 0x7f4fb8075710 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fdc15f4e0 0x7f4fdc09c0a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fdc0b4620 0x7f4fdc0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 --2- 172.21.15.16:0/2154275994 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4fdc06e4e0 0x7f4fdc0cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 >> 172.21.15.16:0/2154275994 conn(0x7f4fdc0d0050 msgr2=0x7f4fdc06dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 shutdown_connections 2024-09-17T05:09:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.369+0000 7f4fe2600640 1 -- 172.21.15.16:0/2154275994 wait complete. 2024-09-17T05:09:22.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.561+0000 7f245a800640 1 -- 172.21.15.16:0/746924411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2454153490 msgr2=0x7f24540d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.561+0000 7f245a800640 1 --2- 172.21.15.16:0/746924411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2454153490 0x7f24540d1ae0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2444009900 tx=0x7f244401c8a0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- 172.21.15.16:0/746924411 shutdown_connections 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 --2- 172.21.15.16:0/746924411 >> v2:172.21.15.104:3300/0 conn(0x7f24540b6c40 0x7f245415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 --2- 172.21.15.16:0/746924411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2454153490 0x7f24540d1ae0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- 172.21.15.16:0/746924411 >> 172.21.15.16:0/746924411 conn(0x7f24540d2240 msgr2=0x7f24540d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- 172.21.15.16:0/746924411 shutdown_connections 2024-09-17T05:09:22.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- 172.21.15.16:0/746924411 wait complete. 2024-09-17T05:09:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 Processor -- start 2024-09-17T05:09:22.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- start start 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 0x7f2454150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24540b7050 0x7f245413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2454153490 0x7f2454155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f245416a1d0 con 0x7f24540b6c40 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f245416a050 con 0x7f24540b7050 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.565+0000 7f245a800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f245416a350 con 0x7f2454153490 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 0x7f2454150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 0x7f2454150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54426/0 (socket says 172.21.15.16:54426) 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 -- 172.21.15.16:0/1921805859 learned_addr learned my addr 172.21.15.16:0/1921805859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f244be00640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24540b7050 0x7f245413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453e00640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2454153490 0x7f2454155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 -- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2454153490 msgr2=0x7f2454155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2454153490 0x7f2454155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 -- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24540b7050 msgr2=0x7f245413b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24540b7050 0x7f245413b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 -- 172.21.15.16:0/1921805859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2454156d40 con 0x7f24540b6c40 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f244be00640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24540b7050 0x7f245413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f2453400640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 0x7f2454150580 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2444009850 tx=0x7f24440a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24440b2040 con 0x7f24540b6c40 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2444004430 con 0x7f24540b6c40 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2454142be0 con 0x7f24540b6c40 2024-09-17T05:09:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444003860 con 0x7f24540b6c40 2024-09-17T05:09:22.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541431a0 con 0x7f24540b6c40 2024-09-17T05:09:22.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245415ffd0 con 0x7f24540b6c40 2024-09-17T05:09:22.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.569+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24440bd060 con 0x7f24540b6c40 2024-09-17T05:09:22.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.573+0000 7f244b400640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2430073200 0x7f2430075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:22.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.573+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5749+0+0 (secure 0 0 0) 0x7f244412d750 con 0x7f24540b6c40 2024-09-17T05:09:22.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.573+0000 7f244be00640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2430073200 0x7f2430075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:22.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.573+0000 7f244be00640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2430073200 0x7f2430075710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f245414fe10 tx=0x7f243c005eb0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.577+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2444016490 con 0x7f24540b6c40 2024-09-17T05:09:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:22.749+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2454153210 con 0x7f24540b6c40 2024-09-17T05:09:23.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.369+0000 7f244b400640 1 -- 172.21.15.16:0/1921805859 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) ==== 169+0+0 (secure 0 0 0) 0x7f24440f87f0 con 0x7f24540b6c40 2024-09-17T05:09:23.369 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:09:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2430073200 msgr2=0x7f2430075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2430073200 0x7f2430075710 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f245414fe10 tx=0x7f243c005eb0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 msgr2=0x7f2454150580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 0x7f2454150580 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2444009850 tx=0x7f24440a5630 comp rx=0 tx=0).stop 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 shutdown_connections 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2430073200 0x7f2430075710 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2454153490 0x7f2454155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24540b7050 0x7f245413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 --2- 172.21.15.16:0/1921805859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24540b6c40 0x7f2454150580 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 >> 172.21.15.16:0/1921805859 conn(0x7f24540d2240 msgr2=0x7f245414d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 shutdown_connections 2024-09-17T05:09:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.373+0000 7f245a800640 1 -- 172.21.15.16:0/1921805859 wait complete. 2024-09-17T05:09:23.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:09:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 -- 172.21.15.16:0/1499391090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c167730 msgr2=0x7f225c169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 --2- 172.21.15.16:0/1499391090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c167730 0x7f225c169b50 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f2254009bc0 tx=0x7f225401cae0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 -- 172.21.15.16:0/1499391090 shutdown_connections 2024-09-17T05:09:23.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 --2- 172.21.15.16:0/1499391090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c167730 0x7f225c169b50 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 -- 172.21.15.16:0/1499391090 >> 172.21.15.16:0/1499391090 conn(0x7f225c09d800 msgr2=0x7f225c0b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 -- 172.21.15.16:0/1499391090 shutdown_connections 2024-09-17T05:09:23.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.581+0000 7f2262a00640 1 -- 172.21.15.16:0/1499391090 wait complete. 2024-09-17T05:09:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 Processor -- start 2024-09-17T05:09:23.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 -- start start 2024-09-17T05:09:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c0d2b30 0x7f225c1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 0x7f225c0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f225c167730 0x7f225c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f225c16ce00 con 0x7f225c0d32f0 2024-09-17T05:09:23.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f225c16cc80 con 0x7f225c0d2b30 2024-09-17T05:09:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f225c16cf80 con 0x7f225c167730 2024-09-17T05:09:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 0x7f225c0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 0x7f225c0cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54450/0 (socket says 172.21.15.16:54450) 2024-09-17T05:09:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 -- 172.21.15.16:0/2458012836 learned_addr learned my addr 172.21.15.16:0/2458012836 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225be00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c0d2b30 0x7f225c1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2260a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f225c167730 0x7f225c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 -- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f225c167730 msgr2=0x7f225c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f225c167730 0x7f225c0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 -- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c0d2b30 msgr2=0x7f225c1509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c0d2b30 0x7f225c1509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 -- 172.21.15.16:0/2458012836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c150f00 con 0x7f225c0d32f0 2024-09-17T05:09:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f225b400640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 0x7f225c0cf760 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f224800c390 tx=0x7f224800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248017070 con 0x7f225c0d32f0 2024-09-17T05:09:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22480053c0 con 0x7f225c0d32f0 2024-09-17T05:09:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c1510f0 con 0x7f225c0d32f0 2024-09-17T05:09:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248003840 con 0x7f225c0d32f0 2024-09-17T05:09:23.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.585+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c167ec0 con 0x7f225c0d32f0 2024-09-17T05:09:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.589+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c15e770 con 0x7f225c0d32f0 2024-09-17T05:09:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.589+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f224801a070 con 0x7f225c0d32f0 2024-09-17T05:09:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.589+0000 7f2253e00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f223c073230 0x7f223c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.589+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f22480933f0 con 0x7f225c0d32f0 2024-09-17T05:09:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.589+0000 7f225be00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f223c073230 0x7f223c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.589+0000 7f225be00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f223c073230 0x7f223c075740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f224c007fd0 tx=0x7f224c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:23.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.593+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f224805de90 con 0x7f225c0d32f0 2024-09-17T05:09:23.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.765+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f225c15a630 con 0x7f225c0d32f0 2024-09-17T05:09:23.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2253e00640 1 -- 172.21.15.16:0/2458012836 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v62) ==== 71+0+0 (secure 0 0 0) 0x7f22480628c0 con 0x7f225c0d32f0 2024-09-17T05:09:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f223c073230 msgr2=0x7f223c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f223c073230 0x7f223c075740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f224c007fd0 tx=0x7f224c009040 comp rx=0 tx=0).stop 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 msgr2=0x7f225c0cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 0x7f225c0cf760 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f224800c390 tx=0x7f224800c850 comp rx=0 tx=0).stop 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 shutdown_connections 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f223c073230 0x7f223c075740 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f225c167730 0x7f225c0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c0d32f0 0x7f225c0cf760 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 --2- 172.21.15.16:0/2458012836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c0d2b30 0x7f225c1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 >> 172.21.15.16:0/2458012836 conn(0x7f225c09d800 msgr2=0x7f225c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 shutdown_connections 2024-09-17T05:09:23.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.773+0000 7f2262a00640 1 -- 172.21.15.16:0/2458012836 wait complete. 2024-09-17T05:09:23.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-09-17T05:09:23.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa47ca00640 1 -- 172.21.15.16:0/910108575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c003fe0 msgr2=0x7fa45c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa47ca00640 1 --2- 172.21.15.16:0/910108575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa45c003fe0 0x7fa45c004810 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c260 tx=0x7fa46001cf60 comp rx=0 tx=0).stop 2024-09-17T05:09:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa476800640 1 -- 172.21.15.16:0/910108575 reap_dead start 2024-09-17T05:09:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa47ca00640 1 -- 172.21.15.16:0/910108575 shutdown_connections 2024-09-17T05:09:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa47ca00640 1 -- 172.21.15.16:0/910108575 >> 172.21.15.16:0/910108575 conn(0x7fa47806f900 msgr2=0x7fa4780b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa47ca00640 1 -- 172.21.15.16:0/910108575 shutdown_connections 2024-09-17T05:09:23.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.953+0000 7fa47ca00640 1 -- 172.21.15.16:0/910108575 wait complete. 2024-09-17T05:09:23.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 Processor -- start 2024-09-17T05:09:23.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 -- start start 2024-09-17T05:09:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45c003fe0 0x7fa47813f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 0x7fa478154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa478160280 0x7fa478155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa478166450 con 0x7fa4781f8630 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781662d0 con 0x7fa45c003fe0 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa47ca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa4781665d0 con 0x7fa478160280 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 0x7fa478154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 0x7fa478154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54466/0 (socket says 172.21.15.16:54466) 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 -- 172.21.15.16:0/622195262 learned_addr learned my addr 172.21.15.16:0/622195262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa475e00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45c003fe0 0x7fa47813f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa476800640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa478160280 0x7fa478155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 -- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa478160280 msgr2=0x7fa478155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa478160280 0x7fa478155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 -- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45c003fe0 msgr2=0x7fa47813f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45c003fe0 0x7fa47813f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.957+0000 7fa46d400640 1 -- 172.21.15.16:0/622195262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa478150130 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46d400640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 0x7fa478154e50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa46001b660 tx=0x7fa460001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460003600 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa460003890 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47813fed0 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460003ca0 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781f9080 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4781f8b10 con 0x7fa4781f8630 2024-09-17T05:09:23.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4600bf060 con 0x7fa4781f8630 2024-09-17T05:09:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46ea00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa454073200 0x7fa454075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.961+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7fa46012e4a0 con 0x7fa4781f8630 2024-09-17T05:09:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.965+0000 7fa475e00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa454073200 0x7fa454075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:23.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.965+0000 7fa475e00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa454073200 0x7fa454075710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa468009ce0 tx=0x7fa4680091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:23.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:23.965+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa460016d10 con 0x7fa4781f8630 2024-09-17T05:09:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.205+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7fa47815e4c0 con 0x7fa4781f8630 2024-09-17T05:09:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.373+0000 7fa46ea00640 1 -- 172.21.15.16:0/622195262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v419) ==== 151+0+0 (secure 0 0 0) 0x7fa4600f9770 con 0x7fa4781f8630 2024-09-17T05:09:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa454073200 msgr2=0x7fa454075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa454073200 0x7fa454075710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa468009ce0 tx=0x7fa4680091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 msgr2=0x7fa478154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 0x7fa478154e50 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fa46001b660 tx=0x7fa460001dd0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 shutdown_connections 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa454073200 0x7fa454075710 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa478160280 0x7fa478155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4781f8630 0x7fa478154e50 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 --2- 172.21.15.16:0/622195262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa45c003fe0 0x7fa47813f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 >> 172.21.15.16:0/622195262 conn(0x7fa47806f900 msgr2=0x7fa47815dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 shutdown_connections 2024-09-17T05:09:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.377+0000 7fa47ca00640 1 -- 172.21.15.16:0/622195262 wait complete. 2024-09-17T05:09:24.397 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-17T05:09:24.398 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:09:24.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 -- 172.21.15.16:0/4107406501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d0010 msgr2=0x7f58200d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 --2- 172.21.15.16:0/4107406501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d0010 0x7f58200d0420 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f582006de50 tx=0x7f581801cd80 comp rx=0 tx=0).stop 2024-09-17T05:09:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 -- 172.21.15.16:0/4107406501 shutdown_connections 2024-09-17T05:09:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 --2- 172.21.15.16:0/4107406501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d0010 0x7f58200d0420 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 --2- 172.21.15.16:0/4107406501 >> v2:172.21.15.104:3300/0 conn(0x7f58200d1580 0x7f58200d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 --2- 172.21.15.16:0/4107406501 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58200cc8c0 0x7f582015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 -- 172.21.15.16:0/4107406501 >> 172.21.15.16:0/4107406501 conn(0x7f58200d2270 msgr2=0x7f58200d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 -- 172.21.15.16:0/4107406501 shutdown_connections 2024-09-17T05:09:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 -- 172.21.15.16:0/4107406501 wait complete. 2024-09-17T05:09:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 Processor -- start 2024-09-17T05:09:24.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 -- start start 2024-09-17T05:09:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.557+0000 7f5828600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58200cc8c0 0x7f5820146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58200d0010 0x7f5820157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 0x7f5820158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58200ca1d0 con 0x7f58200d1580 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f58200ca050 con 0x7f58200d0010 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f58200ca350 con 0x7f58200cc8c0 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 0x7f5820158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 0x7f5820158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54476/0 (socket says 172.21.15.16:54476) 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 -- 172.21.15.16:0/1088377359 learned_addr learned my addr 172.21.15.16:0/1088377359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5825200640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58200d0010 0x7f5820157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5825c00640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58200cc8c0 0x7f5820146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 -- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58200cc8c0 msgr2=0x7f5820146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58200cc8c0 0x7f5820146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 -- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58200d0010 msgr2=0x7f5820157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58200d0010 0x7f5820157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 -- 172.21.15.16:0/1088377359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582006fa70 con 0x7f58200d1580 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5826600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 0x7f5820158210 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5818009bd0 tx=0x7f5818005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180bb070 con 0x7f58200d1580 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5818004d40 con 0x7f58200d1580 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582006fd50 con 0x7f58200d1580 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180b2040 con 0x7f58200d1580 2024-09-17T05:09:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5820070310 con 0x7f58200d1580 2024-09-17T05:09:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f582006f7e0 con 0x7f58200d1580 2024-09-17T05:09:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.561+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f58180be070 con 0x7f58200d1580 2024-09-17T05:09:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.565+0000 7f581d600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5800073200 0x7f5800075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.565+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f581812dd80 con 0x7f58200d1580 2024-09-17T05:09:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.565+0000 7f5825c00640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5800073200 0x7f5800075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.565+0000 7f5825c00640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5800073200 0x7f5800075710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f58140058d0 tx=0x7f5814005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:24.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.569+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5818016970 con 0x7f58200d1580 2024-09-17T05:09:24.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.745+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58200d0420 con 0x7f58200d1580 2024-09-17T05:09:24.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.745+0000 7f581d600640 1 -- 172.21.15.16:0/1088377359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+13303 (secure 0 0 0) 0x7f58180f8e70 con 0x7f58200d1580 2024-09-17T05:09:24.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:24.747 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":128,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:09:23.370077+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6835/892094550":"2024-09-18T05:09:20.359978+0000","172.21.15.104:6834/892094550":"2024-09-18T05:09:20.359978+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:24.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5800073200 msgr2=0x7f5800075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5800073200 0x7f5800075710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f58140058d0 tx=0x7f5814005860 comp rx=0 tx=0).stop 2024-09-17T05:09:24.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 msgr2=0x7f5820158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 0x7f5820158210 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5818009bd0 tx=0x7f5818005b00 comp rx=0 tx=0).stop 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 shutdown_connections 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5800073200 0x7f5800075710 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58200d1580 0x7f5820158210 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58200d0010 0x7f5820157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 --2- 172.21.15.16:0/1088377359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58200cc8c0 0x7f5820146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 >> 172.21.15.16:0/1088377359 conn(0x7f58200d2270 msgr2=0x7f58200ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 shutdown_connections 2024-09-17T05:09:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.749+0000 7f5828600640 1 -- 172.21.15.16:0/1088377359 wait complete. 2024-09-17T05:09:24.769 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:09:24.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 -- 172.21.15.16:0/1157317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074e30 msgr2=0x7f11880ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 --2- 172.21.15.16:0/1157317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074e30 0x7f11880ce5d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1188156030 tx=0x7f117401c800 comp rx=0 tx=0).stop 2024-09-17T05:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 -- 172.21.15.16:0/1157317218 shutdown_connections 2024-09-17T05:09:24.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 --2- 172.21.15.16:0/1157317218 >> v2:172.21.15.104:3300/0 conn(0x7f118815f6f0 0x7f11880ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 --2- 172.21.15.16:0/1157317218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074e30 0x7f11880ce5d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 -- 172.21.15.16:0/1157317218 >> 172.21.15.16:0/1157317218 conn(0x7f11880b2250 msgr2=0x7f11880d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.929+0000 7f118ca00640 1 -- 172.21.15.16:0/1157317218 shutdown_connections 2024-09-17T05:09:24.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 -- 172.21.15.16:0/1157317218 wait complete. 2024-09-17T05:09:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 Processor -- start 2024-09-17T05:09:24.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 -- start start 2024-09-17T05:09:24.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 0x7f11880b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1188074e30 0x7f11881fce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f118815f6f0 0x7f118815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1188168e50 con 0x7f1188074a20 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1188168cd0 con 0x7f118815f6f0 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f118ca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1188168fd0 con 0x7f1188074e30 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.933+0000 7f1185e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 0x7f11880b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 0x7f11880b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54492/0 (socket says 172.21.15.16:54492) 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 -- 172.21.15.16:0/3362702390 learned_addr learned my addr 172.21.15.16:0/3362702390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1186800640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f118815f6f0 0x7f118815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185400640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1188074e30 0x7f11881fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 -- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1188074e30 msgr2=0x7f11881fce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1188074e30 0x7f11881fce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 -- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f118815f6f0 msgr2=0x7f118815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f118815f6f0 0x7f118815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 -- 172.21.15.16:0/3362702390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118815b510 con 0x7f1188074a20 2024-09-17T05:09:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1186800640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f118815f6f0 0x7f118815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:24.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185400640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1188074e30 0x7f11881fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f1185e00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 0x7f11880b4170 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f117800c3c0 tx=0x7f117800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178011790 con 0x7f1188074a20 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1178017070 con 0x7f1188074a20 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178011d40 con 0x7f1188074a20 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11880b33b0 con 0x7f1188074a20 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11880b3970 con 0x7f1188074a20 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118815efe0 con 0x7f1188074a20 2024-09-17T05:09:24.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.937+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f117801c460 con 0x7f1188074a20 2024-09-17T05:09:24.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.941+0000 7f117e000640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1168073230 0x7f1168075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.941+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f1178093ed0 con 0x7f1188074a20 2024-09-17T05:09:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.941+0000 7f1185400640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1168073230 0x7f1168075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:24.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.941+0000 7f1185400640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1168073230 0x7f1168075740 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f11740af000 tx=0x7f117401b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:24.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:24.945+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f117805efd0 con 0x7f1188074a20 2024-09-17T05:09:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.121+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1188150530 con 0x7f1188074a20 2024-09-17T05:09:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.121+0000 7f117e000640 1 -- 172.21.15.16:0/3362702390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f117805e8d0 con 0x7f1188074a20 2024-09-17T05:09:25.122 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:09:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.121+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1168073230 msgr2=0x7f1168075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.121+0000 7f118ca00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1168073230 0x7f1168075740 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f11740af000 tx=0x7f117401b040 comp rx=0 tx=0).stop 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.121+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 msgr2=0x7f11880b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.121+0000 7f118ca00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 0x7f11880b4170 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f117800c3c0 tx=0x7f117800c880 comp rx=0 tx=0).stop 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 shutdown_connections 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1168073230 0x7f1168075740 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f118815f6f0 0x7f118815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1188074e30 0x7f11881fce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 --2- 172.21.15.16:0/3362702390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1188074a20 0x7f11880b4170 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 >> 172.21.15.16:0/3362702390 conn(0x7f11880b2250 msgr2=0x7f1188158980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 shutdown_connections 2024-09-17T05:09:25.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.125+0000 7f118ca00640 1 -- 172.21.15.16:0/3362702390 wait complete. 2024-09-17T05:09:25.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-09-17T05:09:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 -- 172.21.15.16:0/648507973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ded0 msgr2=0x7f1b700af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 --2- 172.21.15.16:0/648507973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ded0 0x7f1b700af550 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009840 tx=0x7f1b6401c760 comp rx=0 tx=0).stop 2024-09-17T05:09:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 -- 172.21.15.16:0/648507973 shutdown_connections 2024-09-17T05:09:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 --2- 172.21.15.16:0/648507973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b7006e2e0 0x7f1b700afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 --2- 172.21.15.16:0/648507973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ded0 0x7f1b700af550 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 -- 172.21.15.16:0/648507973 >> 172.21.15.16:0/648507973 conn(0x7f1b70073330 msgr2=0x7f1b700b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 -- 172.21.15.16:0/648507973 shutdown_connections 2024-09-17T05:09:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.301+0000 7f1b76000640 1 -- 172.21.15.16:0/648507973 wait complete. 2024-09-17T05:09:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 Processor -- start 2024-09-17T05:09:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 -- start start 2024-09-17T05:09:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 0x7f1b700cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:25.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b7006ded0 0x7f1b700ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7006e2e0 0x7f1b701fa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70169db0 con 0x7f1b7006ce90 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70169c30 con 0x7f1b7006e2e0 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b76000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1b70169f30 con 0x7f1b7006ded0 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b74c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 0x7f1b700cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b74c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 0x7f1b700cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54510/0 (socket says 172.21.15.16:54510) 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b74c00640 1 -- 172.21.15.16:0/1278774238 learned_addr learned my addr 172.21.15.16:0/1278774238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:25.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b6fe00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b7006ded0 0x7f1b700ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.305+0000 7f1b75600640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7006e2e0 0x7f1b701fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b74c00640 1 -- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b7006ded0 msgr2=0x7f1b700ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b74c00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b7006ded0 0x7f1b700ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b74c00640 1 -- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7006e2e0 msgr2=0x7f1b701fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b74c00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7006e2e0 0x7f1b701fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b74c00640 1 -- 172.21.15.16:0/1278774238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7014fc30 con 0x7f1b7006ce90 2024-09-17T05:09:25.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b74c00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 0x7f1b700cdda0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009790 tx=0x7f1b640a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640bb070 con 0x7f1b7006ce90 2024-09-17T05:09:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b64004470 con 0x7f1b7006ce90 2024-09-17T05:09:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b70150130 con 0x7f1b7006ce90 2024-09-17T05:09:25.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b640b2040 con 0x7f1b7006ce90 2024-09-17T05:09:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701506f0 con 0x7f1b7006ce90 2024-09-17T05:09:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b70150db0 con 0x7f1b7006ce90 2024-09-17T05:09:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1b640be070 con 0x7f1b7006ce90 2024-09-17T05:09:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.309+0000 7f1b6cc00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b50073230 0x7f1b50075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:25.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.313+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f1b6412db30 con 0x7f1b7006ce90 2024-09-17T05:09:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.313+0000 7f1b6fe00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b50073230 0x7f1b50075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:25.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.313+0000 7f1b6fe00640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b50073230 0x7f1b50075740 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1b7015ffe0 tx=0x7f1b5c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:25.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.317+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1b70150db0 con 0x7f1b7006ce90 2024-09-17T05:09:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:25.553+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f1b7006d2a0 con 0x7f1b7006ce90 2024-09-17T05:09:26.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.381+0000 7f1b6cc00640 1 -- 172.21.15.16:0/1278774238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v421) ==== 151+0+0 (secure 0 0 0) 0x7f1b64016410 con 0x7f1b7006ce90 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b50073230 msgr2=0x7f1b50075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b50073230 0x7f1b50075740 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1b7015ffe0 tx=0x7f1b5c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 msgr2=0x7f1b700cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 0x7f1b700cdda0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009790 tx=0x7f1b640a56c0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 shutdown_connections 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b50073230 0x7f1b50075740 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b7006e2e0 0x7f1b701fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b7006ded0 0x7f1b700ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 --2- 172.21.15.16:0/1278774238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7006ce90 0x7f1b700cdda0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 >> 172.21.15.16:0/1278774238 conn(0x7f1b70073330 msgr2=0x7f1b70160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 shutdown_connections 2024-09-17T05:09:26.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.385+0000 7f1b76000640 1 -- 172.21.15.16:0/1278774238 wait complete. 2024-09-17T05:09:26.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:09:26.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 -- 172.21.15.16:0/356086676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf40cf6b0 msgr2=0x7f2cf40b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/356086676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf40cf6b0 0x7f2cf40b7cb0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009c80 tx=0x7f2cec01cda0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 -- 172.21.15.16:0/356086676 shutdown_connections 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/356086676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf40cf6b0 0x7f2cf40b7cb0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/356086676 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf4162df0 0x7f2cf40b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 -- 172.21.15.16:0/356086676 >> 172.21.15.16:0/356086676 conn(0x7f2cf40b4160 msgr2=0x7f2cf40b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 -- 172.21.15.16:0/356086676 shutdown_connections 2024-09-17T05:09:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.573+0000 7f2cfaa00640 1 -- 172.21.15.16:0/356086676 wait complete. 2024-09-17T05:09:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 Processor -- start 2024-09-17T05:09:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 -- start start 2024-09-17T05:09:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf40b78a0 0x7f2cf414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 0x7f2cf414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cf4143190 0x7f2cf4153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf4144a90 con 0x7f2cf4162df0 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf4144910 con 0x7f2cf4143190 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfaa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf4144c10 con 0x7f2cf40b78a0 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 0x7f2cf414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 0x7f2cf414dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54522/0 (socket says 172.21.15.16:54522) 2024-09-17T05:09:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 -- 172.21.15.16:0/2227214917 learned_addr learned my addr 172.21.15.16:0/2227214917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 -- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf40b78a0 msgr2=0x7f2cf414dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf9600640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf40b78a0 0x7f2cf414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfa000640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cf4143190 0x7f2cf4153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf40b78a0 0x7f2cf414dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 -- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cf4143190 msgr2=0x7f2cf4153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cf4143190 0x7f2cf4153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf8c00640 1 -- 172.21.15.16:0/2227214917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf4152de0 con 0x7f2cf4162df0 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cf9600640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf40b78a0 0x7f2cf414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.577+0000 7f2cfa000640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cf4143190 0x7f2cf4153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf8c00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 0x7f2cf414dff0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c3c0 tx=0x7f2ce000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00052d0 con 0x7f2cf4162df0 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ce0007020 con 0x7f2cf4162df0 2024-09-17T05:09:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0007470 con 0x7f2cf4162df0 2024-09-17T05:09:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf40c4d90 con 0x7f2cf4162df0 2024-09-17T05:09:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf4161f10 con 0x7f2cf4162df0 2024-09-17T05:09:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf40b7cb0 con 0x7f2cf4162df0 2024-09-17T05:09:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ce00067e0 con 0x7f2cf4162df0 2024-09-17T05:09:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf1600640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2cd4073230 0x7f2cd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f2ce00934f0 con 0x7f2cf4162df0 2024-09-17T05:09:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.581+0000 7f2cf9600640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2cd4073230 0x7f2cd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.585+0000 7f2cf9600640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2cd4073230 0x7f2cd4075740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009ac0 tx=0x7f2ce80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.585+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ce005e700 con 0x7f2cf4162df0 2024-09-17T05:09:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.761+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f2cf4076230 con 0x7f2cf4162df0 2024-09-17T05:09:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.761+0000 7f2cf1600640 1 -- 172.21.15.16:0/2227214917 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v62) ==== 59+0+2294 (secure 0 0 0) 0x7f2ce0001090 con 0x7f2cf4162df0 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:09:26.764 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:09:26.765 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:09:26.766 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:09:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2cd4073230 msgr2=0x7f2cd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2cd4073230 0x7f2cd4075740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009ac0 tx=0x7f2ce80091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 msgr2=0x7f2cf414dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 0x7f2cf414dff0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c3c0 tx=0x7f2ce000c880 comp rx=0 tx=0).stop 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 shutdown_connections 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2cd4073230 0x7f2cd4075740 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cf4143190 0x7f2cf4153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cf4162df0 0x7f2cf414dff0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 --2- 172.21.15.16:0/2227214917 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2cf40b78a0 0x7f2cf414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 >> 172.21.15.16:0/2227214917 conn(0x7f2cf40b4160 msgr2=0x7f2cf4142a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 shutdown_connections 2024-09-17T05:09:26.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.765+0000 7f2cfaa00640 1 -- 172.21.15.16:0/2227214917 wait complete. 2024-09-17T05:09:26.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:09:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 -- 172.21.15.16:0/3281019329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2015f6f0 msgr2=0x7fae200cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 --2- 172.21.15.16:0/3281019329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2015f6f0 0x7fae200cece0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fae18009bc0 tx=0x7fae1801cba0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 -- 172.21.15.16:0/3281019329 shutdown_connections 2024-09-17T05:09:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 --2- 172.21.15.16:0/3281019329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae2015f6f0 0x7fae200cece0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 -- 172.21.15.16:0/3281019329 >> 172.21.15.16:0/3281019329 conn(0x7fae200b2250 msgr2=0x7fae200d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 -- 172.21.15.16:0/3281019329 shutdown_connections 2024-09-17T05:09:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.945+0000 7fae27000640 1 -- 172.21.15.16:0/3281019329 wait complete. 2024-09-17T05:09:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 Processor -- start 2024-09-17T05:09:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 -- start start 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae20074a20 0x7fae200caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 0x7fae201faa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae2015f6f0 0x7fae201faf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fae2016a100 con 0x7fae20074e30 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae20169f80 con 0x7fae20074a20 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae27000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fae2016a280 con 0x7fae2015f6f0 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.949+0000 7fae17400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 0x7fae201faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 0x7fae201faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54534/0 (socket says 172.21.15.16:54534) 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 -- 172.21.15.16:0/2040884204 learned_addr learned my addr 172.21.15.16:0/2040884204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae25000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae2015f6f0 0x7fae201faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae1fe00640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae20074a20 0x7fae200caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 -- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae2015f6f0 msgr2=0x7fae201faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae2015f6f0 0x7fae201faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 -- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae20074a20 msgr2=0x7fae200caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae20074a20 0x7fae200caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 -- 172.21.15.16:0/2040884204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae200c4a80 con 0x7fae20074e30 2024-09-17T05:09:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae17400640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 0x7fae201faa50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c3f0 tx=0x7fae0800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08017070 con 0x7fae20074e30 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae080053c0 con 0x7fae20074e30 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2014b990 con 0x7fae20074e30 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08003860 con 0x7fae20074e30 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2014c9d0 con 0x7fae20074e30 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2014c7c0 con 0x7fae20074e30 2024-09-17T05:09:26.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.953+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fae0801a070 con 0x7fae20074e30 2024-09-17T05:09:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.957+0000 7fae1cc00640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadfc073200 0x7fadfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.957+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7fae08093480 con 0x7fae20074e30 2024-09-17T05:09:26.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.957+0000 7fae1fe00640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadfc073200 0x7fadfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:26.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.957+0000 7fae1fe00640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadfc073200 0x7fadfc075710 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fae100046c0 tx=0x7fae10007ec0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:26.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:26.961+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fae0805df20 con 0x7fae20074e30 2024-09-17T05:09:27.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.137+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fae2015efe0 con 0x7fae20074e30 2024-09-17T05:09:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.141+0000 7fae1cc00640 1 -- 172.21.15.16:0/2040884204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v63)=0 v63) ==== 122+0+0 (secure 0 0 0) 0x7fae08062950 con 0x7fae20074e30 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadfc073200 msgr2=0x7fadfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadfc073200 0x7fadfc075710 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fae100046c0 tx=0x7fae10007ec0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 msgr2=0x7fae201faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 0x7fae201faa50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fae0800c3f0 tx=0x7fae0800c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 shutdown_connections 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadfc073200 0x7fadfc075710 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fae2015f6f0 0x7fae201faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae20074e30 0x7fae201faa50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 --2- 172.21.15.16:0/2040884204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae20074a20 0x7fae200caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 >> 172.21.15.16:0/2040884204 conn(0x7fae200b2250 msgr2=0x7fae20151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 shutdown_connections 2024-09-17T05:09:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.145+0000 7fae27000640 1 -- 172.21.15.16:0/2040884204 wait complete. 2024-09-17T05:09:27.166 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:09:27.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:09:27.275 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:09:27.367 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:09:27.368 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:27.368 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:27.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:09:27.377 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:27.377 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:27.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:09:27.387 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:27.388 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:27.432 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:09:27.433 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:27.433 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:27.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:09:27.442 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:09:27.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 -- 172.21.15.16:0/806965483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c074b00 msgr2=0x7f4a3c0b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 --2- 172.21.15.16:0/806965483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c074b00 0x7f4a3c0b2f50 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0b2cf0 tx=0x7f4a2c01c7a0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 -- 172.21.15.16:0/806965483 shutdown_connections 2024-09-17T05:09:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 --2- 172.21.15.16:0/806965483 >> v2:172.21.15.104:3300/0 conn(0x7f4a3c073240 0x7f4a3c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 --2- 172.21.15.16:0/806965483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c074b00 0x7f4a3c0b2f50 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 -- 172.21.15.16:0/806965483 >> 172.21.15.16:0/806965483 conn(0x7f4a3c06f900 msgr2=0x7f4a3c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 -- 172.21.15.16:0/806965483 shutdown_connections 2024-09-17T05:09:27.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.633+0000 7f4a42600640 1 -- 172.21.15.16:0/806965483 wait complete. 2024-09-17T05:09:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 Processor -- start 2024-09-17T05:09:27.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 -- start start 2024-09-17T05:09:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 0x7f4a3c1fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a3c074b00 0x7f4a3c1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a3c15f080 0x7f4a3c1fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:27.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c168b90 con 0x7f4a3c073240 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c168a10 con 0x7f4a3c15f080 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c168d10 con 0x7f4a3c074b00 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 0x7f4a3c1fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 0x7f4a3c1fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36130/0 (socket says 172.21.15.16:36130) 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 -- 172.21.15.16:0/3730639596 learned_addr learned my addr 172.21.15.16:0/3730639596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3be00640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a3c15f080 0x7f4a3c1fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3aa00640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a3c074b00 0x7f4a3c1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 -- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a3c074b00 msgr2=0x7f4a3c1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a3c074b00 0x7f4a3c1601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 -- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a3c15f080 msgr2=0x7f4a3c1fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a3c15f080 0x7f4a3c1fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 -- 172.21.15.16:0/3730639596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c160e50 con 0x7f4a3c073240 2024-09-17T05:09:27.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3be00640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a3c15f080 0x7f4a3c1fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:27.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a3b400640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 0x7f4a3c1fc4b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b0040 tx=0x7f4a2c0a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:27.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0b2040 con 0x7f4a3c073240 2024-09-17T05:09:27.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a2c0043b0 con 0x7f4a3c073240 2024-09-17T05:09:27.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c161130 con 0x7f4a3c073240 2024-09-17T05:09:27.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c005290 con 0x7f4a3c073240 2024-09-17T05:09:27.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.637+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1616f0 con 0x7f4a3c073240 2024-09-17T05:09:27.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.641+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c073650 con 0x7f4a3c073240 2024-09-17T05:09:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.641+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a2c0bd060 con 0x7f4a3c073240 2024-09-17T05:09:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.641+0000 7f4a33400640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a1c073200 0x7f4a1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.641+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f4a2c12d620 con 0x7f4a3c073240 2024-09-17T05:09:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.641+0000 7f4a3aa00640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a1c073200 0x7f4a1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:27.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.641+0000 7f4a3aa00640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a1c073200 0x7f4a1c075710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c15b630 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:27.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.645+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4a2c0164b0 con 0x7f4a3c073240 2024-09-17T05:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.853+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a3c0d0210 con 0x7f4a3c073240 2024-09-17T05:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.853+0000 7f4a33400640 1 -- 172.21.15.16:0/3730639596 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3914 (secure 0 0 0) 0x7f4a2c0f8720 con 0x7f4a3c073240 2024-09-17T05:09:27.855 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:27.855 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":91,"btime":"2024-09-17T05:09:21:362203+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2518879067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2518879067},{"type":"v1","addr":"172.21.15.104:6837","nonce":2518879067}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6551,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/587683813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":587683813},{"type":"v1","addr":"172.21.15.16:6834","nonce":587683813}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7312,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1569119240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1569119240},{"type":"v1","addr":"172.21.15.16:6835","nonce":1569119240}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7570,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1914729007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1914729007},{"type":"v1","addr":"172.21.15.104:6835","nonce":1914729007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-09-17T05:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 91 2024-09-17T05:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a1c073200 msgr2=0x7f4a1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a1c073200 0x7f4a1c075710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c15b630 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2024-09-17T05:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 msgr2=0x7f4a3c1fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 0x7f4a3c1fc4b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0b0040 tx=0x7f4a2c0a56a0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 shutdown_connections 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4a1c073200 0x7f4a1c075710 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a3c15f080 0x7f4a3c1fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4a3c074b00 0x7f4a3c1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 --2- 172.21.15.16:0/3730639596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a3c073240 0x7f4a3c1fc4b0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 >> 172.21.15.16:0/3730639596 conn(0x7f4a3c06f900 msgr2=0x7f4a3c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 shutdown_connections 2024-09-17T05:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:27.857+0000 7f4a42600640 1 -- 172.21.15.16:0/3730639596 wait complete. 2024-09-17T05:09:27.878 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:09:27.878 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:09:27.878 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:09:27.878 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:09:27.878 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:09:27.879 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:09:27.879 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:09:27.879 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:09:27.879 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:09:27.879 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:09:27.879 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:09:27.879 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:09:27.880 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:09:27.880 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:09:27.880 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:09:27.880 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:09:27.893 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:09:27.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:09:27.894 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:09:27.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:09:27.896 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:09:27.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:09:27.901 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:09:27.901 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:09:27.943 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:09:27.943 DEBUG:teuthology.parallel:result is None 2024-09-17T05:09:27.944 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:09:27.944 DEBUG:teuthology.parallel:result is None 2024-09-17T05:09:27.951 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:09:27.951 DEBUG:teuthology.parallel:result is None 2024-09-17T05:09:27.954 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:09:27.955 DEBUG:teuthology.parallel:result is None 2024-09-17T05:09:27.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:09:28.043 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:09:28.044 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:09:28.048 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:09:28.053 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 -- 172.21.15.16:0/2575286672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c15c130 msgr2=0x7f386c073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 --2- 172.21.15.16:0/2575286672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c15c130 0x7f386c073bc0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f386c072f60 tx=0x7f385801c800 comp rx=0 tx=0).stop 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 -- 172.21.15.16:0/2575286672 shutdown_connections 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 --2- 172.21.15.16:0/2575286672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c15c130 0x7f386c073bc0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 --2- 172.21.15.16:0/2575286672 >> v2:172.21.15.104:3300/0 conn(0x7f386c158e40 0x7f386c15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 -- 172.21.15.16:0/2575286672 >> 172.21.15.16:0/2575286672 conn(0x7f386c0d1170 msgr2=0x7f386c0734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:28.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 -- 172.21.15.16:0/2575286672 shutdown_connections 2024-09-17T05:09:28.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.121+0000 7f3872600640 1 -- 172.21.15.16:0/2575286672 wait complete. 2024-09-17T05:09:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 Processor -- start 2024-09-17T05:09:28.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 -- start start 2024-09-17T05:09:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 0x7f386c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f386c15c130 0x7f386c1fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f386c15c540 0x7f386c15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f386c169340 con 0x7f386c158e40 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1691c0 con 0x7f386c15c540 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f386c1694c0 con 0x7f386c15c130 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 0x7f386c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 0x7f386c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36164/0 (socket says 172.21.15.16:36164) 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 -- 172.21.15.16:0/16112411 learned_addr learned my addr 172.21.15.16:0/16112411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f386be00640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f386c15c130 0x7f386c1fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871c00640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f386c15c540 0x7f386c15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 -- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f386c15c130 msgr2=0x7f386c1fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f386c15c130 0x7f386c1fcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 -- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f386c15c540 msgr2=0x7f386c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f386c15c540 0x7f386c15e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 -- 172.21.15.16:0/16112411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c1f9670 con 0x7f386c158e40 2024-09-17T05:09:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3871200640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 0x7f386c15d5d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f386000e7c0 tx=0x7f386000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386000ccc0 con 0x7f386c158e40 2024-09-17T05:09:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f386000ce80 con 0x7f386c158e40 2024-09-17T05:09:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1f9950 con 0x7f386c158e40 2024-09-17T05:09:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860005280 con 0x7f386c158e40 2024-09-17T05:09:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.125+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c15de00 con 0x7f386c158e40 2024-09-17T05:09:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.129+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386c156bf0 con 0x7f386c158e40 2024-09-17T05:09:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.129+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3860005420 con 0x7f386c158e40 2024-09-17T05:09:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.129+0000 7f3868c00640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f384c073200 0x7f384c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.129+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5335+0+0 (secure 0 0 0) 0x7f3860094aa0 con 0x7f386c158e40 2024-09-17T05:09:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.129+0000 7f386be00640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f384c073200 0x7f384c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.129+0000 7f386be00640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f384c073200 0x7f384c075710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f38580af000 tx=0x7f385801b040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:28.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.133+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f386005fcb0 con 0x7f386c158e40 2024-09-17T05:09:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.321+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f386c156910 con 0x7f386c158e40 2024-09-17T05:09:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.393+0000 7f3868c00640 1 -- 172.21.15.16:0/16112411 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v130) ==== 97+0+0 (secure 0 0 0) 0x7f386005f5b0 con 0x7f386c158e40 2024-09-17T05:09:28.396 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:09:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f384c073200 msgr2=0x7f384c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f384c073200 0x7f384c075710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f38580af000 tx=0x7f385801b040 comp rx=0 tx=0).stop 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 msgr2=0x7f386c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 0x7f386c15d5d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f386000e7c0 tx=0x7f386000ec80 comp rx=0 tx=0).stop 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 shutdown_connections 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f384c073200 0x7f384c075710 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f386c15c540 0x7f386c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f386c15c130 0x7f386c1fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 --2- 172.21.15.16:0/16112411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f386c158e40 0x7f386c15d5d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 >> 172.21.15.16:0/16112411 conn(0x7f386c0d1170 msgr2=0x7f386c0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 shutdown_connections 2024-09-17T05:09:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.397+0000 7f3872600640 1 -- 172.21.15.16:0/16112411 wait complete. 2024-09-17T05:09:28.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 -- 172.21.15.16:0/3902781680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c15d600 msgr2=0x7f443c0cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 --2- 172.21.15.16:0/3902781680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c15d600 0x7f443c0cee50 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f443c072860 tx=0x7f442c01c880 comp rx=0 tx=0).stop 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 -- 172.21.15.16:0/3902781680 shutdown_connections 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 --2- 172.21.15.16:0/3902781680 >> v2:172.21.15.104:3300/0 conn(0x7f443c160880 0x7f443c167500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 --2- 172.21.15.16:0/3902781680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c15d600 0x7f443c0cee50 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 -- 172.21.15.16:0/3902781680 >> 172.21.15.16:0/3902781680 conn(0x7f443c06f900 msgr2=0x7f443c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 -- 172.21.15.16:0/3902781680 shutdown_connections 2024-09-17T05:09:28.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.577+0000 7f4442600640 1 -- 172.21.15.16:0/3902781680 wait complete. 2024-09-17T05:09:28.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 Processor -- start 2024-09-17T05:09:28.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 -- start start 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 0x7f443c1406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f443c15d600 0x7f443c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f443c160880 0x7f443c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f443c16a260 con 0x7f443c0cf390 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f443c0b5280 con 0x7f443c15d600 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f443c16a3e0 con 0x7f443c160880 2024-09-17T05:09:28.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 0x7f443c1406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 0x7f443c1406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36184/0 (socket says 172.21.15.16:36184) 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 -- 172.21.15.16:0/1516143213 learned_addr learned my addr 172.21.15.16:0/1516143213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4432a00640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f443c15d600 0x7f443c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443be00640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f443c160880 0x7f443c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 -- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f443c160880 msgr2=0x7f443c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f443c160880 0x7f443c15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 -- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f443c15d600 msgr2=0x7f443c15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f443c15d600 0x7f443c15acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 -- 172.21.15.16:0/1516143213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c140c20 con 0x7f443c0cf390 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4432a00640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f443c15d600 0x7f443c15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:28.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f443b400640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 0x7f443c1406e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f442c0b0040 tx=0x7f442c0a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:28.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0b2040 con 0x7f443c0cf390 2024-09-17T05:09:28.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442c0044b0 con 0x7f443c0cf390 2024-09-17T05:09:28.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c003860 con 0x7f443c0cf390 2024-09-17T05:09:28.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c073800 con 0x7f443c0cf390 2024-09-17T05:09:28.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.581+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c073dc0 con 0x7f443c0cf390 2024-09-17T05:09:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.585+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c0cee50 con 0x7f443c0cf390 2024-09-17T05:09:28.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.585+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f442c0bd060 con 0x7f443c0cf390 2024-09-17T05:09:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.585+0000 7f4433e00640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4418073200 0x7f4418075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:28.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.585+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5249+0+0 (secure 0 0 0) 0x7f442c12d700 con 0x7f443c0cf390 2024-09-17T05:09:28.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.585+0000 7f4432a00640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4418073200 0x7f4418075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:28.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.589+0000 7f4432a00640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4418073200 0x7f4418075710 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f443c1fbf80 tx=0x7f4424008040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.589+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f442c016470 con 0x7f443c0cf390 2024-09-17T05:09:28.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f443c0b3530 con 0x7f443c0cf390 2024-09-17T05:09:28.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4433e00640 1 -- 172.21.15.16:0/1516143213 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) ==== 80+0+5304 (secure 0 0 0) 0x7f442c0f8840 con 0x7f443c0cf390 2024-09-17T05:09:28.826 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:28.826 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:09:28.826 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.827 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.829 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:09:28.830 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.831 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.832 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:09:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.837 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:09:28.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:09:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:09:28.842 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:09:28.844 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:09:28.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4418073200 msgr2=0x7f4418075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4418073200 0x7f4418075710 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f443c1fbf80 tx=0x7f4424008040 comp rx=0 tx=0).stop 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 msgr2=0x7f443c1406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 0x7f443c1406e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f442c0b0040 tx=0x7f442c0a5750 comp rx=0 tx=0).stop 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 shutdown_connections 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4418073200 0x7f4418075710 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f443c160880 0x7f443c15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.825+0000 7f4442600640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f443c15d600 0x7f443c15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.829+0000 7f4442600640 1 --2- 172.21.15.16:0/1516143213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443c0cf390 0x7f443c1406e0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.829+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 >> 172.21.15.16:0/1516143213 conn(0x7f443c06f900 msgr2=0x7f443c1454f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.829+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 shutdown_connections 2024-09-17T05:09:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:28.829+0000 7f4442600640 1 -- 172.21.15.16:0/1516143213 wait complete. 2024-09-17T05:09:28.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:09:29.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 -- 172.21.15.16:0/3447786228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 msgr2=0x7ffa70073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 --2- 172.21.15.16:0/3447786228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa70073650 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009bf0 tx=0x7ffa6801ca80 comp rx=0 tx=0).stop 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 -- 172.21.15.16:0/3447786228 shutdown_connections 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 --2- 172.21.15.16:0/3447786228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa70073650 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 --2- 172.21.15.16:0/3447786228 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa7015f790 0x7ffa700b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 -- 172.21.15.16:0/3447786228 >> 172.21.15.16:0/3447786228 conn(0x7ffa7006f900 msgr2=0x7ffa700b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 -- 172.21.15.16:0/3447786228 shutdown_connections 2024-09-17T05:09:29.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.005+0000 7ffa78200640 1 -- 172.21.15.16:0/3447786228 wait complete. 2024-09-17T05:09:29.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 Processor -- start 2024-09-17T05:09:29.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 -- start start 2024-09-17T05:09:29.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa701f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa70074b00 0x7ffa70142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa70142e50 0x7ffa7014e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70168c10 con 0x7ffa70073240 2024-09-17T05:09:29.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70168a90 con 0x7ffa70142e50 2024-09-17T05:09:29.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa78200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ffa70168d90 con 0x7ffa70074b00 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa75800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa701f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.009+0000 7ffa75800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa701f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36210/0 (socket says 172.21.15.16:36210) 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 -- 172.21.15.16:0/812395289 learned_addr learned my addr 172.21.15.16:0/812395289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa74e00640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa70074b00 0x7ffa70142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa76200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa70142e50 0x7ffa7014e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 -- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa70074b00 msgr2=0x7ffa70142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa70074b00 0x7ffa70142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 -- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa70142e50 msgr2=0x7ffa7014e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa70142e50 0x7ffa7014e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 -- 172.21.15.16:0/812395289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa70146c10 con 0x7ffa70073240 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa74e00640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa70074b00 0x7ffa70142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa76200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa70142e50 0x7ffa7014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa75800640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa701f7200 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c400 tx=0x7ffa6400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64017070 con 0x7ffa70073240 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa64003600 con 0x7ffa70073240 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa70146ef0 con 0x7ffa70073240 2024-09-17T05:09:29.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003a50 con 0x7ffa70073240 2024-09-17T05:09:29.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701474b0 con 0x7ffa70073240 2024-09-17T05:09:29.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa70073650 con 0x7ffa70073240 2024-09-17T05:09:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.013+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa6401a070 con 0x7ffa70073240 2024-09-17T05:09:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.017+0000 7ffa6d600640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa50073200 0x7ffa50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.017+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5249+0+0 (secure 0 0 0) 0x7ffa64093390 con 0x7ffa70073240 2024-09-17T05:09:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.017+0000 7ffa74e00640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa50073200 0x7ffa50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.017+0000 7ffa74e00640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa50073200 0x7ffa50075710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffa7014d460 tx=0x7ffa5c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.021+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffa6405e570 con 0x7ffa70073240 2024-09-17T05:09:29.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffa700d0900 con 0x7ffa70073240 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa6d600640 1 -- 172.21.15.16:0/812395289 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) ==== 84+0+65 (secure 0 0 0) 0x7ffa6405de70 con 0x7ffa70073240 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:09:29.198 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:09:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa50073200 msgr2=0x7ffa50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa50073200 0x7ffa50075710 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffa7014d460 tx=0x7ffa5c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 msgr2=0x7ffa701f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa701f7200 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400c400 tx=0x7ffa6400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 shutdown_connections 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa50073200 0x7ffa50075710 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa70142e50 0x7ffa7014e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa70074b00 0x7ffa70142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 --2- 172.21.15.16:0/812395289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa70073240 0x7ffa701f7200 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 >> 172.21.15.16:0/812395289 conn(0x7ffa7006f900 msgr2=0x7ffa7014f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 shutdown_connections 2024-09-17T05:09:29.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.197+0000 7ffa78200640 1 -- 172.21.15.16:0/812395289 wait complete. 2024-09-17T05:09:29.220 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:09:29.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:09:29.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 -- 172.21.15.16:0/268546203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed41606e0 msgr2=0x7f5ed406de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 --2- 172.21.15.16:0/268546203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed41606e0 0x7f5ed406de60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009bc0 tx=0x7f5ecc01cb40 comp rx=0 tx=0).stop 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 -- 172.21.15.16:0/268546203 shutdown_connections 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 --2- 172.21.15.16:0/268546203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed41606e0 0x7f5ed406de60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 --2- 172.21.15.16:0/268546203 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ed4156a90 0x7f5ed41601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 -- 172.21.15.16:0/268546203 >> 172.21.15.16:0/268546203 conn(0x7f5ed40d0050 msgr2=0x7f5ed40d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 -- 172.21.15.16:0/268546203 shutdown_connections 2024-09-17T05:09:29.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.381+0000 7f5edc000640 1 -- 172.21.15.16:0/268546203 wait complete. 2024-09-17T05:09:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 Processor -- start 2024-09-17T05:09:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 -- start start 2024-09-17T05:09:29.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 0x7f5ed41469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed4156a90 0x7f5ed4150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ed41606e0 0x7f5ed41506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40cf270 con 0x7f5ed40cfa10 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed406e9c0 con 0x7f5ed4156a90 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5edc000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed40cf3f0 con 0x7f5ed41606e0 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 0x7f5ed41469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 0x7f5ed41469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36222/0 (socket says 172.21.15.16:36222) 2024-09-17T05:09:29.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 -- 172.21.15.16:0/3144004516 learned_addr learned my addr 172.21.15.16:0/3144004516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5eda000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ed41606e0 0x7f5ed41506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed8c00640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed4156a90 0x7f5ed4150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 -- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ed41606e0 msgr2=0x7f5ed41506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ed41606e0 0x7f5ed41506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 -- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed4156a90 msgr2=0x7f5ed4150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed4156a90 0x7f5ed4150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:29.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 -- 172.21.15.16:0/3144004516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed415d3c0 con 0x7f5ed40cfa10 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.385+0000 7f5ed9600640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 0x7f5ed41469c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c430 tx=0x7f5ec800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011790 con 0x7f5ed40cfa10 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec8017070 con 0x7f5ed40cfa10 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed415d6a0 con 0x7f5ed40cfa10 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011cc0 con 0x7f5ed40cfa10 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed415b9d0 con 0x7f5ed40cfa10 2024-09-17T05:09:29.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed41601a0 con 0x7f5ed40cfa10 2024-09-17T05:09:29.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ec801c460 con 0x7f5ed40cfa10 2024-09-17T05:09:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed1600640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5249+0+0 (secure 0 0 0) 0x7f5ec8093c50 con 0x7f5ed40cfa10 2024-09-17T05:09:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.389+0000 7f5ed8c00640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:29.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.393+0000 7f5ed8c00640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ed409db40 tx=0x7f5ec00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.393+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ec8099020 con 0x7f5ed40cfa10 2024-09-17T05:09:29.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:29.573+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5ed40adfc0 con 0x7f5ed40cfa10 2024-09-17T05:09:30.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.413+0000 7f5ed1600640 1 -- 172.21.15.16:0/3144004516 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) ==== 138+0+0 (secure 0 0 0) 0x7f5ed4074f50 con 0x7f5ed40cfa10 2024-09-17T05:09:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:09:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 msgr2=0x7f5eb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:30.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ed409db40 tx=0x7f5ec00091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 msgr2=0x7f5ed41469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 0x7f5ed41469c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c430 tx=0x7f5ec800c8f0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 shutdown_connections 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ed41606e0 0x7f5ed41506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed4156a90 0x7f5ed4150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 --2- 172.21.15.16:0/3144004516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40cfa10 0x7f5ed41469c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 >> 172.21.15.16:0/3144004516 conn(0x7f5ed40d0050 msgr2=0x7f5ed4155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 shutdown_connections 2024-09-17T05:09:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.417+0000 7f5edc000640 1 -- 172.21.15.16:0/3144004516 wait complete. 2024-09-17T05:09:30.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:09:30.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 -- 172.21.15.16:0/153667456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815c0a0 msgr2=0x7f3a3815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/153667456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815c0a0 0x7f3a3815c4b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3a38071ce0 tx=0x7f3a2401c770 comp rx=0 tx=0).stop 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 -- 172.21.15.16:0/153667456 shutdown_connections 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/153667456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815c0a0 0x7f3a3815c4b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/153667456 >> v2:172.21.15.104:3300/0 conn(0x7f3a380b2f50 0x7f3a3815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 -- 172.21.15.16:0/153667456 >> 172.21.15.16:0/153667456 conn(0x7f3a3806f900 msgr2=0x7f3a380b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 -- 172.21.15.16:0/153667456 shutdown_connections 2024-09-17T05:09:30.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.617+0000 7f3a3ec00640 1 -- 172.21.15.16:0/153667456 wait complete. 2024-09-17T05:09:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 Processor -- start 2024-09-17T05:09:30.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 -- start start 2024-09-17T05:09:30.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a380b2f50 0x7f3a3813c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a3815c0a0 0x7f3a381f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 0x7f3a381fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38169040 con 0x7f3a3815f150 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38168ec0 con 0x7f3a3815c0a0 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3ec00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3a381691c0 con 0x7f3a380b2f50 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 0x7f3a381fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 0x7f3a381fd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36238/0 (socket says 172.21.15.16:36238) 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 -- 172.21.15.16:0/3820162751 learned_addr learned my addr 172.21.15.16:0/3820162751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a37400640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a3815c0a0 0x7f3a381f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a37e00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a380b2f50 0x7f3a3813c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:30.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 -- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a380b2f50 msgr2=0x7f3a3813c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a380b2f50 0x7f3a3813c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 -- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a3815c0a0 msgr2=0x7f3a381f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a3815c0a0 0x7f3a381f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.621+0000 7f3a3cc00640 1 -- 172.21.15.16:0/3820162751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3813c910 con 0x7f3a3815f150 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a3cc00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 0x7f3a381fd000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000cb10 tx=0x7f3a3000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30016020 con 0x7f3a3815f150 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a3000ce50 con 0x7f3a3815f150 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3813efa0 con 0x7f3a3815f150 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000ade0 con 0x7f3a3815f150 2024-09-17T05:09:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381fc800 con 0x7f3a3815f150 2024-09-17T05:09:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3815bb60 con 0x7f3a3815f150 2024-09-17T05:09:30.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a30019020 con 0x7f3a3815f150 2024-09-17T05:09:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.625+0000 7f3a2fe00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a18073200 0x7f3a18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.629+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5619+0+0 (secure 0 0 0) 0x7f3a300943a0 con 0x7f3a3815f150 2024-09-17T05:09:30.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.629+0000 7f3a37e00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a18073200 0x7f3a18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:30.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.629+0000 7f3a37e00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a18073200 0x7f3a18075710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009b40 tx=0x7f3a280091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:30.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.629+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a30020030 con 0x7f3a3815f150 2024-09-17T05:09:30.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:30.805+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3a3815d800 con 0x7f3a3815f150 2024-09-17T05:09:31.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.417+0000 7f3a2fe00640 1 -- 172.21.15.16:0/3820162751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) ==== 170+0+0 (secure 0 0 0) 0x7f3a3005eda0 con 0x7f3a3815f150 2024-09-17T05:09:31.418 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:09:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a18073200 msgr2=0x7f3a18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a18073200 0x7f3a18075710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009b40 tx=0x7f3a280091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 msgr2=0x7f3a381fd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:31.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 0x7f3a381fd000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000cb10 tx=0x7f3a3000cc00 comp rx=0 tx=0).stop 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 shutdown_connections 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3a18073200 0x7f3a18075710 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3815f150 0x7f3a381fd000 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a3815c0a0 0x7f3a381f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 --2- 172.21.15.16:0/3820162751 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3a380b2f50 0x7f3a3813c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 >> 172.21.15.16:0/3820162751 conn(0x7f3a3806f900 msgr2=0x7f3a3815f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 shutdown_connections 2024-09-17T05:09:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.421+0000 7f3a3ec00640 1 -- 172.21.15.16:0/3820162751 wait complete. 2024-09-17T05:09:31.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:09:31.459 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:09:31.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:09:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 -- 172.21.15.16:0/474075432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff69015f6f0 msgr2=0x7ff6900cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 --2- 172.21.15.16:0/474075432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff69015f6f0 0x7ff6900cece0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff688009bf0 tx=0x7ff68801cb00 comp rx=0 tx=0).stop 2024-09-17T05:09:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 -- 172.21.15.16:0/474075432 shutdown_connections 2024-09-17T05:09:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 --2- 172.21.15.16:0/474075432 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff69015f6f0 0x7ff6900cece0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 -- 172.21.15.16:0/474075432 >> 172.21.15.16:0/474075432 conn(0x7ff6900b2250 msgr2=0x7ff6900d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 -- 172.21.15.16:0/474075432 shutdown_connections 2024-09-17T05:09:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.717+0000 7ff698600640 1 -- 172.21.15.16:0/474075432 wait complete. 2024-09-17T05:09:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 Processor -- start 2024-09-17T05:09:31.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 -- start start 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff690074a20 0x7ff690149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff690074e30 0x7ff6901fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 0x7ff6901fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff69016a1b0 con 0x7ff69015f6f0 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff69016a030 con 0x7ff690074a20 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff69016a330 con 0x7ff690074e30 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 0x7ff6901fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:31.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff695200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff690074e30 0x7ff6901fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 0x7ff6901fc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36240/0 (socket says 172.21.15.16:36240) 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 -- 172.21.15.16:0/492742323 learned_addr learned my addr 172.21.15.16:0/492742323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff695c00640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff690074a20 0x7ff690149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 -- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff690074e30 msgr2=0x7ff6901fbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff690074e30 0x7ff6901fbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 -- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff690074a20 msgr2=0x7ff690149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff690074a20 0x7ff690149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 -- 172.21.15.16:0/492742323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6900af060 con 0x7ff69015f6f0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff696600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 0x7ff6901fc510 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff6880b0040 tx=0x7ff688005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6880b2040 con 0x7ff69015f6f0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6880bb070 con 0x7ff69015f6f0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688004040 con 0x7ff69015f6f0 2024-09-17T05:09:31.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6900af340 con 0x7ff69015f6f0 2024-09-17T05:09:31.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.721+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6900af900 con 0x7ff69015f6f0 2024-09-17T05:09:31.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.725+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69015efe0 con 0x7ff69015f6f0 2024-09-17T05:09:31.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.725+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff688004200 con 0x7ff69015f6f0 2024-09-17T05:09:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.725+0000 7ff68d600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff670073200 0x7ff670075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.725+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6001+0+0 (secure 0 0 0) 0x7ff68812f2a0 con 0x7ff69015f6f0 2024-09-17T05:09:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.729+0000 7ff695c00640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff670073200 0x7ff670075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:31.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.729+0000 7ff695c00640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff670073200 0x7ff670075710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff684007fd0 tx=0x7ff684009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.729+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6880166f0 con 0x7ff69015f6f0 2024-09-17T05:09:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:31.957+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff690074590 con 0x7ff69015f6f0 2024-09-17T05:09:32.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.441+0000 7ff68d600640 1 -- 172.21.15.16:0/492742323 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v94) ==== 174+0+0 (secure 0 0 0) 0x7ff6880f9b20 con 0x7ff69015f6f0 2024-09-17T05:09:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 21 and data pool 22 2024-09-17T05:09:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.457+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff670073200 msgr2=0x7ff670075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.457+0000 7ff698600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff670073200 0x7ff670075710 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff684007fd0 tx=0x7ff684009040 comp rx=0 tx=0).stop 2024-09-17T05:09:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.457+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 msgr2=0x7ff6901fc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.457+0000 7ff698600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 0x7ff6901fc510 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7ff6880b0040 tx=0x7ff688005a60 comp rx=0 tx=0).stop 2024-09-17T05:09:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 shutdown_connections 2024-09-17T05:09:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff670073200 0x7ff670075710 secure :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff684007fd0 tx=0x7ff684009040 comp rx=0 tx=0).stop 2024-09-17T05:09:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff69015f6f0 0x7ff6901fc510 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff690074e30 0x7ff6901fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 --2- 172.21.15.16:0/492742323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff690074a20 0x7ff690149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 >> 172.21.15.16:0/492742323 conn(0x7ff6900b2250 msgr2=0x7ff69015cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 shutdown_connections 2024-09-17T05:09:32.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.461+0000 7ff698600640 1 -- 172.21.15.16:0/492742323 wait complete. 2024-09-17T05:09:32.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 -- 172.21.15.16:0/3511608523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090415f580 msgr2=0x7f090415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 --2- 172.21.15.16:0/3511608523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090415f580 0x7f090415f990 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f09040cd260 tx=0x7f090001cd50 comp rx=0 tx=0).stop 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 -- 172.21.15.16:0/3511608523 shutdown_connections 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 --2- 172.21.15.16:0/3511608523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090415f580 0x7f090415f990 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 --2- 172.21.15.16:0/3511608523 >> v2:172.21.15.104:3300/0 conn(0x7f09040cf860 0x7f090415f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 --2- 172.21.15.16:0/3511608523 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090406ce90 0x7f09040cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 -- 172.21.15.16:0/3511608523 >> 172.21.15.16:0/3511608523 conn(0x7f09040d0860 msgr2=0x7f09040d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 -- 172.21.15.16:0/3511608523 shutdown_connections 2024-09-17T05:09:32.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 -- 172.21.15.16:0/3511608523 wait complete. 2024-09-17T05:09:32.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.657+0000 7f090c800640 1 Processor -- start 2024-09-17T05:09:32.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- start start 2024-09-17T05:09:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 0x7f09041fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:32.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09040cf860 0x7f0904150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090415f580 0x7f09040b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f09040cfe70 con 0x7f090406ce90 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09040b5a80 con 0x7f09040cf860 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0904073e20 con 0x7f090415f580 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 0x7f09041fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 0x7f09041fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36250/0 (socket says 172.21.15.16:36250) 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 -- 172.21.15.16:0/3005182556 learned_addr learned my addr 172.21.15.16:0/3005182556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909400640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09040cf860 0x7f0904150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 -- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090415f580 msgr2=0x7f09040b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090a800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090415f580 0x7f09040b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090415f580 0x7f09040b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 -- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09040cf860 msgr2=0x7f0904150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09040cf860 0x7f0904150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 -- 172.21.15.16:0/3005182556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09040af3a0 con 0x7f090406ce90 2024-09-17T05:09:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909400640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09040cf860 0x7f0904150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090a800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090415f580 0x7f09040b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:32.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f0909e00640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 0x7f09041fc5d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f08f800d750 tx=0x7f08f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8017070 con 0x7f090406ce90 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08f8009d20 con 0x7f090406ce90 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f800de60 con 0x7f090406ce90 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090415bb20 con 0x7f090406ce90 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090415c560 con 0x7f090406ce90 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.661+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0904074310 con 0x7f090406ce90 2024-09-17T05:09:32.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.665+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08f8004280 con 0x7f090406ce90 2024-09-17T05:09:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.665+0000 7f08fe000640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f08e8073200 0x7f08e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.665+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 6101+0+0 (secure 0 0 0) 0x7f08f8094820 con 0x7f090406ce90 2024-09-17T05:09:32.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.665+0000 7f0909400640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f08e8073200 0x7f08e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:32.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.665+0000 7f0909400640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f08e8073200 0x7f08e8075710 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0904154450 tx=0x7f08f40091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.669+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08f8020020 con 0x7f090406ce90 2024-09-17T05:09:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.845+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f090415f360 con 0x7f090406ce90 2024-09-17T05:09:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.845+0000 7f08fe000640 1 -- 172.21.15.16:0/3005182556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+17017 (secure 0 0 0) 0x7f08f805f6a0 con 0x7f090406ce90 2024-09-17T05:09:32.847 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:32.847 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":135,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:09:31.961008+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:09:29.587461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-17T05:09:30.820166+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f08e8073200 msgr2=0x7f08e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f08e8073200 0x7f08e8075710 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0904154450 tx=0x7f08f40091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 msgr2=0x7f09041fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 0x7f09041fc5d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f08f800d750 tx=0x7f08f800dc10 comp rx=0 tx=0).stop 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 shutdown_connections 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f08e8073200 0x7f08e8075710 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f090415f580 0x7f09040b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09040cf860 0x7f0904150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 --2- 172.21.15.16:0/3005182556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f090406ce90 0x7f09041fc5d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 >> 172.21.15.16:0/3005182556 conn(0x7f09040d0860 msgr2=0x7f09040b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 shutdown_connections 2024-09-17T05:09:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:32.849+0000 7f090c800640 1 -- 172.21.15.16:0/3005182556 wait complete. 2024-09-17T05:09:32.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 -- 172.21.15.16:0/1505491410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 msgr2=0x7ff4a815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 --2- 172.21.15.16:0/1505491410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a815c4b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8071ce0 tx=0x7ff49401c830 comp rx=0 tx=0).stop 2024-09-17T05:09:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 -- 172.21.15.16:0/1505491410 shutdown_connections 2024-09-17T05:09:33.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 --2- 172.21.15.16:0/1505491410 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4a815f150 0x7ff4a815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 --2- 172.21.15.16:0/1505491410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a815c4b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 --2- 172.21.15.16:0/1505491410 >> v2:172.21.15.104:3300/0 conn(0x7ff4a80b2f50 0x7ff4a815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 -- 172.21.15.16:0/1505491410 >> 172.21.15.16:0/1505491410 conn(0x7ff4a806f900 msgr2=0x7ff4a80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:33.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 -- 172.21.15.16:0/1505491410 shutdown_connections 2024-09-17T05:09:33.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 -- 172.21.15.16:0/1505491410 wait complete. 2024-09-17T05:09:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 Processor -- start 2024-09-17T05:09:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.029+0000 7ff4af800640 1 -- start start 2024-09-17T05:09:33.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4a80b2f50 0x7ff4a8151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a8167a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4a815f150 0x7ff4a815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a806e3d0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a806e250 con 0x7ff4a80b2f50 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a806e550 con 0x7ff4a815f150 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a8167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a8167a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36268/0 (socket says 172.21.15.16:36268) 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 -- 172.21.15.16:0/818777403 learned_addr learned my addr 172.21.15.16:0/818777403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4ace00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4a80b2f50 0x7ff4a8151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4ad800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4a815f150 0x7ff4a815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 -- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4a815f150 msgr2=0x7ff4a815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4a815f150 0x7ff4a815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 -- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4a80b2f50 msgr2=0x7ff4a8151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4a80b2f50 0x7ff4a8151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 -- 172.21.15.16:0/818777403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a8163f00 con 0x7ff4a815c0a0 2024-09-17T05:09:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a7e00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a8167a40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff4940b0040 tx=0x7ff4940a5650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:33.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940b2040 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4940042c0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4940052d0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81605f0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a8160bb0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.033+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a815c4b0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.037+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff494005470 con 0x7ff4a815c0a0 2024-09-17T05:09:33.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.037+0000 7ff4a4c00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff488073200 0x7ff488075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.037+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 6101+0+0 (secure 0 0 0) 0x7ff49412dd20 con 0x7ff4a815c0a0 2024-09-17T05:09:33.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.037+0000 7ff4ace00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff488073200 0x7ff488075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.037+0000 7ff4ace00640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff488073200 0x7ff488075710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00bad0 tx=0x7ff49c00b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:33.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.041+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff4940164b0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.217+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4a8155ca0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.217+0000 7ff4a4c00640 1 -- 172.21.15.16:0/818777403 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+17017 (secure 0 0 0) 0x7ff4940044b0 con 0x7ff4a815c0a0 2024-09-17T05:09:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:33.219 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":135,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:09:31.961008+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:09:29.587461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-17T05:09:30.820166+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff488073200 msgr2=0x7ff488075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff488073200 0x7ff488075710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00bad0 tx=0x7ff49c00b260 comp rx=0 tx=0).stop 2024-09-17T05:09:33.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 msgr2=0x7ff4a8167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a8167a40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff4940b0040 tx=0x7ff4940a5650 comp rx=0 tx=0).stop 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 shutdown_connections 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff488073200 0x7ff488075710 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff4a815f150 0x7ff4a815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a815c0a0 0x7ff4a8167a40 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 --2- 172.21.15.16:0/818777403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4a80b2f50 0x7ff4a8151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 >> 172.21.15.16:0/818777403 conn(0x7ff4a806f900 msgr2=0x7ff4a80ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 shutdown_connections 2024-09-17T05:09:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.221+0000 7ff4af800640 1 -- 172.21.15.16:0/818777403 wait complete. 2024-09-17T05:09:33.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:09:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 -- 172.21.15.16:0/3862573121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5415ee90 msgr2=0x7fbe540ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 --2- 172.21.15.16:0/3862573121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5415ee90 0x7fbe540ca6b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00a950 tx=0x7fbe3c01dae0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 -- 172.21.15.16:0/3862573121 shutdown_connections 2024-09-17T05:09:33.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 --2- 172.21.15.16:0/3862573121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5415ee90 0x7fbe540ca6b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 -- 172.21.15.16:0/3862573121 >> 172.21.15.16:0/3862573121 conn(0x7fbe540b2250 msgr2=0x7fbe540d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 -- 172.21.15.16:0/3862573121 shutdown_connections 2024-09-17T05:09:33.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.401+0000 7fbe59600640 1 -- 172.21.15.16:0/3862573121 wait complete. 2024-09-17T05:09:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 Processor -- start 2024-09-17T05:09:33.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 -- start start 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 0x7fbe5415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbe540cabf0 0x7fbe540c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe54155070 0x7fbe540c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54168f80 con 0x7fbe5406e1d0 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe54168e00 con 0x7fbe54155070 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe59600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbe54169100 con 0x7fbe540cabf0 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 0x7fbe5415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 0x7fbe5415baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36290/0 (socket says 172.21.15.16:36290) 2024-09-17T05:09:33.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.405+0000 7fbe52a00640 1 -- 172.21.15.16:0/2707977384 learned_addr learned my addr 172.21.15.16:0/2707977384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe4a000640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbe540cabf0 0x7fbe540c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe53400640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe54155070 0x7fbe540c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe52a00640 1 -- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbe540cabf0 msgr2=0x7fbe540c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe52a00640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbe540cabf0 0x7fbe540c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe52a00640 1 -- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe54155070 msgr2=0x7fbe540c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe52a00640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe54155070 0x7fbe540c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe52a00640 1 -- 172.21.15.16:0/2707977384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe541522f0 con 0x7fbe5406e1d0 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe4a000640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbe540cabf0 0x7fbe540c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe53400640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe54155070 0x7fbe540c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe52a00640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 0x7fbe5415baf0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c400 tx=0x7fbe4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44011790 con 0x7fbe5406e1d0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe44017070 con 0x7fbe5406e1d0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe541598f0 con 0x7fbe5406e1d0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44011cc0 con 0x7fbe5406e1d0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe54152870 con 0x7fbe5406e1d0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe541593e0 con 0x7fbe5406e1d0 2024-09-17T05:09:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.409+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe4401c460 con 0x7fbe5406e1d0 2024-09-17T05:09:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.413+0000 7fbe4b400640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe30073200 0x7fbe30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.413+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 6101+0+0 (secure 0 0 0) 0x7fbe440941e0 con 0x7fbe5406e1d0 2024-09-17T05:09:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.413+0000 7fbe4a000640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe30073200 0x7fbe30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.413+0000 7fbe4a000640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe30073200 0x7fbe30075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0b0000 tx=0x7fbe3c01c320 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.417+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe4405f070 con 0x7fbe5406e1d0 2024-09-17T05:09:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:33.637+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbe5415dab0 con 0x7fbe5406e1d0 2024-09-17T05:09:34.417 INFO:tasks.ceph.mds.b.smithi104.stderr:2024-09-17T05:09:34.414+0000 7fd35e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:09:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.453+0000 7fbe4b400640 1 -- 172.21.15.16:0/2707977384 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v97) ==== 118+0+0 (secure 0 0 0) 0x7fbe4405e970 con 0x7fbe5406e1d0 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe30073200 msgr2=0x7fbe30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe30073200 0x7fbe30075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0b0000 tx=0x7fbe3c01c320 comp rx=0 tx=0).stop 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 msgr2=0x7fbe5415baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 0x7fbe5415baf0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c400 tx=0x7fbe4400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 shutdown_connections 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbe30073200 0x7fbe30075710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe54155070 0x7fbe540c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbe540cabf0 0x7fbe540c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 --2- 172.21.15.16:0/2707977384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5406e1d0 0x7fbe5415baf0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 >> 172.21.15.16:0/2707977384 conn(0x7fbe540b2250 msgr2=0x7fbe5415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 shutdown_connections 2024-09-17T05:09:34.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.457+0000 7fbe59600640 1 -- 172.21.15.16:0/2707977384 wait complete. 2024-09-17T05:09:34.481 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:09:34.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 -- 172.21.15.16:0/644300078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 msgr2=0x7f1ce400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 --2- 172.21.15.16:0/644300078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1ce400b9d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80017c0 tx=0x7f1cf801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 -- 172.21.15.16:0/644300078 shutdown_connections 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 --2- 172.21.15.16:0/644300078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1ce400b9d0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 -- 172.21.15.16:0/644300078 >> 172.21.15.16:0/644300078 conn(0x7f1d000cd980 msgr2=0x7f1d000cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 -- 172.21.15.16:0/644300078 shutdown_connections 2024-09-17T05:09:34.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 -- 172.21.15.16:0/644300078 wait complete. 2024-09-17T05:09:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 Processor -- start 2024-09-17T05:09:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.649+0000 7f1d06000640 1 -- start start 2024-09-17T05:09:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1d001f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00151b50 0x7f1d00157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d001f5830 0x7f1d001f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d000cb8e0 con 0x7f1ce40051d0 2024-09-17T05:09:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d000cb760 con 0x7f1d00151b50 2024-09-17T05:09:34.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1d000cba60 con 0x7f1d001f5830 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1d001f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1d001f98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36308/0 (socket says 172.21.15.16:36308) 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 -- 172.21.15.16:0/1060520004 learned_addr learned my addr 172.21.15.16:0/1060520004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cffe00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d001f5830 0x7f1d001f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cf6a00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00151b50 0x7f1d00157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 -- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d001f5830 msgr2=0x7f1d001f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d001f5830 0x7f1d001f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 -- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00151b50 msgr2=0x7f1d00157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00151b50 0x7f1d00157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 -- 172.21.15.16:0/1060520004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d000caac0 con 0x7f1ce40051d0 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cffe00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d001f5830 0x7f1d001f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:34.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cf6a00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00151b50 0x7f1d00157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cff400640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1d001f98d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0011d60 tx=0x7f1cf000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf001e070 con 0x7f1ce40051d0 2024-09-17T05:09:34.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cf0011e50 con 0x7f1ce40051d0 2024-09-17T05:09:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf000ea40 con 0x7f1ce40051d0 2024-09-17T05:09:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d000cada0 con 0x7f1ce40051d0 2024-09-17T05:09:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d001f90d0 con 0x7f1ce40051d0 2024-09-17T05:09:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.653+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d000706b0 con 0x7f1ce40051d0 2024-09-17T05:09:34.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.657+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1cf0021070 con 0x7f1ce40051d0 2024-09-17T05:09:34.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.657+0000 7f1cf7e00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1cdc073230 0x7f1cdc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:34.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.657+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f1cf00951e0 con 0x7f1ce40051d0 2024-09-17T05:09:34.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.657+0000 7f1cf6a00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1cdc073230 0x7f1cdc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:34.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.657+0000 7f1cf6a00640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1cdc073230 0x7f1cdc075740 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8001870 tx=0x7f1ce8001120 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:34.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.661+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1cf005fa10 con 0x7f1ce40051d0 2024-09-17T05:09:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:34.869+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1d0006a180 con 0x7f1ce40051d0 2024-09-17T05:09:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.457+0000 7f1cf7e00640 1 -- 172.21.15.16:0/1060520004 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v98) ==== 122+0+0 (secure 0 0 0) 0x7f1cf0064440 con 0x7f1ce40051d0 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1cdc073230 msgr2=0x7f1cdc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1cdc073230 0x7f1cdc075740 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8001870 tx=0x7f1ce8001120 comp rx=0 tx=0).stop 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 msgr2=0x7f1d001f98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1d001f98d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0011d60 tx=0x7f1cf000f790 comp rx=0 tx=0).stop 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 shutdown_connections 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1cdc073230 0x7f1cdc075740 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1d001f5830 0x7f1d001f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d00151b50 0x7f1d00157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 --2- 172.21.15.16:0/1060520004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce40051d0 0x7f1d001f98d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 >> 172.21.15.16:0/1060520004 conn(0x7f1d000cd980 msgr2=0x7f1d00155be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 shutdown_connections 2024-09-17T05:09:35.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.461+0000 7f1d06000640 1 -- 172.21.15.16:0/1060520004 wait complete. 2024-09-17T05:09:35.485 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:09:35.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 -- 172.21.15.16:0/478351550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 msgr2=0x7f44e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 --2- 172.21.15.16:0/478351550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44e0004810 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c2e0 tx=0x7f44e401f6e0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fe400640 1 -- 172.21.15.16:0/478351550 reap_dead start 2024-09-17T05:09:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 -- 172.21.15.16:0/478351550 shutdown_connections 2024-09-17T05:09:35.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 -- 172.21.15.16:0/478351550 >> 172.21.15.16:0/478351550 conn(0x7f44f80b4000 msgr2=0x7f44f80b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 -- 172.21.15.16:0/478351550 shutdown_connections 2024-09-17T05:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 -- 172.21.15.16:0/478351550 wait complete. 2024-09-17T05:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.649+0000 7f44fee00640 1 Processor -- start 2024-09-17T05:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 -- start start 2024-09-17T05:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44f80cbfd0 0x7f44f80ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f8153840 0x7f44f8155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81657a0 con 0x7f44e0004080 2024-09-17T05:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8165620 con 0x7f44f8153840 2024-09-17T05:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f44f8165920 con 0x7f44f80cbfd0 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44f80b4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36324/0 (socket says 172.21.15.16:36324) 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 -- 172.21.15.16:0/2962851690 learned_addr learned my addr 172.21.15.16:0/2962851690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fd000640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44f80cbfd0 0x7f44f80ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fe400640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f8153840 0x7f44f8155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 -- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44f80cbfd0 msgr2=0x7f44f80ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44f80cbfd0 0x7f44f80ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 -- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f8153840 msgr2=0x7f44f8155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f8153840 0x7f44f8155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 -- 172.21.15.16:0/2962851690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f80ca980 con 0x7f44e0004080 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fda00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44f80b4f00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f44ec003f20 tx=0x7f44ec001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec011640 con 0x7f44e0004080 2024-09-17T05:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44ec016020 con 0x7f44e0004080 2024-09-17T05:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f80cac60 con 0x7f44e0004080 2024-09-17T05:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec011a80 con 0x7f44e0004080 2024-09-17T05:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.653+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f8154000 con 0x7f44e0004080 2024-09-17T05:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.657+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f8156770 con 0x7f44e0004080 2024-09-17T05:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.657+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f44ec011c20 con 0x7f44e0004080 2024-09-17T05:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.657+0000 7f44f5600640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f44d8073200 0x7f44d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.657+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f44ec0950d0 con 0x7f44e0004080 2024-09-17T05:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.657+0000 7f44fd000640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f44d8073200 0x7f44d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.657+0000 7f44fd000640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f44d8073200 0x7f44d8075710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f44e40b0000 tx=0x7f44e401c520 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:35.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.661+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f44ec020020 con 0x7f44e0004080 2024-09-17T05:09:35.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.869+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44f80b5440 con 0x7f44e0004080 2024-09-17T05:09:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.869+0000 7f44f5600640 1 -- 172.21.15.16:0/2962851690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5233 (secure 0 0 0) 0x7f44ec05fff0 con 0x7f44e0004080 2024-09-17T05:09:35.871 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:35.871 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"btime":"2024-09-17T05:09:35:459466+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:34.875103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 98 2024-09-17T05:09:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f44d8073200 msgr2=0x7f44d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f44d8073200 0x7f44d8075710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f44e40b0000 tx=0x7f44e401c520 comp rx=0 tx=0).stop 2024-09-17T05:09:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 msgr2=0x7f44f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44f80b4f00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f44ec003f20 tx=0x7f44ec001700 comp rx=0 tx=0).stop 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 shutdown_connections 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f44d8073200 0x7f44d8075710 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44f8153840 0x7f44f8155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f44f80cbfd0 0x7f44f80ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 --2- 172.21.15.16:0/2962851690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44e0004080 0x7f44f80b4f00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 >> 172.21.15.16:0/2962851690 conn(0x7f44f80b4000 msgr2=0x7f44f80d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 shutdown_connections 2024-09-17T05:09:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:35.873+0000 7f44fee00640 1 -- 172.21.15.16:0/2962851690 wait complete. 2024-09-17T05:09:35.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 -- 172.21.15.16:0/2507863389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244073ec0 msgr2=0x7fc2440742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 --2- 172.21.15.16:0/2507863389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244073ec0 0x7fc2440742d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc234009900 tx=0x7fc23401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 -- 172.21.15.16:0/2507863389 shutdown_connections 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 --2- 172.21.15.16:0/2507863389 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc24406ce90 0x7fc24406d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 --2- 172.21.15.16:0/2507863389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244073ec0 0x7fc2440742d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 -- 172.21.15.16:0/2507863389 >> 172.21.15.16:0/2507863389 conn(0x7fc2440d10c0 msgr2=0x7fc2440d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 -- 172.21.15.16:0/2507863389 shutdown_connections 2024-09-17T05:09:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.049+0000 7fc24b800640 1 -- 172.21.15.16:0/2507863389 wait complete. 2024-09-17T05:09:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 Processor -- start 2024-09-17T05:09:36.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 -- start start 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 0x7fc2441f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc244073ec0 0x7fc2440cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc24415ca90 0x7fc2441fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc244169e80 con 0x7fc24406ce90 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc244169d00 con 0x7fc244073ec0 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc24b800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc24416a000 con 0x7fc24415ca90 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc248e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 0x7fc2441f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc248e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 0x7fc2441f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36344/0 (socket says 172.21.15.16:36344) 2024-09-17T05:09:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc248e00640 1 -- 172.21.15.16:0/94600228 learned_addr learned my addr 172.21.15.16:0/94600228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc249800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc24415ca90 0x7fc2441fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc23be00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc244073ec0 0x7fc2440cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.053+0000 7fc248e00640 1 -- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc24415ca90 msgr2=0x7fc2441fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc248e00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc24415ca90 0x7fc2441fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc248e00640 1 -- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc244073ec0 msgr2=0x7fc2440cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc248e00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc244073ec0 0x7fc2440cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc248e00640 1 -- 172.21.15.16:0/94600228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2441fcc60 con 0x7fc24406ce90 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc23be00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc244073ec0 0x7fc2440cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc248e00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 0x7fc2441f9680 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc234009850 tx=0x7fc2340a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340b2040 con 0x7fc24406ce90 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2340043b0 con 0x7fc24406ce90 2024-09-17T05:09:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2441fcf40 con 0x7fc24406ce90 2024-09-17T05:09:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234005280 con 0x7fc24406ce90 2024-09-17T05:09:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2441f8e80 con 0x7fc24406ce90 2024-09-17T05:09:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24406d2a0 con 0x7fc24406ce90 2024-09-17T05:09:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.057+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc234005420 con 0x7fc24406ce90 2024-09-17T05:09:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.061+0000 7fc241600640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc220073200 0x7fc220075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.061+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fc23412e650 con 0x7fc24406ce90 2024-09-17T05:09:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.061+0000 7fc23be00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc220073200 0x7fc220075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.061+0000 7fc23be00640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc220073200 0x7fc220075710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc24415f710 tx=0x7fc22c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.065+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc2340164d0 con 0x7fc24406ce90 2024-09-17T05:09:36.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.241+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc24415d2b0 con 0x7fc24406ce90 2024-09-17T05:09:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.241+0000 7fc241600640 1 -- 172.21.15.16:0/94600228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+17004 (secure 0 0 0) 0x7fc2340f94f0 con 0x7fc24406ce90 2024-09-17T05:09:36.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:36.243 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":136,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:09:33.443596+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:09:29.587461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-17T05:09:30.820166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc220073200 msgr2=0x7fc220075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc220073200 0x7fc220075710 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc24415f710 tx=0x7fc22c009040 comp rx=0 tx=0).stop 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 msgr2=0x7fc2441f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 0x7fc2441f9680 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fc234009850 tx=0x7fc2340a5620 comp rx=0 tx=0).stop 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 shutdown_connections 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc220073200 0x7fc220075710 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc24415ca90 0x7fc2441fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc244073ec0 0x7fc2440cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 --2- 172.21.15.16:0/94600228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc24406ce90 0x7fc2441f9680 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 >> 172.21.15.16:0/94600228 conn(0x7fc2440d10c0 msgr2=0x7fc24406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 shutdown_connections 2024-09-17T05:09:36.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.245+0000 7fc24b800640 1 -- 172.21.15.16:0/94600228 wait complete. 2024-09-17T05:09:36.265 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:09:36.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:09:36.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 -- 172.21.15.16:0/150686497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b9c0053b0 msgr2=0x7f4b9c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 --2- 172.21.15.16:0/150686497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b9c0053b0 0x7f4b9c00ba60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4bb81561f0 tx=0x7f4bb001c750 comp rx=0 tx=0).stop 2024-09-17T05:09:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 -- 172.21.15.16:0/150686497 shutdown_connections 2024-09-17T05:09:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 --2- 172.21.15.16:0/150686497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b9c0053b0 0x7f4b9c00ba60 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 -- 172.21.15.16:0/150686497 >> 172.21.15.16:0/150686497 conn(0x7f4bb806f900 msgr2=0x7f4bb80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 -- 172.21.15.16:0/150686497 shutdown_connections 2024-09-17T05:09:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.425+0000 7f4bbde00640 1 -- 172.21.15.16:0/150686497 wait complete. 2024-09-17T05:09:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 Processor -- start 2024-09-17T05:09:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 -- start start 2024-09-17T05:09:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b9c0053b0 0x7f4bb81f1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 0x7f4bb81f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bb813d0b0 0x7f4bb81401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb815bef0 con 0x7f4bb814caa0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb815bd70 con 0x7f4bb813d0b0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb815c4b0 con 0x7f4b9c0053b0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 0x7f4bb81f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 0x7f4bb81f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36360/0 (socket says 172.21.15.16:36360) 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 -- 172.21.15.16:0/1299045650 learned_addr learned my addr 172.21.15.16:0/1299045650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bb6a00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b9c0053b0 0x7f4bb81f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bb7400640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bb813d0b0 0x7f4bb81401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 -- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b9c0053b0 msgr2=0x7f4bb81f1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b9c0053b0 0x7f4bb81f1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 -- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bb813d0b0 msgr2=0x7f4bb81401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bb813d0b0 0x7f4bb81401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 -- 172.21.15.16:0/1299045650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81f4d30 con 0x7f4bb814caa0 2024-09-17T05:09:36.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bb6a00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b9c0053b0 0x7f4bb81f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bb7400640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bb813d0b0 0x7f4bb81401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bae000640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 0x7f4bb81f47f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00126c0 tx=0x7f4ba0012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0010b40 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ba0010ce0 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00136b0 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81f1a70 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.429+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81f2030 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.433+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb8073650 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.433+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4ba0013850 con 0x7f4bb814caa0 2024-09-17T05:09:36.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.433+0000 7f4bafe00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b94073230 0x7f4b94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.433+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f4ba0098090 con 0x7f4bb814caa0 2024-09-17T05:09:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.433+0000 7f4bb6a00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b94073230 0x7f4b94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.433+0000 7f4bb6a00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b94073230 0x7f4b94075740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009ea0 tx=0x7f4ba80094f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:36.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.437+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4ba0022020 con 0x7f4bb814caa0 2024-09-17T05:09:36.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.613+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bb81f1800 con 0x7f4b94073230 2024-09-17T05:09:36.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.617+0000 7f4bafe00640 1 -- 172.21.15.16:0/1299045650 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252468 (secure 0 0 0) 0x7f4bb81f1800 con 0x7f4b94073230 2024-09-17T05:09:36.620 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:36.622 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:09:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b94073230 msgr2=0x7f4b94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b94073230 0x7f4b94075740 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009ea0 tx=0x7f4ba80094f0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 msgr2=0x7f4bb81f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 0x7f4bb81f47f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00126c0 tx=0x7f4ba0012b80 comp rx=0 tx=0).stop 2024-09-17T05:09:36.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 shutdown_connections 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4b94073230 0x7f4b94075740 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4bb813d0b0 0x7f4bb81401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bb814caa0 0x7f4bb81f47f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 --2- 172.21.15.16:0/1299045650 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4b9c0053b0 0x7f4bb81f1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 >> 172.21.15.16:0/1299045650 conn(0x7f4bb806f900 msgr2=0x7f4bb815afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 shutdown_connections 2024-09-17T05:09:36.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.621+0000 7f4bbde00640 1 -- 172.21.15.16:0/1299045650 wait complete. 2024-09-17T05:09:36.642 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-17T05:09:35.602049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276336,"kb_used_data":14312,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754698384,"statfs":{"total":773094113280,"available":772811145216,"internally_reserved":0,"allocated":14655488,"data_stored":7387118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.185278"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.463890+0000","last_change":"2024-09-17T05:09:32.459554+0000","last_active":"2024-09-17T05:09:32.463890+0000","last_peered":"2024-09-17T05:09:32.463890+0000","last_clean":"2024-09-17T05:09:32.463890+0000","last_became_active":"2024-09-17T05:09:32.459174+0000","last_became_peered":"2024-09-17T05:09:32.459174+0000","last_unstale":"2024-09-17T05:09:32.463890+0000","last_undegraded":"2024-09-17T05:09:32.463890+0000","last_fullsized":"2024-09-17T05:09:32.463890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:32.194720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471973+0000","last_change":"2024-09-17T05:09:32.466630+0000","last_active":"2024-09-17T05:09:32.471973+0000","last_peered":"2024-09-17T05:09:32.471973+0000","last_clean":"2024-09-17T05:09:32.471973+0000","last_became_active":"2024-09-17T05:09:32.466252+0000","last_became_peered":"2024-09-17T05:09:32.466252+0000","last_unstale":"2024-09-17T05:09:32.471973+0000","last_undegraded":"2024-09-17T05:09:32.471973+0000","last_fullsized":"2024-09-17T05:09:32.471973+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:36.452709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461615+0000","last_change":"2024-09-17T05:09:32.456378+0000","last_active":"2024-09-17T05:09:32.461615+0000","last_peered":"2024-09-17T05:09:32.461615+0000","last_clean":"2024-09-17T05:09:32.461615+0000","last_became_active":"2024-09-17T05:09:32.453939+0000","last_became_peered":"2024-09-17T05:09:32.453939+0000","last_unstale":"2024-09-17T05:09:32.461615+0000","last_undegraded":"2024-09-17T05:09:32.461615+0000","last_fullsized":"2024-09-17T05:09:32.461615+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:49.314414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454670+0000","last_change":"2024-09-17T05:09:32.451234+0000","last_active":"2024-09-17T05:09:32.454670+0000","last_peered":"2024-09-17T05:09:32.454670+0000","last_clean":"2024-09-17T05:09:32.454670+0000","last_became_active":"2024-09-17T05:09:32.450925+0000","last_became_peered":"2024-09-17T05:09:32.450925+0000","last_unstale":"2024-09-17T05:09:32.454670+0000","last_undegraded":"2024-09-17T05:09:32.454670+0000","last_fullsized":"2024-09-17T05:09:32.454670+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:36.938546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471876+0000","last_change":"2024-09-17T05:09:32.468365+0000","last_active":"2024-09-17T05:09:32.471876+0000","last_peered":"2024-09-17T05:09:32.471876+0000","last_clean":"2024-09-17T05:09:32.471876+0000","last_became_active":"2024-09-17T05:09:32.468058+0000","last_became_peered":"2024-09-17T05:09:32.468058+0000","last_unstale":"2024-09-17T05:09:32.471876+0000","last_undegraded":"2024-09-17T05:09:32.471876+0000","last_fullsized":"2024-09-17T05:09:32.471876+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:48.315681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.449029+0000","last_change":"2024-09-17T05:09:32.444055+0000","last_active":"2024-09-17T05:09:32.449029+0000","last_peered":"2024-09-17T05:09:32.449029+0000","last_clean":"2024-09-17T05:09:32.449029+0000","last_became_active":"2024-09-17T05:09:32.443675+0000","last_became_peered":"2024-09-17T05:09:32.443675+0000","last_unstale":"2024-09-17T05:09:32.449029+0000","last_undegraded":"2024-09-17T05:09:32.449029+0000","last_fullsized":"2024-09-17T05:09:32.449029+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:49.588209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454074+0000","last_change":"2024-09-17T05:09:32.449877+0000","last_active":"2024-09-17T05:09:32.454074+0000","last_peered":"2024-09-17T05:09:32.454074+0000","last_clean":"2024-09-17T05:09:32.454074+0000","last_became_active":"2024-09-17T05:09:32.449569+0000","last_became_peered":"2024-09-17T05:09:32.449569+0000","last_unstale":"2024-09-17T05:09:32.454074+0000","last_undegraded":"2024-09-17T05:09:32.454074+0000","last_fullsized":"2024-09-17T05:09:32.454074+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:25.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454905+0000","last_change":"2024-09-17T05:09:32.449875+0000","last_active":"2024-09-17T05:09:32.454905+0000","last_peered":"2024-09-17T05:09:32.454905+0000","last_clean":"2024-09-17T05:09:32.454905+0000","last_became_active":"2024-09-17T05:09:32.449574+0000","last_became_peered":"2024-09-17T05:09:32.449574+0000","last_unstale":"2024-09-17T05:09:32.454905+0000","last_undegraded":"2024-09-17T05:09:32.454905+0000","last_fullsized":"2024-09-17T05:09:32.454905+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:48.569088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471958+0000","last_change":"2024-09-17T05:09:32.469198+0000","last_active":"2024-09-17T05:09:32.471958+0000","last_peered":"2024-09-17T05:09:32.471958+0000","last_clean":"2024-09-17T05:09:32.471958+0000","last_became_active":"2024-09-17T05:09:32.466681+0000","last_became_peered":"2024-09-17T05:09:32.466681+0000","last_unstale":"2024-09-17T05:09:32.471958+0000","last_undegraded":"2024-09-17T05:09:32.471958+0000","last_fullsized":"2024-09-17T05:09:32.471958+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:57.395931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.462177+0000","last_change":"2024-09-17T05:09:32.455532+0000","last_active":"2024-09-17T05:09:32.462177+0000","last_peered":"2024-09-17T05:09:32.462177+0000","last_clean":"2024-09-17T05:09:32.462177+0000","last_became_active":"2024-09-17T05:09:32.455208+0000","last_became_peered":"2024-09-17T05:09:32.455208+0000","last_unstale":"2024-09-17T05:09:32.462177+0000","last_undegraded":"2024-09-17T05:09:32.462177+0000","last_fullsized":"2024-09-17T05:09:32.462177+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:49.588209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458110+0000","last_change":"2024-09-17T05:09:32.455174+0000","last_active":"2024-09-17T05:09:33.458110+0000","last_peered":"2024-09-17T05:09:33.458110+0000","last_clean":"2024-09-17T05:09:33.458110+0000","last_became_active":"2024-09-17T05:09:32.454777+0000","last_became_peered":"2024-09-17T05:09:32.454777+0000","last_unstale":"2024-09-17T05:09:33.458110+0000","last_undegraded":"2024-09-17T05:09:33.458110+0000","last_fullsized":"2024-09-17T05:09:33.458110+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:57.523419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.472315+0000","last_change":"2024-09-17T05:09:32.470815+0000","last_active":"2024-09-17T05:09:32.472315+0000","last_peered":"2024-09-17T05:09:32.472315+0000","last_clean":"2024-09-17T05:09:32.472315+0000","last_became_active":"2024-09-17T05:09:32.470408+0000","last_became_peered":"2024-09-17T05:09:32.470408+0000","last_unstale":"2024-09-17T05:09:32.472315+0000","last_undegraded":"2024-09-17T05:09:32.472315+0000","last_fullsized":"2024-09-17T05:09:32.472315+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:58.233748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.468597+0000","last_change":"2024-09-17T05:09:32.459999+0000","last_active":"2024-09-17T05:09:32.468597+0000","last_peered":"2024-09-17T05:09:32.468597+0000","last_clean":"2024-09-17T05:09:32.468597+0000","last_became_active":"2024-09-17T05:09:32.459672+0000","last_became_peered":"2024-09-17T05:09:32.459672+0000","last_unstale":"2024-09-17T05:09:32.468597+0000","last_undegraded":"2024-09-17T05:09:32.468597+0000","last_fullsized":"2024-09-17T05:09:32.468597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:15.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.466109+0000","last_change":"2024-09-17T05:09:32.463346+0000","last_active":"2024-09-17T05:09:32.466109+0000","last_peered":"2024-09-17T05:09:32.466109+0000","last_clean":"2024-09-17T05:09:32.466109+0000","last_became_active":"2024-09-17T05:09:32.462997+0000","last_became_peered":"2024-09-17T05:09:32.462997+0000","last_unstale":"2024-09-17T05:09:32.466109+0000","last_undegraded":"2024-09-17T05:09:32.466109+0000","last_fullsized":"2024-09-17T05:09:32.466109+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:55.099404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457630+0000","last_change":"2024-09-17T05:09:32.460653+0000","last_active":"2024-09-17T05:09:33.457630+0000","last_peered":"2024-09-17T05:09:33.457630+0000","last_clean":"2024-09-17T05:09:33.457630+0000","last_became_active":"2024-09-17T05:09:32.460313+0000","last_became_peered":"2024-09-17T05:09:32.460313+0000","last_unstale":"2024-09-17T05:09:33.457630+0000","last_undegraded":"2024-09-17T05:09:33.457630+0000","last_fullsized":"2024-09-17T05:09:33.457630+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:14.951678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457753+0000","last_change":"2024-09-17T05:09:32.469224+0000","last_active":"2024-09-17T05:09:33.457753+0000","last_peered":"2024-09-17T05:09:33.457753+0000","last_clean":"2024-09-17T05:09:33.457753+0000","last_became_active":"2024-09-17T05:09:32.468901+0000","last_became_peered":"2024-09-17T05:09:32.468901+0000","last_unstale":"2024-09-17T05:09:33.457753+0000","last_undegraded":"2024-09-17T05:09:33.457753+0000","last_fullsized":"2024-09-17T05:09:33.457753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:59.352039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469901+0000","last_change":"2024-09-17T05:09:32.465545+0000","last_active":"2024-09-17T05:09:32.469901+0000","last_peered":"2024-09-17T05:09:32.469901+0000","last_clean":"2024-09-17T05:09:32.469901+0000","last_became_active":"2024-09-17T05:09:32.465231+0000","last_became_peered":"2024-09-17T05:09:32.465231+0000","last_unstale":"2024-09-17T05:09:32.469901+0000","last_undegraded":"2024-09-17T05:09:32.469901+0000","last_fullsized":"2024-09-17T05:09:32.469901+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:18.950422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.470645+0000","last_change":"2024-09-17T05:09:32.464472+0000","last_active":"2024-09-17T05:09:32.470645+0000","last_peered":"2024-09-17T05:09:32.470645+0000","last_clean":"2024-09-17T05:09:32.470645+0000","last_became_active":"2024-09-17T05:09:32.464124+0000","last_became_peered":"2024-09-17T05:09:32.464124+0000","last_unstale":"2024-09-17T05:09:32.470645+0000","last_undegraded":"2024-09-17T05:09:32.470645+0000","last_fullsized":"2024-09-17T05:09:32.470645+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:41.186636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455851+0000","last_change":"2024-09-17T05:09:32.450170+0000","last_active":"2024-09-17T05:09:32.455851+0000","last_peered":"2024-09-17T05:09:32.455851+0000","last_clean":"2024-09-17T05:09:32.455851+0000","last_became_active":"2024-09-17T05:09:32.449562+0000","last_became_peered":"2024-09-17T05:09:32.449562+0000","last_unstale":"2024-09-17T05:09:32.455851+0000","last_undegraded":"2024-09-17T05:09:32.455851+0000","last_fullsized":"2024-09-17T05:09:32.455851+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:01.783185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457972+0000","last_change":"2024-09-17T05:09:32.461392+0000","last_active":"2024-09-17T05:09:33.457972+0000","last_peered":"2024-09-17T05:09:33.457972+0000","last_clean":"2024-09-17T05:09:33.457972+0000","last_became_active":"2024-09-17T05:09:32.460956+0000","last_became_peered":"2024-09-17T05:09:32.460956+0000","last_unstale":"2024-09-17T05:09:33.457972+0000","last_undegraded":"2024-09-17T05:09:33.457972+0000","last_fullsized":"2024-09-17T05:09:33.457972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:26.680267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458127+0000","last_change":"2024-09-17T05:09:32.460007+0000","last_active":"2024-09-17T05:09:33.458127+0000","last_peered":"2024-09-17T05:09:33.458127+0000","last_clean":"2024-09-17T05:09:33.458127+0000","last_became_active":"2024-09-17T05:09:32.459679+0000","last_became_peered":"2024-09-17T05:09:32.459679+0000","last_unstale":"2024-09-17T05:09:33.458127+0000","last_undegraded":"2024-09-17T05:09:33.458127+0000","last_fullsized":"2024-09-17T05:09:33.458127+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:32.819161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.458419+0000","last_change":"2024-09-17T05:09:32.455151+0000","last_active":"2024-09-17T05:09:32.458419+0000","last_peered":"2024-09-17T05:09:32.458419+0000","last_clean":"2024-09-17T05:09:32.458419+0000","last_became_active":"2024-09-17T05:09:32.454832+0000","last_became_peered":"2024-09-17T05:09:32.454832+0000","last_unstale":"2024-09-17T05:09:32.458419+0000","last_undegraded":"2024-09-17T05:09:32.458419+0000","last_fullsized":"2024-09-17T05:09:32.458419+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:03.750851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.460257+0000","last_change":"2024-09-17T05:09:32.458627+0000","last_active":"2024-09-17T05:09:32.460257+0000","last_peered":"2024-09-17T05:09:32.460257+0000","last_clean":"2024-09-17T05:09:32.460257+0000","last_became_active":"2024-09-17T05:09:32.458294+0000","last_became_peered":"2024-09-17T05:09:32.458294+0000","last_unstale":"2024-09-17T05:09:32.460257+0000","last_undegraded":"2024-09-17T05:09:32.460257+0000","last_fullsized":"2024-09-17T05:09:32.460257+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:55.206745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.466825+0000","last_change":"2024-09-17T05:09:32.459107+0000","last_active":"2024-09-17T05:09:32.466825+0000","last_peered":"2024-09-17T05:09:32.466825+0000","last_clean":"2024-09-17T05:09:32.466825+0000","last_became_active":"2024-09-17T05:09:32.458688+0000","last_became_peered":"2024-09-17T05:09:32.458688+0000","last_unstale":"2024-09-17T05:09:32.466825+0000","last_undegraded":"2024-09-17T05:09:32.466825+0000","last_fullsized":"2024-09-17T05:09:32.466825+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:22.411822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.450093+0000","last_change":"2024-09-17T05:09:32.446846+0000","last_active":"2024-09-17T05:09:32.450093+0000","last_peered":"2024-09-17T05:09:32.450093+0000","last_clean":"2024-09-17T05:09:32.450093+0000","last_became_active":"2024-09-17T05:09:32.446531+0000","last_became_peered":"2024-09-17T05:09:32.446531+0000","last_unstale":"2024-09-17T05:09:32.450093+0000","last_undegraded":"2024-09-17T05:09:32.450093+0000","last_fullsized":"2024-09-17T05:09:32.450093+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:33.775785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.456907+0000","last_change":"2024-09-17T05:09:32.455249+0000","last_active":"2024-09-17T05:09:32.456907+0000","last_peered":"2024-09-17T05:09:32.456907+0000","last_clean":"2024-09-17T05:09:32.456907+0000","last_became_active":"2024-09-17T05:09:32.454941+0000","last_became_peered":"2024-09-17T05:09:32.454941+0000","last_unstale":"2024-09-17T05:09:32.456907+0000","last_undegraded":"2024-09-17T05:09:32.456907+0000","last_fullsized":"2024-09-17T05:09:32.456907+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:06.317385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467318+0000","last_change":"2024-09-17T05:09:32.461732+0000","last_active":"2024-09-17T05:09:32.467318+0000","last_peered":"2024-09-17T05:09:32.467318+0000","last_clean":"2024-09-17T05:09:32.467318+0000","last_became_active":"2024-09-17T05:09:32.457414+0000","last_became_peered":"2024-09-17T05:09:32.457414+0000","last_unstale":"2024-09-17T05:09:32.467318+0000","last_undegraded":"2024-09-17T05:09:32.467318+0000","last_fullsized":"2024-09-17T05:09:32.467318+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:33.775785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469304+0000","last_change":"2024-09-17T05:09:32.460278+0000","last_active":"2024-09-17T05:09:32.469304+0000","last_peered":"2024-09-17T05:09:32.469304+0000","last_clean":"2024-09-17T05:09:32.469304+0000","last_became_active":"2024-09-17T05:09:32.459952+0000","last_became_peered":"2024-09-17T05:09:32.459952+0000","last_unstale":"2024-09-17T05:09:32.469304+0000","last_undegraded":"2024-09-17T05:09:32.469304+0000","last_fullsized":"2024-09-17T05:09:32.469304+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:30.041172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455632+0000","last_change":"2024-09-17T05:09:32.447811+0000","last_active":"2024-09-17T05:09:32.455632+0000","last_peered":"2024-09-17T05:09:32.455632+0000","last_clean":"2024-09-17T05:09:32.455632+0000","last_became_active":"2024-09-17T05:09:32.446396+0000","last_became_peered":"2024-09-17T05:09:32.446396+0000","last_unstale":"2024-09-17T05:09:32.455632+0000","last_undegraded":"2024-09-17T05:09:32.455632+0000","last_fullsized":"2024-09-17T05:09:32.455632+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:50.635619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.470382+0000","last_change":"2024-09-17T05:09:32.461757+0000","last_active":"2024-09-17T05:09:32.470382+0000","last_peered":"2024-09-17T05:09:32.470382+0000","last_clean":"2024-09-17T05:09:32.470382+0000","last_became_active":"2024-09-17T05:09:32.457538+0000","last_became_peered":"2024-09-17T05:09:32.457538+0000","last_unstale":"2024-09-17T05:09:32.470382+0000","last_undegraded":"2024-09-17T05:09:32.470382+0000","last_fullsized":"2024-09-17T05:09:32.470382+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:04.289018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457733+0000","last_change":"2024-09-17T05:09:32.461664+0000","last_active":"2024-09-17T05:09:33.457733+0000","last_peered":"2024-09-17T05:09:33.457733+0000","last_clean":"2024-09-17T05:09:33.457733+0000","last_became_active":"2024-09-17T05:09:32.461128+0000","last_became_peered":"2024-09-17T05:09:32.461128+0000","last_unstale":"2024-09-17T05:09:33.457733+0000","last_undegraded":"2024-09-17T05:09:33.457733+0000","last_fullsized":"2024-09-17T05:09:33.457733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:28.231172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467140+0000","last_change":"2024-09-17T05:09:32.463336+0000","last_active":"2024-09-17T05:09:32.467140+0000","last_peered":"2024-09-17T05:09:32.467140+0000","last_clean":"2024-09-17T05:09:32.467140+0000","last_became_active":"2024-09-17T05:09:32.463014+0000","last_became_peered":"2024-09-17T05:09:32.463014+0000","last_unstale":"2024-09-17T05:09:32.467140+0000","last_undegraded":"2024-09-17T05:09:32.467140+0000","last_fullsized":"2024-09-17T05:09:32.467140+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:43.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458274+0000","last_change":"2024-09-17T05:09:32.458522+0000","last_active":"2024-09-17T05:09:33.458274+0000","last_peered":"2024-09-17T05:09:33.458274+0000","last_clean":"2024-09-17T05:09:33.458274+0000","last_became_active":"2024-09-17T05:09:32.458105+0000","last_became_peered":"2024-09-17T05:09:32.458105+0000","last_unstale":"2024-09-17T05:09:33.458274+0000","last_undegraded":"2024-09-17T05:09:33.458274+0000","last_fullsized":"2024-09-17T05:09:33.458274+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:27.892551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"135'4","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457749+0000","last_change":"2024-09-17T05:09:31.437545+0000","last_active":"2024-09-17T05:09:33.457749+0000","last_peered":"2024-09-17T05:09:33.457749+0000","last_clean":"2024-09-17T05:09:33.457749+0000","last_became_active":"2024-09-17T05:09:31.437095+0000","last_became_peered":"2024-09-17T05:09:31.437095+0000","last_unstale":"2024-09-17T05:09:33.457749+0000","last_undegraded":"2024-09-17T05:09:33.457749+0000","last_fullsized":"2024-09-17T05:09:33.457749+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:41.971283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.472048+0000","last_change":"2024-09-17T05:09:32.470759+0000","last_active":"2024-09-17T05:09:32.472048+0000","last_peered":"2024-09-17T05:09:32.472048+0000","last_clean":"2024-09-17T05:09:32.472048+0000","last_became_active":"2024-09-17T05:09:32.470366+0000","last_became_peered":"2024-09-17T05:09:32.470366+0000","last_unstale":"2024-09-17T05:09:32.472048+0000","last_undegraded":"2024-09-17T05:09:32.472048+0000","last_fullsized":"2024-09-17T05:09:32.472048+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:03.034788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.480928+0000","last_change":"2024-09-17T05:09:31.450928+0000","last_active":"2024-09-17T05:09:32.480928+0000","last_peered":"2024-09-17T05:09:32.480928+0000","last_clean":"2024-09-17T05:09:32.480928+0000","last_became_active":"2024-09-17T05:09:31.450418+0000","last_became_peered":"2024-09-17T05:09:31.450418+0000","last_unstale":"2024-09-17T05:09:32.480928+0000","last_undegraded":"2024-09-17T05:09:32.480928+0000","last_fullsized":"2024-09-17T05:09:32.480928+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:52.304829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469861+0000","last_change":"2024-09-17T05:09:32.466182+0000","last_active":"2024-09-17T05:09:32.469861+0000","last_peered":"2024-09-17T05:09:32.469861+0000","last_clean":"2024-09-17T05:09:32.469861+0000","last_became_active":"2024-09-17T05:09:32.465861+0000","last_became_peered":"2024-09-17T05:09:32.465861+0000","last_unstale":"2024-09-17T05:09:32.469861+0000","last_undegraded":"2024-09-17T05:09:32.469861+0000","last_fullsized":"2024-09-17T05:09:32.469861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:10.599638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.482094+0000","last_change":"2024-09-17T05:09:31.454091+0000","last_active":"2024-09-17T05:09:32.482094+0000","last_peered":"2024-09-17T05:09:32.482094+0000","last_clean":"2024-09-17T05:09:32.482094+0000","last_became_active":"2024-09-17T05:09:31.453764+0000","last_became_peered":"2024-09-17T05:09:31.453764+0000","last_unstale":"2024-09-17T05:09:32.482094+0000","last_undegraded":"2024-09-17T05:09:32.482094+0000","last_fullsized":"2024-09-17T05:09:32.482094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:44.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.460678+0000","last_change":"2024-09-17T05:09:32.456594+0000","last_active":"2024-09-17T05:09:32.460678+0000","last_peered":"2024-09-17T05:09:32.460678+0000","last_clean":"2024-09-17T05:09:32.460678+0000","last_became_active":"2024-09-17T05:09:32.456255+0000","last_became_peered":"2024-09-17T05:09:32.456255+0000","last_unstale":"2024-09-17T05:09:32.460678+0000","last_undegraded":"2024-09-17T05:09:32.460678+0000","last_fullsized":"2024-09-17T05:09:32.460678+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:17.519917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.477378+0000","last_change":"2024-09-17T05:09:31.454908+0000","last_active":"2024-09-17T05:09:32.477378+0000","last_peered":"2024-09-17T05:09:32.477378+0000","last_clean":"2024-09-17T05:09:32.477378+0000","last_became_active":"2024-09-17T05:09:31.454597+0000","last_became_peered":"2024-09-17T05:09:31.454597+0000","last_unstale":"2024-09-17T05:09:32.477378+0000","last_undegraded":"2024-09-17T05:09:32.477378+0000","last_fullsized":"2024-09-17T05:09:32.477378+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:41.623332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457596+0000","last_change":"2024-09-17T05:09:32.450284+0000","last_active":"2024-09-17T05:09:33.457596+0000","last_peered":"2024-09-17T05:09:33.457596+0000","last_clean":"2024-09-17T05:09:33.457596+0000","last_became_active":"2024-09-17T05:09:32.449934+0000","last_became_peered":"2024-09-17T05:09:32.449934+0000","last_unstale":"2024-09-17T05:09:33.457596+0000","last_undegraded":"2024-09-17T05:09:33.457596+0000","last_fullsized":"2024-09-17T05:09:33.457596+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:00.319736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.481809+0000","last_change":"2024-09-17T05:09:31.460131+0000","last_active":"2024-09-17T05:09:32.481809+0000","last_peered":"2024-09-17T05:09:32.481809+0000","last_clean":"2024-09-17T05:09:32.481809+0000","last_became_active":"2024-09-17T05:09:31.459807+0000","last_became_peered":"2024-09-17T05:09:31.459807+0000","last_unstale":"2024-09-17T05:09:32.481809+0000","last_undegraded":"2024-09-17T05:09:32.481809+0000","last_fullsized":"2024-09-17T05:09:32.481809+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:27.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.460949+0000","last_change":"2024-09-17T05:09:32.457002+0000","last_active":"2024-09-17T05:09:32.460949+0000","last_peered":"2024-09-17T05:09:32.460949+0000","last_clean":"2024-09-17T05:09:32.460949+0000","last_became_active":"2024-09-17T05:09:32.456714+0000","last_became_peered":"2024-09-17T05:09:32.456714+0000","last_unstale":"2024-09-17T05:09:32.460949+0000","last_undegraded":"2024-09-17T05:09:32.460949+0000","last_fullsized":"2024-09-17T05:09:32.460949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:49.108183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.478700+0000","last_change":"2024-09-17T05:09:31.448013+0000","last_active":"2024-09-17T05:09:32.478700+0000","last_peered":"2024-09-17T05:09:32.478700+0000","last_clean":"2024-09-17T05:09:32.478700+0000","last_became_active":"2024-09-17T05:09:31.447712+0000","last_became_peered":"2024-09-17T05:09:31.447712+0000","last_unstale":"2024-09-17T05:09:32.478700+0000","last_undegraded":"2024-09-17T05:09:32.478700+0000","last_fullsized":"2024-09-17T05:09:32.478700+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:02.943997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.452967+0000","last_change":"2024-09-17T05:09:32.451003+0000","last_active":"2024-09-17T05:09:32.452967+0000","last_peered":"2024-09-17T05:09:32.452967+0000","last_clean":"2024-09-17T05:09:32.452967+0000","last_became_active":"2024-09-17T05:09:32.450673+0000","last_became_peered":"2024-09-17T05:09:32.450673+0000","last_unstale":"2024-09-17T05:09:32.452967+0000","last_undegraded":"2024-09-17T05:09:32.452967+0000","last_fullsized":"2024-09-17T05:09:32.452967+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:42.630264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.447958+0000","last_change":"2024-09-17T05:09:31.455527+0000","last_active":"2024-09-17T05:09:32.447958+0000","last_peered":"2024-09-17T05:09:32.447958+0000","last_clean":"2024-09-17T05:09:32.447958+0000","last_became_active":"2024-09-17T05:09:31.449706+0000","last_became_peered":"2024-09-17T05:09:31.449706+0000","last_unstale":"2024-09-17T05:09:32.447958+0000","last_undegraded":"2024-09-17T05:09:32.447958+0000","last_fullsized":"2024-09-17T05:09:32.447958+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:24.848250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.470659+0000","last_change":"2024-09-17T05:09:32.468898+0000","last_active":"2024-09-17T05:09:32.470659+0000","last_peered":"2024-09-17T05:09:32.470659+0000","last_clean":"2024-09-17T05:09:32.470659+0000","last_became_active":"2024-09-17T05:09:32.467892+0000","last_became_peered":"2024-09-17T05:09:32.467892+0000","last_unstale":"2024-09-17T05:09:32.470659+0000","last_undegraded":"2024-09-17T05:09:32.470659+0000","last_fullsized":"2024-09-17T05:09:32.470659+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:11.595839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448653+0000","last_change":"2024-09-17T05:09:31.448830+0000","last_active":"2024-09-17T05:09:32.448653+0000","last_peered":"2024-09-17T05:09:32.448653+0000","last_clean":"2024-09-17T05:09:32.448653+0000","last_became_active":"2024-09-17T05:09:31.447529+0000","last_became_peered":"2024-09-17T05:09:31.447529+0000","last_unstale":"2024-09-17T05:09:32.448653+0000","last_undegraded":"2024-09-17T05:09:32.448653+0000","last_fullsized":"2024-09-17T05:09:32.448653+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:42.448158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461418+0000","last_change":"2024-09-17T05:09:32.458761+0000","last_active":"2024-09-17T05:09:32.461418+0000","last_peered":"2024-09-17T05:09:32.461418+0000","last_clean":"2024-09-17T05:09:32.461418+0000","last_became_active":"2024-09-17T05:09:32.458369+0000","last_became_peered":"2024-09-17T05:09:32.458369+0000","last_unstale":"2024-09-17T05:09:32.461418+0000","last_undegraded":"2024-09-17T05:09:32.461418+0000","last_fullsized":"2024-09-17T05:09:32.461418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:28.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461521+0000","last_change":"2024-09-17T05:09:31.450785+0000","last_active":"2024-09-17T05:09:32.461521+0000","last_peered":"2024-09-17T05:09:32.461521+0000","last_clean":"2024-09-17T05:09:32.461521+0000","last_became_active":"2024-09-17T05:09:31.450424+0000","last_became_peered":"2024-09-17T05:09:31.450424+0000","last_unstale":"2024-09-17T05:09:32.461521+0000","last_undegraded":"2024-09-17T05:09:32.461521+0000","last_fullsized":"2024-09-17T05:09:32.461521+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:01.909901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458369+0000","last_change":"2024-09-17T05:09:32.460398+0000","last_active":"2024-09-17T05:09:33.458369+0000","last_peered":"2024-09-17T05:09:33.458369+0000","last_clean":"2024-09-17T05:09:33.458369+0000","last_became_active":"2024-09-17T05:09:32.459882+0000","last_became_peered":"2024-09-17T05:09:32.459882+0000","last_unstale":"2024-09-17T05:09:33.458369+0000","last_undegraded":"2024-09-17T05:09:33.458369+0000","last_fullsized":"2024-09-17T05:09:33.458369+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:08.017571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448050+0000","last_change":"2024-09-17T05:09:31.447599+0000","last_active":"2024-09-17T05:09:32.448050+0000","last_peered":"2024-09-17T05:09:32.448050+0000","last_clean":"2024-09-17T05:09:32.448050+0000","last_became_active":"2024-09-17T05:09:31.447249+0000","last_became_peered":"2024-09-17T05:09:31.447249+0000","last_unstale":"2024-09-17T05:09:32.448050+0000","last_undegraded":"2024-09-17T05:09:32.448050+0000","last_fullsized":"2024-09-17T05:09:32.448050+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:06.979107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.472884+0000","last_change":"2024-09-17T05:09:32.471747+0000","last_active":"2024-09-17T05:09:32.472884+0000","last_peered":"2024-09-17T05:09:32.472884+0000","last_clean":"2024-09-17T05:09:32.472884+0000","last_became_active":"2024-09-17T05:09:32.470371+0000","last_became_peered":"2024-09-17T05:09:32.470371+0000","last_unstale":"2024-09-17T05:09:32.472884+0000","last_undegraded":"2024-09-17T05:09:32.472884+0000","last_fullsized":"2024-09-17T05:09:32.472884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:37.643590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"135'3","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458215+0000","last_change":"2024-09-17T05:09:31.440159+0000","last_active":"2024-09-17T05:09:33.458215+0000","last_peered":"2024-09-17T05:09:33.458215+0000","last_clean":"2024-09-17T05:09:33.458215+0000","last_became_active":"2024-09-17T05:09:31.439748+0000","last_became_peered":"2024-09-17T05:09:31.439748+0000","last_unstale":"2024-09-17T05:09:33.458215+0000","last_undegraded":"2024-09-17T05:09:33.458215+0000","last_fullsized":"2024-09-17T05:09:33.458215+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.879610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.457385+0000","last_change":"2024-09-17T05:09:32.451818+0000","last_active":"2024-09-17T05:09:32.457385+0000","last_peered":"2024-09-17T05:09:32.457385+0000","last_clean":"2024-09-17T05:09:32.457385+0000","last_became_active":"2024-09-17T05:09:32.451475+0000","last_became_peered":"2024-09-17T05:09:32.451475+0000","last_unstale":"2024-09-17T05:09:32.457385+0000","last_undegraded":"2024-09-17T05:09:32.457385+0000","last_fullsized":"2024-09-17T05:09:32.457385+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:36.986054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.480613+0000","last_change":"2024-09-17T05:09:31.445981+0000","last_active":"2024-09-17T05:09:32.480613+0000","last_peered":"2024-09-17T05:09:32.480613+0000","last_clean":"2024-09-17T05:09:32.480613+0000","last_became_active":"2024-09-17T05:09:31.445647+0000","last_became_peered":"2024-09-17T05:09:31.445647+0000","last_unstale":"2024-09-17T05:09:32.480613+0000","last_undegraded":"2024-09-17T05:09:32.480613+0000","last_fullsized":"2024-09-17T05:09:32.480613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:29.465023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467360+0000","last_change":"2024-09-17T05:09:32.463019+0000","last_active":"2024-09-17T05:09:32.467360+0000","last_peered":"2024-09-17T05:09:32.467360+0000","last_clean":"2024-09-17T05:09:32.467360+0000","last_became_active":"2024-09-17T05:09:32.458897+0000","last_became_peered":"2024-09-17T05:09:32.458897+0000","last_unstale":"2024-09-17T05:09:32.467360+0000","last_undegraded":"2024-09-17T05:09:32.467360+0000","last_fullsized":"2024-09-17T05:09:32.467360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:01.783185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.447977+0000","last_change":"2024-09-17T05:09:31.458968+0000","last_active":"2024-09-17T05:09:32.447977+0000","last_peered":"2024-09-17T05:09:32.447977+0000","last_clean":"2024-09-17T05:09:32.447977+0000","last_became_active":"2024-09-17T05:09:31.458583+0000","last_became_peered":"2024-09-17T05:09:31.458583+0000","last_unstale":"2024-09-17T05:09:32.447977+0000","last_undegraded":"2024-09-17T05:09:32.447977+0000","last_fullsized":"2024-09-17T05:09:32.447977+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:28.957448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469424+0000","last_change":"2024-09-17T05:09:32.462939+0000","last_active":"2024-09-17T05:09:32.469424+0000","last_peered":"2024-09-17T05:09:32.469424+0000","last_clean":"2024-09-17T05:09:32.469424+0000","last_became_active":"2024-09-17T05:09:32.459569+0000","last_became_peered":"2024-09-17T05:09:32.459569+0000","last_unstale":"2024-09-17T05:09:32.469424+0000","last_undegraded":"2024-09-17T05:09:32.469424+0000","last_fullsized":"2024-09-17T05:09:32.469424+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:32.399711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448369+0000","last_change":"2024-09-17T05:09:31.447239+0000","last_active":"2024-09-17T05:09:32.448369+0000","last_peered":"2024-09-17T05:09:32.448369+0000","last_clean":"2024-09-17T05:09:32.448369+0000","last_became_active":"2024-09-17T05:09:31.446850+0000","last_became_peered":"2024-09-17T05:09:31.446850+0000","last_unstale":"2024-09-17T05:09:32.448369+0000","last_undegraded":"2024-09-17T05:09:32.448369+0000","last_fullsized":"2024-09-17T05:09:32.448369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:21.178937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.463610+0000","last_change":"2024-09-17T05:09:32.456766+0000","last_active":"2024-09-17T05:09:32.463610+0000","last_peered":"2024-09-17T05:09:32.463610+0000","last_clean":"2024-09-17T05:09:32.463610+0000","last_became_active":"2024-09-17T05:09:32.456453+0000","last_became_peered":"2024-09-17T05:09:32.456453+0000","last_unstale":"2024-09-17T05:09:32.463610+0000","last_undegraded":"2024-09-17T05:09:32.463610+0000","last_fullsized":"2024-09-17T05:09:32.463610+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:22.411822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"135'8","reported_seq":21,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.484068+0000","last_change":"2024-09-17T05:09:31.454775+0000","last_active":"2024-09-17T05:09:32.484068+0000","last_peered":"2024-09-17T05:09:32.484068+0000","last_clean":"2024-09-17T05:09:32.484068+0000","last_became_active":"2024-09-17T05:09:31.454471+0000","last_became_peered":"2024-09-17T05:09:31.454471+0000","last_unstale":"2024-09-17T05:09:32.484068+0000","last_undegraded":"2024-09-17T05:09:32.484068+0000","last_fullsized":"2024-09-17T05:09:32.484068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:33.992646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.451840+0000","last_change":"2024-09-17T05:09:32.448684+0000","last_active":"2024-09-17T05:09:32.451840+0000","last_peered":"2024-09-17T05:09:32.451840+0000","last_clean":"2024-09-17T05:09:32.451840+0000","last_became_active":"2024-09-17T05:09:32.448382+0000","last_became_peered":"2024-09-17T05:09:32.448382+0000","last_unstale":"2024-09-17T05:09:32.451840+0000","last_undegraded":"2024-09-17T05:09:32.451840+0000","last_fullsized":"2024-09-17T05:09:32.451840+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:04.289018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.479423+0000","last_change":"2024-09-17T05:09:31.449297+0000","last_active":"2024-09-17T05:09:32.479423+0000","last_peered":"2024-09-17T05:09:32.479423+0000","last_clean":"2024-09-17T05:09:32.479423+0000","last_became_active":"2024-09-17T05:09:31.447500+0000","last_became_peered":"2024-09-17T05:09:31.447500+0000","last_unstale":"2024-09-17T05:09:32.479423+0000","last_undegraded":"2024-09-17T05:09:32.479423+0000","last_fullsized":"2024-09-17T05:09:32.479423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:01.980845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.468353+0000","last_change":"2024-09-17T05:09:32.461007+0000","last_active":"2024-09-17T05:09:32.468353+0000","last_peered":"2024-09-17T05:09:32.468353+0000","last_clean":"2024-09-17T05:09:32.468353+0000","last_became_active":"2024-09-17T05:09:32.460012+0000","last_became_peered":"2024-09-17T05:09:32.460012+0000","last_unstale":"2024-09-17T05:09:32.468353+0000","last_undegraded":"2024-09-17T05:09:32.468353+0000","last_fullsized":"2024-09-17T05:09:32.468353+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:02.182467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.482598+0000","last_change":"2024-09-17T05:09:31.451787+0000","last_active":"2024-09-17T05:09:32.482598+0000","last_peered":"2024-09-17T05:09:32.482598+0000","last_clean":"2024-09-17T05:09:32.482598+0000","last_became_active":"2024-09-17T05:09:31.451435+0000","last_became_peered":"2024-09-17T05:09:31.451435+0000","last_unstale":"2024-09-17T05:09:32.482598+0000","last_undegraded":"2024-09-17T05:09:32.482598+0000","last_fullsized":"2024-09-17T05:09:32.482598+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:58.085012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469829+0000","last_change":"2024-09-17T05:09:32.465410+0000","last_active":"2024-09-17T05:09:32.469829+0000","last_peered":"2024-09-17T05:09:32.469829+0000","last_clean":"2024-09-17T05:09:32.469829+0000","last_became_active":"2024-09-17T05:09:32.465095+0000","last_became_peered":"2024-09-17T05:09:32.465095+0000","last_unstale":"2024-09-17T05:09:32.469829+0000","last_undegraded":"2024-09-17T05:09:32.469829+0000","last_fullsized":"2024-09-17T05:09:32.469829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:12.572279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.475027+0000","last_change":"2024-09-17T05:09:31.447005+0000","last_active":"2024-09-17T05:09:32.475027+0000","last_peered":"2024-09-17T05:09:32.475027+0000","last_clean":"2024-09-17T05:09:32.475027+0000","last_became_active":"2024-09-17T05:09:31.446621+0000","last_became_peered":"2024-09-17T05:09:31.446621+0000","last_unstale":"2024-09-17T05:09:32.475027+0000","last_undegraded":"2024-09-17T05:09:32.475027+0000","last_fullsized":"2024-09-17T05:09:32.475027+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:24.437307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454680+0000","last_change":"2024-09-17T05:09:32.452255+0000","last_active":"2024-09-17T05:09:32.454680+0000","last_peered":"2024-09-17T05:09:32.454680+0000","last_clean":"2024-09-17T05:09:32.454680+0000","last_became_active":"2024-09-17T05:09:32.451890+0000","last_became_peered":"2024-09-17T05:09:32.451890+0000","last_unstale":"2024-09-17T05:09:32.454680+0000","last_undegraded":"2024-09-17T05:09:32.454680+0000","last_fullsized":"2024-09-17T05:09:32.454680+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:37.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.438128+0000","last_change":"2024-09-17T05:09:31.455533+0000","last_active":"2024-09-17T05:09:32.438128+0000","last_peered":"2024-09-17T05:09:32.438128+0000","last_clean":"2024-09-17T05:09:32.438128+0000","last_became_active":"2024-09-17T05:09:31.455000+0000","last_became_peered":"2024-09-17T05:09:31.455000+0000","last_unstale":"2024-09-17T05:09:32.438128+0000","last_undegraded":"2024-09-17T05:09:32.438128+0000","last_fullsized":"2024-09-17T05:09:32.438128+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:34.644350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.473332+0000","last_change":"2024-09-17T05:09:32.471957+0000","last_active":"2024-09-17T05:09:32.473332+0000","last_peered":"2024-09-17T05:09:32.473332+0000","last_clean":"2024-09-17T05:09:32.473332+0000","last_became_active":"2024-09-17T05:09:32.471469+0000","last_became_peered":"2024-09-17T05:09:32.471469+0000","last_unstale":"2024-09-17T05:09:32.473332+0000","last_undegraded":"2024-09-17T05:09:32.473332+0000","last_fullsized":"2024-09-17T05:09:32.473332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:09.796220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.479630+0000","last_change":"2024-09-17T05:09:31.449205+0000","last_active":"2024-09-17T05:09:32.479630+0000","last_peered":"2024-09-17T05:09:32.479630+0000","last_clean":"2024-09-17T05:09:32.479630+0000","last_became_active":"2024-09-17T05:09:31.448489+0000","last_became_peered":"2024-09-17T05:09:31.448489+0000","last_unstale":"2024-09-17T05:09:32.479630+0000","last_undegraded":"2024-09-17T05:09:32.479630+0000","last_fullsized":"2024-09-17T05:09:32.479630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:33.407660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.451985+0000","last_change":"2024-09-17T05:09:32.449063+0000","last_active":"2024-09-17T05:09:32.451985+0000","last_peered":"2024-09-17T05:09:32.451985+0000","last_clean":"2024-09-17T05:09:32.451985+0000","last_became_active":"2024-09-17T05:09:32.448648+0000","last_became_peered":"2024-09-17T05:09:32.448648+0000","last_unstale":"2024-09-17T05:09:32.451985+0000","last_undegraded":"2024-09-17T05:09:32.451985+0000","last_fullsized":"2024-09-17T05:09:32.451985+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:49.314414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461739+0000","last_change":"2024-09-17T05:09:31.442155+0000","last_active":"2024-09-17T05:09:32.461739+0000","last_peered":"2024-09-17T05:09:32.461739+0000","last_clean":"2024-09-17T05:09:32.461739+0000","last_became_active":"2024-09-17T05:09:31.441840+0000","last_became_peered":"2024-09-17T05:09:31.441840+0000","last_unstale":"2024-09-17T05:09:32.461739+0000","last_undegraded":"2024-09-17T05:09:32.461739+0000","last_fullsized":"2024-09-17T05:09:32.461739+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:42.627880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.463854+0000","last_change":"2024-09-17T05:09:32.458594+0000","last_active":"2024-09-17T05:09:32.463854+0000","last_peered":"2024-09-17T05:09:32.463854+0000","last_clean":"2024-09-17T05:09:32.463854+0000","last_became_active":"2024-09-17T05:09:32.458232+0000","last_became_peered":"2024-09-17T05:09:32.458232+0000","last_unstale":"2024-09-17T05:09:32.463854+0000","last_undegraded":"2024-09-17T05:09:32.463854+0000","last_fullsized":"2024-09-17T05:09:32.463854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:16.847253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448590+0000","last_change":"2024-09-17T05:09:31.448324+0000","last_active":"2024-09-17T05:09:32.448590+0000","last_peered":"2024-09-17T05:09:32.448590+0000","last_clean":"2024-09-17T05:09:32.448590+0000","last_became_active":"2024-09-17T05:09:31.447136+0000","last_became_peered":"2024-09-17T05:09:31.447136+0000","last_unstale":"2024-09-17T05:09:32.448590+0000","last_undegraded":"2024-09-17T05:09:32.448590+0000","last_fullsized":"2024-09-17T05:09:32.448590+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:08.708278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457600+0000","last_change":"2024-09-17T05:09:32.452984+0000","last_active":"2024-09-17T05:09:33.457600+0000","last_peered":"2024-09-17T05:09:33.457600+0000","last_clean":"2024-09-17T05:09:33.457600+0000","last_became_active":"2024-09-17T05:09:32.452616+0000","last_became_peered":"2024-09-17T05:09:32.452616+0000","last_unstale":"2024-09-17T05:09:33.457600+0000","last_undegraded":"2024-09-17T05:09:33.457600+0000","last_fullsized":"2024-09-17T05:09:33.457600+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:56.310567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461692+0000","last_change":"2024-09-17T05:09:31.447582+0000","last_active":"2024-09-17T05:09:32.461692+0000","last_peered":"2024-09-17T05:09:32.461692+0000","last_clean":"2024-09-17T05:09:32.461692+0000","last_became_active":"2024-09-17T05:09:31.447206+0000","last_became_peered":"2024-09-17T05:09:31.447206+0000","last_unstale":"2024-09-17T05:09:32.461692+0000","last_undegraded":"2024-09-17T05:09:32.461692+0000","last_fullsized":"2024-09-17T05:09:32.461692+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:07.596266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458278+0000","last_change":"2024-09-17T05:09:32.450008+0000","last_active":"2024-09-17T05:09:33.458278+0000","last_peered":"2024-09-17T05:09:33.458278+0000","last_clean":"2024-09-17T05:09:33.458278+0000","last_became_active":"2024-09-17T05:09:32.449687+0000","last_became_peered":"2024-09-17T05:09:32.449687+0000","last_unstale":"2024-09-17T05:09:33.458278+0000","last_undegraded":"2024-09-17T05:09:33.458278+0000","last_fullsized":"2024-09-17T05:09:33.458278+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:36.057783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.481921+0000","last_change":"2024-09-17T05:09:31.459463+0000","last_active":"2024-09-17T05:09:32.481921+0000","last_peered":"2024-09-17T05:09:32.481921+0000","last_clean":"2024-09-17T05:09:32.481921+0000","last_became_active":"2024-09-17T05:09:31.459167+0000","last_became_peered":"2024-09-17T05:09:31.459167+0000","last_unstale":"2024-09-17T05:09:32.481921+0000","last_undegraded":"2024-09-17T05:09:32.481921+0000","last_fullsized":"2024-09-17T05:09:32.481921+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:24.567345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.465977+0000","last_change":"2024-09-17T05:09:32.464067+0000","last_active":"2024-09-17T05:09:32.465977+0000","last_peered":"2024-09-17T05:09:32.465977+0000","last_clean":"2024-09-17T05:09:32.465977+0000","last_became_active":"2024-09-17T05:09:32.463766+0000","last_became_peered":"2024-09-17T05:09:32.463766+0000","last_unstale":"2024-09-17T05:09:32.465977+0000","last_undegraded":"2024-09-17T05:09:32.465977+0000","last_fullsized":"2024-09-17T05:09:32.465977+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:07.316592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.449405+0000","last_change":"2024-09-17T05:09:31.446794+0000","last_active":"2024-09-17T05:09:32.449405+0000","last_peered":"2024-09-17T05:09:32.449405+0000","last_clean":"2024-09-17T05:09:32.449405+0000","last_became_active":"2024-09-17T05:09:31.446459+0000","last_became_peered":"2024-09-17T05:09:31.446459+0000","last_unstale":"2024-09-17T05:09:32.449405+0000","last_undegraded":"2024-09-17T05:09:32.449405+0000","last_fullsized":"2024-09-17T05:09:32.449405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:47.193757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.464104+0000","last_change":"2024-09-17T05:09:32.459109+0000","last_active":"2024-09-17T05:09:32.464104+0000","last_peered":"2024-09-17T05:09:32.464104+0000","last_clean":"2024-09-17T05:09:32.464104+0000","last_became_active":"2024-09-17T05:09:32.457310+0000","last_became_peered":"2024-09-17T05:09:32.457310+0000","last_unstale":"2024-09-17T05:09:32.464104+0000","last_undegraded":"2024-09-17T05:09:32.464104+0000","last_fullsized":"2024-09-17T05:09:32.464104+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:42.479724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.438705+0000","last_change":"2024-09-17T05:09:31.447285+0000","last_active":"2024-09-17T05:09:32.438705+0000","last_peered":"2024-09-17T05:09:32.438705+0000","last_clean":"2024-09-17T05:09:32.438705+0000","last_became_active":"2024-09-17T05:09:31.446984+0000","last_became_peered":"2024-09-17T05:09:31.446984+0000","last_unstale":"2024-09-17T05:09:32.438705+0000","last_undegraded":"2024-09-17T05:09:32.438705+0000","last_fullsized":"2024-09-17T05:09:32.438705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:48.726773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471895+0000","last_change":"2024-09-17T05:09:32.468753+0000","last_active":"2024-09-17T05:09:32.471895+0000","last_peered":"2024-09-17T05:09:32.471895+0000","last_clean":"2024-09-17T05:09:32.471895+0000","last_became_active":"2024-09-17T05:09:32.466179+0000","last_became_peered":"2024-09-17T05:09:32.466179+0000","last_unstale":"2024-09-17T05:09:32.471895+0000","last_undegraded":"2024-09-17T05:09:32.471895+0000","last_fullsized":"2024-09-17T05:09:32.471895+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:58.022190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.438649+0000","last_change":"2024-09-17T05:09:31.436964+0000","last_active":"2024-09-17T05:09:32.438649+0000","last_peered":"2024-09-17T05:09:32.438649+0000","last_clean":"2024-09-17T05:09:32.438649+0000","last_became_active":"2024-09-17T05:09:31.436618+0000","last_became_peered":"2024-09-17T05:09:31.436618+0000","last_unstale":"2024-09-17T05:09:32.438649+0000","last_undegraded":"2024-09-17T05:09:32.438649+0000","last_fullsized":"2024-09-17T05:09:32.438649+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:28.054297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467673+0000","last_change":"2024-09-17T05:09:32.464524+0000","last_active":"2024-09-17T05:09:32.467673+0000","last_peered":"2024-09-17T05:09:32.467673+0000","last_clean":"2024-09-17T05:09:32.467673+0000","last_became_active":"2024-09-17T05:09:32.464219+0000","last_became_peered":"2024-09-17T05:09:32.464219+0000","last_unstale":"2024-09-17T05:09:32.467673+0000","last_undegraded":"2024-09-17T05:09:32.467673+0000","last_fullsized":"2024-09-17T05:09:32.467673+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:28.074106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.445388+0000","last_change":"2024-09-17T05:09:31.447897+0000","last_active":"2024-09-17T05:09:32.445388+0000","last_peered":"2024-09-17T05:09:32.445388+0000","last_clean":"2024-09-17T05:09:32.445388+0000","last_became_active":"2024-09-17T05:09:31.447469+0000","last_became_peered":"2024-09-17T05:09:31.447469+0000","last_unstale":"2024-09-17T05:09:32.445388+0000","last_undegraded":"2024-09-17T05:09:32.445388+0000","last_fullsized":"2024-09-17T05:09:32.445388+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.175535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471000+0000","last_change":"2024-09-17T05:09:32.469495+0000","last_active":"2024-09-17T05:09:32.471000+0000","last_peered":"2024-09-17T05:09:32.471000+0000","last_clean":"2024-09-17T05:09:32.471000+0000","last_became_active":"2024-09-17T05:09:32.467144+0000","last_became_peered":"2024-09-17T05:09:32.467144+0000","last_unstale":"2024-09-17T05:09:32.471000+0000","last_undegraded":"2024-09-17T05:09:32.471000+0000","last_fullsized":"2024-09-17T05:09:32.471000+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:43.239306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457374+0000","last_change":"2024-09-17T05:09:31.448040+0000","last_active":"2024-09-17T05:09:33.457374+0000","last_peered":"2024-09-17T05:09:33.457374+0000","last_clean":"2024-09-17T05:09:33.457374+0000","last_became_active":"2024-09-17T05:09:31.447684+0000","last_became_peered":"2024-09-17T05:09:31.447684+0000","last_unstale":"2024-09-17T05:09:33.457374+0000","last_undegraded":"2024-09-17T05:09:33.457374+0000","last_fullsized":"2024-09-17T05:09:33.457374+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:19.060445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455829+0000","last_change":"2024-09-17T05:09:32.450277+0000","last_active":"2024-09-17T05:09:32.455829+0000","last_peered":"2024-09-17T05:09:32.455829+0000","last_clean":"2024-09-17T05:09:32.455829+0000","last_became_active":"2024-09-17T05:09:32.449952+0000","last_became_peered":"2024-09-17T05:09:32.449952+0000","last_unstale":"2024-09-17T05:09:32.455829+0000","last_undegraded":"2024-09-17T05:09:32.455829+0000","last_fullsized":"2024-09-17T05:09:32.455829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:30.471955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.445326+0000","last_change":"2024-09-17T05:09:31.435307+0000","last_active":"2024-09-17T05:09:32.445326+0000","last_peered":"2024-09-17T05:09:32.445326+0000","last_clean":"2024-09-17T05:09:32.445326+0000","last_became_active":"2024-09-17T05:09:31.434992+0000","last_became_peered":"2024-09-17T05:09:31.434992+0000","last_unstale":"2024-09-17T05:09:32.445326+0000","last_undegraded":"2024-09-17T05:09:32.445326+0000","last_fullsized":"2024-09-17T05:09:32.445326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:21.404890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455598+0000","last_change":"2024-09-17T05:09:32.447512+0000","last_active":"2024-09-17T05:09:32.455598+0000","last_peered":"2024-09-17T05:09:32.455598+0000","last_clean":"2024-09-17T05:09:32.455598+0000","last_became_active":"2024-09-17T05:09:32.445980+0000","last_became_peered":"2024-09-17T05:09:32.445980+0000","last_unstale":"2024-09-17T05:09:32.455598+0000","last_undegraded":"2024-09-17T05:09:32.455598+0000","last_fullsized":"2024-09-17T05:09:32.455598+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:28.065744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.444688+0000","last_change":"2024-09-17T05:09:31.439519+0000","last_active":"2024-09-17T05:09:32.444688+0000","last_peered":"2024-09-17T05:09:32.444688+0000","last_clean":"2024-09-17T05:09:32.444688+0000","last_became_active":"2024-09-17T05:09:31.439187+0000","last_became_peered":"2024-09-17T05:09:31.439187+0000","last_unstale":"2024-09-17T05:09:32.444688+0000","last_undegraded":"2024-09-17T05:09:32.444688+0000","last_fullsized":"2024-09-17T05:09:32.444688+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:54.199813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.458900+0000","last_change":"2024-09-17T05:09:32.454379+0000","last_active":"2024-09-17T05:09:32.458900+0000","last_peered":"2024-09-17T05:09:32.458900+0000","last_clean":"2024-09-17T05:09:32.458900+0000","last_became_active":"2024-09-17T05:09:32.453001+0000","last_became_peered":"2024-09-17T05:09:32.453001+0000","last_unstale":"2024-09-17T05:09:32.458900+0000","last_undegraded":"2024-09-17T05:09:32.458900+0000","last_fullsized":"2024-09-17T05:09:32.458900+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:34.999578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"135'6","reported_seq":21,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458352+0000","last_change":"2024-09-17T05:09:31.449828+0000","last_active":"2024-09-17T05:09:33.458352+0000","last_peered":"2024-09-17T05:09:33.458352+0000","last_clean":"2024-09-17T05:09:33.458352+0000","last_became_active":"2024-09-17T05:09:31.449463+0000","last_became_peered":"2024-09-17T05:09:31.449463+0000","last_unstale":"2024-09-17T05:09:33.458352+0000","last_undegraded":"2024-09-17T05:09:33.458352+0000","last_fullsized":"2024-09-17T05:09:33.458352+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:45.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":331,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.437820+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:09:32.437820+0000","last_peered":"2024-09-17T05:09:32.437820+0000","last_clean":"2024-09-17T05:09:32.437820+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:09:32.437820+0000","last_undegraded":"2024-09-17T05:09:32.437820+0000","last_fullsized":"2024-09-17T05:09:32.437820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509547,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1695744,"data_stored":780590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36984,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334856,"statfs":{"total":96636764160,"available":96598892544,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36992,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334848,"statfs":{"total":96636764160,"available":96598884352,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37564,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334276,"statfs":{"total":96636764160,"available":96598298624,"internally_reserved":0,"allocated":2269184,"data_stored":1363175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2314240,"data_stored":1379318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:09:36.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:09:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 -- 172.21.15.16:0/582130395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc0b2f50 msgr2=0x7fcefc073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 --2- 172.21.15.16:0/582130395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc0b2f50 0x7fcefc073950 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009bc0 tx=0x7fcef401ca80 comp rx=0 tx=0).stop 2024-09-17T05:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 -- 172.21.15.16:0/582130395 shutdown_connections 2024-09-17T05:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 --2- 172.21.15.16:0/582130395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc0b2f50 0x7fcefc073950 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 -- 172.21.15.16:0/582130395 >> 172.21.15.16:0/582130395 conn(0x7fcefc06f900 msgr2=0x7fcefc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 -- 172.21.15.16:0/582130395 shutdown_connections 2024-09-17T05:09:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.841+0000 7fcf02000640 1 -- 172.21.15.16:0/582130395 wait complete. 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 Processor -- start 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 -- start start 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcefc074b00 0x7fcefc1535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcefc0b2f50 0x7fcefc13c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 0x7fcefc154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc169b60 con 0x7fcefc14f2d0 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1699e0 con 0x7fcefc0b2f50 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcefc169ce0 con 0x7fcefc074b00 2024-09-17T05:09:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 0x7fcefc154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 0x7fcefc154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36378/0 (socket says 172.21.15.16:36378) 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 -- 172.21.15.16:0/2493661070 learned_addr learned my addr 172.21.15.16:0/2493661070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcef3e00640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcefc0b2f50 0x7fcefc13c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefb400640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcefc074b00 0x7fcefc1535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 -- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcefc074b00 msgr2=0x7fcefc1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcefc074b00 0x7fcefc1535c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 -- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcefc0b2f50 msgr2=0x7fcefc13c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcefc0b2f50 0x7fcefc13c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 -- 172.21.15.16:0/2493661070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc0af8a0 con 0x7fcefc14f2d0 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcef3e00640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcefc0b2f50 0x7fcefc13c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefb400640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcefc074b00 0x7fcefc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcefbe00640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 0x7fcefc154d10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcef401a6b0 tx=0x7fcef40a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef40ba020 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcef4004d40 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc153df0 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef40b2040 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.845+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc1543b0 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.849+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcefc15f080 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.849+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcef4004320 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.849+0000 7fcef3400640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fced8073200 0x7fced8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.849+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fcef412ebb0 con 0x7fcefc14f2d0 2024-09-17T05:09:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.849+0000 7fcefb400640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fced8073200 0x7fced8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:36.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.849+0000 7fcefb400640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fced8073200 0x7fced8075710 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fceec000f70 tx=0x7fceec009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:36.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:36.853+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcef40c4020 con 0x7fcefc14f2d0 2024-09-17T05:09:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.025+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcefc06a7c0 con 0x7fced8073200 2024-09-17T05:09:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.029+0000 7fcef3400640 1 -- 172.21.15.16:0/2493661070 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252468 (secure 0 0 0) 0x7fcefc06a7c0 con 0x7fced8073200 2024-09-17T05:09:37.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:09:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fced8073200 msgr2=0x7fced8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fced8073200 0x7fced8075710 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fceec000f70 tx=0x7fceec009040 comp rx=0 tx=0).stop 2024-09-17T05:09:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 msgr2=0x7fcefc154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 0x7fcefc154d10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcef401a6b0 tx=0x7fcef40a5840 comp rx=0 tx=0).stop 2024-09-17T05:09:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 shutdown_connections 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fced8073200 0x7fced8075710 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcefc14f2d0 0x7fcefc154d10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcefc0b2f50 0x7fcefc13c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 --2- 172.21.15.16:0/2493661070 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcefc074b00 0x7fcefc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 >> 172.21.15.16:0/2493661070 conn(0x7fcefc06f900 msgr2=0x7fcefc15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 shutdown_connections 2024-09-17T05:09:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.033+0000 7fcf02000640 1 -- 172.21.15.16:0/2493661070 wait complete. 2024-09-17T05:09:37.054 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-17T05:09:35.602049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276336,"kb_used_data":14312,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754698384,"statfs":{"total":773094113280,"available":772811145216,"internally_reserved":0,"allocated":14655488,"data_stored":7387118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.185278"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.463890+0000","last_change":"2024-09-17T05:09:32.459554+0000","last_active":"2024-09-17T05:09:32.463890+0000","last_peered":"2024-09-17T05:09:32.463890+0000","last_clean":"2024-09-17T05:09:32.463890+0000","last_became_active":"2024-09-17T05:09:32.459174+0000","last_became_peered":"2024-09-17T05:09:32.459174+0000","last_unstale":"2024-09-17T05:09:32.463890+0000","last_undegraded":"2024-09-17T05:09:32.463890+0000","last_fullsized":"2024-09-17T05:09:32.463890+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:32.194720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471973+0000","last_change":"2024-09-17T05:09:32.466630+0000","last_active":"2024-09-17T05:09:32.471973+0000","last_peered":"2024-09-17T05:09:32.471973+0000","last_clean":"2024-09-17T05:09:32.471973+0000","last_became_active":"2024-09-17T05:09:32.466252+0000","last_became_peered":"2024-09-17T05:09:32.466252+0000","last_unstale":"2024-09-17T05:09:32.471973+0000","last_undegraded":"2024-09-17T05:09:32.471973+0000","last_fullsized":"2024-09-17T05:09:32.471973+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:36.452709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461615+0000","last_change":"2024-09-17T05:09:32.456378+0000","last_active":"2024-09-17T05:09:32.461615+0000","last_peered":"2024-09-17T05:09:32.461615+0000","last_clean":"2024-09-17T05:09:32.461615+0000","last_became_active":"2024-09-17T05:09:32.453939+0000","last_became_peered":"2024-09-17T05:09:32.453939+0000","last_unstale":"2024-09-17T05:09:32.461615+0000","last_undegraded":"2024-09-17T05:09:32.461615+0000","last_fullsized":"2024-09-17T05:09:32.461615+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:49.314414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454670+0000","last_change":"2024-09-17T05:09:32.451234+0000","last_active":"2024-09-17T05:09:32.454670+0000","last_peered":"2024-09-17T05:09:32.454670+0000","last_clean":"2024-09-17T05:09:32.454670+0000","last_became_active":"2024-09-17T05:09:32.450925+0000","last_became_peered":"2024-09-17T05:09:32.450925+0000","last_unstale":"2024-09-17T05:09:32.454670+0000","last_undegraded":"2024-09-17T05:09:32.454670+0000","last_fullsized":"2024-09-17T05:09:32.454670+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:36.938546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471876+0000","last_change":"2024-09-17T05:09:32.468365+0000","last_active":"2024-09-17T05:09:32.471876+0000","last_peered":"2024-09-17T05:09:32.471876+0000","last_clean":"2024-09-17T05:09:32.471876+0000","last_became_active":"2024-09-17T05:09:32.468058+0000","last_became_peered":"2024-09-17T05:09:32.468058+0000","last_unstale":"2024-09-17T05:09:32.471876+0000","last_undegraded":"2024-09-17T05:09:32.471876+0000","last_fullsized":"2024-09-17T05:09:32.471876+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:48.315681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.449029+0000","last_change":"2024-09-17T05:09:32.444055+0000","last_active":"2024-09-17T05:09:32.449029+0000","last_peered":"2024-09-17T05:09:32.449029+0000","last_clean":"2024-09-17T05:09:32.449029+0000","last_became_active":"2024-09-17T05:09:32.443675+0000","last_became_peered":"2024-09-17T05:09:32.443675+0000","last_unstale":"2024-09-17T05:09:32.449029+0000","last_undegraded":"2024-09-17T05:09:32.449029+0000","last_fullsized":"2024-09-17T05:09:32.449029+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:49.588209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454074+0000","last_change":"2024-09-17T05:09:32.449877+0000","last_active":"2024-09-17T05:09:32.454074+0000","last_peered":"2024-09-17T05:09:32.454074+0000","last_clean":"2024-09-17T05:09:32.454074+0000","last_became_active":"2024-09-17T05:09:32.449569+0000","last_became_peered":"2024-09-17T05:09:32.449569+0000","last_unstale":"2024-09-17T05:09:32.454074+0000","last_undegraded":"2024-09-17T05:09:32.454074+0000","last_fullsized":"2024-09-17T05:09:32.454074+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:25.444239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454905+0000","last_change":"2024-09-17T05:09:32.449875+0000","last_active":"2024-09-17T05:09:32.454905+0000","last_peered":"2024-09-17T05:09:32.454905+0000","last_clean":"2024-09-17T05:09:32.454905+0000","last_became_active":"2024-09-17T05:09:32.449574+0000","last_became_peered":"2024-09-17T05:09:32.449574+0000","last_unstale":"2024-09-17T05:09:32.454905+0000","last_undegraded":"2024-09-17T05:09:32.454905+0000","last_fullsized":"2024-09-17T05:09:32.454905+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:48.569088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471958+0000","last_change":"2024-09-17T05:09:32.469198+0000","last_active":"2024-09-17T05:09:32.471958+0000","last_peered":"2024-09-17T05:09:32.471958+0000","last_clean":"2024-09-17T05:09:32.471958+0000","last_became_active":"2024-09-17T05:09:32.466681+0000","last_became_peered":"2024-09-17T05:09:32.466681+0000","last_unstale":"2024-09-17T05:09:32.471958+0000","last_undegraded":"2024-09-17T05:09:32.471958+0000","last_fullsized":"2024-09-17T05:09:32.471958+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:57.395931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.462177+0000","last_change":"2024-09-17T05:09:32.455532+0000","last_active":"2024-09-17T05:09:32.462177+0000","last_peered":"2024-09-17T05:09:32.462177+0000","last_clean":"2024-09-17T05:09:32.462177+0000","last_became_active":"2024-09-17T05:09:32.455208+0000","last_became_peered":"2024-09-17T05:09:32.455208+0000","last_unstale":"2024-09-17T05:09:32.462177+0000","last_undegraded":"2024-09-17T05:09:32.462177+0000","last_fullsized":"2024-09-17T05:09:32.462177+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:51:49.588209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458110+0000","last_change":"2024-09-17T05:09:32.455174+0000","last_active":"2024-09-17T05:09:33.458110+0000","last_peered":"2024-09-17T05:09:33.458110+0000","last_clean":"2024-09-17T05:09:33.458110+0000","last_became_active":"2024-09-17T05:09:32.454777+0000","last_became_peered":"2024-09-17T05:09:32.454777+0000","last_unstale":"2024-09-17T05:09:33.458110+0000","last_undegraded":"2024-09-17T05:09:33.458110+0000","last_fullsized":"2024-09-17T05:09:33.458110+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:49:57.523419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.472315+0000","last_change":"2024-09-17T05:09:32.470815+0000","last_active":"2024-09-17T05:09:32.472315+0000","last_peered":"2024-09-17T05:09:32.472315+0000","last_clean":"2024-09-17T05:09:32.472315+0000","last_became_active":"2024-09-17T05:09:32.470408+0000","last_became_peered":"2024-09-17T05:09:32.470408+0000","last_unstale":"2024-09-17T05:09:32.472315+0000","last_undegraded":"2024-09-17T05:09:32.472315+0000","last_fullsized":"2024-09-17T05:09:32.472315+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:58.233748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.468597+0000","last_change":"2024-09-17T05:09:32.459999+0000","last_active":"2024-09-17T05:09:32.468597+0000","last_peered":"2024-09-17T05:09:32.468597+0000","last_clean":"2024-09-17T05:09:32.468597+0000","last_became_active":"2024-09-17T05:09:32.459672+0000","last_became_peered":"2024-09-17T05:09:32.459672+0000","last_unstale":"2024-09-17T05:09:32.468597+0000","last_undegraded":"2024-09-17T05:09:32.468597+0000","last_fullsized":"2024-09-17T05:09:32.468597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:15.979580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.466109+0000","last_change":"2024-09-17T05:09:32.463346+0000","last_active":"2024-09-17T05:09:32.466109+0000","last_peered":"2024-09-17T05:09:32.466109+0000","last_clean":"2024-09-17T05:09:32.466109+0000","last_became_active":"2024-09-17T05:09:32.462997+0000","last_became_peered":"2024-09-17T05:09:32.462997+0000","last_unstale":"2024-09-17T05:09:32.466109+0000","last_undegraded":"2024-09-17T05:09:32.466109+0000","last_fullsized":"2024-09-17T05:09:32.466109+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:55.099404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457630+0000","last_change":"2024-09-17T05:09:32.460653+0000","last_active":"2024-09-17T05:09:33.457630+0000","last_peered":"2024-09-17T05:09:33.457630+0000","last_clean":"2024-09-17T05:09:33.457630+0000","last_became_active":"2024-09-17T05:09:32.460313+0000","last_became_peered":"2024-09-17T05:09:32.460313+0000","last_unstale":"2024-09-17T05:09:33.457630+0000","last_undegraded":"2024-09-17T05:09:33.457630+0000","last_fullsized":"2024-09-17T05:09:33.457630+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:36:14.951678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457753+0000","last_change":"2024-09-17T05:09:32.469224+0000","last_active":"2024-09-17T05:09:33.457753+0000","last_peered":"2024-09-17T05:09:33.457753+0000","last_clean":"2024-09-17T05:09:33.457753+0000","last_became_active":"2024-09-17T05:09:32.468901+0000","last_became_peered":"2024-09-17T05:09:32.468901+0000","last_unstale":"2024-09-17T05:09:33.457753+0000","last_undegraded":"2024-09-17T05:09:33.457753+0000","last_fullsized":"2024-09-17T05:09:33.457753+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:59.352039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469901+0000","last_change":"2024-09-17T05:09:32.465545+0000","last_active":"2024-09-17T05:09:32.469901+0000","last_peered":"2024-09-17T05:09:32.469901+0000","last_clean":"2024-09-17T05:09:32.469901+0000","last_became_active":"2024-09-17T05:09:32.465231+0000","last_became_peered":"2024-09-17T05:09:32.465231+0000","last_unstale":"2024-09-17T05:09:32.469901+0000","last_undegraded":"2024-09-17T05:09:32.469901+0000","last_fullsized":"2024-09-17T05:09:32.469901+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:18.950422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.470645+0000","last_change":"2024-09-17T05:09:32.464472+0000","last_active":"2024-09-17T05:09:32.470645+0000","last_peered":"2024-09-17T05:09:32.470645+0000","last_clean":"2024-09-17T05:09:32.470645+0000","last_became_active":"2024-09-17T05:09:32.464124+0000","last_became_peered":"2024-09-17T05:09:32.464124+0000","last_unstale":"2024-09-17T05:09:32.470645+0000","last_undegraded":"2024-09-17T05:09:32.470645+0000","last_fullsized":"2024-09-17T05:09:32.470645+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:41.186636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455851+0000","last_change":"2024-09-17T05:09:32.450170+0000","last_active":"2024-09-17T05:09:32.455851+0000","last_peered":"2024-09-17T05:09:32.455851+0000","last_clean":"2024-09-17T05:09:32.455851+0000","last_became_active":"2024-09-17T05:09:32.449562+0000","last_became_peered":"2024-09-17T05:09:32.449562+0000","last_unstale":"2024-09-17T05:09:32.455851+0000","last_undegraded":"2024-09-17T05:09:32.455851+0000","last_fullsized":"2024-09-17T05:09:32.455851+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:01.783185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457972+0000","last_change":"2024-09-17T05:09:32.461392+0000","last_active":"2024-09-17T05:09:33.457972+0000","last_peered":"2024-09-17T05:09:33.457972+0000","last_clean":"2024-09-17T05:09:33.457972+0000","last_became_active":"2024-09-17T05:09:32.460956+0000","last_became_peered":"2024-09-17T05:09:32.460956+0000","last_unstale":"2024-09-17T05:09:33.457972+0000","last_undegraded":"2024-09-17T05:09:33.457972+0000","last_fullsized":"2024-09-17T05:09:33.457972+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:26.680267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458127+0000","last_change":"2024-09-17T05:09:32.460007+0000","last_active":"2024-09-17T05:09:33.458127+0000","last_peered":"2024-09-17T05:09:33.458127+0000","last_clean":"2024-09-17T05:09:33.458127+0000","last_became_active":"2024-09-17T05:09:32.459679+0000","last_became_peered":"2024-09-17T05:09:32.459679+0000","last_unstale":"2024-09-17T05:09:33.458127+0000","last_undegraded":"2024-09-17T05:09:33.458127+0000","last_fullsized":"2024-09-17T05:09:33.458127+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:32.819161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.458419+0000","last_change":"2024-09-17T05:09:32.455151+0000","last_active":"2024-09-17T05:09:32.458419+0000","last_peered":"2024-09-17T05:09:32.458419+0000","last_clean":"2024-09-17T05:09:32.458419+0000","last_became_active":"2024-09-17T05:09:32.454832+0000","last_became_peered":"2024-09-17T05:09:32.454832+0000","last_unstale":"2024-09-17T05:09:32.458419+0000","last_undegraded":"2024-09-17T05:09:32.458419+0000","last_fullsized":"2024-09-17T05:09:32.458419+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:03.750851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.460257+0000","last_change":"2024-09-17T05:09:32.458627+0000","last_active":"2024-09-17T05:09:32.460257+0000","last_peered":"2024-09-17T05:09:32.460257+0000","last_clean":"2024-09-17T05:09:32.460257+0000","last_became_active":"2024-09-17T05:09:32.458294+0000","last_became_peered":"2024-09-17T05:09:32.458294+0000","last_unstale":"2024-09-17T05:09:32.460257+0000","last_undegraded":"2024-09-17T05:09:32.460257+0000","last_fullsized":"2024-09-17T05:09:32.460257+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:55.206745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.466825+0000","last_change":"2024-09-17T05:09:32.459107+0000","last_active":"2024-09-17T05:09:32.466825+0000","last_peered":"2024-09-17T05:09:32.466825+0000","last_clean":"2024-09-17T05:09:32.466825+0000","last_became_active":"2024-09-17T05:09:32.458688+0000","last_became_peered":"2024-09-17T05:09:32.458688+0000","last_unstale":"2024-09-17T05:09:32.466825+0000","last_undegraded":"2024-09-17T05:09:32.466825+0000","last_fullsized":"2024-09-17T05:09:32.466825+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:22.411822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.450093+0000","last_change":"2024-09-17T05:09:32.446846+0000","last_active":"2024-09-17T05:09:32.450093+0000","last_peered":"2024-09-17T05:09:32.450093+0000","last_clean":"2024-09-17T05:09:32.450093+0000","last_became_active":"2024-09-17T05:09:32.446531+0000","last_became_peered":"2024-09-17T05:09:32.446531+0000","last_unstale":"2024-09-17T05:09:32.450093+0000","last_undegraded":"2024-09-17T05:09:32.450093+0000","last_fullsized":"2024-09-17T05:09:32.450093+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:33.775785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.456907+0000","last_change":"2024-09-17T05:09:32.455249+0000","last_active":"2024-09-17T05:09:32.456907+0000","last_peered":"2024-09-17T05:09:32.456907+0000","last_clean":"2024-09-17T05:09:32.456907+0000","last_became_active":"2024-09-17T05:09:32.454941+0000","last_became_peered":"2024-09-17T05:09:32.454941+0000","last_unstale":"2024-09-17T05:09:32.456907+0000","last_undegraded":"2024-09-17T05:09:32.456907+0000","last_fullsized":"2024-09-17T05:09:32.456907+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:06.317385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467318+0000","last_change":"2024-09-17T05:09:32.461732+0000","last_active":"2024-09-17T05:09:32.467318+0000","last_peered":"2024-09-17T05:09:32.467318+0000","last_clean":"2024-09-17T05:09:32.467318+0000","last_became_active":"2024-09-17T05:09:32.457414+0000","last_became_peered":"2024-09-17T05:09:32.457414+0000","last_unstale":"2024-09-17T05:09:32.467318+0000","last_undegraded":"2024-09-17T05:09:32.467318+0000","last_fullsized":"2024-09-17T05:09:32.467318+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:38:33.775785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469304+0000","last_change":"2024-09-17T05:09:32.460278+0000","last_active":"2024-09-17T05:09:32.469304+0000","last_peered":"2024-09-17T05:09:32.469304+0000","last_clean":"2024-09-17T05:09:32.469304+0000","last_became_active":"2024-09-17T05:09:32.459952+0000","last_became_peered":"2024-09-17T05:09:32.459952+0000","last_unstale":"2024-09-17T05:09:32.469304+0000","last_undegraded":"2024-09-17T05:09:32.469304+0000","last_fullsized":"2024-09-17T05:09:32.469304+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:04:30.041172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455632+0000","last_change":"2024-09-17T05:09:32.447811+0000","last_active":"2024-09-17T05:09:32.455632+0000","last_peered":"2024-09-17T05:09:32.455632+0000","last_clean":"2024-09-17T05:09:32.455632+0000","last_became_active":"2024-09-17T05:09:32.446396+0000","last_became_peered":"2024-09-17T05:09:32.446396+0000","last_unstale":"2024-09-17T05:09:32.455632+0000","last_undegraded":"2024-09-17T05:09:32.455632+0000","last_fullsized":"2024-09-17T05:09:32.455632+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:50.635619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.470382+0000","last_change":"2024-09-17T05:09:32.461757+0000","last_active":"2024-09-17T05:09:32.470382+0000","last_peered":"2024-09-17T05:09:32.470382+0000","last_clean":"2024-09-17T05:09:32.470382+0000","last_became_active":"2024-09-17T05:09:32.457538+0000","last_became_peered":"2024-09-17T05:09:32.457538+0000","last_unstale":"2024-09-17T05:09:32.470382+0000","last_undegraded":"2024-09-17T05:09:32.470382+0000","last_fullsized":"2024-09-17T05:09:32.470382+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:04.289018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457733+0000","last_change":"2024-09-17T05:09:32.461664+0000","last_active":"2024-09-17T05:09:33.457733+0000","last_peered":"2024-09-17T05:09:33.457733+0000","last_clean":"2024-09-17T05:09:33.457733+0000","last_became_active":"2024-09-17T05:09:32.461128+0000","last_became_peered":"2024-09-17T05:09:32.461128+0000","last_unstale":"2024-09-17T05:09:33.457733+0000","last_undegraded":"2024-09-17T05:09:33.457733+0000","last_fullsized":"2024-09-17T05:09:33.457733+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:28.231172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467140+0000","last_change":"2024-09-17T05:09:32.463336+0000","last_active":"2024-09-17T05:09:32.467140+0000","last_peered":"2024-09-17T05:09:32.467140+0000","last_clean":"2024-09-17T05:09:32.467140+0000","last_became_active":"2024-09-17T05:09:32.463014+0000","last_became_peered":"2024-09-17T05:09:32.463014+0000","last_unstale":"2024-09-17T05:09:32.467140+0000","last_undegraded":"2024-09-17T05:09:32.467140+0000","last_fullsized":"2024-09-17T05:09:32.467140+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:43.337496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458274+0000","last_change":"2024-09-17T05:09:32.458522+0000","last_active":"2024-09-17T05:09:33.458274+0000","last_peered":"2024-09-17T05:09:33.458274+0000","last_clean":"2024-09-17T05:09:33.458274+0000","last_became_active":"2024-09-17T05:09:32.458105+0000","last_became_peered":"2024-09-17T05:09:32.458105+0000","last_unstale":"2024-09-17T05:09:33.458274+0000","last_undegraded":"2024-09-17T05:09:33.458274+0000","last_fullsized":"2024-09-17T05:09:33.458274+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:27.892551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"135'4","reported_seq":19,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457749+0000","last_change":"2024-09-17T05:09:31.437545+0000","last_active":"2024-09-17T05:09:33.457749+0000","last_peered":"2024-09-17T05:09:33.457749+0000","last_clean":"2024-09-17T05:09:33.457749+0000","last_became_active":"2024-09-17T05:09:31.437095+0000","last_became_peered":"2024-09-17T05:09:31.437095+0000","last_unstale":"2024-09-17T05:09:33.457749+0000","last_undegraded":"2024-09-17T05:09:33.457749+0000","last_fullsized":"2024-09-17T05:09:33.457749+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:00:41.971283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.472048+0000","last_change":"2024-09-17T05:09:32.470759+0000","last_active":"2024-09-17T05:09:32.472048+0000","last_peered":"2024-09-17T05:09:32.472048+0000","last_clean":"2024-09-17T05:09:32.472048+0000","last_became_active":"2024-09-17T05:09:32.470366+0000","last_became_peered":"2024-09-17T05:09:32.470366+0000","last_unstale":"2024-09-17T05:09:32.472048+0000","last_undegraded":"2024-09-17T05:09:32.472048+0000","last_fullsized":"2024-09-17T05:09:32.472048+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:03.034788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.480928+0000","last_change":"2024-09-17T05:09:31.450928+0000","last_active":"2024-09-17T05:09:32.480928+0000","last_peered":"2024-09-17T05:09:32.480928+0000","last_clean":"2024-09-17T05:09:32.480928+0000","last_became_active":"2024-09-17T05:09:31.450418+0000","last_became_peered":"2024-09-17T05:09:31.450418+0000","last_unstale":"2024-09-17T05:09:32.480928+0000","last_undegraded":"2024-09-17T05:09:32.480928+0000","last_fullsized":"2024-09-17T05:09:32.480928+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:56:52.304829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469861+0000","last_change":"2024-09-17T05:09:32.466182+0000","last_active":"2024-09-17T05:09:32.469861+0000","last_peered":"2024-09-17T05:09:32.469861+0000","last_clean":"2024-09-17T05:09:32.469861+0000","last_became_active":"2024-09-17T05:09:32.465861+0000","last_became_peered":"2024-09-17T05:09:32.465861+0000","last_unstale":"2024-09-17T05:09:32.469861+0000","last_undegraded":"2024-09-17T05:09:32.469861+0000","last_fullsized":"2024-09-17T05:09:32.469861+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:10.599638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.482094+0000","last_change":"2024-09-17T05:09:31.454091+0000","last_active":"2024-09-17T05:09:32.482094+0000","last_peered":"2024-09-17T05:09:32.482094+0000","last_clean":"2024-09-17T05:09:32.482094+0000","last_became_active":"2024-09-17T05:09:31.453764+0000","last_became_peered":"2024-09-17T05:09:31.453764+0000","last_unstale":"2024-09-17T05:09:32.482094+0000","last_undegraded":"2024-09-17T05:09:32.482094+0000","last_fullsized":"2024-09-17T05:09:32.482094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:07:44.684155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.460678+0000","last_change":"2024-09-17T05:09:32.456594+0000","last_active":"2024-09-17T05:09:32.460678+0000","last_peered":"2024-09-17T05:09:32.460678+0000","last_clean":"2024-09-17T05:09:32.460678+0000","last_became_active":"2024-09-17T05:09:32.456255+0000","last_became_peered":"2024-09-17T05:09:32.456255+0000","last_unstale":"2024-09-17T05:09:32.460678+0000","last_undegraded":"2024-09-17T05:09:32.460678+0000","last_fullsized":"2024-09-17T05:09:32.460678+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:17.519917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.477378+0000","last_change":"2024-09-17T05:09:31.454908+0000","last_active":"2024-09-17T05:09:32.477378+0000","last_peered":"2024-09-17T05:09:32.477378+0000","last_clean":"2024-09-17T05:09:32.477378+0000","last_became_active":"2024-09-17T05:09:31.454597+0000","last_became_peered":"2024-09-17T05:09:31.454597+0000","last_unstale":"2024-09-17T05:09:32.477378+0000","last_undegraded":"2024-09-17T05:09:32.477378+0000","last_fullsized":"2024-09-17T05:09:32.477378+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:41.623332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457596+0000","last_change":"2024-09-17T05:09:32.450284+0000","last_active":"2024-09-17T05:09:33.457596+0000","last_peered":"2024-09-17T05:09:33.457596+0000","last_clean":"2024-09-17T05:09:33.457596+0000","last_became_active":"2024-09-17T05:09:32.449934+0000","last_became_peered":"2024-09-17T05:09:32.449934+0000","last_unstale":"2024-09-17T05:09:33.457596+0000","last_undegraded":"2024-09-17T05:09:33.457596+0000","last_fullsized":"2024-09-17T05:09:33.457596+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:00.319736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.481809+0000","last_change":"2024-09-17T05:09:31.460131+0000","last_active":"2024-09-17T05:09:32.481809+0000","last_peered":"2024-09-17T05:09:32.481809+0000","last_clean":"2024-09-17T05:09:32.481809+0000","last_became_active":"2024-09-17T05:09:31.459807+0000","last_became_peered":"2024-09-17T05:09:31.459807+0000","last_unstale":"2024-09-17T05:09:32.481809+0000","last_undegraded":"2024-09-17T05:09:32.481809+0000","last_fullsized":"2024-09-17T05:09:32.481809+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:27.151412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.460949+0000","last_change":"2024-09-17T05:09:32.457002+0000","last_active":"2024-09-17T05:09:32.460949+0000","last_peered":"2024-09-17T05:09:32.460949+0000","last_clean":"2024-09-17T05:09:32.460949+0000","last_became_active":"2024-09-17T05:09:32.456714+0000","last_became_peered":"2024-09-17T05:09:32.456714+0000","last_unstale":"2024-09-17T05:09:32.460949+0000","last_undegraded":"2024-09-17T05:09:32.460949+0000","last_fullsized":"2024-09-17T05:09:32.460949+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:49.108183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.478700+0000","last_change":"2024-09-17T05:09:31.448013+0000","last_active":"2024-09-17T05:09:32.478700+0000","last_peered":"2024-09-17T05:09:32.478700+0000","last_clean":"2024-09-17T05:09:32.478700+0000","last_became_active":"2024-09-17T05:09:31.447712+0000","last_became_peered":"2024-09-17T05:09:31.447712+0000","last_unstale":"2024-09-17T05:09:32.478700+0000","last_undegraded":"2024-09-17T05:09:32.478700+0000","last_fullsized":"2024-09-17T05:09:32.478700+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:02.943997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.452967+0000","last_change":"2024-09-17T05:09:32.451003+0000","last_active":"2024-09-17T05:09:32.452967+0000","last_peered":"2024-09-17T05:09:32.452967+0000","last_clean":"2024-09-17T05:09:32.452967+0000","last_became_active":"2024-09-17T05:09:32.450673+0000","last_became_peered":"2024-09-17T05:09:32.450673+0000","last_unstale":"2024-09-17T05:09:32.452967+0000","last_undegraded":"2024-09-17T05:09:32.452967+0000","last_fullsized":"2024-09-17T05:09:32.452967+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:42.630264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.447958+0000","last_change":"2024-09-17T05:09:31.455527+0000","last_active":"2024-09-17T05:09:32.447958+0000","last_peered":"2024-09-17T05:09:32.447958+0000","last_clean":"2024-09-17T05:09:32.447958+0000","last_became_active":"2024-09-17T05:09:31.449706+0000","last_became_peered":"2024-09-17T05:09:31.449706+0000","last_unstale":"2024-09-17T05:09:32.447958+0000","last_undegraded":"2024-09-17T05:09:32.447958+0000","last_fullsized":"2024-09-17T05:09:32.447958+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:24.848250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.470659+0000","last_change":"2024-09-17T05:09:32.468898+0000","last_active":"2024-09-17T05:09:32.470659+0000","last_peered":"2024-09-17T05:09:32.470659+0000","last_clean":"2024-09-17T05:09:32.470659+0000","last_became_active":"2024-09-17T05:09:32.467892+0000","last_became_peered":"2024-09-17T05:09:32.467892+0000","last_unstale":"2024-09-17T05:09:32.470659+0000","last_undegraded":"2024-09-17T05:09:32.470659+0000","last_fullsized":"2024-09-17T05:09:32.470659+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:11.595839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448653+0000","last_change":"2024-09-17T05:09:31.448830+0000","last_active":"2024-09-17T05:09:32.448653+0000","last_peered":"2024-09-17T05:09:32.448653+0000","last_clean":"2024-09-17T05:09:32.448653+0000","last_became_active":"2024-09-17T05:09:31.447529+0000","last_became_peered":"2024-09-17T05:09:31.447529+0000","last_unstale":"2024-09-17T05:09:32.448653+0000","last_undegraded":"2024-09-17T05:09:32.448653+0000","last_fullsized":"2024-09-17T05:09:32.448653+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:42.448158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461418+0000","last_change":"2024-09-17T05:09:32.458761+0000","last_active":"2024-09-17T05:09:32.461418+0000","last_peered":"2024-09-17T05:09:32.461418+0000","last_clean":"2024-09-17T05:09:32.461418+0000","last_became_active":"2024-09-17T05:09:32.458369+0000","last_became_peered":"2024-09-17T05:09:32.458369+0000","last_unstale":"2024-09-17T05:09:32.461418+0000","last_undegraded":"2024-09-17T05:09:32.461418+0000","last_fullsized":"2024-09-17T05:09:32.461418+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:10:28.158344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461521+0000","last_change":"2024-09-17T05:09:31.450785+0000","last_active":"2024-09-17T05:09:32.461521+0000","last_peered":"2024-09-17T05:09:32.461521+0000","last_clean":"2024-09-17T05:09:32.461521+0000","last_became_active":"2024-09-17T05:09:31.450424+0000","last_became_peered":"2024-09-17T05:09:31.450424+0000","last_unstale":"2024-09-17T05:09:32.461521+0000","last_undegraded":"2024-09-17T05:09:32.461521+0000","last_fullsized":"2024-09-17T05:09:32.461521+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:26:01.909901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458369+0000","last_change":"2024-09-17T05:09:32.460398+0000","last_active":"2024-09-17T05:09:33.458369+0000","last_peered":"2024-09-17T05:09:33.458369+0000","last_clean":"2024-09-17T05:09:33.458369+0000","last_became_active":"2024-09-17T05:09:32.459882+0000","last_became_peered":"2024-09-17T05:09:32.459882+0000","last_unstale":"2024-09-17T05:09:33.458369+0000","last_undegraded":"2024-09-17T05:09:33.458369+0000","last_fullsized":"2024-09-17T05:09:33.458369+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:08.017571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448050+0000","last_change":"2024-09-17T05:09:31.447599+0000","last_active":"2024-09-17T05:09:32.448050+0000","last_peered":"2024-09-17T05:09:32.448050+0000","last_clean":"2024-09-17T05:09:32.448050+0000","last_became_active":"2024-09-17T05:09:31.447249+0000","last_became_peered":"2024-09-17T05:09:31.447249+0000","last_unstale":"2024-09-17T05:09:32.448050+0000","last_undegraded":"2024-09-17T05:09:32.448050+0000","last_fullsized":"2024-09-17T05:09:32.448050+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:33:06.979107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.472884+0000","last_change":"2024-09-17T05:09:32.471747+0000","last_active":"2024-09-17T05:09:32.472884+0000","last_peered":"2024-09-17T05:09:32.472884+0000","last_clean":"2024-09-17T05:09:32.472884+0000","last_became_active":"2024-09-17T05:09:32.470371+0000","last_became_peered":"2024-09-17T05:09:32.470371+0000","last_unstale":"2024-09-17T05:09:32.472884+0000","last_undegraded":"2024-09-17T05:09:32.472884+0000","last_fullsized":"2024-09-17T05:09:32.472884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:13:37.643590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"135'3","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458215+0000","last_change":"2024-09-17T05:09:31.440159+0000","last_active":"2024-09-17T05:09:33.458215+0000","last_peered":"2024-09-17T05:09:33.458215+0000","last_clean":"2024-09-17T05:09:33.458215+0000","last_became_active":"2024-09-17T05:09:31.439748+0000","last_became_peered":"2024-09-17T05:09:31.439748+0000","last_unstale":"2024-09-17T05:09:33.458215+0000","last_undegraded":"2024-09-17T05:09:33.458215+0000","last_fullsized":"2024-09-17T05:09:33.458215+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.879610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.457385+0000","last_change":"2024-09-17T05:09:32.451818+0000","last_active":"2024-09-17T05:09:32.457385+0000","last_peered":"2024-09-17T05:09:32.457385+0000","last_clean":"2024-09-17T05:09:32.457385+0000","last_became_active":"2024-09-17T05:09:32.451475+0000","last_became_peered":"2024-09-17T05:09:32.451475+0000","last_unstale":"2024-09-17T05:09:32.457385+0000","last_undegraded":"2024-09-17T05:09:32.457385+0000","last_fullsized":"2024-09-17T05:09:32.457385+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:36.986054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.480613+0000","last_change":"2024-09-17T05:09:31.445981+0000","last_active":"2024-09-17T05:09:32.480613+0000","last_peered":"2024-09-17T05:09:32.480613+0000","last_clean":"2024-09-17T05:09:32.480613+0000","last_became_active":"2024-09-17T05:09:31.445647+0000","last_became_peered":"2024-09-17T05:09:31.445647+0000","last_unstale":"2024-09-17T05:09:32.480613+0000","last_undegraded":"2024-09-17T05:09:32.480613+0000","last_fullsized":"2024-09-17T05:09:32.480613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:29.465023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467360+0000","last_change":"2024-09-17T05:09:32.463019+0000","last_active":"2024-09-17T05:09:32.467360+0000","last_peered":"2024-09-17T05:09:32.467360+0000","last_clean":"2024-09-17T05:09:32.467360+0000","last_became_active":"2024-09-17T05:09:32.458897+0000","last_became_peered":"2024-09-17T05:09:32.458897+0000","last_unstale":"2024-09-17T05:09:32.467360+0000","last_undegraded":"2024-09-17T05:09:32.467360+0000","last_fullsized":"2024-09-17T05:09:32.467360+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:01.783185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.447977+0000","last_change":"2024-09-17T05:09:31.458968+0000","last_active":"2024-09-17T05:09:32.447977+0000","last_peered":"2024-09-17T05:09:32.447977+0000","last_clean":"2024-09-17T05:09:32.447977+0000","last_became_active":"2024-09-17T05:09:31.458583+0000","last_became_peered":"2024-09-17T05:09:31.458583+0000","last_unstale":"2024-09-17T05:09:32.447977+0000","last_undegraded":"2024-09-17T05:09:32.447977+0000","last_fullsized":"2024-09-17T05:09:32.447977+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:28.957448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469424+0000","last_change":"2024-09-17T05:09:32.462939+0000","last_active":"2024-09-17T05:09:32.469424+0000","last_peered":"2024-09-17T05:09:32.469424+0000","last_clean":"2024-09-17T05:09:32.469424+0000","last_became_active":"2024-09-17T05:09:32.459569+0000","last_became_peered":"2024-09-17T05:09:32.459569+0000","last_unstale":"2024-09-17T05:09:32.469424+0000","last_undegraded":"2024-09-17T05:09:32.469424+0000","last_fullsized":"2024-09-17T05:09:32.469424+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:32.399711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448369+0000","last_change":"2024-09-17T05:09:31.447239+0000","last_active":"2024-09-17T05:09:32.448369+0000","last_peered":"2024-09-17T05:09:32.448369+0000","last_clean":"2024-09-17T05:09:32.448369+0000","last_became_active":"2024-09-17T05:09:31.446850+0000","last_became_peered":"2024-09-17T05:09:31.446850+0000","last_unstale":"2024-09-17T05:09:32.448369+0000","last_undegraded":"2024-09-17T05:09:32.448369+0000","last_fullsized":"2024-09-17T05:09:32.448369+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:10:21.178937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.463610+0000","last_change":"2024-09-17T05:09:32.456766+0000","last_active":"2024-09-17T05:09:32.463610+0000","last_peered":"2024-09-17T05:09:32.463610+0000","last_clean":"2024-09-17T05:09:32.463610+0000","last_became_active":"2024-09-17T05:09:32.456453+0000","last_became_peered":"2024-09-17T05:09:32.456453+0000","last_unstale":"2024-09-17T05:09:32.463610+0000","last_undegraded":"2024-09-17T05:09:32.463610+0000","last_fullsized":"2024-09-17T05:09:32.463610+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:22.411822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"135'8","reported_seq":21,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.484068+0000","last_change":"2024-09-17T05:09:31.454775+0000","last_active":"2024-09-17T05:09:32.484068+0000","last_peered":"2024-09-17T05:09:32.484068+0000","last_clean":"2024-09-17T05:09:32.484068+0000","last_became_active":"2024-09-17T05:09:31.454471+0000","last_became_peered":"2024-09-17T05:09:31.454471+0000","last_unstale":"2024-09-17T05:09:32.484068+0000","last_undegraded":"2024-09-17T05:09:32.484068+0000","last_fullsized":"2024-09-17T05:09:32.484068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:33.992646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.451840+0000","last_change":"2024-09-17T05:09:32.448684+0000","last_active":"2024-09-17T05:09:32.451840+0000","last_peered":"2024-09-17T05:09:32.451840+0000","last_clean":"2024-09-17T05:09:32.451840+0000","last_became_active":"2024-09-17T05:09:32.448382+0000","last_became_peered":"2024-09-17T05:09:32.448382+0000","last_unstale":"2024-09-17T05:09:32.451840+0000","last_undegraded":"2024-09-17T05:09:32.451840+0000","last_fullsized":"2024-09-17T05:09:32.451840+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:04.289018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.479423+0000","last_change":"2024-09-17T05:09:31.449297+0000","last_active":"2024-09-17T05:09:32.479423+0000","last_peered":"2024-09-17T05:09:32.479423+0000","last_clean":"2024-09-17T05:09:32.479423+0000","last_became_active":"2024-09-17T05:09:31.447500+0000","last_became_peered":"2024-09-17T05:09:31.447500+0000","last_unstale":"2024-09-17T05:09:32.479423+0000","last_undegraded":"2024-09-17T05:09:32.479423+0000","last_fullsized":"2024-09-17T05:09:32.479423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:01.980845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.468353+0000","last_change":"2024-09-17T05:09:32.461007+0000","last_active":"2024-09-17T05:09:32.468353+0000","last_peered":"2024-09-17T05:09:32.468353+0000","last_clean":"2024-09-17T05:09:32.468353+0000","last_became_active":"2024-09-17T05:09:32.460012+0000","last_became_peered":"2024-09-17T05:09:32.460012+0000","last_unstale":"2024-09-17T05:09:32.468353+0000","last_undegraded":"2024-09-17T05:09:32.468353+0000","last_fullsized":"2024-09-17T05:09:32.468353+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:02.182467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.482598+0000","last_change":"2024-09-17T05:09:31.451787+0000","last_active":"2024-09-17T05:09:32.482598+0000","last_peered":"2024-09-17T05:09:32.482598+0000","last_clean":"2024-09-17T05:09:32.482598+0000","last_became_active":"2024-09-17T05:09:31.451435+0000","last_became_peered":"2024-09-17T05:09:31.451435+0000","last_unstale":"2024-09-17T05:09:32.482598+0000","last_undegraded":"2024-09-17T05:09:32.482598+0000","last_fullsized":"2024-09-17T05:09:32.482598+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:58.085012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.469829+0000","last_change":"2024-09-17T05:09:32.465410+0000","last_active":"2024-09-17T05:09:32.469829+0000","last_peered":"2024-09-17T05:09:32.469829+0000","last_clean":"2024-09-17T05:09:32.469829+0000","last_became_active":"2024-09-17T05:09:32.465095+0000","last_became_peered":"2024-09-17T05:09:32.465095+0000","last_unstale":"2024-09-17T05:09:32.469829+0000","last_undegraded":"2024-09-17T05:09:32.469829+0000","last_fullsized":"2024-09-17T05:09:32.469829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:12.572279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"135'1","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.475027+0000","last_change":"2024-09-17T05:09:31.447005+0000","last_active":"2024-09-17T05:09:32.475027+0000","last_peered":"2024-09-17T05:09:32.475027+0000","last_clean":"2024-09-17T05:09:32.475027+0000","last_became_active":"2024-09-17T05:09:31.446621+0000","last_became_peered":"2024-09-17T05:09:31.446621+0000","last_unstale":"2024-09-17T05:09:32.475027+0000","last_undegraded":"2024-09-17T05:09:32.475027+0000","last_fullsized":"2024-09-17T05:09:32.475027+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:24.437307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.454680+0000","last_change":"2024-09-17T05:09:32.452255+0000","last_active":"2024-09-17T05:09:32.454680+0000","last_peered":"2024-09-17T05:09:32.454680+0000","last_clean":"2024-09-17T05:09:32.454680+0000","last_became_active":"2024-09-17T05:09:32.451890+0000","last_became_peered":"2024-09-17T05:09:32.451890+0000","last_unstale":"2024-09-17T05:09:32.454680+0000","last_undegraded":"2024-09-17T05:09:32.454680+0000","last_fullsized":"2024-09-17T05:09:32.454680+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:37.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.438128+0000","last_change":"2024-09-17T05:09:31.455533+0000","last_active":"2024-09-17T05:09:32.438128+0000","last_peered":"2024-09-17T05:09:32.438128+0000","last_clean":"2024-09-17T05:09:32.438128+0000","last_became_active":"2024-09-17T05:09:31.455000+0000","last_became_peered":"2024-09-17T05:09:31.455000+0000","last_unstale":"2024-09-17T05:09:32.438128+0000","last_undegraded":"2024-09-17T05:09:32.438128+0000","last_fullsized":"2024-09-17T05:09:32.438128+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:39:34.644350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.473332+0000","last_change":"2024-09-17T05:09:32.471957+0000","last_active":"2024-09-17T05:09:32.473332+0000","last_peered":"2024-09-17T05:09:32.473332+0000","last_clean":"2024-09-17T05:09:32.473332+0000","last_became_active":"2024-09-17T05:09:32.471469+0000","last_became_peered":"2024-09-17T05:09:32.471469+0000","last_unstale":"2024-09-17T05:09:32.473332+0000","last_undegraded":"2024-09-17T05:09:32.473332+0000","last_fullsized":"2024-09-17T05:09:32.473332+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:09.796220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.479630+0000","last_change":"2024-09-17T05:09:31.449205+0000","last_active":"2024-09-17T05:09:32.479630+0000","last_peered":"2024-09-17T05:09:32.479630+0000","last_clean":"2024-09-17T05:09:32.479630+0000","last_became_active":"2024-09-17T05:09:31.448489+0000","last_became_peered":"2024-09-17T05:09:31.448489+0000","last_unstale":"2024-09-17T05:09:32.479630+0000","last_undegraded":"2024-09-17T05:09:32.479630+0000","last_fullsized":"2024-09-17T05:09:32.479630+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:33.407660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.451985+0000","last_change":"2024-09-17T05:09:32.449063+0000","last_active":"2024-09-17T05:09:32.451985+0000","last_peered":"2024-09-17T05:09:32.451985+0000","last_clean":"2024-09-17T05:09:32.451985+0000","last_became_active":"2024-09-17T05:09:32.448648+0000","last_became_peered":"2024-09-17T05:09:32.448648+0000","last_unstale":"2024-09-17T05:09:32.451985+0000","last_undegraded":"2024-09-17T05:09:32.451985+0000","last_fullsized":"2024-09-17T05:09:32.451985+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:49.314414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461739+0000","last_change":"2024-09-17T05:09:31.442155+0000","last_active":"2024-09-17T05:09:32.461739+0000","last_peered":"2024-09-17T05:09:32.461739+0000","last_clean":"2024-09-17T05:09:32.461739+0000","last_became_active":"2024-09-17T05:09:31.441840+0000","last_became_peered":"2024-09-17T05:09:31.441840+0000","last_unstale":"2024-09-17T05:09:32.461739+0000","last_undegraded":"2024-09-17T05:09:32.461739+0000","last_fullsized":"2024-09-17T05:09:32.461739+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:40:42.627880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.463854+0000","last_change":"2024-09-17T05:09:32.458594+0000","last_active":"2024-09-17T05:09:32.463854+0000","last_peered":"2024-09-17T05:09:32.463854+0000","last_clean":"2024-09-17T05:09:32.463854+0000","last_became_active":"2024-09-17T05:09:32.458232+0000","last_became_peered":"2024-09-17T05:09:32.458232+0000","last_unstale":"2024-09-17T05:09:32.463854+0000","last_undegraded":"2024-09-17T05:09:32.463854+0000","last_fullsized":"2024-09-17T05:09:32.463854+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:15:16.847253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.448590+0000","last_change":"2024-09-17T05:09:31.448324+0000","last_active":"2024-09-17T05:09:32.448590+0000","last_peered":"2024-09-17T05:09:32.448590+0000","last_clean":"2024-09-17T05:09:32.448590+0000","last_became_active":"2024-09-17T05:09:31.447136+0000","last_became_peered":"2024-09-17T05:09:31.447136+0000","last_unstale":"2024-09-17T05:09:32.448590+0000","last_undegraded":"2024-09-17T05:09:32.448590+0000","last_fullsized":"2024-09-17T05:09:32.448590+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:08.708278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457600+0000","last_change":"2024-09-17T05:09:32.452984+0000","last_active":"2024-09-17T05:09:33.457600+0000","last_peered":"2024-09-17T05:09:33.457600+0000","last_clean":"2024-09-17T05:09:33.457600+0000","last_became_active":"2024-09-17T05:09:32.452616+0000","last_became_peered":"2024-09-17T05:09:32.452616+0000","last_unstale":"2024-09-17T05:09:33.457600+0000","last_undegraded":"2024-09-17T05:09:33.457600+0000","last_fullsized":"2024-09-17T05:09:33.457600+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:56.310567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.461692+0000","last_change":"2024-09-17T05:09:31.447582+0000","last_active":"2024-09-17T05:09:32.461692+0000","last_peered":"2024-09-17T05:09:32.461692+0000","last_clean":"2024-09-17T05:09:32.461692+0000","last_became_active":"2024-09-17T05:09:31.447206+0000","last_became_peered":"2024-09-17T05:09:31.447206+0000","last_unstale":"2024-09-17T05:09:32.461692+0000","last_undegraded":"2024-09-17T05:09:32.461692+0000","last_fullsized":"2024-09-17T05:09:32.461692+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:07.596266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":13,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458278+0000","last_change":"2024-09-17T05:09:32.450008+0000","last_active":"2024-09-17T05:09:33.458278+0000","last_peered":"2024-09-17T05:09:33.458278+0000","last_clean":"2024-09-17T05:09:33.458278+0000","last_became_active":"2024-09-17T05:09:32.449687+0000","last_became_peered":"2024-09-17T05:09:32.449687+0000","last_unstale":"2024-09-17T05:09:33.458278+0000","last_undegraded":"2024-09-17T05:09:33.458278+0000","last_fullsized":"2024-09-17T05:09:33.458278+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:09:36.057783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"135'2","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.481921+0000","last_change":"2024-09-17T05:09:31.459463+0000","last_active":"2024-09-17T05:09:32.481921+0000","last_peered":"2024-09-17T05:09:32.481921+0000","last_clean":"2024-09-17T05:09:32.481921+0000","last_became_active":"2024-09-17T05:09:31.459167+0000","last_became_peered":"2024-09-17T05:09:31.459167+0000","last_unstale":"2024-09-17T05:09:32.481921+0000","last_undegraded":"2024-09-17T05:09:32.481921+0000","last_fullsized":"2024-09-17T05:09:32.481921+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:03:24.567345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.465977+0000","last_change":"2024-09-17T05:09:32.464067+0000","last_active":"2024-09-17T05:09:32.465977+0000","last_peered":"2024-09-17T05:09:32.465977+0000","last_clean":"2024-09-17T05:09:32.465977+0000","last_became_active":"2024-09-17T05:09:32.463766+0000","last_became_peered":"2024-09-17T05:09:32.463766+0000","last_unstale":"2024-09-17T05:09:32.465977+0000","last_undegraded":"2024-09-17T05:09:32.465977+0000","last_fullsized":"2024-09-17T05:09:32.465977+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:07.316592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.449405+0000","last_change":"2024-09-17T05:09:31.446794+0000","last_active":"2024-09-17T05:09:32.449405+0000","last_peered":"2024-09-17T05:09:32.449405+0000","last_clean":"2024-09-17T05:09:32.449405+0000","last_became_active":"2024-09-17T05:09:31.446459+0000","last_became_peered":"2024-09-17T05:09:31.446459+0000","last_unstale":"2024-09-17T05:09:32.449405+0000","last_undegraded":"2024-09-17T05:09:32.449405+0000","last_fullsized":"2024-09-17T05:09:32.449405+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:43:47.193757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.464104+0000","last_change":"2024-09-17T05:09:32.459109+0000","last_active":"2024-09-17T05:09:32.464104+0000","last_peered":"2024-09-17T05:09:32.464104+0000","last_clean":"2024-09-17T05:09:32.464104+0000","last_became_active":"2024-09-17T05:09:32.457310+0000","last_became_peered":"2024-09-17T05:09:32.457310+0000","last_unstale":"2024-09-17T05:09:32.464104+0000","last_undegraded":"2024-09-17T05:09:32.464104+0000","last_fullsized":"2024-09-17T05:09:32.464104+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:45:42.479724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.438705+0000","last_change":"2024-09-17T05:09:31.447285+0000","last_active":"2024-09-17T05:09:32.438705+0000","last_peered":"2024-09-17T05:09:32.438705+0000","last_clean":"2024-09-17T05:09:32.438705+0000","last_became_active":"2024-09-17T05:09:31.446984+0000","last_became_peered":"2024-09-17T05:09:31.446984+0000","last_unstale":"2024-09-17T05:09:32.438705+0000","last_undegraded":"2024-09-17T05:09:32.438705+0000","last_fullsized":"2024-09-17T05:09:32.438705+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:59:48.726773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471895+0000","last_change":"2024-09-17T05:09:32.468753+0000","last_active":"2024-09-17T05:09:32.471895+0000","last_peered":"2024-09-17T05:09:32.471895+0000","last_clean":"2024-09-17T05:09:32.471895+0000","last_became_active":"2024-09-17T05:09:32.466179+0000","last_became_peered":"2024-09-17T05:09:32.466179+0000","last_unstale":"2024-09-17T05:09:32.471895+0000","last_undegraded":"2024-09-17T05:09:32.471895+0000","last_fullsized":"2024-09-17T05:09:32.471895+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:17:58.022190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.438649+0000","last_change":"2024-09-17T05:09:31.436964+0000","last_active":"2024-09-17T05:09:32.438649+0000","last_peered":"2024-09-17T05:09:32.438649+0000","last_clean":"2024-09-17T05:09:32.438649+0000","last_became_active":"2024-09-17T05:09:31.436618+0000","last_became_peered":"2024-09-17T05:09:31.436618+0000","last_unstale":"2024-09-17T05:09:32.438649+0000","last_undegraded":"2024-09-17T05:09:32.438649+0000","last_fullsized":"2024-09-17T05:09:32.438649+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:28.054297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.467673+0000","last_change":"2024-09-17T05:09:32.464524+0000","last_active":"2024-09-17T05:09:32.467673+0000","last_peered":"2024-09-17T05:09:32.467673+0000","last_clean":"2024-09-17T05:09:32.467673+0000","last_became_active":"2024-09-17T05:09:32.464219+0000","last_became_peered":"2024-09-17T05:09:32.464219+0000","last_unstale":"2024-09-17T05:09:32.467673+0000","last_undegraded":"2024-09-17T05:09:32.467673+0000","last_fullsized":"2024-09-17T05:09:32.467673+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:28.074106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.445388+0000","last_change":"2024-09-17T05:09:31.447897+0000","last_active":"2024-09-17T05:09:32.445388+0000","last_peered":"2024-09-17T05:09:32.445388+0000","last_clean":"2024-09-17T05:09:32.445388+0000","last_became_active":"2024-09-17T05:09:31.447469+0000","last_became_peered":"2024-09-17T05:09:31.447469+0000","last_unstale":"2024-09-17T05:09:32.445388+0000","last_undegraded":"2024-09-17T05:09:32.445388+0000","last_fullsized":"2024-09-17T05:09:32.445388+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:01.175535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.471000+0000","last_change":"2024-09-17T05:09:32.469495+0000","last_active":"2024-09-17T05:09:32.471000+0000","last_peered":"2024-09-17T05:09:32.471000+0000","last_clean":"2024-09-17T05:09:32.471000+0000","last_became_active":"2024-09-17T05:09:32.467144+0000","last_became_peered":"2024-09-17T05:09:32.467144+0000","last_unstale":"2024-09-17T05:09:32.471000+0000","last_undegraded":"2024-09-17T05:09:32.471000+0000","last_fullsized":"2024-09-17T05:09:32.471000+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:43.239306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.457374+0000","last_change":"2024-09-17T05:09:31.448040+0000","last_active":"2024-09-17T05:09:33.457374+0000","last_peered":"2024-09-17T05:09:33.457374+0000","last_clean":"2024-09-17T05:09:33.457374+0000","last_became_active":"2024-09-17T05:09:31.447684+0000","last_became_peered":"2024-09-17T05:09:31.447684+0000","last_unstale":"2024-09-17T05:09:33.457374+0000","last_undegraded":"2024-09-17T05:09:33.457374+0000","last_fullsized":"2024-09-17T05:09:33.457374+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:19.060445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455829+0000","last_change":"2024-09-17T05:09:32.450277+0000","last_active":"2024-09-17T05:09:32.455829+0000","last_peered":"2024-09-17T05:09:32.455829+0000","last_clean":"2024-09-17T05:09:32.455829+0000","last_became_active":"2024-09-17T05:09:32.449952+0000","last_became_peered":"2024-09-17T05:09:32.449952+0000","last_unstale":"2024-09-17T05:09:32.455829+0000","last_undegraded":"2024-09-17T05:09:32.455829+0000","last_fullsized":"2024-09-17T05:09:32.455829+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:30.471955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.445326+0000","last_change":"2024-09-17T05:09:31.435307+0000","last_active":"2024-09-17T05:09:32.445326+0000","last_peered":"2024-09-17T05:09:32.445326+0000","last_clean":"2024-09-17T05:09:32.445326+0000","last_became_active":"2024-09-17T05:09:31.434992+0000","last_became_peered":"2024-09-17T05:09:31.434992+0000","last_unstale":"2024-09-17T05:09:32.445326+0000","last_undegraded":"2024-09-17T05:09:32.445326+0000","last_fullsized":"2024-09-17T05:09:32.445326+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:21.404890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.455598+0000","last_change":"2024-09-17T05:09:32.447512+0000","last_active":"2024-09-17T05:09:32.455598+0000","last_peered":"2024-09-17T05:09:32.455598+0000","last_clean":"2024-09-17T05:09:32.455598+0000","last_became_active":"2024-09-17T05:09:32.445980+0000","last_became_peered":"2024-09-17T05:09:32.445980+0000","last_unstale":"2024-09-17T05:09:32.455598+0000","last_undegraded":"2024-09-17T05:09:32.455598+0000","last_fullsized":"2024-09-17T05:09:32.455598+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:28.065744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.444688+0000","last_change":"2024-09-17T05:09:31.439519+0000","last_active":"2024-09-17T05:09:32.444688+0000","last_peered":"2024-09-17T05:09:32.444688+0000","last_clean":"2024-09-17T05:09:32.444688+0000","last_became_active":"2024-09-17T05:09:31.439187+0000","last_became_peered":"2024-09-17T05:09:31.439187+0000","last_unstale":"2024-09-17T05:09:32.444688+0000","last_undegraded":"2024-09-17T05:09:32.444688+0000","last_fullsized":"2024-09-17T05:09:32.444688+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:54.199813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.458900+0000","last_change":"2024-09-17T05:09:32.454379+0000","last_active":"2024-09-17T05:09:32.458900+0000","last_peered":"2024-09-17T05:09:32.458900+0000","last_clean":"2024-09-17T05:09:32.458900+0000","last_became_active":"2024-09-17T05:09:32.453001+0000","last_became_peered":"2024-09-17T05:09:32.453001+0000","last_unstale":"2024-09-17T05:09:32.458900+0000","last_undegraded":"2024-09-17T05:09:32.458900+0000","last_fullsized":"2024-09-17T05:09:32.458900+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:31.417624+0000","last_clean_scrub_stamp":"2024-09-17T05:09:31.417624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:17:34.999578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"135'6","reported_seq":21,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-17T05:09:33.458352+0000","last_change":"2024-09-17T05:09:31.449828+0000","last_active":"2024-09-17T05:09:33.458352+0000","last_peered":"2024-09-17T05:09:33.458352+0000","last_clean":"2024-09-17T05:09:33.458352+0000","last_became_active":"2024-09-17T05:09:31.449463+0000","last_became_peered":"2024-09-17T05:09:31.449463+0000","last_unstale":"2024-09-17T05:09:33.458352+0000","last_undegraded":"2024-09-17T05:09:33.458352+0000","last_fullsized":"2024-09-17T05:09:33.458352+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:09:30.410692+0000","last_clean_scrub_stamp":"2024-09-17T05:09:30.410692+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:41:45.042614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":331,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-17T05:09:32.437820+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:09:32.437820+0000","last_peered":"2024-09-17T05:09:32.437820+0000","last_clean":"2024-09-17T05:09:32.437820+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:09:32.437820+0000","last_undegraded":"2024-09-17T05:09:32.437820+0000","last_fullsized":"2024-09-17T05:09:32.437820+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509547,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509546,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1695744,"data_stored":780590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36984,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334856,"statfs":{"total":96636764160,"available":96598892544,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36992,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334848,"statfs":{"total":96636764160,"available":96598884352,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37564,"kb_used_data":2216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334276,"statfs":{"total":96636764160,"available":96598298624,"internally_reserved":0,"allocated":2269184,"data_stored":1363175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":772807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2314240,"data_stored":1379318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:09:37.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:09:37.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:09:37.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 -- 172.21.15.16:0/3736386720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 msgr2=0x7f04c81600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 --2- 172.21.15.16:0/3736386720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c81600e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f04c80b8340 tx=0x7f04b801c830 comp rx=0 tx=0).stop 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 -- 172.21.15.16:0/3736386720 shutdown_connections 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 --2- 172.21.15.16:0/3736386720 >> v2:172.21.15.104:3300/0 conn(0x7f04c815caf0 0x7f04c815cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 --2- 172.21.15.16:0/3736386720 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04c80d1ce0 0x7f04c8160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 --2- 172.21.15.16:0/3736386720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c81600e0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 -- 172.21.15.16:0/3736386720 >> 172.21.15.16:0/3736386720 conn(0x7f04c80cecc0 msgr2=0x7f04c80cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 -- 172.21.15.16:0/3736386720 shutdown_connections 2024-09-17T05:09:37.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 -- 172.21.15.16:0/3736386720 wait complete. 2024-09-17T05:09:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 Processor -- start 2024-09-17T05:09:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.261+0000 7f04cce00640 1 -- start start 2024-09-17T05:09:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c8071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04c815caf0 0x7f04c80700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04c8154280 0x7f04c80b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04c8169270 con 0x7f04c815a300 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04c80b28d0 con 0x7f04c8154280 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f04c81693f0 con 0x7f04c815caf0 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c8071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c8071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45038/0 (socket says 172.21.15.16:45038) 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 -- 172.21.15.16:0/2408630861 learned_addr learned my addr 172.21.15.16:0/2408630861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c6800640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04c8154280 0x7f04c80b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04bfe00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04c815caf0 0x7f04c80700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 -- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04c815caf0 msgr2=0x7f04c80700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04c815caf0 0x7f04c80700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 -- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04c8154280 msgr2=0x7f04c80b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04c8154280 0x7f04c80b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 -- 172.21.15.16:0/2408630861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c806f380 con 0x7f04c815a300 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04c5e00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c8071b40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f04b80b0040 tx=0x7f04b80a67e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b80bc070 con 0x7f04c815a300 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b8004410 con 0x7f04c815a300 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c806e9b0 con 0x7f04c815a300 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8005290 con 0x7f04c815a300 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c806f8d0 con 0x7f04c815a300 2024-09-17T05:09:37.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.265+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c80d1ce0 con 0x7f04c815a300 2024-09-17T05:09:37.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.269+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04b80bf070 con 0x7f04c815a300 2024-09-17T05:09:37.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.269+0000 7f04be000640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04a4073200 0x7f04a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.269+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f04b812e540 con 0x7f04c815a300 2024-09-17T05:09:37.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.269+0000 7f04bfe00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04a4073200 0x7f04a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.269+0000 7f04bfe00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04a4073200 0x7f04a4075710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f04b0005fd0 tx=0x7f04b0005d80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:37.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.273+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f04b80164b0 con 0x7f04c815a300 2024-09-17T05:09:37.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.509+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f04c815ab00 con 0x7f04c815a300 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04be000640 1 -- 172.21.15.16:0/2408630861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) ==== 190+0+147 (secure 0 0 0) 0x7f04b80f9330 con 0x7f04c815a300 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:09:37.513 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:09:37.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04a4073200 msgr2=0x7f04a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04a4073200 0x7f04a4075710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f04b0005fd0 tx=0x7f04b0005d80 comp rx=0 tx=0).stop 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 msgr2=0x7f04c8071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c8071b40 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f04b80b0040 tx=0x7f04b80a67e0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 shutdown_connections 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f04a4073200 0x7f04a4075710 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04c8154280 0x7f04c80b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f04c815caf0 0x7f04c80700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 --2- 172.21.15.16:0/2408630861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c815a300 0x7f04c8071b40 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 >> 172.21.15.16:0/2408630861 conn(0x7f04c80cecc0 msgr2=0x7f04c815f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:37.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 shutdown_connections 2024-09-17T05:09:37.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.513+0000 7f04cce00640 1 -- 172.21.15.16:0/2408630861 wait complete. 2024-09-17T05:09:37.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 -- 172.21.15.16:0/2855575029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 msgr2=0x7fcb6006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 --2- 172.21.15.16:0/2855575029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb6006dc60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009bc0 tx=0x7fcb5801cac0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 -- 172.21.15.16:0/2855575029 shutdown_connections 2024-09-17T05:09:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 --2- 172.21.15.16:0/2855575029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb6006dc60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 --2- 172.21.15.16:0/2855575029 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb6015ba00 0x7fcb6006d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 -- 172.21.15.16:0/2855575029 >> 172.21.15.16:0/2855575029 conn(0x7fcb6006f900 msgr2=0x7fcb600b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 -- 172.21.15.16:0/2855575029 shutdown_connections 2024-09-17T05:09:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.693+0000 7fcb66c00640 1 -- 172.21.15.16:0/2855575029 wait complete. 2024-09-17T05:09:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 Processor -- start 2024-09-17T05:09:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 -- start start 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 0x7fcb60162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb6015ba00 0x7fcb600cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb600cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60167f60 con 0x7fcb600cc550 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60167de0 con 0x7fcb6015be10 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb66c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcb601680e0 con 0x7fcb6015ba00 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 0x7fcb60162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 0x7fcb60162ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45046/0 (socket says 172.21.15.16:45046) 2024-09-17T05:09:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.697+0000 7fcb5fe00640 1 -- 172.21.15.16:0/1754955986 learned_addr learned my addr 172.21.15.16:0/1754955986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5fe00640 1 -- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb6015ba00 msgr2=0x7fcb600cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5fe00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb6015ba00 0x7fcb600cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5fe00640 1 -- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 msgr2=0x7fcb600cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb64c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb600cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5fe00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb600cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5fe00640 1 -- 172.21.15.16:0/1754955986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb600ceec0 con 0x7fcb600cc550 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb64c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb600cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5fe00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 0x7fcb60162ef0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c530 tx=0x7fcb5000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50016020 con 0x7fcb600cc550 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb50012940 con 0x7fcb600cc550 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb600ce620 con 0x7fcb600cc550 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb50012d90 con 0x7fcb600cc550 2024-09-17T05:09:37.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb600cebe0 con 0x7fcb600cc550 2024-09-17T05:09:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6015e810 con 0x7fcb600cc550 2024-09-17T05:09:37.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.701+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb50019020 con 0x7fcb600cc550 2024-09-17T05:09:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.705+0000 7fcb5cc00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073230 0x7fcb3c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.705+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fcb50093e70 con 0x7fcb600cc550 2024-09-17T05:09:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.705+0000 7fcb57400640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073230 0x7fcb3c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.705+0000 7fcb57400640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073230 0x7fcb3c075740 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcb480058d0 tx=0x7fcb48005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.709+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb50020070 con 0x7fcb600cc550 2024-09-17T05:09:37.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb6015ece0 con 0x7fcb600cc550 2024-09-17T05:09:37.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb5cc00640 1 -- 172.21.15.16:0/1754955986 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5233 (secure 0 0 0) 0x7fcb5005ed00 con 0x7fcb600cc550 2024-09-17T05:09:37.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:37.921 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"btime":"2024-09-17T05:09:35:459466+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:34.875103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:37.922 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 98 2024-09-17T05:09:37.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073230 msgr2=0x7fcb3c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073230 0x7fcb3c075740 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcb480058d0 tx=0x7fcb48005860 comp rx=0 tx=0).stop 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 msgr2=0x7fcb60162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 0x7fcb60162ef0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000c530 tx=0x7fcb5000c9f0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 shutdown_connections 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcb3c073230 0x7fcb3c075740 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb6015be10 0x7fcb600cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcb6015ba00 0x7fcb600cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 --2- 172.21.15.16:0/1754955986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb600cc550 0x7fcb60162ef0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 >> 172.21.15.16:0/1754955986 conn(0x7fcb6006f900 msgr2=0x7fcb600b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 shutdown_connections 2024-09-17T05:09:37.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:37.921+0000 7fcb66c00640 1 -- 172.21.15.16:0/1754955986 wait complete. 2024-09-17T05:09:38.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:09:31.960966+0000', 'modified': '2024-09-17T05:09:34.875103+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7648}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7648': {'gid': 7648, 'name': 'b', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6837/1269584978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 1269584978}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 1269584978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7648, 'qdb_cluster': [7648]} 2024-09-17T05:09:38.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:09:38.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T05:09:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 -- 172.21.15.16:0/3271385353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee0074a20 msgr2=0x7fcee015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 --2- 172.21.15.16:0/3271385353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee0074a20 0x7fcee015efe0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fcee00cca90 tx=0x7fced001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 -- 172.21.15.16:0/3271385353 shutdown_connections 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 --2- 172.21.15.16:0/3271385353 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcee015ba50 0x7fcee015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 --2- 172.21.15.16:0/3271385353 >> v2:172.21.15.104:3300/0 conn(0x7fcee0074e30 0x7fcee015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 --2- 172.21.15.16:0/3271385353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee0074a20 0x7fcee015efe0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 -- 172.21.15.16:0/3271385353 >> 172.21.15.16:0/3271385353 conn(0x7fcee00b2250 msgr2=0x7fcee00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 -- 172.21.15.16:0/3271385353 shutdown_connections 2024-09-17T05:09:38.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 -- 172.21.15.16:0/3271385353 wait complete. 2024-09-17T05:09:38.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 Processor -- start 2024-09-17T05:09:38.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.509+0000 7fcee5600640 1 -- start start 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcee5600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 0x7fcee015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcee5600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcee0074e30 0x7fcee0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcee5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee015ba50 0x7fcee0150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcee5600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcee006dbd0 con 0x7fcee015ba50 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcee5600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcee006da50 con 0x7fcee0074a20 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcee5600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcee00ce5d0 con 0x7fcee0074e30 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 0x7fcee015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee015ba50 0x7fcee0150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 0x7fcee015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:37092/0 (socket says 172.21.15.16:37092) 2024-09-17T05:09:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 -- 172.21.15.16:0/2504968531 learned_addr learned my addr 172.21.15.16:0/2504968531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fced7e00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcee0074e30 0x7fcee0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 -- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcee0074e30 msgr2=0x7fcee0152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcee0074e30 0x7fcee0152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 -- 172.21.15.16:0/2504968531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee015ba50 msgr2=0x7fcee0150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee015ba50 0x7fcee0150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 -- 172.21.15.16:0/2504968531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee015d480 con 0x7fcee0074a20 2024-09-17T05:09:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedf400640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee015ba50 0x7fcee0150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:09:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.513+0000 7fcedea00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 0x7fcee015cf40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fced001af70 tx=0x7fced00a64d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.517+0000 7fced6a00640 1 -- 172.21.15.16:0/2504968531 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00ac070 con 0x7fcee0074a20 2024-09-17T05:09:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.517+0000 7fced6a00640 1 -- 172.21.15.16:0/2504968531 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced0004320 con 0x7fcee0074a20 2024-09-17T05:09:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.517+0000 7fced6a00640 1 -- 172.21.15.16:0/2504968531 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0005320 con 0x7fcee0074a20 2024-09-17T05:09:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.517+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee01f9680 con 0x7fcee0074a20 2024-09-17T05:09:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.517+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee01f9c40 con 0x7fcee0074a20 2024-09-17T05:09:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.517+0000 7fced6a00640 1 -- 172.21.15.16:0/2504968531 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fced00af070 con 0x7fcee0074a20 2024-09-17T05:09:38.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced6a00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcebc073200 0x7fcebc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced6a00640 1 -- 172.21.15.16:0/2504968531 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fced012e2d0 con 0x7fcee0074a20 2024-09-17T05:09:38.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcebc073200 0x7fcebc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 Processor -- start 2024-09-17T05:09:38.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 -- start start 2024-09-17T05:09:38.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcebc073200 0x7fcebc075710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcee0153f90 tx=0x7fcec800b260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563881489410 con 0x563881463950 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5638813c6b40 con 0x563881073460 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x5638813b5f70 con 0x56388144cca0 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fcedea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fcedea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:37098/0 (socket says 172.21.15.16:37098) 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fcedea00640 1 -- 172.21.15.16:0/1870931660 learned_addr learned my addr 172.21.15.16:0/1870931660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fcedf400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 -- 172.21.15.16:0/1870931660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 msgr2=0x5638815b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 -- 172.21.15.16:0/1870931660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 msgr2=0x5638815b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.521+0000 7fced7e00640 1 -- 172.21.15.16:0/1870931660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388137d5b0 con 0x563881463950 2024-09-17T05:09:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fced7e00640 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815b4be0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fcec80043f0 tx=0x7fcec8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fceaa7ce2fb702c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fcecf000640 1 -- 172.21.15.16:0/1870931660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8011bc0 con 0x563881463950 2024-09-17T05:09:38.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fcecf000640 1 -- 172.21.15.16:0/1870931660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec8011e50 con 0x563881463950 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fcecf000640 1 -- 172.21.15.16:0/1870931660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec801e620 con 0x563881463950 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 -- 172.21.15.16:0/1870931660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 msgr2=0x5638815b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815b4be0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fcec80043f0 tx=0x7fcec8013000 comp rx=0 tx=0).stop 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 -- 172.21.15.16:0/1870931660 shutdown_connections 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815b4be0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 --2- 172.21.15.16:0/1870931660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 -- 172.21.15.16:0/1870931660 >> 172.21.15.16:0/1870931660 conn(0x563881539d90 msgr2=0x563881569690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 -- 172.21.15.16:0/1870931660 shutdown_connections 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 -- 172.21.15.16:0/1870931660 wait complete. 2024-09-17T05:09:38.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 -- start start 2024-09-17T05:09:38.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.525+0000 7fceece68000 1 Processor -- start 2024-09-17T05:09:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 0x5638815bb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638815bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5638813505f0 con 0x563881073460 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5638813b9d80 con 0x563881463950 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x56388135c520 con 0x56388144cca0 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 10 client.0 _ll_register_callbacks cb 0x563881360830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 0x5638815bb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 0x5638815bb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45078/0 (socket says 172.21.15.16:45078) 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 -- 172.21.15.16:0/3907624372 learned_addr learned my addr 172.21.15.16:0/3907624372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedf400640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638815bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fced7e00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 -- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 msgr2=0x5638815bbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815bbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 -- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 msgr2=0x5638815bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638815bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 -- 172.21.15.16:0/3907624372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563881354190 con 0x563881073460 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fced7e00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcedea00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 0x5638815bb920 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fced0000c10 tx=0x7fced00fd470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1375f01cfdfec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 1 -- 172.21.15.16:0/3907624372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00f8280 con 0x563881073460 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 1 -- 172.21.15.16:0/3907624372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fced0102170 con 0x563881073460 2024-09-17T05:09:38.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56388134e110 con 0x563881073460 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 1 -- 172.21.15.16:0/3907624372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fced00a6740 con 0x563881073460 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 10 client.8253 fetch_fsmap learned FSMap version 98 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x56388143af10 con 0x563881073460 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 1 -- 172.21.15.16:0/3907624372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56388134e110 con 0x563881073460 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 1 -- 172.21.15.16:0/3907624372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 98) ==== 3502+0+0 (secure 0 0 0) 0x7fced00b9070 con 0x563881073460 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 10 client.8253 fetch_fsmap finished waiting for FSMap version 98 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 10 client.8253 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 20 client.8253 populate_metadata read hostname 'smithi016' 2024-09-17T05:09:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x5638810587d0 0x5638815c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 4 client.8253 mds_command: new command op to 7648 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:09:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 --> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638813c8a90 con 0x5638810587d0 2024-09-17T05:09:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fced7e00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x5638810587d0 0x5638815c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fced7e00640 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x5638810587d0 0x5638815c5d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3519d92393c9b0a server_cookie=f217e7ec369a98ae in_seq=0 out_seq=0 2024-09-17T05:09:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.529+0000 7fcec3400640 10 client.8253 ms_handle_connect on v2:172.21.15.104:6836/1269584978 2024-09-17T05:09:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fcec3400640 1 -- 172.21.15.16:0/3907624372 <== mds.0 v2:172.21.15.104:6836/1269584978 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638813c8a90 con 0x5638810587d0 2024-09-17T05:09:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fcec3400640 10 client.8253 handle_command_reply: tid=0 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 client.8253 shutdown 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 msgr2=0x5638815bb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 0x5638815bb920 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fced0000c10 tx=0x7fced00fd470 comp rx=0 tx=0).stop 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 shutdown_connections 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x5638810587d0 0x5638815c5d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638815bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 --2- 172.21.15.16:0/3907624372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881073460 0x5638815bb920 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fcec3400640 0 client.8253 ms_handle_reset on v2:172.21.15.104:6836/1269584978 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 >> 172.21.15.16:0/3907624372 conn(0x563881539d90 msgr2=0x563881570440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 shutdown_connections 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 1 -- 172.21.15.16:0/3907624372 wait complete. 2024-09-17T05:09:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.533+0000 7fceece68000 20 client.8253 trim_cache size 0 max 16384 2024-09-17T05:09:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.549+0000 7fceece68000 1 Processor -- start 2024-09-17T05:09:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.549+0000 7fceece68000 1 -- start start 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x563881073460 0x563881626840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 0x563881626d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638816272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0205be0 con 0x56388144cca0 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0204cd0 con 0x563881463950 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcee01fcda0 con 0x563881073460 2024-09-17T05:09:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 0x563881626d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 0x563881626d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45086/0 (socket says 172.21.15.16:45086) 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 -- 172.21.15.16:0/618785707 learned_addr learned my addr 172.21.15.16:0/618785707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcedf400640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638816272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcedea00640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x563881073460 0x563881626840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 -- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x563881073460 msgr2=0x563881626840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x563881073460 0x563881626840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 -- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 msgr2=0x5638816272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638816272c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 -- 172.21.15.16:0/618785707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563881411db0 con 0x56388144cca0 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcedea00640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x563881073460 0x563881626840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcedf400640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638816272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fced7e00640 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 0x563881626d80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fcec8004300 tx=0x7fcec8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24eea09cc963d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcece600640 1 -- 172.21.15.16:0/618785707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec803b680 con 0x56388144cca0 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcece600640 1 -- 172.21.15.16:0/618785707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec802c040 con 0x56388144cca0 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fcece600640 1 -- 172.21.15.16:0/618785707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec803bbb0 con 0x56388144cca0 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- 172.21.15.16:0/618785707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 msgr2=0x563881626d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 0x563881626d80 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fcec8004300 tx=0x7fcec8004740 comp rx=0 tx=0).stop 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- 172.21.15.16:0/618785707 shutdown_connections 2024-09-17T05:09:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881463950 0x5638816272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56388144cca0 0x563881626d80 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 --2- 172.21.15.16:0/618785707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x563881073460 0x563881626840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- 172.21.15.16:0/618785707 >> 172.21.15.16:0/618785707 conn(0x563881539d90 msgr2=0x563881568cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- 172.21.15.16:0/618785707 shutdown_connections 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.553+0000 7fceece68000 1 -- 172.21.15.16:0/618785707 wait complete. 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 -- start start 2024-09-17T05:09:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 Processor -- start 2024-09-17T05:09:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56388135c520 con 0x563881463950 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0204820 con 0x563881073460 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x5638813505f0 con 0x56388144cca0 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 10 client.0 _ll_register_callbacks cb 0x7fced012c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45092/0 (socket says 172.21.15.16:45092) 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fced7e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 -- 172.21.15.16:0/3617537664 learned_addr learned my addr 172.21.15.16:0/3617537664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 -- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 msgr2=0x5638815e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 -- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 msgr2=0x5638815e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815e5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 -- 172.21.15.16:0/3617537664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388137cab0 con 0x563881463950 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcedf400640 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815e66b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fced8012c10 tx=0x7fced8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3637d6d23f4c3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcec3400640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcec3400640 1 -- 172.21.15.16:0/3617537664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced801d410 con 0x563881463950 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcec3400640 1 -- 172.21.15.16:0/3617537664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fced801d5b0 con 0x563881463950 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec802c040 con 0x563881463950 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcec3400640 1 -- 172.21.15.16:0/3617537664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fced8029050 con 0x563881463950 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 10 client.8259 fetch_fsmap learned FSMap version 98 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=2+}) -- 0x56388135b070 con 0x563881463950 2024-09-17T05:09:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcec3400640 1 -- 172.21.15.16:0/3617537664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec802c040 con 0x563881463950 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fcec3400640 1 -- 172.21.15.16:0/3617537664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 98) ==== 3502+0+0 (secure 0 0 0) 0x7fced802c020 con 0x563881463950 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.557+0000 7fceece68000 10 client.8259 fetch_fsmap finished waiting for FSMap version 98 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 10 client.8259 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 20 client.8259 populate_metadata read hostname 'smithi016' 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x563881046830 0x5638815e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 4 client.8259 mds_command: new command op to 7648 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 --> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] -- command(tid 0: {"prefix": "status"}) -- 0x5638813c4540 con 0x563881046830 2024-09-17T05:09:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcedea00640 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x563881046830 0x5638815e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcec3400640 10 client.8259 ms_handle_connect on v2:172.21.15.104:6836/1269584978 2024-09-17T05:09:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcedea00640 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x563881046830 0x5638815e6bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd573732fc9f6b7 server_cookie=daa9b0bd8a81800f in_seq=0 out_seq=0 2024-09-17T05:09:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcec3400640 1 -- 172.21.15.16:0/3617537664 <== mds.0 v2:172.21.15.104:6836/1269584978 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5638813c4540 con 0x563881046830 2024-09-17T05:09:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcec3400640 10 client.8259 handle_command_reply: tid=0 2024-09-17T05:09:38.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 client.8259 shutdown 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 msgr2=0x5638815e66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815e66b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fced8012c10 tx=0x7fced8011810 comp rx=0 tx=0).stop 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 shutdown_connections 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:6836/1269584978,v1:172.21.15.104:6837/1269584978] conn(0x563881046830 0x5638815e6bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563881463950 0x5638815e66b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x56388144cca0 0x5638815e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 --2- 172.21.15.16:0/3617537664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563881073460 0x5638815e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcec3400640 0 client.8259 ms_handle_reset on v2:172.21.15.104:6836/1269584978 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 >> 172.21.15.16:0/3617537664 conn(0x563881539d90 msgr2=0x563881569690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 shutdown_connections 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 1 -- 172.21.15.16:0/3617537664 wait complete. 2024-09-17T05:09:38.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fceece68000 20 client.8259 trim_cache size 0 max 16384 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 7648, 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1093309250000001, 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 98, 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 136, 2024-09-17T05:09:38.566 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 136, 2024-09-17T05:09:38.567 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.504180960999999 2024-09-17T05:09:38.567 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:09:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcebc073200 msgr2=0x7fcebc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcebc073200 0x7fcebc075710 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcee0153f90 tx=0x7fcec800b260 comp rx=0 tx=0).stop 2024-09-17T05:09:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 msgr2=0x7fcee015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 0x7fcee015cf40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fced001af70 tx=0x7fced00a64d0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 shutdown_connections 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcebc073200 0x7fcebc075710 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee015ba50 0x7fcee0150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcee0074e30 0x7fcee0152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 --2- 172.21.15.16:0/2504968531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcee0074a20 0x7fcee015cf40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 >> 172.21.15.16:0/2504968531 conn(0x7fcee00b2250 msgr2=0x7fcee00af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 shutdown_connections 2024-09-17T05:09:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.561+0000 7fcee5600640 1 -- 172.21.15.16:0/2504968531 wait complete. 2024-09-17T05:09:38.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:09:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 -- 172.21.15.16:0/4131238643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240004810 msgr2=0x7f6240004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 --2- 172.21.15.16:0/4131238643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240004810 0x7f6240004c20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f624400c3c0 tx=0x7f624401cec0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 -- 172.21.15.16:0/4131238643 shutdown_connections 2024-09-17T05:09:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 --2- 172.21.15.16:0/4131238643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240004810 0x7f6240004c20 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 -- 172.21.15.16:0/4131238643 >> 172.21.15.16:0/4131238643 conn(0x7f62580d0860 msgr2=0x7f62580d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 -- 172.21.15.16:0/4131238643 shutdown_connections 2024-09-17T05:09:38.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 -- 172.21.15.16:0/4131238643 wait complete. 2024-09-17T05:09:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 Processor -- start 2024-09-17T05:09:38.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.741+0000 7f6260600640 1 -- start start 2024-09-17T05:09:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6240004810 0x7f6258157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 0x7f6258153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6258155ae0 0x7f6258147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f62580b59f0 con 0x7f6258154bf0 2024-09-17T05:09:38.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f62580b5870 con 0x7f6258155ae0 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f625815f040 con 0x7f6240004810 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 0x7f6258153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 0x7f6258153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45116/0 (socket says 172.21.15.16:45116) 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 -- 172.21.15.16:0/3477526790 learned_addr learned my addr 172.21.15.16:0/3477526790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625dc00640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6240004810 0x7f6258157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625e600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6258155ae0 0x7f6258147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 -- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6240004810 msgr2=0x7f6258157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6240004810 0x7f6258157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 -- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6258155ae0 msgr2=0x7f6258147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6258155ae0 0x7f6258147120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 -- 172.21.15.16:0/3477526790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6258152df0 con 0x7f6258154bf0 2024-09-17T05:09:38.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625e600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6258155ae0 0x7f6258147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:38.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f625d200640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 0x7f6258153e90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f6244098870 tx=0x7f62440a5b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244001260 con 0x7f6258154bf0 2024-09-17T05:09:38.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6244001690 con 0x7f6258154bf0 2024-09-17T05:09:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62581530d0 con 0x7f6258154bf0 2024-09-17T05:09:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62440060d0 con 0x7f6258154bf0 2024-09-17T05:09:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6258153690 con 0x7f6258154bf0 2024-09-17T05:09:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.745+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625815f4a0 con 0x7f6258154bf0 2024-09-17T05:09:38.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.749+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6244006270 con 0x7f6258154bf0 2024-09-17T05:09:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.749+0000 7f6255600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6238073200 0x7f6238075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.749+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f624412dec0 con 0x7f6258154bf0 2024-09-17T05:09:38.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.749+0000 7f625dc00640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6238073200 0x7f6238075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:38.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.749+0000 7f625dc00640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6238073200 0x7f6238075710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f625806bbd0 tx=0x7f624c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:38.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.753+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f625806d2a0 con 0x7f6258154bf0 2024-09-17T05:09:38.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.961+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6258156880 con 0x7f6258154bf0 2024-09-17T05:09:38.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.961+0000 7f6255600640 1 -- 172.21.15.16:0/3477526790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7f62440f8dd0 con 0x7f6258154bf0 2024-09-17T05:09:38.964 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:09:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6238073200 msgr2=0x7f6238075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6238073200 0x7f6238075710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f625806bbd0 tx=0x7f624c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 msgr2=0x7f6258153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 0x7f6258153e90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f6244098870 tx=0x7f62440a5b50 comp rx=0 tx=0).stop 2024-09-17T05:09:38.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 shutdown_connections 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6238073200 0x7f6238075710 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6258155ae0 0x7f6258147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6258154bf0 0x7f6258153e90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 --2- 172.21.15.16:0/3477526790 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6240004810 0x7f6258157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 >> 172.21.15.16:0/3477526790 conn(0x7f62580d0860 msgr2=0x7f6258156d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 shutdown_connections 2024-09-17T05:09:38.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:38.965+0000 7f6260600640 1 -- 172.21.15.16:0/3477526790 wait complete. 2024-09-17T05:09:38.987 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:09:38.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:09:38.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:09:38.987 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:09:38.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:38.988 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:09:38.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:09:38.988 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:09:38.988 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:09:38.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:38.989 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:09:38.998 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:09:38.998 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:09:38.998 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:09:38.998 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:09:38.999 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41914sec preferred_lft 41914sec 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:39.000 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:39.001 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:09:39.002 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:09:39.002 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:39.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:39.003 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:09:39.066 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:09:39.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:39.068 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:09:39.121 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:09:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:09:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:09:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:09:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:09:39.122 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:09:39.122 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:09:39.122 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:09:39.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:09:39.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:09:39.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:09:39.123 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:09:39.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:09:39.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:39.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:39.261 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:09:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:09:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:09:39.270 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:09:39.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:39.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:09:39.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:39.333 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:09:39.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:39.384 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:39.440 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:09:39.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:39.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:39.550 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:39.550 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:09:39.550 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:09:39.568 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:39.568 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:09:39.568 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:09:39.568 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:39.568 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:09:39.578 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:09:39.582 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:09:39.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:39.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:39.603 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:09:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:09:39.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:39.717 INFO:teuthology.orchestra.run.smithi016.stdout:client6887 2024-09-17T05:09:39.717 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:39.717 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:39.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:09:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 -- 172.21.15.16:0/3220188656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40b2f50 msgr2=0x7f6db415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 --2- 172.21.15.16:0/3220188656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40b2f50 0x7f6db415bb60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009840 tx=0x7f6da401c790 comp rx=0 tx=0).stop 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 -- 172.21.15.16:0/3220188656 shutdown_connections 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 --2- 172.21.15.16:0/3220188656 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6db415c0a0 0x7f6db415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 --2- 172.21.15.16:0/3220188656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40b2f50 0x7f6db415bb60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 -- 172.21.15.16:0/3220188656 >> 172.21.15.16:0/3220188656 conn(0x7f6db406f900 msgr2=0x7f6db40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 -- 172.21.15.16:0/3220188656 shutdown_connections 2024-09-17T05:09:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 -- 172.21.15.16:0/3220188656 wait complete. 2024-09-17T05:09:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 Processor -- start 2024-09-17T05:09:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.929+0000 7f6db8a00640 1 -- start start 2024-09-17T05:09:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6db40b2f50 0x7f6db414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 0x7f6db41fc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6db415f150 0x7f6db41f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4169120 con 0x7f6db415c0a0 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4168fa0 con 0x7f6db415f150 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6db41692a0 con 0x7f6db40b2f50 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 0x7f6db41fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 0x7f6db41fc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45140/0 (socket says 172.21.15.16:45140) 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 -- 172.21.15.16:0/1457830127 learned_addr learned my addr 172.21.15.16:0/1457830127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db1e00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6db40b2f50 0x7f6db414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db2800640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6db415f150 0x7f6db41f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 -- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6db40b2f50 msgr2=0x7f6db414d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6db40b2f50 0x7f6db414d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 -- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6db415f150 msgr2=0x7f6db41f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6db415f150 0x7f6db41f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 -- 172.21.15.16:0/1457830127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db414dd80 con 0x7f6db415c0a0 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6dabe00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 0x7f6db41fc670 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00e740 tx=0x7f6d9c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00ccc0 con 0x7f6db415c0a0 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d9c00ce60 con 0x7f6db415c0a0 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c005260 con 0x7f6db415c0a0 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db41fcbb0 con 0x7f6db415c0a0 2024-09-17T05:09:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db41fd310 con 0x7f6db415c0a0 2024-09-17T05:09:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.933+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db41fcd80 con 0x7f6db415c0a0 2024-09-17T05:09:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.937+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d9c005400 con 0x7f6db415c0a0 2024-09-17T05:09:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.937+0000 7f6daa000640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d90073200 0x7f6d90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.937+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f6d9c094c40 con 0x7f6db415c0a0 2024-09-17T05:09:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.937+0000 7f6db1e00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d90073200 0x7f6d90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.937+0000 7f6db1e00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d90073200 0x7f6d90075710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6da40af000 tx=0x7f6da401afd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:39.941+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d9c05f470 con 0x7f6db415c0a0 2024-09-17T05:09:40.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.157+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6db415bb60 con 0x7f6db415c0a0 2024-09-17T05:09:40.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.157+0000 7f6daa000640 1 -- 172.21.15.16:0/1457830127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v98) ==== 53+0+73 (secure 0 0 0) 0x7f6d9c063ea0 con 0x7f6db415c0a0 2024-09-17T05:09:40.158 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d90073200 msgr2=0x7f6d90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d90073200 0x7f6d90075710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6da40af000 tx=0x7f6da401afd0 comp rx=0 tx=0).stop 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 msgr2=0x7f6db41fc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 0x7f6db41fc670 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00e740 tx=0x7f6d9c00ec00 comp rx=0 tx=0).stop 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 shutdown_connections 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d90073200 0x7f6d90075710 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6db415f150 0x7f6db41f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db415c0a0 0x7f6db41fc670 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 --2- 172.21.15.16:0/1457830127 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6db40b2f50 0x7f6db414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 >> 172.21.15.16:0/1457830127 conn(0x7f6db406f900 msgr2=0x7f6db4068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 shutdown_connections 2024-09-17T05:09:40.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:40.161+0000 7f6db8a00640 1 -- 172.21.15.16:0/1457830127 wait complete. 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:09:40.181 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:09:40.182 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:09:40.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.182 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.192 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 41908sec preferred_lft 41908sec 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:09:40.193 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:09:40.194 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:09:40.195 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.195 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:09:40.195 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:09:40.195 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:09:40.195 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:09:40.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.196 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:09:40.257 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:09:40.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.259 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:09:40.312 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:09:40.312 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:09:40.313 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:09:40.314 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:09:40.314 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:09:40.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:09:40.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:09:40.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:09:40.314 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:09:40.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:09:40.452 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:40.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.460 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:09:40.469 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:09:40.469 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:09:40.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:09:40.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.470 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:09:40.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:40.533 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:40.588 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:09:40.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:09:40.589 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:40.640 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:09:40.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:40.741 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:40.741 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:09:40.741 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:09:40.758 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:40.758 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:09:40.759 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:09:40.759 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:40.759 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:09:40.769 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:09:40.772 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:09:40.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:40.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:40.792 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:09:40.882 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:09:40.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:09:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:40.891 INFO:teuthology.orchestra.run.smithi104.stdout:client8277 2024-09-17T05:09:40.892 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:40.892 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:40.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-09-17T05:09:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 -- 172.21.15.16:0/1973523285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 msgr2=0x7f369c0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 --2- 172.21.15.16:0/1973523285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0ce5d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f369c0ce2c0 tx=0x7f368801c8e0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 -- 172.21.15.16:0/1973523285 shutdown_connections 2024-09-17T05:09:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 --2- 172.21.15.16:0/1973523285 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c15f6f0 0x7f369c0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 --2- 172.21.15.16:0/1973523285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0ce5d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 -- 172.21.15.16:0/1973523285 >> 172.21.15.16:0/1973523285 conn(0x7f369c0b2250 msgr2=0x7f369c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 -- 172.21.15.16:0/1973523285 shutdown_connections 2024-09-17T05:09:41.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 -- 172.21.15.16:0/1973523285 wait complete. 2024-09-17T05:09:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 Processor -- start 2024-09-17T05:09:41.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.069+0000 7f36a3e00640 1 -- start start 2024-09-17T05:09:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c074a20 0x7f369c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f369c15f6f0 0x7f369c157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f369c168f30 con 0x7f369c074e30 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f369c168db0 con 0x7f369c15f6f0 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f369c1690b0 con 0x7f369c074a20 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45158/0 (socket says 172.21.15.16:45158) 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 -- 172.21.15.16:0/189394009 learned_addr learned my addr 172.21.15.16:0/189394009 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 -- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c074a20 msgr2=0x7f369c0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a1400640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c074a20 0x7f369c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a1e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f369c15f6f0 0x7f369c157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c074a20 0x7f369c0b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 -- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f369c15f6f0 msgr2=0x7f369c157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f369c15f6f0 0x7f369c157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 -- 172.21.15.16:0/189394009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c158870 con 0x7f369c074e30 2024-09-17T05:09:41.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a1e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f369c15f6f0 0x7f369c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a1400640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c074a20 0x7f369c0b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a0a00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0c5fb0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f36880b0040 tx=0x7f36880a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880b2040 con 0x7f369c074e30 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3688004330 con 0x7f369c074e30 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880052d0 con 0x7f369c074e30 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c158b50 con 0x7f369c074e30 2024-09-17T05:09:41.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c159110 con 0x7f369c074e30 2024-09-17T05:09:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.073+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c15efe0 con 0x7f369c074e30 2024-09-17T05:09:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.077+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3688005470 con 0x7f369c074e30 2024-09-17T05:09:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.077+0000 7f3699600640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.077+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f368812d0e0 con 0x7f369c074e30 2024-09-17T05:09:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.077+0000 7f36a1400640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.077+0000 7f36a1400640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f367c073200 0x7f367c075710 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f369c0c70e0 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:41.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.081+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f36880164d0 con 0x7f369c074e30 2024-09-17T05:09:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.253+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f369c158600 con 0x7f369c074e30 2024-09-17T05:09:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.257+0000 7f3699600640 1 -- 172.21.15.16:0/189394009 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v64)=0 v64) ==== 121+0+0 (secure 0 0 0) 0x7f36880f8000 con 0x7f369c074e30 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f367c073200 msgr2=0x7f367c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f367c073200 0x7f367c075710 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f369c0c70e0 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 msgr2=0x7f369c0c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0c5fb0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f36880b0040 tx=0x7f36880a6f60 comp rx=0 tx=0).stop 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 shutdown_connections 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f367c073200 0x7f367c075710 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f369c15f6f0 0x7f369c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f369c074e30 0x7f369c0c5fb0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 --2- 172.21.15.16:0/189394009 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f369c074a20 0x7f369c0b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 >> 172.21.15.16:0/189394009 conn(0x7f369c0b2250 msgr2=0x7f369c1553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 shutdown_connections 2024-09-17T05:09:41.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.261+0000 7f36a3e00640 1 -- 172.21.15.16:0/189394009 wait complete. 2024-09-17T05:09:41.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-09-17T05:09:41.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 -- 172.21.15.16:0/2696859900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 msgr2=0x7f6d5406e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 --2- 172.21.15.16:0/2696859900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d5406e3b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009c80 tx=0x7f6d4c01cd40 comp rx=0 tx=0).stop 2024-09-17T05:09:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 -- 172.21.15.16:0/2696859900 shutdown_connections 2024-09-17T05:09:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 --2- 172.21.15.16:0/2696859900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d5406e3b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 --2- 172.21.15.16:0/2696859900 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d540744d0 0x7f6d540cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 -- 172.21.15.16:0/2696859900 >> 172.21.15.16:0/2696859900 conn(0x7f6d540b2dd0 msgr2=0x7f6d540b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 -- 172.21.15.16:0/2696859900 shutdown_connections 2024-09-17T05:09:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.441+0000 7f6d59800640 1 -- 172.21.15.16:0/2696859900 wait complete. 2024-09-17T05:09:41.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 Processor -- start 2024-09-17T05:09:41.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 -- start start 2024-09-17T05:09:41.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d540b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d540cce00 0x7f6d5414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5414aba0 0x7f6d541fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54168a60 con 0x7f6d5406dfa0 2024-09-17T05:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d541688e0 con 0x7f6d5414aba0 2024-09-17T05:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d59800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6d54168be0 con 0x7f6d540cce00 2024-09-17T05:09:41.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d540b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d52a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d540b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45180/0 (socket says 172.21.15.16:45180) 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d52a00640 1 -- 172.21.15.16:0/259272493 learned_addr learned my addr 172.21.15.16:0/259272493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d53400640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5414aba0 0x7f6d541fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.445+0000 7f6d4a000640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d540cce00 0x7f6d5414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d52a00640 1 -- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d540cce00 msgr2=0x7f6d5414d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d52a00640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d540cce00 0x7f6d5414d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d52a00640 1 -- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5414aba0 msgr2=0x7f6d541fb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d52a00640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5414aba0 0x7f6d541fb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d52a00640 1 -- 172.21.15.16:0/259272493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5414d910 con 0x7f6d5406dfa0 2024-09-17T05:09:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d53400640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5414aba0 0x7f6d541fb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:09:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d52a00640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d540b52f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c400 tx=0x7f6d4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44017070 con 0x7f6d5406dfa0 2024-09-17T05:09:41.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d44003600 con 0x7f6d5406dfa0 2024-09-17T05:09:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d44003a50 con 0x7f6d5406dfa0 2024-09-17T05:09:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5414dbf0 con 0x7f6d5406dfa0 2024-09-17T05:09:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d541f8430 con 0x7f6d5406dfa0 2024-09-17T05:09:41.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5406e3b0 con 0x7f6d5406dfa0 2024-09-17T05:09:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.449+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6d4401a070 con 0x7f6d5406dfa0 2024-09-17T05:09:41.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.453+0000 7f6d4b400640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d30073200 0x7f6d30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.453+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f6d440936e0 con 0x7f6d5406dfa0 2024-09-17T05:09:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.453+0000 7f6d4a000640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d30073200 0x7f6d30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.453+0000 7f6d4a000640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d30073200 0x7f6d30075710 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6d5414d170 tx=0x7f6d3c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.457+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6d4405de80 con 0x7f6d5406dfa0 2024-09-17T05:09:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.629+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f6d540b5840 con 0x7f6d5406dfa0 2024-09-17T05:09:41.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.633+0000 7f6d4b400640 1 -- 172.21.15.16:0/259272493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v65)=0 v65) ==== 119+0+0 (secure 0 0 0) 0x7f6d440628b0 con 0x7f6d5406dfa0 2024-09-17T05:09:41.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d30073200 msgr2=0x7f6d30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d30073200 0x7f6d30075710 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6d5414d170 tx=0x7f6d3c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 msgr2=0x7f6d540b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d540b52f0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f6d4400c400 tx=0x7f6d4400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 shutdown_connections 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6d30073200 0x7f6d30075710 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5414aba0 0x7f6d541fb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6d540cce00 0x7f6d5414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 --2- 172.21.15.16:0/259272493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5406dfa0 0x7f6d540b52f0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 >> 172.21.15.16:0/259272493 conn(0x7f6d540b2dd0 msgr2=0x7f6d54068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 shutdown_connections 2024-09-17T05:09:41.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.637+0000 7f6d59800640 1 -- 172.21.15.16:0/259272493 wait complete. 2024-09-17T05:09:41.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-09-17T05:09:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 -- 172.21.15.16:0/114789785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 msgr2=0x7f9f6815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/114789785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f6815efe0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9f680ce2c0 tx=0x7f9f5801cda0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 -- 172.21.15.16:0/114789785 shutdown_connections 2024-09-17T05:09:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/114789785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 0x7f9f6815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/114789785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f6815efe0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 -- 172.21.15.16:0/114789785 >> 172.21.15.16:0/114789785 conn(0x7f9f680b2250 msgr2=0x7f9f680d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 -- 172.21.15.16:0/114789785 shutdown_connections 2024-09-17T05:09:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 -- 172.21.15.16:0/114789785 wait complete. 2024-09-17T05:09:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.821+0000 7f9f6ca00640 1 Processor -- start 2024-09-17T05:09:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 -- start start 2024-09-17T05:09:41.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f68165680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f68074e30 0x7f9f681f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 0x7f9f68151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68168f30 con 0x7f9f68074a20 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68168db0 con 0x7f9f68074e30 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9f681690b0 con 0x7f9f6815f520 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f68165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f68165680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45196/0 (socket says 172.21.15.16:45196) 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 -- 172.21.15.16:0/1935202922 learned_addr learned my addr 172.21.15.16:0/1935202922 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f5d400640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f68074e30 0x7f9f681f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f66800640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 0x7f9f68151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 -- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 msgr2=0x7f9f68151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 0x7f9f68151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 -- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f68074e30 msgr2=0x7f9f681f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:41.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f68074e30 0x7f9f681f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:41.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 -- 172.21.15.16:0/1935202922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f680b3060 con 0x7f9f68074a20 2024-09-17T05:09:41.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f66800640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 0x7f9f68151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:41.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f65e00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f68165680 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9f58000f90 tx=0x7f9f580092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580bb070 con 0x7f9f68074a20 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f58004c50 con 0x7f9f68074a20 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f680b3340 con 0x7f9f68074a20 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580b2040 con 0x7f9f68074a20 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.825+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f680b3900 con 0x7f9f68074a20 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.829+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6815efe0 con 0x7f9f68074a20 2024-09-17T05:09:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.829+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f580be070 con 0x7f9f68074a20 2024-09-17T05:09:41.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.829+0000 7f9f5ea00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f44073220 0x7f9f44075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:41.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.829+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f9f5812dda0 con 0x7f9f68074a20 2024-09-17T05:09:41.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.829+0000 7f9f5d400640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f44073220 0x7f9f44075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:41.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.829+0000 7f9f5d400640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f44073220 0x7f9f44075730 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f68150a80 tx=0x7f9f50009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:41.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:41.833+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9f58016990 con 0x7f9f68074a20 2024-09-17T05:09:42.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.009+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f9f68150ab0 con 0x7f9f68074a20 2024-09-17T05:09:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f5ea00640 1 -- 172.21.15.16:0/1935202922 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v66)=0 v66) ==== 128+0+0 (secure 0 0 0) 0x7f9f580f85d0 con 0x7f9f68074a20 2024-09-17T05:09:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f44073220 msgr2=0x7f9f44075730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f44073220 0x7f9f44075730 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9f68150a80 tx=0x7f9f50009040 comp rx=0 tx=0).stop 2024-09-17T05:09:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 msgr2=0x7f9f68165680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f68165680 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9f58000f90 tx=0x7f9f580092c0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 shutdown_connections 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9f44073220 0x7f9f44075730 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9f6815f520 0x7f9f68151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f68074e30 0x7f9f681f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 --2- 172.21.15.16:0/1935202922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f68074a20 0x7f9f68165680 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 >> 172.21.15.16:0/1935202922 conn(0x7f9f680b2250 msgr2=0x7f9f680ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 shutdown_connections 2024-09-17T05:09:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.013+0000 7f9f6ca00640 1 -- 172.21.15.16:0/1935202922 wait complete. 2024-09-17T05:09:42.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-09-17T05:09:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 -- 172.21.15.16:0/1222713739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 msgr2=0x7fb88015aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 --2- 172.21.15.16:0/1222713739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb88015aac0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fb870009900 tx=0x7fb87001c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 -- 172.21.15.16:0/1222713739 shutdown_connections 2024-09-17T05:09:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 --2- 172.21.15.16:0/1222713739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb880074920 0x7fb8800cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 --2- 172.21.15.16:0/1222713739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb88015aac0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 -- 172.21.15.16:0/1222713739 >> 172.21.15.16:0/1222713739 conn(0x7fb8800b2250 msgr2=0x7fb8800d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 -- 172.21.15.16:0/1222713739 shutdown_connections 2024-09-17T05:09:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 -- 172.21.15.16:0/1222713739 wait complete. 2024-09-17T05:09:42.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 Processor -- start 2024-09-17T05:09:42.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.209+0000 7fb887200640 1 -- start start 2024-09-17T05:09:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb880074510 0x7fb880158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb880074920 0x7fb8801fc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb8801fca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb880165d10 con 0x7fb8800ceea0 2024-09-17T05:09:42.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb880165b90 con 0x7fb880074920 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb880165e90 con 0x7fb880074510 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb8801fca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb8801fca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45214/0 (socket says 172.21.15.16:45214) 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 -- 172.21.15.16:0/3036955892 learned_addr learned my addr 172.21.15.16:0/3036955892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb880074510 0x7fb880158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb87f400640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb880074920 0x7fb8801fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 -- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb880074510 msgr2=0x7fb880158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb880074510 0x7fb880158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 -- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb880074920 msgr2=0x7fb8801fc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb880074920 0x7fb8801fc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 -- 172.21.15.16:0/3036955892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8800c6fb0 con 0x7fb8800ceea0 2024-09-17T05:09:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb885200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb8801fca60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb87800e970 tx=0x7fb87800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878004d30 con 0x7fb8800ceea0 2024-09-17T05:09:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb87800cb20 con 0x7fb8800ceea0 2024-09-17T05:09:42.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878005020 con 0x7fb8800ceea0 2024-09-17T05:09:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88015bbc0 con 0x7fb8800ceea0 2024-09-17T05:09:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8800c87f0 con 0x7fb8800ceea0 2024-09-17T05:09:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.213+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8800cb2c0 con 0x7fb8800ceea0 2024-09-17T05:09:42.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.217+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb8780051c0 con 0x7fb8800ceea0 2024-09-17T05:09:42.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.217+0000 7fb877e00640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb860073200 0x7fb860075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.217+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fb878094ff0 con 0x7fb8800ceea0 2024-09-17T05:09:42.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.217+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb860073200 0x7fb860075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.217+0000 7fb87fe00640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb860073200 0x7fb860075710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb8700af000 tx=0x7fb87001b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:42.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.221+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb878016030 con 0x7fb8800ceea0 2024-09-17T05:09:42.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fb8800b1a30 con 0x7fb8800ceea0 2024-09-17T05:09:42.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb877e00640 1 -- 172.21.15.16:0/3036955892 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v66) ==== 106+0+4 (secure 0 0 0) 0x7fb87805f8e0 con 0x7fb8800ceea0 2024-09-17T05:09:42.397 INFO:teuthology.orchestra.run.smithi016.stdout:100 2024-09-17T05:09:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb860073200 msgr2=0x7fb860075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb860073200 0x7fb860075710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb8700af000 tx=0x7fb87001b0f0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 msgr2=0x7fb8801fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb8801fca60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb87800e970 tx=0x7fb87800ee30 comp rx=0 tx=0).stop 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 shutdown_connections 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb860073200 0x7fb860075710 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8800ceea0 0x7fb8801fca60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb880074920 0x7fb8801fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 --2- 172.21.15.16:0/3036955892 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb880074510 0x7fb880158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 >> 172.21.15.16:0/3036955892 conn(0x7fb8800b2250 msgr2=0x7fb880154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 shutdown_connections 2024-09-17T05:09:42.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.397+0000 7fb887200640 1 -- 172.21.15.16:0/3036955892 wait complete. 2024-09-17T05:09:42.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-09-17T05:09:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- 172.21.15.16:0/3852811682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d80cadc0 msgr2=0x7f76d81590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 --2- 172.21.15.16:0/3852811682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d80cadc0 0x7f76d81590a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009840 tx=0x7f76c001c820 comp rx=0 tx=0).stop 2024-09-17T05:09:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- 172.21.15.16:0/3852811682 shutdown_connections 2024-09-17T05:09:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 --2- 172.21.15.16:0/3852811682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d80cadc0 0x7f76d81590a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- 172.21.15.16:0/3852811682 >> 172.21.15.16:0/3852811682 conn(0x7f76d80b2250 msgr2=0x7f76d80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- 172.21.15.16:0/3852811682 shutdown_connections 2024-09-17T05:09:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- 172.21.15.16:0/3852811682 wait complete. 2024-09-17T05:09:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 Processor -- start 2024-09-17T05:09:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- start start 2024-09-17T05:09:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 0x7f76d814d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d80d0940 0x7f76d80c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76d80c6c50 0x7f76d80b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76d816a040 con 0x7f76d8073f30 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76d8169ec0 con 0x7f76d80d0940 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.581+0000 7f76de800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f76d816a1c0 con 0x7f76d80c6c50 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 0x7f76d814d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 0x7f76d814d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45226/0 (socket says 172.21.15.16:45226) 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 -- 172.21.15.16:0/2673000272 learned_addr learned my addr 172.21.15.16:0/2673000272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76cfe00640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d80d0940 0x7f76d80c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7e00640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76d80c6c50 0x7f76d80b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 -- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76d80c6c50 msgr2=0x7f76d80b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76d80c6c50 0x7f76d80b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 -- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d80d0940 msgr2=0x7f76d80c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d80d0940 0x7f76d80c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 -- 172.21.15.16:0/2673000272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d814dde0 con 0x7f76d8073f30 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76d7400640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 0x7f76d814d8a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c450 tx=0x7f76c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011790 con 0x7f76d8073f30 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76c8017070 con 0x7f76d8073f30 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d814eb90 con 0x7f76d8073f30 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c8011ca0 con 0x7f76d8073f30 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d814e5d0 con 0x7f76d8073f30 2024-09-17T05:09:42.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d81590a0 con 0x7f76d8073f30 2024-09-17T05:09:42.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.585+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76c801c460 con 0x7f76d8073f30 2024-09-17T05:09:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.589+0000 7f76cf400640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76b4073200 0x7f76b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.589+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f76c8094150 con 0x7f76d8073f30 2024-09-17T05:09:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.589+0000 7f76cfe00640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76b4073200 0x7f76b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.589+0000 7f76cfe00640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76b4073200 0x7f76b4075710 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f76c00af000 tx=0x7f76c001b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.593+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76c805e8f0 con 0x7f76d8073f30 2024-09-17T05:09:42.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.769+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f76d8159aa0 con 0x7f76d8073f30 2024-09-17T05:09:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.773+0000 7f76cf400640 1 -- 172.21.15.16:0/2673000272 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v67)=0 v67) ==== 124+0+0 (secure 0 0 0) 0x7f76c8063320 con 0x7f76d8073f30 2024-09-17T05:09:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76b4073200 msgr2=0x7f76b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76b4073200 0x7f76b4075710 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f76c00af000 tx=0x7f76c001b060 comp rx=0 tx=0).stop 2024-09-17T05:09:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 msgr2=0x7f76d814d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 0x7f76d814d8a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f76c800c450 tx=0x7f76c800c910 comp rx=0 tx=0).stop 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 shutdown_connections 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f76b4073200 0x7f76b4075710 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f76d80c6c50 0x7f76d80b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76d80d0940 0x7f76d80c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 --2- 172.21.15.16:0/2673000272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76d8073f30 0x7f76d814d8a0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 >> 172.21.15.16:0/2673000272 conn(0x7f76d80b2250 msgr2=0x7f76d815b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 shutdown_connections 2024-09-17T05:09:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.777+0000 7f76de800640 1 -- 172.21.15.16:0/2673000272 wait complete. 2024-09-17T05:09:42.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 -- 172.21.15.16:0/3425101385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e015f520 msgr2=0x7f34e015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 --2- 172.21.15.16:0/3425101385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e015f520 0x7f34e015c160 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f34e0072860 tx=0x7f34d801cd80 comp rx=0 tx=0).stop 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 -- 172.21.15.16:0/3425101385 shutdown_connections 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 --2- 172.21.15.16:0/3425101385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e015f520 0x7f34e015c160 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 --2- 172.21.15.16:0/3425101385 >> v2:172.21.15.104:3300/0 conn(0x7f34e0074a20 0x7f34e015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 -- 172.21.15.16:0/3425101385 >> 172.21.15.16:0/3425101385 conn(0x7f34e00b2250 msgr2=0x7f34e00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 -- 172.21.15.16:0/3425101385 shutdown_connections 2024-09-17T05:09:42.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 -- 172.21.15.16:0/3425101385 wait complete. 2024-09-17T05:09:42.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 Processor -- start 2024-09-17T05:09:42.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 -- start start 2024-09-17T05:09:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.957+0000 7f34e7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 0x7f34e0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0074e30 0x7f34e00ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e015f520 0x7f34e00cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0168e20 con 0x7f34e0074a20 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0168ca0 con 0x7f34e015f520 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f34e0168fa0 con 0x7f34e0074e30 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 0x7f34e0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 0x7f34e0151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45242/0 (socket says 172.21.15.16:45242) 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 -- 172.21.15.16:0/589145598 learned_addr learned my addr 172.21.15.16:0/589145598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e5200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e015f520 0x7f34e00cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34d7e00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0074e30 0x7f34e00ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 -- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0074e30 msgr2=0x7f34e00ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0074e30 0x7f34e00ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 -- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e015f520 msgr2=0x7f34e00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e015f520 0x7f34e00cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 -- 172.21.15.16:0/589145598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e0154190 con 0x7f34e0074a20 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34d7e00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0074e30 0x7f34e00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dfe00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 0x7f34e0151aa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f34d000d750 tx=0x7f34d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0017070 con 0x7f34e0074a20 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34d0009d20 con 0x7f34e0074a20 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e0151fe0 con 0x7f34e0074a20 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0005280 con 0x7f34e0074a20 2024-09-17T05:09:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e0154710 con 0x7f34e0074a20 2024-09-17T05:09:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e015efe0 con 0x7f34e0074a20 2024-09-17T05:09:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.961+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f34d0005420 con 0x7f34e0074a20 2024-09-17T05:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.965+0000 7f34dcc00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34bc073200 0x7f34bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.965+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f34d0094590 con 0x7f34e0074a20 2024-09-17T05:09:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.965+0000 7f34d7e00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34bc073200 0x7f34bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.965+0000 7f34d7e00640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34bc073200 0x7f34bc075710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f34e0155a00 tx=0x7f34c8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:42.969+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34d000de60 con 0x7f34e0074a20 2024-09-17T05:09:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.145+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f34e0165b30 con 0x7f34e0074a20 2024-09-17T05:09:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.145+0000 7f34dcc00640 1 -- 172.21.15.16:0/589145598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v67) ==== 106+0+8 (secure 0 0 0) 0x7f34d005f410 con 0x7f34e0074a20 2024-09-17T05:09:43.148 INFO:teuthology.orchestra.run.smithi016.stdout:1048576 2024-09-17T05:09:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34bc073200 msgr2=0x7f34bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34bc073200 0x7f34bc075710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f34e0155a00 tx=0x7f34c8009040 comp rx=0 tx=0).stop 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 msgr2=0x7f34e0151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 0x7f34e0151aa0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f34d000d750 tx=0x7f34d000dc10 comp rx=0 tx=0).stop 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 shutdown_connections 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f34bc073200 0x7f34bc075710 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e015f520 0x7f34e00cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f34e0074e30 0x7f34e00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 --2- 172.21.15.16:0/589145598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0074a20 0x7f34e0151aa0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 >> 172.21.15.16:0/589145598 conn(0x7f34e00b2250 msgr2=0x7f34e015ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 shutdown_connections 2024-09-17T05:09:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.149+0000 7f34e7200640 1 -- 172.21.15.16:0/589145598 wait complete. 2024-09-17T05:09:43.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 -- 172.21.15.16:0/1328869819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0bc003a10 msgr2=0x7fc0bc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 --2- 172.21.15.16:0/1328869819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0bc003a10 0x7fc0bc003e80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009b40 tx=0x7fc0c801d270 comp rx=0 tx=0).stop 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0d6800640 1 -- 172.21.15.16:0/1328869819 reap_dead start 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 -- 172.21.15.16:0/1328869819 shutdown_connections 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 -- 172.21.15.16:0/1328869819 >> 172.21.15.16:0/1328869819 conn(0x7fc0d80b2250 msgr2=0x7fc0d80d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 -- 172.21.15.16:0/1328869819 shutdown_connections 2024-09-17T05:09:43.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 -- 172.21.15.16:0/1328869819 wait complete. 2024-09-17T05:09:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 Processor -- start 2024-09-17T05:09:43.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.333+0000 7fc0dca00640 1 -- start start 2024-09-17T05:09:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc0bc003a10 0x7fc0d81f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 0x7fc0d8164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0d8164e40 0x7fc0d80b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d8166b20 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d806e820 con 0x7fc0d8164e40 2024-09-17T05:09:43.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d8166ca0 con 0x7fc0bc003a10 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 0x7fc0d8164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 0x7fc0d8164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45254/0 (socket says 172.21.15.16:45254) 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 -- 172.21.15.16:0/2115583143 learned_addr learned my addr 172.21.15.16:0/2115583143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0d5e00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc0bc003a10 0x7fc0d81f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0d6800640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0d8164e40 0x7fc0d80b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 -- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc0bc003a10 msgr2=0x7fc0d81f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc0bc003a10 0x7fc0d81f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 -- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0d8164e40 msgr2=0x7fc0d80b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0d8164e40 0x7fc0d80b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 -- 172.21.15.16:0/2115583143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d8165250 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cd400640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 0x7fc0d8164900 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0011a80 tx=0x7fc0c0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000fe10 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0c0016070 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d80b47e0 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0012780 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d80b4a80 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.337+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d80af2b0 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.341+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc0c0018030 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.341+0000 7fc0cea00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc0b4073230 0x7fc0b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.341+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fc0c0094c50 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.341+0000 7fc0d5e00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc0b4073230 0x7fc0b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.341+0000 7fc0d5e00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc0b4073230 0x7fc0b4075740 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80ad000 tx=0x7fc0c801a010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:43.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.345+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc0d8165250 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7fc0d815eb90 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0cea00640 1 -- 172.21.15.16:0/2115583143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v67) ==== 112+0+10 (secure 0 0 0) 0x7fc0c005fa40 con 0x7fc0d81fcdc0 2024-09-17T05:09:43.522 INFO:teuthology.orchestra.run.smithi016.stdout:60.000000 2024-09-17T05:09:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc0b4073230 msgr2=0x7fc0b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0dca00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc0b4073230 0x7fc0b4075740 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80ad000 tx=0x7fc0c801a010 comp rx=0 tx=0).stop 2024-09-17T05:09:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 msgr2=0x7fc0d8164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0dca00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 0x7fc0d8164900 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0011a80 tx=0x7fc0c0011f40 comp rx=0 tx=0).stop 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.521+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 shutdown_connections 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc0b4073230 0x7fc0b4075740 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0d8164e40 0x7fc0d80b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0d81fcdc0 0x7fc0d8164900 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 --2- 172.21.15.16:0/2115583143 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc0bc003a10 0x7fc0d81f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 >> 172.21.15.16:0/2115583143 conn(0x7fc0d80b2250 msgr2=0x7fc0d815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 shutdown_connections 2024-09-17T05:09:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.525+0000 7fc0dca00640 1 -- 172.21.15.16:0/2115583143 wait complete. 2024-09-17T05:09:43.545 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:43.545 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:43.554 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:09:43.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:09:43.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:43.655 INFO:teuthology.orchestra.run.smithi016.stdout:client6887 2024-09-17T05:09:43.655 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:43.655 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:43.664 DEBUG:teuthology.orchestra.run.smithi016:> stdin-killer -- python3 -c ' 2024-09-17T05:09:43.664 DEBUG:teuthology.orchestra.run.smithi016:> import fcntl 2024-09-17T05:09:43.664 DEBUG:teuthology.orchestra.run.smithi016:> import sys 2024-09-17T05:09:43.664 DEBUG:teuthology.orchestra.run.smithi016:> import time 2024-09-17T05:09:43.664 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> n = 400 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> if not os.path.exists(abs_path): 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> os.makedirs(abs_path) 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> handles = [] 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> for i in range(0, n): 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> fname = "file_"+str(i) 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> path = os.path.join(abs_path, fname) 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> handles.append(open(path, '"'"'w'"'"')) 2024-09-17T05:09:43.665 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> print("waiting with handles open", file=sys.stderr) 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> while True: 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> print("open_n_background: keeping files open", file=sys.stderr) 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> try: 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> if os.read(0, 4096) == b"": 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> break 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> except BlockingIOError: 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> pass 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> time.sleep(2) 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> print("stdin closed, goodbye!", file=sys.stderr) 2024-09-17T05:09:43.666 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-17T05:09:43.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:43.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 -- 172.21.15.16:0/2753257561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b006dfa0 msgr2=0x7f06b006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 --2- 172.21.15.16:0/2753257561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b006dfa0 0x7f06b006e3b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f06b0073890 tx=0x7f069c01c7a0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 -- 172.21.15.16:0/2753257561 shutdown_connections 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 --2- 172.21.15.16:0/2753257561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b006dfa0 0x7f06b006e3b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 --2- 172.21.15.16:0/2753257561 >> v2:172.21.15.104:3300/0 conn(0x7f06b00749d0 0x7f06b0152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 -- 172.21.15.16:0/2753257561 >> 172.21.15.16:0/2753257561 conn(0x7f06b00cdb80 msgr2=0x7f06b00cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.873+0000 7f06b6200640 1 -- 172.21.15.16:0/2753257561 shutdown_connections 2024-09-17T05:09:43.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 -- 172.21.15.16:0/2753257561 wait complete. 2024-09-17T05:09:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 Processor -- start 2024-09-17T05:09:43.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 -- start start 2024-09-17T05:09:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06b006dfa0 0x7f06b01455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 0x7f06b01f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b00cc6f0 0x7f06b00b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06b015c370 con 0x7f06b00cc6f0 2024-09-17T05:09:43.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f06b015c1f0 con 0x7f06b00749d0 2024-09-17T05:09:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.877+0000 7f06b6200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f06b015c4f0 con 0x7f06b006dfa0 2024-09-17T05:09:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b00cc6f0 0x7f06b00b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 0x7f06b01f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06af400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06b006dfa0 0x7f06b01455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 0x7f06b01f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:37244/0 (socket says 172.21.15.16:37244) 2024-09-17T05:09:43.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 -- 172.21.15.16:0/2048926413 learned_addr learned my addr 172.21.15.16:0/2048926413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 -- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06b006dfa0 msgr2=0x7f06b01455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06b006dfa0 0x7f06b01455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 -- 172.21.15.16:0/2048926413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b00cc6f0 msgr2=0x7f06b00b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b00cc6f0 0x7f06b00b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 -- 172.21.15.16:0/2048926413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b0151a20 con 0x7f06b00749d0 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06afe00640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b00cc6f0 0x7f06b00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06aea00640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 0x7f06b01f86d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f069c01cc80 tx=0x7f069c0a5620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c0b2040 con 0x7f06b00749d0 2024-09-17T05:09:43.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f069c0042c0 con 0x7f06b00749d0 2024-09-17T05:09:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c005290 con 0x7f06b00749d0 2024-09-17T05:09:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b01f9280 con 0x7f06b00749d0 2024-09-17T05:09:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b01f9840 con 0x7f06b00749d0 2024-09-17T05:09:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b01f8ff0 con 0x7f06b00749d0 2024-09-17T05:09:43.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.881+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f069c005430 con 0x7f06b00749d0 2024-09-17T05:09:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.885+0000 7f06a7400640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0690073200 0x7f0690075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.885+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f069c12e340 con 0x7f06b00749d0 2024-09-17T05:09:43.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.885+0000 7f06af400640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0690073200 0x7f0690075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:43.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.885+0000 7f06af400640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0690073200 0x7f0690075710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009b40 tx=0x7f06a00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:43.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:43.889+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f069c0c4060 con 0x7f06b00749d0 2024-09-17T05:09:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.121+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06b006e3b0 con 0x7f06b00749d0 2024-09-17T05:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.121+0000 7f06a7400640 1 -- 172.21.15.16:0/2048926413 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+5233 (secure 0 0 0) 0x7f069c0164b0 con 0x7f06b00749d0 2024-09-17T05:09:44.122 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:44.122 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":98,"btime":"2024-09-17T05:09:35:459466+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:34.875103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 98 2024-09-17T05:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.121+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0690073200 msgr2=0x7f0690075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.121+0000 7f06b6200640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0690073200 0x7f0690075710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009b40 tx=0x7f06a00091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 msgr2=0x7f06b01f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 0x7f06b01f86d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f069c01cc80 tx=0x7f069c0a5620 comp rx=0 tx=0).stop 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 shutdown_connections 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0690073200 0x7f0690075710 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b00cc6f0 0x7f06b00b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06b00749d0 0x7f06b01f86d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 --2- 172.21.15.16:0/2048926413 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f06b006dfa0 0x7f06b01455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 >> 172.21.15.16:0/2048926413 conn(0x7f06b00cdb80 msgr2=0x7f06b006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 shutdown_connections 2024-09-17T05:09:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.125+0000 7f06b6200640 1 -- 172.21.15.16:0/2048926413 wait complete. 2024-09-17T05:09:44.146 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-17T05:09:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:waiting with handles open 2024-09-17T05:09:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:44.330 INFO:teuthology.orchestra.run.smithi104.stdout:[{"id":8277,"entity":{"name":{"type":"client","num":8277},"addr":{"type":"v1","addr":"172.21.15.104:0","nonce":1326080318}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":3.551953873,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.989360713008359,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8277 v1:172.21.15.104:0/1326080318","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":6887,"entity":{"name":{"type":"client","num":6887},"addr":{"type":"v1","addr":"172.21.15.16:0","nonce":3729639820}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":424,"uptime":4.7399384060000003,"requests_in_flight":399,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1201.2427210630003,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6887 v1:172.21.15.16:0/3729639820","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:09:44.331 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.104:0", "nonce": 1326080318, "type": "v1" }, "name": { "num": 8277, "type": "client" } }, "id": 8277, "inst": "client.8277 v1:172.21.15.104:0/1326080318", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.989360713008359 }, "state": "open", "uptime": 3.551953873 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 3729639820, "type": "v1" }, "name": { "num": 6887, "type": "client" } }, "id": 6887, "inst": "client.6887 v1:172.21.15.16:0/3729639820", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 424, "requests_in_flight": 399, "session_cache_liveness": { "halflife": 300, "value": 1201.2427210630003 }, "state": "open", "uptime": 4.739938406 } ] 2024-09-17T05:09:44.331 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-09-17T05:09:44.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:09:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 -- 172.21.15.16:0/300494878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0d0830 msgr2=0x7f5b9c070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 --2- 172.21.15.16:0/300494878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0d0830 0x7f5b9c070490 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009840 tx=0x7f5b8801c700 comp rx=0 tx=0).stop 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 -- 172.21.15.16:0/300494878 shutdown_connections 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 --2- 172.21.15.16:0/300494878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0d0830 0x7f5b9c070490 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 --2- 172.21.15.16:0/300494878 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b9c0d1aa0 0x7f5b9c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 -- 172.21.15.16:0/300494878 >> 172.21.15.16:0/300494878 conn(0x7f5b9c0b4160 msgr2=0x7f5b9c0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 -- 172.21.15.16:0/300494878 shutdown_connections 2024-09-17T05:09:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.489+0000 7f5ba2000640 1 -- 172.21.15.16:0/300494878 wait complete. 2024-09-17T05:09:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 Processor -- start 2024-09-17T05:09:44.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 -- start start 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 0x7f5b9c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b9c0d0830 0x7f5b9c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b9c0d1aa0 0x7f5b9c0b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c16a110 con 0x7f5b9c0709d0 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c169f90 con 0x7f5b9c0d1aa0 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c16a290 con 0x7f5b9c0d0830 2024-09-17T05:09:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 0x7f5b9c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 0x7f5b9c14c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45296/0 (socket says 172.21.15.16:45296) 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 -- 172.21.15.16:0/3287094575 learned_addr learned my addr 172.21.15.16:0/3287094575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5b9be00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b9c0d0830 0x7f5b9c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba1600640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b9c0d1aa0 0x7f5b9c0b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 -- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b9c0d0830 msgr2=0x7f5b9c156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b9c0d0830 0x7f5b9c156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 -- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b9c0d1aa0 msgr2=0x7f5b9c0b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b9c0d1aa0 0x7f5b9c0b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 -- 172.21.15.16:0/3287094575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c1fa6d0 con 0x7f5b9c0709d0 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba1600640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b9c0d1aa0 0x7f5b9c0b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba0c00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 0x7f5b9c14c180 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000d750 tx=0x7f5b9000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90017070 con 0x7f5b9c0709d0 2024-09-17T05:09:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b90009d20 con 0x7f5b9c0709d0 2024-09-17T05:09:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90005280 con 0x7f5b9c0709d0 2024-09-17T05:09:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1fa9b0 con 0x7f5b9c0709d0 2024-09-17T05:09:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.493+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c1faf70 con 0x7f5b9c0709d0 2024-09-17T05:09:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.497+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c1fb350 con 0x7f5b9c0709d0 2024-09-17T05:09:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.497+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5b90005420 con 0x7f5b9c0709d0 2024-09-17T05:09:44.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.497+0000 7f5b98c00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b7c073230 0x7f5b7c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.497+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7f5b90094430 con 0x7f5b9c0709d0 2024-09-17T05:09:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.497+0000 7f5b9be00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b7c073230 0x7f5b7c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.497+0000 7f5b9be00640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b7c073230 0x7f5b7c075740 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5b880af000 tx=0x7f5b8801afa0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.501+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5b9c1fb350 con 0x7f5b9c0709d0 2024-09-17T05:09:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5b9c070490 con 0x7f5b9c0709d0 2024-09-17T05:09:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5b98c00640 1 -- 172.21.15.16:0/3287094575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5b9000de60 con 0x7f5b9c0709d0 2024-09-17T05:09:44.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:44.717 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T05:09:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b7c073230 msgr2=0x7f5b7c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b7c073230 0x7f5b7c075740 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5b880af000 tx=0x7f5b8801afa0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 msgr2=0x7f5b9c14c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 0x7f5b9c14c180 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000d750 tx=0x7f5b9000dc10 comp rx=0 tx=0).stop 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 shutdown_connections 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5b7c073230 0x7f5b7c075740 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b9c0d1aa0 0x7f5b9c0b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5b9c0d0830 0x7f5b9c156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 --2- 172.21.15.16:0/3287094575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b9c0709d0 0x7f5b9c14c180 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 >> 172.21.15.16:0/3287094575 conn(0x7f5b9c0b4160 msgr2=0x7f5b9c070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 shutdown_connections 2024-09-17T05:09:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:44.717+0000 7f5ba2000640 1 -- 172.21.15.16:0/3287094575 wait complete. 2024-09-17T05:09:44.740 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-09-17T05:09:44.740 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-09-17T05:09:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:49.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.909+0000 7faddc200640 1 -- 172.21.15.16:0/862113873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4073240 msgr2=0x7fadd4073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.909+0000 7faddc200640 1 --2- 172.21.15.16:0/862113873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4073240 0x7fadd4073650 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009c80 tx=0x7fadcc01cd10 comp rx=0 tx=0).stop 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- 172.21.15.16:0/862113873 shutdown_connections 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 --2- 172.21.15.16:0/862113873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd4073240 0x7fadd4073650 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 --2- 172.21.15.16:0/862113873 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd415c270 0x7fadd415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- 172.21.15.16:0/862113873 >> 172.21.15.16:0/862113873 conn(0x7fadd406f900 msgr2=0x7fadd40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- 172.21.15.16:0/862113873 shutdown_connections 2024-09-17T05:09:49.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- 172.21.15.16:0/862113873 wait complete. 2024-09-17T05:09:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 Processor -- start 2024-09-17T05:09:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- start start 2024-09-17T05:09:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd4073240 0x7fadd4158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 0x7fadd4161a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd415c270 0x7fadd4160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4168e00 con 0x7fadd40b2f50 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.913+0000 7faddc200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4168c80 con 0x7fadd415c270 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7faddc200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fadd4168f80 con 0x7fadd4073240 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 0x7fadd4161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 0x7fadd4161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41770/0 (socket says 172.21.15.16:41770) 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 -- 172.21.15.16:0/2819414777 learned_addr learned my addr 172.21.15.16:0/2819414777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadda200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd415c270 0x7fadd4160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd9800640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd4073240 0x7fadd4158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 -- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd4073240 msgr2=0x7fadd4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd4073240 0x7fadd4158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 -- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd415c270 msgr2=0x7fadd4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd415c270 0x7fadd4160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:49.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 -- 172.21.15.16:0/2819414777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd4158220 con 0x7fadd40b2f50 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd9800640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd4073240 0x7fadd4158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd8e00640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 0x7fadd4161a80 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d680 tx=0x7fadc000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0017070 con 0x7fadd40b2f50 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadc0005310 con 0x7fadd40b2f50 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd4161fc0 con 0x7fadd40b2f50 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0009820 con 0x7fadd40b2f50 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd41584c0 con 0x7fadd40b2f50 2024-09-17T05:09:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd4073650 con 0x7fadd40b2f50 2024-09-17T05:09:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.917+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fadc00099c0 con 0x7fadd40b2f50 2024-09-17T05:09:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.921+0000 7fadd1600640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadb4073230 0x7fadb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.921+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 6101+0+0 (secure 0 0 0) 0x7fadc0094430 con 0x7fadd40b2f50 2024-09-17T05:09:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.921+0000 7fadd9800640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadb4073230 0x7fadb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.921+0000 7fadd9800640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadb4073230 0x7fadb4075740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fadc80016b0 tx=0x7fadc8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:49.925+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fadc005ec60 con 0x7fadd40b2f50 2024-09-17T05:09:50.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fadd4158010 con 0x7fadd40b2f50 2024-09-17T05:09:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7fadd1600640 1 -- 172.21.15.16:0/2819414777 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fadc0063690 con 0x7fadd40b2f50 2024-09-17T05:09:50.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:50.142 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:09:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadb4073230 msgr2=0x7fadb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadb4073230 0x7fadb4075740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fadc80016b0 tx=0x7fadc8009040 comp rx=0 tx=0).stop 2024-09-17T05:09:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 msgr2=0x7fadd4161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 0x7fadd4161a80 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fadc000d680 tx=0x7fadc000db40 comp rx=0 tx=0).stop 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 shutdown_connections 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fadb4073230 0x7fadb4075740 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd415c270 0x7fadd4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd40b2f50 0x7fadd4161a80 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 --2- 172.21.15.16:0/2819414777 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fadd4073240 0x7fadd4158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.141+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 >> 172.21.15.16:0/2819414777 conn(0x7fadd406f900 msgr2=0x7fadd415ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.145+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 shutdown_connections 2024-09-17T05:09:50.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.145+0000 7faddc200640 1 -- 172.21.15.16:0/2819414777 wait complete. 2024-09-17T05:09:50.165 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-17T05:09:50.165 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-09-17T05:09:50.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T05:09:50.316 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 -- 172.21.15.16:0/262094357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 msgr2=0x7fbd4015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 --2- 172.21.15.16:0/262094357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd4015bb60 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009900 tx=0x7fbd3401c850 comp rx=0 tx=0).stop 2024-09-17T05:09:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 -- 172.21.15.16:0/262094357 shutdown_connections 2024-09-17T05:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 --2- 172.21.15.16:0/262094357 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd4015c0a0 0x7fbd4015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 --2- 172.21.15.16:0/262094357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd4015bb60 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 -- 172.21.15.16:0/262094357 >> 172.21.15.16:0/262094357 conn(0x7fbd4006f900 msgr2=0x7fbd400b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 -- 172.21.15.16:0/262094357 shutdown_connections 2024-09-17T05:09:50.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 -- 172.21.15.16:0/262094357 wait complete. 2024-09-17T05:09:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 Processor -- start 2024-09-17T05:09:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.329+0000 7fbd48000640 1 -- start start 2024-09-17T05:09:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd40159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd4015c0a0 0x7fbd4015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:50.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd4015f150 0x7fbd401f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40169150 con 0x7fbd400b2f50 2024-09-17T05:09:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd40168fd0 con 0x7fbd4015f150 2024-09-17T05:09:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbd401692d0 con 0x7fbd4015c0a0 2024-09-17T05:09:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd40159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:50.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd40159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41788/0 (socket says 172.21.15.16:41788) 2024-09-17T05:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 -- 172.21.15.16:0/1047994686 learned_addr learned my addr 172.21.15.16:0/1047994686 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd44c00640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd4015c0a0 0x7fbd4015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd46000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd4015f150 0x7fbd401f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 -- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd4015c0a0 msgr2=0x7fbd4015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd4015c0a0 0x7fbd4015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 -- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd4015f150 msgr2=0x7fbd401f7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd4015f150 0x7fbd401f7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 -- 172.21.15.16:0/1047994686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd401f6f50 con 0x7fbd400b2f50 2024-09-17T05:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd45600640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd40159d10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b0040 tx=0x7fbd340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340bb070 con 0x7fbd400b2f50 2024-09-17T05:09:50.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd34005030 con 0x7fbd400b2f50 2024-09-17T05:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd40164a60 con 0x7fbd400b2f50 2024-09-17T05:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b2040 con 0x7fbd400b2f50 2024-09-17T05:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd40165020 con 0x7fbd400b2f50 2024-09-17T05:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.333+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4015bb60 con 0x7fbd400b2f50 2024-09-17T05:09:50.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.337+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbd340be070 con 0x7fbd400b2f50 2024-09-17T05:09:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.337+0000 7fbd3d600640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd20073200 0x7fbd20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.337+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 6130+0+0 (secure 0 0 0) 0x7fbd3412dbd0 con 0x7fbd400b2f50 2024-09-17T05:09:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.337+0000 7fbd44c00640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd20073200 0x7fbd20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.337+0000 7fbd44c00640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd20073200 0x7fbd20075710 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbd400722a0 tx=0x7fbd2c00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:50.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.341+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbd340164d0 con 0x7fbd400b2f50 2024-09-17T05:09:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.553+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbd400cc4b0 con 0x7fbd400b2f50 2024-09-17T05:09:50.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.553+0000 7fbd3d600640 1 -- 172.21.15.16:0/1047994686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fbd340f83e0 con 0x7fbd400b2f50 2024-09-17T05:09:50.560 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:50.560 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-09-17T05:09:50.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd20073200 msgr2=0x7fbd20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd20073200 0x7fbd20075710 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbd400722a0 tx=0x7fbd2c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 msgr2=0x7fbd40159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd40159d10 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b0040 tx=0x7fbd340040c0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 shutdown_connections 2024-09-17T05:09:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbd20073200 0x7fbd20075710 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd4015f150 0x7fbd401f7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbd4015c0a0 0x7fbd4015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 --2- 172.21.15.16:0/1047994686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd400b2f50 0x7fbd40159d10 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 >> 172.21.15.16:0/1047994686 conn(0x7fbd4006f900 msgr2=0x7fbd40152c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 shutdown_connections 2024-09-17T05:09:50.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50.557+0000 7fbd48000640 1 -- 172.21.15.16:0/1047994686 wait complete. 2024-09-17T05:09:50.578 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:09:50.578 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-09-17T05:09:50.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:50 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-09-17T05:09:52.316 INFO:teuthology.orchestra.run.smithi016.stderr:open_n_background: keeping files open 2024-09-17T05:09:52.316 INFO:teuthology.orchestra.run.smithi016.stderr:stdin closed, goodbye! 2024-09-17T05:09:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:09:52.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 -- 172.21.15.16:0/2352166475 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 msgr2=0x7fd60c0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 --2- 172.21.15.16:0/2352166475 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 0x7fd60c0b2050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009840 tx=0x7fd5f401c7f0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 -- 172.21.15.16:0/2352166475 shutdown_connections 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 --2- 172.21.15.16:0/2352166475 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 0x7fd60c0b2050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 --2- 172.21.15.16:0/2352166475 >> v2:172.21.15.104:3300/0 conn(0x7fd60c15a050 0x7fd60c0cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 -- 172.21.15.16:0/2352166475 >> 172.21.15.16:0/2352166475 conn(0x7fd60c0d2240 msgr2=0x7fd60c0d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:52.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 -- 172.21.15.16:0/2352166475 shutdown_connections 2024-09-17T05:09:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 -- 172.21.15.16:0/2352166475 wait complete. 2024-09-17T05:09:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 Processor -- start 2024-09-17T05:09:52.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.553+0000 7fd611e00640 1 -- start start 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 0x7fd60c140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 0x7fd60c1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd60c15a050 0x7fd60c1f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c146190 con 0x7fd60c0b6a90 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c146010 con 0x7fd60c15a050 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd60c16a250 con 0x7fd60c0b1c40 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 0x7fd60c1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:52.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 0x7fd60c1f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41800/0 (socket says 172.21.15.16:41800) 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 -- 172.21.15.16:0/1428288449 learned_addr learned my addr 172.21.15.16:0/1428288449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd60aa00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 0x7fd60c140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd60b400640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd60c15a050 0x7fd60c1f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 -- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 msgr2=0x7fd60c140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 0x7fd60c140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 -- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd60c15a050 msgr2=0x7fd60c1f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd60c15a050 0x7fd60c1f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 -- 172.21.15.16:0/1428288449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c0c71e0 con 0x7fd60c0b6a90 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd60b400640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd60c15a050 0x7fd60c1f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:52.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 0x7fd60c1f6e90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009790 tx=0x7fd5f4005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40bb070 con 0x7fd60c0b6a90 2024-09-17T05:09:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5f4004330 con 0x7fd60c0b6a90 2024-09-17T05:09:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c1f7c30 con 0x7fd60c0b6a90 2024-09-17T05:09:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40b2040 con 0x7fd60c0b6a90 2024-09-17T05:09:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c140e90 con 0x7fd60c0b6a90 2024-09-17T05:09:52.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.557+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c1f79a0 con 0x7fd60c0b6a90 2024-09-17T05:09:52.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.561+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd5f4003820 con 0x7fd60c0b6a90 2024-09-17T05:09:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.561+0000 7fd603400640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5e8073230 0x7fd5e8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:52.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.561+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7fd5f412dd60 con 0x7fd60c0b6a90 2024-09-17T05:09:52.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.561+0000 7fd60aa00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5e8073230 0x7fd5e8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:52.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.561+0000 7fd60aa00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5e8073230 0x7fd5e8075740 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009ad0 tx=0x7fd5fc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:52.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.565+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd5f4016410 con 0x7fd60c0b6a90 2024-09-17T05:09:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd60c0b2050 con 0x7fd60c0b6a90 2024-09-17T05:09:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd603400640 1 -- 172.21.15.16:0/1428288449 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7fd5f40f8ab0 con 0x7fd60c0b6a90 2024-09-17T05:09:52.777 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:52.777 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"btime":"2024-09-17T05:09:49:296334+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:48.485972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 100 2024-09-17T05:09:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5e8073230 msgr2=0x7fd5e8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5e8073230 0x7fd5e8075740 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009ad0 tx=0x7fd5fc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 msgr2=0x7fd60c1f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 0x7fd60c1f6e90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009790 tx=0x7fd5f4005ca0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 shutdown_connections 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd5e8073230 0x7fd5e8075740 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd60c15a050 0x7fd60c1f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd60c0b6a90 0x7fd60c1f6e90 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 --2- 172.21.15.16:0/1428288449 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd60c0b1c40 0x7fd60c140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 >> 172.21.15.16:0/1428288449 conn(0x7fd60c0d2240 msgr2=0x7fd60c13eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 shutdown_connections 2024-09-17T05:09:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:52.777+0000 7fd611e00640 1 -- 172.21.15.16:0/1428288449 wait complete. 2024-09-17T05:09:52.800 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-09-17T05:09:52.980 INFO:teuthology.orchestra.run.smithi104.stdout:[{"id":8277,"entity":{"name":{"type":"client","num":8277},"addr":{"type":"v1","addr":"172.21.15.104:0","nonce":1326080318}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":12.199842268999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.930223474611596,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.8277 v1:172.21.15.104:0/1326080318","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi104","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}},{"id":6887,"entity":{"name":{"type":"client","num":6887},"addr":{"type":"v1","addr":"172.21.15.16:0","nonce":3729639820}},"state":"open","num_leases":400,"num_caps":201,"request_load_avg":384,"uptime":13.387826801999999,"requests_in_flight":0,"num_completed_requests":399,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1510.8794968472846,"halflife":60},"release_caps":{"value":198.99793048720971,"halflife":60},"recall_caps_throttle":{"value":425.83312987866515,"halflife":1.5},"recall_caps_throttle2o":{"value":137.03519871027268,"halflife":0.5},"session_cache_liveness":{"value":1177.4790525104725,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6887 v1:172.21.15.16:0/3729639820","completed_requests":[{"tid":29,"created_ino":"0x10000000001"},{"tid":30,"created_ino":"0x100000001f6"},{"tid":31,"created_ino":"0x100000001f7"},{"tid":32,"created_ino":"0x100000001f8"},{"tid":33,"created_ino":"0x100000001f9"},{"tid":34,"created_ino":"0x100000001fa"},{"tid":35,"created_ino":"0x100000001fb"},{"tid":36,"created_ino":"0x100000001fc"},{"tid":37,"created_ino":"0x100000001fd"},{"tid":38,"created_ino":"0x100000001fe"},{"tid":39,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":41,"created_ino":"0x10000000201"},{"tid":42,"created_ino":"0x10000000202"},{"tid":43,"created_ino":"0x10000000203"},{"tid":44,"created_ino":"0x10000000204"},{"tid":45,"created_ino":"0x10000000205"},{"tid":46,"created_ino":"0x10000000206"},{"tid":47,"created_ino":"0x10000000207"},{"tid":48,"created_ino":"0x10000000208"},{"tid":49,"created_ino":"0x10000000209"},{"tid":50,"created_ino":"0x1000000020a"},{"tid":51,"created_ino":"0x1000000020b"},{"tid":52,"created_ino":"0x1000000020c"},{"tid":53,"created_ino":"0x1000000020d"},{"tid":54,"created_ino":"0x1000000020e"},{"tid":55,"created_ino":"0x1000000020f"},{"tid":56,"created_ino":"0x10000000210"},{"tid":57,"created_ino":"0x10000000211"},{"tid":58,"created_ino":"0x10000000212"},{"tid":59,"created_ino":"0x10000000213"},{"tid":60,"created_ino":"0x10000000214"},{"tid":61,"created_ino":"0x10000000215"},{"tid":62,"created_ino":"0x10000000216"},{"tid":63,"created_ino":"0x10000000217"},{"tid":64,"created_ino":"0x10000000218"},{"tid":65,"created_ino":"0x10000000219"},{"tid":66,"created_ino":"0x1000000021a"},{"tid":67,"created_ino":"0x1000000021b"},{"tid":68,"created_ino":"0x1000000021c"},{"tid":69,"created_ino":"0x1000000021d"},{"tid":70,"created_ino":"0x1000000021e"},{"tid":71,"created_ino":"0x1000000021f"},{"tid":72,"created_ino":"0x10000000220"},{"tid":73,"created_ino":"0x10000000221"},{"tid":74,"created_ino":"0x10000000222"},{"tid":75,"created_ino":"0x10000000223"},{"tid":76,"created_ino":"0x10000000224"},{"tid":77,"created_ino":"0x10000000225"},{"tid":78,"created_ino":"0x10000000226"},{"tid":79,"created_ino":"0x10000000227"},{"tid":80,"created_ino":"0x10000000228"},{"tid":81,"created_ino":"0x10000000229"},{"tid":82,"created_ino":"0x1000000022a"},{"tid":83,"created_ino":"0x1000000022b"},{"tid":84,"created_ino":"0x1000000022c"},{"tid":85,"created_ino":"0x1000000022d"},{"tid":86,"created_ino":"0x1000000022e"},{"tid":87,"created_ino":"0x1000000022f"},{"tid":88,"created_ino":"0x10000000230"},{"tid":89,"created_ino":"0x10000000231"},{"tid":90,"created_ino":"0x10000000232"},{"tid":91,"created_ino":"0x10000000233"},{"tid":92,"created_ino":"0x10000000234"},{"tid":93,"created_ino":"0x10000000235"},{"tid":94,"created_ino":"0x10000000236"},{"tid":95,"created_ino":"0x10000000237"},{"tid":96,"created_ino":"0x10000000238"},{"tid":97,"created_ino":"0x10000000239"},{"tid":98,"created_ino":"0x1000000023a"},{"tid":99,"created_ino":"0x1000000023b"},{"tid":100,"created_ino":"0x1000000023c"},{"tid":101,"created_ino":"0x1000000023d"},{"tid":102,"created_ino":"0x1000000023e"},{"tid":103,"created_ino":"0x1000000023f"},{"tid":104,"created_ino":"0x10000000240"},{"tid":105,"created_ino":"0x10000000241"},{"tid":106,"created_ino":"0x10000000242"},{"tid":107,"created_ino":"0x10000000243"},{"tid":108,"created_ino":"0x10000000244"},{"tid":109,"created_ino":"0x10000000245"},{"tid":110,"created_ino":"0x10000000246"},{"tid":111,"created_ino":"0x10000000247"},{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"},{"tid":229,"created_ino":"0x100000002bd"},{"tid":230,"created_ino":"0x100000002be"},{"tid":231,"created_ino":"0x100000002bf"},{"tid":232,"created_ino":"0x100000002c0"},{"tid":233,"created_ino":"0x100000002c1"},{"tid":234,"created_ino":"0x100000002c2"},{"tid":235,"created_ino":"0x100000002c3"},{"tid":236,"created_ino":"0x100000002c4"},{"tid":237,"created_ino":"0x100000002c5"},{"tid":238,"created_ino":"0x100000002c6"},{"tid":239,"created_ino":"0x100000002c7"},{"tid":240,"created_ino":"0x100000002c8"},{"tid":241,"created_ino":"0x100000002c9"},{"tid":242,"created_ino":"0x100000002ca"},{"tid":243,"created_ino":"0x100000002cb"},{"tid":244,"created_ino":"0x100000002cc"},{"tid":245,"created_ino":"0x100000002cd"},{"tid":246,"created_ino":"0x100000002ce"},{"tid":247,"created_ino":"0x100000002cf"},{"tid":248,"created_ino":"0x100000002d0"},{"tid":249,"created_ino":"0x100000002d1"},{"tid":250,"created_ino":"0x100000002d2"},{"tid":251,"created_ino":"0x100000002d3"},{"tid":252,"created_ino":"0x100000002d4"},{"tid":253,"created_ino":"0x100000002d5"},{"tid":254,"created_ino":"0x100000002d6"},{"tid":255,"created_ino":"0x100000002d7"},{"tid":256,"created_ino":"0x100000002d8"},{"tid":257,"created_ino":"0x100000002d9"},{"tid":258,"created_ino":"0x100000002da"},{"tid":259,"created_ino":"0x100000002db"},{"tid":260,"created_ino":"0x100000002dc"},{"tid":261,"created_ino":"0x100000002dd"},{"tid":262,"created_ino":"0x100000002de"},{"tid":263,"created_ino":"0x100000002df"},{"tid":264,"created_ino":"0x100000002e0"},{"tid":265,"created_ino":"0x100000002e1"},{"tid":266,"created_ino":"0x100000002e2"},{"tid":267,"created_ino":"0x100000002e3"},{"tid":268,"created_ino":"0x100000002e4"},{"tid":269,"created_ino":"0x100000002e5"},{"tid":270,"created_ino":"0x100000002e6"},{"tid":271,"created_ino":"0x100000002e7"},{"tid":272,"created_ino":"0x100000002e8"},{"tid":273,"created_ino":"0x100000002e9"},{"tid":274,"created_ino":"0x100000002ea"},{"tid":275,"created_ino":"0x100000002eb"},{"tid":276,"created_ino":"0x100000002ec"},{"tid":277,"created_ino":"0x100000002ed"},{"tid":278,"created_ino":"0x100000002ee"},{"tid":279,"created_ino":"0x100000002ef"},{"tid":280,"created_ino":"0x100000002f0"},{"tid":281,"created_ino":"0x100000002f1"},{"tid":282,"created_ino":"0x100000002f2"},{"tid":283,"created_ino":"0x100000002f3"},{"tid":284,"created_ino":"0x100000002f4"},{"tid":285,"created_ino":"0x100000002f5"},{"tid":286,"created_ino":"0x100000002f6"},{"tid":287,"created_ino":"0x100000002f7"},{"tid":288,"created_ino":"0x100000002f8"},{"tid":289,"created_ino":"0x100000002f9"},{"tid":290,"created_ino":"0x100000002fa"},{"tid":291,"created_ino":"0x100000002fb"},{"tid":292,"created_ino":"0x100000002fc"},{"tid":293,"created_ino":"0x100000002fd"},{"tid":294,"created_ino":"0x100000002fe"},{"tid":295,"created_ino":"0x100000002ff"},{"tid":296,"created_ino":"0x10000000300"},{"tid":297,"created_ino":"0x10000000301"},{"tid":298,"created_ino":"0x10000000302"},{"tid":299,"created_ino":"0x10000000303"},{"tid":300,"created_ino":"0x10000000304"},{"tid":301,"created_ino":"0x10000000305"},{"tid":302,"created_ino":"0x10000000306"},{"tid":303,"created_ino":"0x10000000307"},{"tid":304,"created_ino":"0x10000000308"},{"tid":305,"created_ino":"0x10000000309"},{"tid":306,"created_ino":"0x1000000030a"},{"tid":307,"created_ino":"0x1000000030b"},{"tid":308,"created_ino":"0x1000000030c"},{"tid":309,"created_ino":"0x1000000030d"},{"tid":310,"created_ino":"0x1000000030e"},{"tid":311,"created_ino":"0x1000000030f"},{"tid":312,"created_ino":"0x10000000310"},{"tid":313,"created_ino":"0x10000000311"},{"tid":314,"created_ino":"0x10000000312"},{"tid":315,"created_ino":"0x10000000313"},{"tid":316,"created_ino":"0x10000000314"},{"tid":317,"created_ino":"0x10000000315"},{"tid":318,"created_ino":"0x10000000316"},{"tid":319,"created_ino":"0x10000000317"},{"tid":320,"created_ino":"0x10000000318"},{"tid":321,"created_ino":"0x10000000319"},{"tid":322,"created_ino":"0x1000000031a"},{"tid":323,"created_ino":"0x1000000031b"},{"tid":324,"created_ino":"0x1000000031c"},{"tid":325,"created_ino":"0x1000000031d"},{"tid":326,"created_ino":"0x1000000031e"},{"tid":327,"created_ino":"0x1000000031f"},{"tid":328,"created_ino":"0x10000000320"},{"tid":329,"created_ino":"0x10000000321"},{"tid":330,"created_ino":"0x10000000322"},{"tid":331,"created_ino":"0x10000000323"},{"tid":332,"created_ino":"0x10000000324"},{"tid":333,"created_ino":"0x10000000325"},{"tid":334,"created_ino":"0x10000000326"},{"tid":335,"created_ino":"0x10000000327"},{"tid":336,"created_ino":"0x10000000328"},{"tid":337,"created_ino":"0x10000000329"},{"tid":338,"created_ino":"0x1000000032a"},{"tid":339,"created_ino":"0x1000000032b"},{"tid":340,"created_ino":"0x1000000032c"},{"tid":341,"created_ino":"0x1000000032d"},{"tid":342,"created_ino":"0x1000000032e"},{"tid":343,"created_ino":"0x1000000032f"},{"tid":344,"created_ino":"0x10000000330"},{"tid":345,"created_ino":"0x10000000331"},{"tid":346,"created_ino":"0x10000000332"},{"tid":347,"created_ino":"0x10000000333"},{"tid":348,"created_ino":"0x10000000334"},{"tid":349,"created_ino":"0x10000000335"},{"tid":350,"created_ino":"0x10000000336"},{"tid":351,"created_ino":"0x10000000337"},{"tid":352,"created_ino":"0x10000000338"},{"tid":353,"created_ino":"0x10000000339"},{"tid":354,"created_ino":"0x1000000033a"},{"tid":355,"created_ino":"0x1000000033b"},{"tid":356,"created_ino":"0x1000000033c"},{"tid":357,"created_ino":"0x1000000033d"},{"tid":358,"created_ino":"0x1000000033e"},{"tid":359,"created_ino":"0x1000000033f"},{"tid":360,"created_ino":"0x10000000340"},{"tid":361,"created_ino":"0x10000000341"},{"tid":362,"created_ino":"0x10000000342"},{"tid":363,"created_ino":"0x10000000343"},{"tid":364,"created_ino":"0x10000000344"},{"tid":365,"created_ino":"0x10000000345"},{"tid":366,"created_ino":"0x10000000346"},{"tid":367,"created_ino":"0x10000000347"},{"tid":368,"created_ino":"0x10000000348"},{"tid":369,"created_ino":"0x10000000349"},{"tid":370,"created_ino":"0x1000000034a"},{"tid":371,"created_ino":"0x1000000034b"},{"tid":372,"created_ino":"0x1000000034c"},{"tid":373,"created_ino":"0x1000000034d"},{"tid":374,"created_ino":"0x1000000034e"},{"tid":375,"created_ino":"0x1000000034f"},{"tid":376,"created_ino":"0x10000000350"},{"tid":377,"created_ino":"0x10000000351"},{"tid":378,"created_ino":"0x10000000352"},{"tid":379,"created_ino":"0x10000000353"},{"tid":380,"created_ino":"0x10000000354"},{"tid":381,"created_ino":"0x10000000355"},{"tid":382,"created_ino":"0x10000000356"},{"tid":383,"created_ino":"0x10000000357"},{"tid":384,"created_ino":"0x10000000358"},{"tid":385,"created_ino":"0x10000000359"},{"tid":386,"created_ino":"0x1000000035a"},{"tid":387,"created_ino":"0x1000000035b"},{"tid":388,"created_ino":"0x1000000035c"},{"tid":389,"created_ino":"0x1000000035d"},{"tid":390,"created_ino":"0x1000000035e"},{"tid":391,"created_ino":"0x1000000035f"},{"tid":392,"created_ino":"0x10000000360"},{"tid":393,"created_ino":"0x10000000361"},{"tid":394,"created_ino":"0x10000000362"},{"tid":395,"created_ino":"0x10000000363"},{"tid":396,"created_ino":"0x10000000364"},{"tid":397,"created_ino":"0x10000000365"},{"tid":398,"created_ino":"0x10000000366"},{"tid":399,"created_ino":"0x10000000367"},{"tid":400,"created_ino":"0x10000000368"},{"tid":401,"created_ino":"0x10000000369"},{"tid":402,"created_ino":"0x1000000036a"},{"tid":403,"created_ino":"0x1000000036b"},{"tid":404,"created_ino":"0x1000000036c"},{"tid":405,"created_ino":"0x1000000036d"},{"tid":406,"created_ino":"0x1000000036e"},{"tid":407,"created_ino":"0x1000000036f"},{"tid":408,"created_ino":"0x10000000370"},{"tid":409,"created_ino":"0x10000000371"},{"tid":410,"created_ino":"0x10000000372"},{"tid":411,"created_ino":"0x10000000373"},{"tid":412,"created_ino":"0x10000000374"},{"tid":413,"created_ino":"0x10000000375"},{"tid":414,"created_ino":"0x10000000376"},{"tid":415,"created_ino":"0x10000000377"},{"tid":416,"created_ino":"0x10000000378"},{"tid":417,"created_ino":"0x10000000379"},{"tid":418,"created_ino":"0x1000000037a"},{"tid":419,"created_ino":"0x1000000037b"},{"tid":420,"created_ino":"0x1000000037c"},{"tid":421,"created_ino":"0x1000000037d"},{"tid":422,"created_ino":"0x1000000037e"},{"tid":423,"created_ino":"0x1000000037f"},{"tid":424,"created_ino":"0x10000000380"},{"tid":425,"created_ino":"0x10000000381"},{"tid":426,"created_ino":"0x10000000382"},{"tid":427,"created_ino":"0x10000000383"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi016","kernel_version":"6.11.0-rc5-g2ca44a2f35ca","root":"/"}}] 2024-09-17T05:09:52.983 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi104", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.104:0", "nonce": 1326080318, "type": "v1" }, "name": { "num": 8277, "type": "client" } }, "id": 8277, "inst": "client.8277 v1:172.21.15.104:0/1326080318", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.930223474611596 }, "state": "open", "uptime": 12.199842269 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi016", "kernel_version": "6.11.0-rc5-g2ca44a2f35ca", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 29 }, { "created_ino": "0x100000001f6", "tid": 30 }, { "created_ino": "0x100000001f7", "tid": 31 }, { "created_ino": "0x100000001f8", "tid": 32 }, { "created_ino": "0x100000001f9", "tid": 33 }, { "created_ino": "0x100000001fa", "tid": 34 }, { "created_ino": "0x100000001fb", "tid": 35 }, { "created_ino": "0x100000001fc", "tid": 36 }, { "created_ino": "0x100000001fd", "tid": 37 }, { "created_ino": "0x100000001fe", "tid": 38 }, { "created_ino": "0x100000001ff", "tid": 39 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 41 }, { "created_ino": "0x10000000202", "tid": 42 }, { "created_ino": "0x10000000203", "tid": 43 }, { "created_ino": "0x10000000204", "tid": 44 }, { "created_ino": "0x10000000205", "tid": 45 }, { "created_ino": "0x10000000206", "tid": 46 }, { "created_ino": "0x10000000207", "tid": 47 }, { "created_ino": "0x10000000208", "tid": 48 }, { "created_ino": "0x10000000209", "tid": 49 }, { "created_ino": "0x1000000020a", "tid": 50 }, { "created_ino": "0x1000000020b", "tid": 51 }, { "created_ino": "0x1000000020c", "tid": 52 }, { "created_ino": "0x1000000020d", "tid": 53 }, { "created_ino": "0x1000000020e", "tid": 54 }, { "created_ino": "0x1000000020f", "tid": 55 }, { "created_ino": "0x10000000210", "tid": 56 }, { "created_ino": "0x10000000211", "tid": 57 }, { "created_ino": "0x10000000212", "tid": 58 }, { "created_ino": "0x10000000213", "tid": 59 }, { "created_ino": "0x10000000214", "tid": 60 }, { "created_ino": "0x10000000215", "tid": 61 }, { "created_ino": "0x10000000216", "tid": 62 }, { "created_ino": "0x10000000217", "tid": 63 }, { "created_ino": "0x10000000218", "tid": 64 }, { "created_ino": "0x10000000219", "tid": 65 }, { "created_ino": "0x1000000021a", "tid": 66 }, { "created_ino": "0x1000000021b", "tid": 67 }, { "created_ino": "0x1000000021c", "tid": 68 }, { "created_ino": "0x1000000021d", "tid": 69 }, { "created_ino": "0x1000000021e", "tid": 70 }, { "created_ino": "0x1000000021f", "tid": 71 }, { "created_ino": "0x10000000220", "tid": 72 }, { "created_ino": "0x10000000221", "tid": 73 }, { "created_ino": "0x10000000222", "tid": 74 }, { "created_ino": "0x10000000223", "tid": 75 }, { "created_ino": "0x10000000224", "tid": 76 }, { "created_ino": "0x10000000225", "tid": 77 }, { "created_ino": "0x10000000226", "tid": 78 }, { "created_ino": "0x10000000227", "tid": 79 }, { "created_ino": "0x10000000228", "tid": 80 }, { "created_ino": "0x10000000229", "tid": 81 }, { "created_ino": "0x1000000022a", "tid": 82 }, { "created_ino": "0x1000000022b", "tid": 83 }, { "created_ino": "0x1000000022c", "tid": 84 }, { "created_ino": "0x1000000022d", "tid": 85 }, { "created_ino": "0x1000000022e", "tid": 86 }, { "created_ino": "0x1000000022f", "tid": 87 }, { "created_ino": "0x10000000230", "tid": 88 }, { "created_ino": "0x10000000231", "tid": 89 }, { "created_ino": "0x10000000232", "tid": 90 }, { "created_ino": "0x10000000233", "tid": 91 }, { "created_ino": "0x10000000234", "tid": 92 }, { "created_ino": "0x10000000235", "tid": 93 }, { "created_ino": "0x10000000236", "tid": 94 }, { "created_ino": "0x10000000237", "tid": 95 }, { "created_ino": "0x10000000238", "tid": 96 }, { "created_ino": "0x10000000239", "tid": 97 }, { "created_ino": "0x1000000023a", "tid": 98 }, { "created_ino": "0x1000000023b", "tid": 99 }, { "created_ino": "0x1000000023c", "tid": 100 }, { "created_ino": "0x1000000023d", "tid": 101 }, { "created_ino": "0x1000000023e", "tid": 102 }, { "created_ino": "0x1000000023f", "tid": 103 }, { "created_ino": "0x10000000240", "tid": 104 }, { "created_ino": "0x10000000241", "tid": 105 }, { "created_ino": "0x10000000242", "tid": 106 }, { "created_ino": "0x10000000243", "tid": 107 }, { "created_ino": "0x10000000244", "tid": 108 }, { "created_ino": "0x10000000245", "tid": 109 }, { "created_ino": "0x10000000246", "tid": 110 }, { "created_ino": "0x10000000247", "tid": 111 }, { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 }, { "created_ino": "0x100000002bd", "tid": 229 }, { "created_ino": "0x100000002be", "tid": 230 }, { "created_ino": "0x100000002bf", "tid": 231 }, { "created_ino": "0x100000002c0", "tid": 232 }, { "created_ino": "0x100000002c1", "tid": 233 }, { "created_ino": "0x100000002c2", "tid": 234 }, { "created_ino": "0x100000002c3", "tid": 235 }, { "created_ino": "0x100000002c4", "tid": 236 }, { "created_ino": "0x100000002c5", "tid": 237 }, { "created_ino": "0x100000002c6", "tid": 238 }, { "created_ino": "0x100000002c7", "tid": 239 }, { "created_ino": "0x100000002c8", "tid": 240 }, { "created_ino": "0x100000002c9", "tid": 241 }, { "created_ino": "0x100000002ca", "tid": 242 }, { "created_ino": "0x100000002cb", "tid": 243 }, { "created_ino": "0x100000002cc", "tid": 244 }, { "created_ino": "0x100000002cd", "tid": 245 }, { "created_ino": "0x100000002ce", "tid": 246 }, { "created_ino": "0x100000002cf", "tid": 247 }, { "created_ino": "0x100000002d0", "tid": 248 }, { "created_ino": "0x100000002d1", "tid": 249 }, { "created_ino": "0x100000002d2", "tid": 250 }, { "created_ino": "0x100000002d3", "tid": 251 }, { "created_ino": "0x100000002d4", "tid": 252 }, { "created_ino": "0x100000002d5", "tid": 253 }, { "created_ino": "0x100000002d6", "tid": 254 }, { "created_ino": "0x100000002d7", "tid": 255 }, { "created_ino": "0x100000002d8", "tid": 256 }, { "created_ino": "0x100000002d9", "tid": 257 }, { "created_ino": "0x100000002da", "tid": 258 }, { "created_ino": "0x100000002db", "tid": 259 }, { "created_ino": "0x100000002dc", "tid": 260 }, { "created_ino": "0x100000002dd", "tid": 261 }, { "created_ino": "0x100000002de", "tid": 262 }, { "created_ino": "0x100000002df", "tid": 263 }, { "created_ino": "0x100000002e0", "tid": 264 }, { "created_ino": "0x100000002e1", "tid": 265 }, { "created_ino": "0x100000002e2", "tid": 266 }, { "created_ino": "0x100000002e3", "tid": 267 }, { "created_ino": "0x100000002e4", "tid": 268 }, { "created_ino": "0x100000002e5", "tid": 269 }, { "created_ino": "0x100000002e6", "tid": 270 }, { "created_ino": "0x100000002e7", "tid": 271 }, { "created_ino": "0x100000002e8", "tid": 272 }, { "created_ino": "0x100000002e9", "tid": 273 }, { "created_ino": "0x100000002ea", "tid": 274 }, { "created_ino": "0x100000002eb", "tid": 275 }, { "created_ino": "0x100000002ec", "tid": 276 }, { "created_ino": "0x100000002ed", "tid": 277 }, { "created_ino": "0x100000002ee", "tid": 278 }, { "created_ino": "0x100000002ef", "tid": 279 }, { "created_ino": "0x100000002f0", "tid": 280 }, { "created_ino": "0x100000002f1", "tid": 281 }, { "created_ino": "0x100000002f2", "tid": 282 }, { "created_ino": "0x100000002f3", "tid": 283 }, { "created_ino": "0x100000002f4", "tid": 284 }, { "created_ino": "0x100000002f5", "tid": 285 }, { "created_ino": "0x100000002f6", "tid": 286 }, { "created_ino": "0x100000002f7", "tid": 287 }, { "created_ino": "0x100000002f8", "tid": 288 }, { "created_ino": "0x100000002f9", "tid": 289 }, { "created_ino": "0x100000002fa", "tid": 290 }, { "created_ino": "0x100000002fb", "tid": 291 }, { "created_ino": "0x100000002fc", "tid": 292 }, { "created_ino": "0x100000002fd", "tid": 293 }, { "created_ino": "0x100000002fe", "tid": 294 }, { "created_ino": "0x100000002ff", "tid": 295 }, { "created_ino": "0x10000000300", "tid": 296 }, { "created_ino": "0x10000000301", "tid": 297 }, { "created_ino": "0x10000000302", "tid": 298 }, { "created_ino": "0x10000000303", "tid": 299 }, { "created_ino": "0x10000000304", "tid": 300 }, { "created_ino": "0x10000000305", "tid": 301 }, { "created_ino": "0x10000000306", "tid": 302 }, { "created_ino": "0x10000000307", "tid": 303 }, { "created_ino": "0x10000000308", "tid": 304 }, { "created_ino": "0x10000000309", "tid": 305 }, { "created_ino": "0x1000000030a", "tid": 306 }, { "created_ino": "0x1000000030b", "tid": 307 }, { "created_ino": "0x1000000030c", "tid": 308 }, { "created_ino": "0x1000000030d", "tid": 309 }, { "created_ino": "0x1000000030e", "tid": 310 }, { "created_ino": "0x1000000030f", "tid": 311 }, { "created_ino": "0x10000000310", "tid": 312 }, { "created_ino": "0x10000000311", "tid": 313 }, { "created_ino": "0x10000000312", "tid": 314 }, { "created_ino": "0x10000000313", "tid": 315 }, { "created_ino": "0x10000000314", "tid": 316 }, { "created_ino": "0x10000000315", "tid": 317 }, { "created_ino": "0x10000000316", "tid": 318 }, { "created_ino": "0x10000000317", "tid": 319 }, { "created_ino": "0x10000000318", "tid": 320 }, { "created_ino": "0x10000000319", "tid": 321 }, { "created_ino": "0x1000000031a", "tid": 322 }, { "created_ino": "0x1000000031b", "tid": 323 }, { "created_ino": "0x1000000031c", "tid": 324 }, { "created_ino": "0x1000000031d", "tid": 325 }, { "created_ino": "0x1000000031e", "tid": 326 }, { "created_ino": "0x1000000031f", "tid": 327 }, { "created_ino": "0x10000000320", "tid": 328 }, { "created_ino": "0x10000000321", "tid": 329 }, { "created_ino": "0x10000000322", "tid": 330 }, { "created_ino": "0x10000000323", "tid": 331 }, { "created_ino": "0x10000000324", "tid": 332 }, { "created_ino": "0x10000000325", "tid": 333 }, { "created_ino": "0x10000000326", "tid": 334 }, { "created_ino": "0x10000000327", "tid": 335 }, { "created_ino": "0x10000000328", "tid": 336 }, { "created_ino": "0x10000000329", "tid": 337 }, { "created_ino": "0x1000000032a", "tid": 338 }, { "created_ino": "0x1000000032b", "tid": 339 }, { "created_ino": "0x1000000032c", "tid": 340 }, { "created_ino": "0x1000000032d", "tid": 341 }, { "created_ino": "0x1000000032e", "tid": 342 }, { "created_ino": "0x1000000032f", "tid": 343 }, { "created_ino": "0x10000000330", "tid": 344 }, { "created_ino": "0x10000000331", "tid": 345 }, { "created_ino": "0x10000000332", "tid": 346 }, { "created_ino": "0x10000000333", "tid": 347 }, { "created_ino": "0x10000000334", "tid": 348 }, { "created_ino": "0x10000000335", "tid": 349 }, { "created_ino": "0x10000000336", "tid": 350 }, { "created_ino": "0x10000000337", "tid": 351 }, { "created_ino": "0x10000000338", "tid": 352 }, { "created_ino": "0x10000000339", "tid": 353 }, { "created_ino": "0x1000000033a", "tid": 354 }, { "created_ino": "0x1000000033b", "tid": 355 }, { "created_ino": "0x1000000033c", "tid": 356 }, { "created_ino": "0x1000000033d", "tid": 357 }, { "created_ino": "0x1000000033e", "tid": 358 }, { "created_ino": "0x1000000033f", "tid": 359 }, { "created_ino": "0x10000000340", "tid": 360 }, { "created_ino": "0x10000000341", "tid": 361 }, { "created_ino": "0x10000000342", "tid": 362 }, { "created_ino": "0x10000000343", "tid": 363 }, { "created_ino": "0x10000000344", "tid": 364 }, { "created_ino": "0x10000000345", "tid": 365 }, { "created_ino": "0x10000000346", "tid": 366 }, { "created_ino": "0x10000000347", "tid": 367 }, { "created_ino": "0x10000000348", "tid": 368 }, { "created_ino": "0x10000000349", "tid": 369 }, { "created_ino": "0x1000000034a", "tid": 370 }, { "created_ino": "0x1000000034b", "tid": 371 }, { "created_ino": "0x1000000034c", "tid": 372 }, { "created_ino": "0x1000000034d", "tid": 373 }, { "created_ino": "0x1000000034e", "tid": 374 }, { "created_ino": "0x1000000034f", "tid": 375 }, { "created_ino": "0x10000000350", "tid": 376 }, { "created_ino": "0x10000000351", "tid": 377 }, { "created_ino": "0x10000000352", "tid": 378 }, { "created_ino": "0x10000000353", "tid": 379 }, { "created_ino": "0x10000000354", "tid": 380 }, { "created_ino": "0x10000000355", "tid": 381 }, { "created_ino": "0x10000000356", "tid": 382 }, { "created_ino": "0x10000000357", "tid": 383 }, { "created_ino": "0x10000000358", "tid": 384 }, { "created_ino": "0x10000000359", "tid": 385 }, { "created_ino": "0x1000000035a", "tid": 386 }, { "created_ino": "0x1000000035b", "tid": 387 }, { "created_ino": "0x1000000035c", "tid": 388 }, { "created_ino": "0x1000000035d", "tid": 389 }, { "created_ino": "0x1000000035e", "tid": 390 }, { "created_ino": "0x1000000035f", "tid": 391 }, { "created_ino": "0x10000000360", "tid": 392 }, { "created_ino": "0x10000000361", "tid": 393 }, { "created_ino": "0x10000000362", "tid": 394 }, { "created_ino": "0x10000000363", "tid": 395 }, { "created_ino": "0x10000000364", "tid": 396 }, { "created_ino": "0x10000000365", "tid": 397 }, { "created_ino": "0x10000000366", "tid": 398 }, { "created_ino": "0x10000000367", "tid": 399 }, { "created_ino": "0x10000000368", "tid": 400 }, { "created_ino": "0x10000000369", "tid": 401 }, { "created_ino": "0x1000000036a", "tid": 402 }, { "created_ino": "0x1000000036b", "tid": 403 }, { "created_ino": "0x1000000036c", "tid": 404 }, { "created_ino": "0x1000000036d", "tid": 405 }, { "created_ino": "0x1000000036e", "tid": 406 }, { "created_ino": "0x1000000036f", "tid": 407 }, { "created_ino": "0x10000000370", "tid": 408 }, { "created_ino": "0x10000000371", "tid": 409 }, { "created_ino": "0x10000000372", "tid": 410 }, { "created_ino": "0x10000000373", "tid": 411 }, { "created_ino": "0x10000000374", "tid": 412 }, { "created_ino": "0x10000000375", "tid": 413 }, { "created_ino": "0x10000000376", "tid": 414 }, { "created_ino": "0x10000000377", "tid": 415 }, { "created_ino": "0x10000000378", "tid": 416 }, { "created_ino": "0x10000000379", "tid": 417 }, { "created_ino": "0x1000000037a", "tid": 418 }, { "created_ino": "0x1000000037b", "tid": 419 }, { "created_ino": "0x1000000037c", "tid": 420 }, { "created_ino": "0x1000000037d", "tid": 421 }, { "created_ino": "0x1000000037e", "tid": 422 }, { "created_ino": "0x1000000037f", "tid": 423 }, { "created_ino": "0x10000000380", "tid": 424 }, { "created_ino": "0x10000000381", "tid": 425 }, { "created_ino": "0x10000000382", "tid": 426 }, { "created_ino": "0x10000000383", "tid": 427 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.16:0", "nonce": 3729639820, "type": "v1" }, "name": { "num": 6887, "type": "client" } }, "id": 6887, "inst": "client.6887 v1:172.21.15.16:0/3729639820", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 201, "num_completed_flushes": 0, "num_completed_requests": 399, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1510.8794968472846 }, "recall_caps_throttle": { "halflife": 1.5, "value": 425.83312987866515 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 137.03519871027268 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 198.99793048720971 }, "request_load_avg": 384, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1177.4790525104725 }, "state": "open", "uptime": 13.387826802 } ] 2024-09-17T05:09:52.983 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T05:09:52.984 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:09:52.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:09:53.083 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:09:53.183 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:09:53.183 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:53.183 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/."\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi016') 2024-09-17T05:09:53.183 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:53.183 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:53.193 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:53.193 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:53.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:09:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 -- 172.21.15.16:0/2580959664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b415f6f0 msgr2=0x7fc6b40cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2580959664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b415f6f0 0x7fc6b40cece0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009bf0 tx=0x7fc6ac01cbe0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 -- 172.21.15.16:0/2580959664 shutdown_connections 2024-09-17T05:09:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2580959664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b415f6f0 0x7fc6b40cece0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 -- 172.21.15.16:0/2580959664 >> 172.21.15.16:0/2580959664 conn(0x7fc6b40b2250 msgr2=0x7fc6b40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 -- 172.21.15.16:0/2580959664 shutdown_connections 2024-09-17T05:09:53.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 -- 172.21.15.16:0/2580959664 wait complete. 2024-09-17T05:09:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 Processor -- start 2024-09-17T05:09:53.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.449+0000 7fc6bc400640 1 -- start start 2024-09-17T05:09:53.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b4074a20 0x7fc6b41fcd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:53.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6b4074e30 0x7fc6b4150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:53.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 0x7fc6b4150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:53.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b416a1b0 con 0x7fc6b415f6f0 2024-09-17T05:09:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b416a030 con 0x7fc6b4074e30 2024-09-17T05:09:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b416a330 con 0x7fc6b4074a20 2024-09-17T05:09:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 0x7fc6b4150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 0x7fc6b4150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41812/0 (socket says 172.21.15.16:41812) 2024-09-17T05:09:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 -- 172.21.15.16:0/2579301579 learned_addr learned my addr 172.21.15.16:0/2579301579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:53.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b9a00640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b4074a20 0x7fc6b41fcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 -- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b4074a20 msgr2=0x7fc6b41fcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b4074a20 0x7fc6b41fcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b9000640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6b4074e30 0x7fc6b4150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 -- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6b4074e30 msgr2=0x7fc6b4150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6b4074e30 0x7fc6b4150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 -- 172.21.15.16:0/2579301579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b40b3ff0 con 0x7fc6b415f6f0 2024-09-17T05:09:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b9a00640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b4074a20 0x7fc6b41fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b9000640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6b4074e30 0x7fc6b4150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6ba400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 0x7fc6b4150ff0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac01c650 tx=0x7fc6ac003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0b2040 con 0x7fc6b415f6f0 2024-09-17T05:09:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6ac0bb070 con 0x7fc6b415f6f0 2024-09-17T05:09:53.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0043d0 con 0x7fc6b415f6f0 2024-09-17T05:09:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b40b2f50 con 0x7fc6b415f6f0 2024-09-17T05:09:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b40b3510 con 0x7fc6b415f6f0 2024-09-17T05:09:53.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.453+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b40b3710 con 0x7fc6b415f6f0 2024-09-17T05:09:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.457+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc6ac004570 con 0x7fc6b415f6f0 2024-09-17T05:09:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.457+0000 7fc6b1600640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc694073230 0x7fc694075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.457+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7fc6ac12f700 con 0x7fc6b415f6f0 2024-09-17T05:09:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.457+0000 7fc6b9a00640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc694073230 0x7fc694075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.457+0000 7fc6b9a00640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc694073230 0x7fc694075740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007fd0 tx=0x7fc6a8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:53.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.461+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc6b40b3710 con 0x7fc6b415f6f0 2024-09-17T05:09:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.661+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc6b40b3d80 con 0x7fc6b415f6f0 2024-09-17T05:09:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.661+0000 7fc6b1600640 1 -- 172.21.15.16:0/2579301579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) ==== 80+0+0 (secure 0 0 0) 0x7fc6ac0167d0 con 0x7fc6b415f6f0 2024-09-17T05:09:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:09:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc694073230 msgr2=0x7fc694075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc694073230 0x7fc694075740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8007fd0 tx=0x7fc6a8009040 comp rx=0 tx=0).stop 2024-09-17T05:09:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 msgr2=0x7fc6b4150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 0x7fc6b4150ff0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac01c650 tx=0x7fc6ac003f40 comp rx=0 tx=0).stop 2024-09-17T05:09:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 shutdown_connections 2024-09-17T05:09:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc694073230 0x7fc694075740 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b415f6f0 0x7fc6b4150ff0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6b4074e30 0x7fc6b4150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 --2- 172.21.15.16:0/2579301579 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc6b4074a20 0x7fc6b41fcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 >> 172.21.15.16:0/2579301579 conn(0x7fc6b40b2250 msgr2=0x7fc6b40cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 shutdown_connections 2024-09-17T05:09:53.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:53.665+0000 7fc6bc400640 1 -- 172.21.15.16:0/2579301579 wait complete. 2024-09-17T05:09:53.686 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:09:53.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:53.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:09:53.786 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:09:53.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:53.786 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:09:53.795 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:53.795 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:53.796 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:53.805 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:53.805 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:53.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:09:54.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 -- 172.21.15.16:0/2419482457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 msgr2=0x7f6b6c0caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 --2- 172.21.15.16:0/2419482457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c0caac0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0b2ca0 tx=0x7f6b5401c740 comp rx=0 tx=0).stop 2024-09-17T05:09:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 -- 172.21.15.16:0/2419482457 shutdown_connections 2024-09-17T05:09:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 --2- 172.21.15.16:0/2419482457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c0caac0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 --2- 172.21.15.16:0/2419482457 >> v2:172.21.15.104:3300/0 conn(0x7f6b6c0d0940 0x7f6b6c1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 -- 172.21.15.16:0/2419482457 >> 172.21.15.16:0/2419482457 conn(0x7f6b6c0b2250 msgr2=0x7f6b6c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 -- 172.21.15.16:0/2419482457 shutdown_connections 2024-09-17T05:09:54.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.013+0000 7f6b72600640 1 -- 172.21.15.16:0/2419482457 wait complete. 2024-09-17T05:09:54.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 Processor -- start 2024-09-17T05:09:54.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 -- start start 2024-09-17T05:09:54.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b6c0d0940 0x7f6b6c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b6c15f100 0x7f6b6c14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c168e10 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c168c90 con 0x7f6b6c0d0940 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b72600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6b6c168f90 con 0x7f6b6c15f100 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c1514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41836/0 (socket says 172.21.15.16:41836) 2024-09-17T05:09:54.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 -- 172.21.15.16:0/1907483271 learned_addr learned my addr 172.21.15.16:0/1907483271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b62a00640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b6c0d0940 0x7f6b6c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6be00640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b6c15f100 0x7f6b6c14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 -- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b6c15f100 msgr2=0x7f6b6c14f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b6c15f100 0x7f6b6c14f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 -- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b6c0d0940 msgr2=0x7f6b6c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b6c0d0940 0x7f6b6c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.017+0000 7f6b6b400640 1 -- 172.21.15.16:0/1907483271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c151d40 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b6b400640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c1514f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00d750 tx=0x7f6b5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c017070 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b5c009d20 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c005280 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c152020 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c153af0 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c0caac0 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6b5c005420 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b63e00640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b48073230 0x7f6b48075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.021+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7f6b5c0943c0 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.025+0000 7f6b62a00640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b48073230 0x7f6b48075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.025+0000 7f6b62a00640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b48073230 0x7f6b48075740 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6b540af000 tx=0x7f6b5401af80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:54.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.029+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6b5c00de60 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.213+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b6c15ee70 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.213+0000 7f6b63e00640 1 -- 172.21.15.16:0/1907483271 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v138) ==== 80+0+0 (secure 0 0 0) 0x7f6b5c05ec60 con 0x7f6b6c0ca6b0 2024-09-17T05:09:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:09:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b48073230 msgr2=0x7f6b48075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b48073230 0x7f6b48075740 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6b540af000 tx=0x7f6b5401af80 comp rx=0 tx=0).stop 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 msgr2=0x7f6b6c1514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c1514f0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00d750 tx=0x7f6b5c00dc10 comp rx=0 tx=0).stop 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 shutdown_connections 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6b48073230 0x7f6b48075740 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6b6c15f100 0x7f6b6c14f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b6c0d0940 0x7f6b6c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 --2- 172.21.15.16:0/1907483271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b6c0ca6b0 0x7f6b6c1514f0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 >> 172.21.15.16:0/1907483271 conn(0x7f6b6c0b2250 msgr2=0x7f6b6c157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 shutdown_connections 2024-09-17T05:09:54.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.217+0000 7f6b72600640 1 -- 172.21.15.16:0/1907483271 wait complete. 2024-09-17T05:09:54.238 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:09:54.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:54.239 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:09:54.265 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:09:54.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:09:54.266 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:09:54.316 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:54.317 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:09:54.317 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:54.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:09:54.326 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:09:54.326 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:09:54.326 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:09:54.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:09:54.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 -- 172.21.15.16:0/2719770415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 msgr2=0x7fc3240cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 --2- 172.21.15.16:0/2719770415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc3240cccd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009840 tx=0x7fc30c01c820 comp rx=0 tx=0).stop 2024-09-17T05:09:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 -- 172.21.15.16:0/2719770415 shutdown_connections 2024-09-17T05:09:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 --2- 172.21.15.16:0/2719770415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc3240cccd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 --2- 172.21.15.16:0/2719770415 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc324154d70 0x7fc32415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 -- 172.21.15.16:0/2719770415 >> 172.21.15.16:0/2719770415 conn(0x7fc32406f9c0 msgr2=0x7fc3240b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 -- 172.21.15.16:0/2719770415 shutdown_connections 2024-09-17T05:09:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.529+0000 7fc329400640 1 -- 172.21.15.16:0/2719770415 wait complete. 2024-09-17T05:09:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 Processor -- start 2024-09-17T05:09:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 -- start start 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc32414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3240cddd0 0x7fc3241f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc324154d70 0x7fc3241637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc324165c20 con 0x7fc324154d70 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240ceb00 con 0x7fc3240cc8c0 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc329400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc324165da0 con 0x7fc3240cddd0 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc323e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc32414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc328a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc324154d70 0x7fc3241637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc323e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc32414db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:38532/0 (socket says 172.21.15.16:38532) 2024-09-17T05:09:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc323e00640 1 -- 172.21.15.16:0/3029961318 learned_addr learned my addr 172.21.15.16:0/3029961318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.533+0000 7fc31be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3240cddd0 0x7fc3241f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc323e00640 1 -- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3240cddd0 msgr2=0x7fc3241f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc323e00640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3240cddd0 0x7fc3241f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc323e00640 1 -- 172.21.15.16:0/3029961318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc324154d70 msgr2=0x7fc3241637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc323e00640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc324154d70 0x7fc3241637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc323e00640 1 -- 172.21.15.16:0/3029961318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3241f99f0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc323e00640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc32414db50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e7f0 tx=0x7fc31400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31400ccc0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc31400ce80 con 0x7fc3240cc8c0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314005280 con 0x7fc3240cc8c0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc324153910 con 0x7fc3240cc8c0 2024-09-17T05:09:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc324153ed0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3240cccd0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.537+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc314005420 con 0x7fc3240cc8c0 2024-09-17T05:09:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.541+0000 7fc320c00640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc300073200 0x7fc300075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.541+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7fc3140951f0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.541+0000 7fc31be00640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc300073200 0x7fc300075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.541+0000 7fc31be00640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc300073200 0x7fc300075710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0af000 tx=0x7fc30c01b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:54.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.545+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc3241f99f0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.753+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc32415bc70 con 0x7fc3240cc8c0 2024-09-17T05:09:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.753+0000 7fc320c00640 1 -- 172.21.15.16:0/3029961318 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7fc31405f9f0 con 0x7fc3240cc8c0 2024-09-17T05:09:54.754 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:54.754 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"btime":"2024-09-17T05:09:49:296334+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:48.485972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 100 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc300073200 msgr2=0x7fc300075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc300073200 0x7fc300075710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc30c0af000 tx=0x7fc30c01b060 comp rx=0 tx=0).stop 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 msgr2=0x7fc32414db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc32414db50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e7f0 tx=0x7fc31400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 shutdown_connections 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc300073200 0x7fc300075710 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc324154d70 0x7fc3241637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc3240cddd0 0x7fc3241f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 --2- 172.21.15.16:0/3029961318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3240cc8c0 0x7fc32414db50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 >> 172.21.15.16:0/3029961318 conn(0x7fc32406f9c0 msgr2=0x7fc3240b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 shutdown_connections 2024-09-17T05:09:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.757+0000 7fc329400640 1 -- 172.21.15.16:0/3029961318 wait complete. 2024-09-17T05:09:54.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 -- 172.21.15.16:0/3403421524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 msgr2=0x7f1e8015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 --2- 172.21.15.16:0/3403421524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 0x7f1e8015b950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009870 tx=0x7f1e7401c7d0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 -- 172.21.15.16:0/3403421524 shutdown_connections 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 --2- 172.21.15.16:0/3403421524 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 0x7f1e8015b950 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 -- 172.21.15.16:0/3403421524 >> 172.21.15.16:0/3403421524 conn(0x7f1e800d10c0 msgr2=0x7f1e800d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 -- 172.21.15.16:0/3403421524 shutdown_connections 2024-09-17T05:09:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 -- 172.21.15.16:0/3403421524 wait complete. 2024-09-17T05:09:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 Processor -- start 2024-09-17T05:09:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.937+0000 7f1e88800640 1 -- start start 2024-09-17T05:09:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8006e7a0 0x7f1e80144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 0x7f1e80165af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 0x7f1e8015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80169ce0 con 0x7f1e8006ebb0 2024-09-17T05:09:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e80169b60 con 0x7f1e8006e7a0 2024-09-17T05:09:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1e80169e60 con 0x7f1e80152540 2024-09-17T05:09:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 0x7f1e80165af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 0x7f1e80165af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41842/0 (socket says 172.21.15.16:41842) 2024-09-17T05:09:54.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 -- 172.21.15.16:0/212786051 learned_addr learned my addr 172.21.15.16:0/212786051 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85e00640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8006e7a0 0x7f1e80144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e86800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 0x7f1e8015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 -- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 msgr2=0x7f1e8015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 0x7f1e8015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 -- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8006e7a0 msgr2=0x7f1e80144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:54.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8006e7a0 0x7f1e80144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 -- 172.21.15.16:0/212786051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e80166570 con 0x7f1e8006ebb0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e85400640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 0x7f1e80165af0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f1e70001ec0 tx=0x7f1e7000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70017070 con 0x7f1e8006ebb0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e70003600 con 0x7f1e8006ebb0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e80164520 con 0x7f1e8006ebb0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70003a50 con 0x7f1e8006ebb0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e80164ae0 con 0x7f1e8006ebb0 2024-09-17T05:09:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8015b950 con 0x7f1e8006ebb0 2024-09-17T05:09:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.941+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e7001a070 con 0x7f1e8006ebb0 2024-09-17T05:09:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.945+0000 7f1e7a000640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e64073200 0x7f1e64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.945+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7f1e70093790 con 0x7f1e8006ebb0 2024-09-17T05:09:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.945+0000 7f1e85e00640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e64073200 0x7f1e64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:54.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.945+0000 7f1e85e00640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e64073200 0x7f1e64075710 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1e740af000 tx=0x7f1e7401b010 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:54.949+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e7005e570 con 0x7f1e8006ebb0 2024-09-17T05:09:55.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.157+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e8015cb80 con 0x7f1e8006ebb0 2024-09-17T05:09:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e7a000640 1 -- 172.21.15.16:0/212786051 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7f1e7005de70 con 0x7f1e8006ebb0 2024-09-17T05:09:55.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:55.160 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"btime":"2024-09-17T05:09:49:296334+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:48.485972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 100 2024-09-17T05:09:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e64073200 msgr2=0x7f1e64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e64073200 0x7f1e64075710 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1e740af000 tx=0x7f1e7401b010 comp rx=0 tx=0).stop 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 msgr2=0x7f1e80165af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 0x7f1e80165af0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f1e70001ec0 tx=0x7f1e7000c890 comp rx=0 tx=0).stop 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 shutdown_connections 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1e64073200 0x7f1e64075710 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1e80152540 0x7f1e8015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e8006ebb0 0x7f1e80165af0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 --2- 172.21.15.16:0/212786051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e8006e7a0 0x7f1e80144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 >> 172.21.15.16:0/212786051 conn(0x7f1e800d10c0 msgr2=0x7f1e8015d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 shutdown_connections 2024-09-17T05:09:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.161+0000 7f1e88800640 1 -- 172.21.15.16:0/212786051 wait complete. 2024-09-17T05:09:55.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:55.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 -- 172.21.15.16:0/336010362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 msgr2=0x7f78500b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 --2- 172.21.15.16:0/336010362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 0x7f78500b4270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7840009840 tx=0x7f784001c740 comp rx=0 tx=0).stop 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 -- 172.21.15.16:0/336010362 shutdown_connections 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 --2- 172.21.15.16:0/336010362 >> v2:172.21.15.104:3300/0 conn(0x7f78500cee10 0x7f78500b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 --2- 172.21.15.16:0/336010362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 0x7f78500b4270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 -- 172.21.15.16:0/336010362 >> 172.21.15.16:0/336010362 conn(0x7f785006f900 msgr2=0x7f78500b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 -- 172.21.15.16:0/336010362 shutdown_connections 2024-09-17T05:09:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 -- 172.21.15.16:0/336010362 wait complete. 2024-09-17T05:09:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 Processor -- start 2024-09-17T05:09:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.341+0000 7f7856600640 1 -- start start 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 0x7f78500af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f785006de60 0x7f7850145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 0x7f78500ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7850165d40 con 0x7f78500cee10 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7850165bc0 con 0x7f785006de60 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f7850165ec0 con 0x7f785006da50 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 0x7f78500ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 0x7f78500ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41854/0 (socket says 172.21.15.16:41854) 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 -- 172.21.15.16:0/36529902 learned_addr learned my addr 172.21.15.16:0/36529902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784f400640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 0x7f78500af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784ea00640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f785006de60 0x7f7850145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 -- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 msgr2=0x7f78500af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 0x7f78500af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 -- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f785006de60 msgr2=0x7f7850145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f785006de60 0x7f7850145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 -- 172.21.15.16:0/36529902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78500ca850 con 0x7f78500cee10 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784ea00640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f785006de60 0x7f7850145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f784fe00640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 0x7f78500ae6c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f784800ec70 tx=0x7f784800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848003b80 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7848003d20 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848004280 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78500cab30 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78500cb0f0 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.345+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78500ca5c0 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.349+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7848004420 con 0x7f78500cee10 2024-09-17T05:09:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.349+0000 7f7847400640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7830073200 0x7f7830075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.349+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7f7848095170 con 0x7f78500cee10 2024-09-17T05:09:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.349+0000 7f784f400640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7830073200 0x7f7830075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.349+0000 7f784f400640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7830073200 0x7f7830075710 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f78400af000 tx=0x7f78400b2040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:55.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.353+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f784805ffd0 con 0x7f78500cee10 2024-09-17T05:09:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.529+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78500b2f50 con 0x7f78500cee10 2024-09-17T05:09:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7847400640 1 -- 172.21.15.16:0/36529902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+17050 (secure 0 0 0) 0x7f784805f8d0 con 0x7f78500cee10 2024-09-17T05:09:55.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:55.533 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":138,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:09:51.310230+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:09:29.587461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-17T05:09:30.820166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.19","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:55.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7830073200 msgr2=0x7f7830075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7830073200 0x7f7830075710 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f78400af000 tx=0x7f78400b2040 comp rx=0 tx=0).stop 2024-09-17T05:09:55.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 msgr2=0x7f78500ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 0x7f78500ae6c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f784800ec70 tx=0x7f784800c6a0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 shutdown_connections 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7830073200 0x7f7830075710 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78500cee10 0x7f78500ae6c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f785006de60 0x7f7850145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 --2- 172.21.15.16:0/36529902 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f785006da50 0x7f78500af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 >> 172.21.15.16:0/36529902 conn(0x7f785006f900 msgr2=0x7f7850158720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 shutdown_connections 2024-09-17T05:09:55.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.533+0000 7f7856600640 1 -- 172.21.15.16:0/36529902 wait complete. 2024-09-17T05:09:55.555 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:09:55.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:55.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0ea200640 1 -- 172.21.15.16:0/2952682474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c8004d50 msgr2=0x7fe0c8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0ea200640 1 --2- 172.21.15.16:0/2952682474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0c8004d50 0x7fe0c8005160 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009eb0 tx=0x7fe0dc01d590 comp rx=0 tx=0).stop 2024-09-17T05:09:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0e3e00640 1 -- 172.21.15.16:0/2952682474 reap_dead start 2024-09-17T05:09:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0ea200640 1 -- 172.21.15.16:0/2952682474 shutdown_connections 2024-09-17T05:09:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0ea200640 1 -- 172.21.15.16:0/2952682474 >> 172.21.15.16:0/2952682474 conn(0x7fe0e406f900 msgr2=0x7fe0e40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0ea200640 1 -- 172.21.15.16:0/2952682474 shutdown_connections 2024-09-17T05:09:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.713+0000 7fe0ea200640 1 -- 172.21.15.16:0/2952682474 wait complete. 2024-09-17T05:09:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 Processor -- start 2024-09-17T05:09:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 -- start start 2024-09-17T05:09:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe0c8004d50 0x7fe0e4148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 0x7fe0e414af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe0e414b4c0 0x7fe0e40aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4073c50 con 0x7fe0e414ab70 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e4073ad0 con 0x7fe0e414b4c0 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e4073dd0 con 0x7fe0c8004d50 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 0x7fe0e414af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 0x7fe0e414af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41874/0 (socket says 172.21.15.16:41874) 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 -- 172.21.15.16:0/223409969 learned_addr learned my addr 172.21.15.16:0/223409969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e3400640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe0c8004d50 0x7fe0e4148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e3e00640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe0e414b4c0 0x7fe0e40aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 -- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe0c8004d50 msgr2=0x7fe0e4148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe0c8004d50 0x7fe0e4148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 -- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe0e414b4c0 msgr2=0x7fe0e40aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe0e414b4c0 0x7fe0e40aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 -- 172.21.15.16:0/223409969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e41f5b80 con 0x7fe0e414ab70 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0e2a00640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 0x7fe0e414af80 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000e650 tx=0x7fe0d000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0015070 con 0x7fe0e414ab70 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0d0010980 con 0x7fe0e414ab70 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0010d70 con 0x7fe0e414ab70 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.717+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e41f5e60 con 0x7fe0e414ab70 2024-09-17T05:09:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.721+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e40abb70 con 0x7fe0e414ab70 2024-09-17T05:09:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.721+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e41f58f0 con 0x7fe0e414ab70 2024-09-17T05:09:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.721+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe0d0018070 con 0x7fe0e414ab70 2024-09-17T05:09:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.721+0000 7fe0db400640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe0c4073200 0x7fe0c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.721+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7fe0d00946f0 con 0x7fe0e414ab70 2024-09-17T05:09:55.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.721+0000 7fe0e3400640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe0c4073200 0x7fe0c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:55.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.725+0000 7fe0e3400640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe0c4073200 0x7fe0c4075710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009c40 tx=0x7fe0d40091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.725+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe0d005ef90 con 0x7fe0e414ab70 2024-09-17T05:09:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.933+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0e40b3690 con 0x7fe0e414ab70 2024-09-17T05:09:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.933+0000 7fe0db400640 1 -- 172.21.15.16:0/223409969 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7fe0d00639c0 con 0x7fe0e414ab70 2024-09-17T05:09:55.935 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:55.936 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"btime":"2024-09-17T05:09:49:296334+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:48.485972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 100 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe0c4073200 msgr2=0x7fe0c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe0c4073200 0x7fe0c4075710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4009c40 tx=0x7fe0d40091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 msgr2=0x7fe0e414af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 0x7fe0e414af80 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe0d000e650 tx=0x7fe0d000ea50 comp rx=0 tx=0).stop 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 shutdown_connections 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe0c4073200 0x7fe0c4075710 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe0e414b4c0 0x7fe0e40aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0e414ab70 0x7fe0e414af80 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 --2- 172.21.15.16:0/223409969 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe0c8004d50 0x7fe0e4148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 >> 172.21.15.16:0/223409969 conn(0x7fe0e406f900 msgr2=0x7fe0e40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 shutdown_connections 2024-09-17T05:09:55.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:55.937+0000 7fe0ea200640 1 -- 172.21.15.16:0/223409969 wait complete. 2024-09-17T05:09:55.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:09:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefe6600640 1 -- 172.21.15.16:0/508597794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc4004da0 msgr2=0x7fefc4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefe6600640 1 --2- 172.21.15.16:0/508597794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc4004da0 0x7fefc4005230 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fefe0072d60 tx=0x7fefd801d590 comp rx=0 tx=0).stop 2024-09-17T05:09:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefdfe00640 1 -- 172.21.15.16:0/508597794 reap_dead start 2024-09-17T05:09:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefe6600640 1 -- 172.21.15.16:0/508597794 shutdown_connections 2024-09-17T05:09:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefe6600640 1 -- 172.21.15.16:0/508597794 >> 172.21.15.16:0/508597794 conn(0x7fefe006f900 msgr2=0x7fefe00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefe6600640 1 -- 172.21.15.16:0/508597794 shutdown_connections 2024-09-17T05:09:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.117+0000 7fefe6600640 1 -- 172.21.15.16:0/508597794 wait complete. 2024-09-17T05:09:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 Processor -- start 2024-09-17T05:09:56.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 -- start start 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc4004da0 0x7fefe015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 0x7fefe01f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe01f5e30 0x7fefe01f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0073ce0 con 0x7fefe015ab70 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0073b60 con 0x7fefc4004da0 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fefe0073e60 con 0x7fefe01f5e30 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 0x7fefe01f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 0x7fefe01f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41884/0 (socket says 172.21.15.16:41884) 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 -- 172.21.15.16:0/2534425732 learned_addr learned my addr 172.21.15.16:0/2534425732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefdfe00640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe01f5e30 0x7fefe01f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefdf400640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc4004da0 0x7fefe015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 -- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe01f5e30 msgr2=0x7fefe01f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe01f5e30 0x7fefe01f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 -- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc4004da0 msgr2=0x7fefe015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc4004da0 0x7fefe015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 -- 172.21.15.16:0/2534425732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe015af80 con 0x7fefe015ab70 2024-09-17T05:09:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefdfe00640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe01f5e30 0x7fefe01f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:56.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd6a00640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 0x7fefe01f58f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006450 tx=0x7fefc80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8015070 con 0x7fefe015ab70 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefc8011900 con 0x7fefe015ab70 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe00ce520 con 0x7fefe015ab70 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8011d50 con 0x7fefe015ab70 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.121+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe00ceae0 con 0x7fefe015ab70 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.125+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe00743c0 con 0x7fefe015ab70 2024-09-17T05:09:56.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.125+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefc8018070 con 0x7fefe015ab70 2024-09-17T05:09:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.125+0000 7fefd7e00640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefbc073230 0x7fefbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.125+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7fefc8095180 con 0x7fefe015ab70 2024-09-17T05:09:56.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.125+0000 7fefdf400640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefbc073230 0x7fefbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.125+0000 7fefdf400640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefbc073230 0x7fefbc075740 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fefd0000c10 tx=0x7fefd0006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.129+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fefe015af80 con 0x7fefe015ab70 2024-09-17T05:09:56.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.337+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefe00ce270 con 0x7fefe015ab70 2024-09-17T05:09:56.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.341+0000 7fefd7e00640 1 -- 172.21.15.16:0/2534425732 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7fefc805ff50 con 0x7fefe015ab70 2024-09-17T05:09:56.342 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:56.342 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"btime":"2024-09-17T05:09:49:296334+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:09:31.960966+0000","modified":"2024-09-17T05:09:48.485972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7648},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7648":{"gid":7648,"name":"b","rank":0,"incarnation":94,"state":"up:active","state_seq":7,"addr":"172.21.15.104:6837/1269584978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1269584978},{"type":"v1","addr":"172.21.15.104:6837","nonce":1269584978}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7648,"qdb_cluster":[7648]},"id":10}]} 2024-09-17T05:09:56.342 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 100 2024-09-17T05:09:56.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.341+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefbc073230 msgr2=0x7fefbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.341+0000 7fefe6600640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefbc073230 0x7fefbc075740 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fefd0000c10 tx=0x7fefd0006b20 comp rx=0 tx=0).stop 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.341+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 msgr2=0x7fefe01f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.341+0000 7fefe6600640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 0x7fefe01f58f0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fefc8006450 tx=0x7fefc80039d0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 shutdown_connections 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fefbc073230 0x7fefbc075740 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fefe01f5e30 0x7fefe01f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefe015ab70 0x7fefe01f58f0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 --2- 172.21.15.16:0/2534425732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fefc4004da0 0x7fefe015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 >> 172.21.15.16:0/2534425732 conn(0x7fefe006f900 msgr2=0x7fefe00b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 shutdown_connections 2024-09-17T05:09:56.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.345+0000 7fefe6600640 1 -- 172.21.15.16:0/2534425732 wait complete. 2024-09-17T05:09:56.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:09:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 -- 172.21.15.16:0/1636930356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b40b2f50 msgr2=0x7ff1b415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1636930356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b40b2f50 0x7ff1b415bb60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009840 tx=0x7ff1a401c790 comp rx=0 tx=0).stop 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 -- 172.21.15.16:0/1636930356 shutdown_connections 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1636930356 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1b415c0a0 0x7ff1b415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1636930356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b40b2f50 0x7ff1b415bb60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 -- 172.21.15.16:0/1636930356 >> 172.21.15.16:0/1636930356 conn(0x7ff1b406f900 msgr2=0x7ff1b40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 -- 172.21.15.16:0/1636930356 shutdown_connections 2024-09-17T05:09:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.525+0000 7ff1bac00640 1 -- 172.21.15.16:0/1636930356 wait complete. 2024-09-17T05:09:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 Processor -- start 2024-09-17T05:09:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 -- start start 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 0x7ff1b40cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b415c0a0 0x7ff1b406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1b415f150 0x7ff1b414f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4169150 con 0x7ff1b40b2f50 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4168fd0 con 0x7ff1b415c0a0 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b41692d0 con 0x7ff1b415f150 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 0x7ff1b40cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 0x7ff1b40cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41900/0 (socket says 172.21.15.16:41900) 2024-09-17T05:09:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 -- 172.21.15.16:0/1550763677 learned_addr learned my addr 172.21.15.16:0/1550763677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1ab400640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b415c0a0 0x7ff1b406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b8c00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1b415f150 0x7ff1b414f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 -- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1b415f150 msgr2=0x7ff1b414f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1b415f150 0x7ff1b414f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 -- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b415c0a0 msgr2=0x7ff1b406d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b415c0a0 0x7ff1b406d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 -- 172.21.15.16:0/1550763677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41f9380 con 0x7ff1b40b2f50 2024-09-17T05:09:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b3e00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 0x7ff1b40cb830 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009790 tx=0x7ff1a40a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40bb070 con 0x7ff1b40b2f50 2024-09-17T05:09:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1a4005030 con 0x7ff1b40b2f50 2024-09-17T05:09:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40b2040 con 0x7ff1b40b2f50 2024-09-17T05:09:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41f9660 con 0x7ff1b40b2f50 2024-09-17T05:09:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.529+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b41f9c20 con 0x7ff1b40b2f50 2024-09-17T05:09:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.533+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b415bb60 con 0x7ff1b40b2f50 2024-09-17T05:09:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.533+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff1a4005460 con 0x7ff1b40b2f50 2024-09-17T05:09:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.533+0000 7ff1b0c00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff190073200 0x7ff190075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.533+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7ff1a412dc90 con 0x7ff1b40b2f50 2024-09-17T05:09:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.533+0000 7ff1ab400640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff190073200 0x7ff190075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.533+0000 7ff1ab400640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff190073200 0x7ff190075710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff1b415df10 tx=0x7ff19c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.537+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff1a40f1db0 con 0x7ff1b40b2f50 2024-09-17T05:09:56.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1b415e840 con 0x7ff1b40b2f50 2024-09-17T05:09:56.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1b0c00640 1 -- 172.21.15.16:0/1550763677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+17050 (secure 0 0 0) 0x7ff1a4016410 con 0x7ff1b40b2f50 2024-09-17T05:09:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:09:56.714 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":138,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:09:51.310230+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:09:29.587461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-09-17T05:09:30.820166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.19","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:09:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff190073200 msgr2=0x7ff190075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff190073200 0x7ff190075710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff1b415df10 tx=0x7ff19c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 msgr2=0x7ff1b40cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 0x7ff1b40cb830 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009790 tx=0x7ff1a40a5780 comp rx=0 tx=0).stop 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 shutdown_connections 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff190073200 0x7ff190075710 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff1b415f150 0x7ff1b414f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1b415c0a0 0x7ff1b406d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 --2- 172.21.15.16:0/1550763677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b40b2f50 0x7ff1b40cb830 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 >> 172.21.15.16:0/1550763677 conn(0x7ff1b406f900 msgr2=0x7ff1b41514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.713+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 shutdown_connections 2024-09-17T05:09:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.717+0000 7ff1bac00640 1 -- 172.21.15.16:0/1550763677 wait complete. 2024-09-17T05:09:56.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:09:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 -- 172.21.15.16:0/2094249203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc300073f30 msgr2=0x7fc300074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 --2- 172.21.15.16:0/2094249203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc300073f30 0x7fc300074340 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b2ca0 tx=0x7fc2f801caf0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 -- 172.21.15.16:0/2094249203 shutdown_connections 2024-09-17T05:09:56.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 --2- 172.21.15.16:0/2094249203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc300073f30 0x7fc300074340 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 --2- 172.21.15.16:0/2094249203 >> v2:172.21.15.104:3300/0 conn(0x7fc3000d0940 0x7fc3001590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 -- 172.21.15.16:0/2094249203 >> 172.21.15.16:0/2094249203 conn(0x7fc3000b2250 msgr2=0x7fc3000d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 -- 172.21.15.16:0/2094249203 shutdown_connections 2024-09-17T05:09:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.893+0000 7fc307800640 1 -- 172.21.15.16:0/2094249203 wait complete. 2024-09-17T05:09:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 Processor -- start 2024-09-17T05:09:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 -- start start 2024-09-17T05:09:56.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc300073f30 0x7fc3001575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3000ca6b0 0x7fc3001f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3000d0940 0x7fc3000b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc300168b60 con 0x7fc3000d0940 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001689e0 con 0x7fc3000ca6b0 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc307800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc300168ce0 con 0x7fc300073f30 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc305800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3000d0940 0x7fc3000b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc305800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3000d0940 0x7fc3000b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41914/0 (socket says 172.21.15.16:41914) 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.897+0000 7fc305800640 1 -- 172.21.15.16:0/2745119557 learned_addr learned my addr 172.21.15.16:0/2745119557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc2f7e00640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3000ca6b0 0x7fc3001f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc304e00640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc300073f30 0x7fc3001575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc305800640 1 -- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc300073f30 msgr2=0x7fc3001575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc305800640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc300073f30 0x7fc3001575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc305800640 1 -- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3000ca6b0 msgr2=0x7fc3001f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc305800640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3000ca6b0 0x7fc3001f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc305800640 1 -- 172.21.15.16:0/2745119557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3000b49e0 con 0x7fc3000d0940 2024-09-17T05:09:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc304e00640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc300073f30 0x7fc3001575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc2f7e00640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3000ca6b0 0x7fc3001f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:09:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc305800640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3000d0940 0x7fc3000b5500 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009b10 tx=0x7fc2f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80bc070 con 0x7fc3000d0940 2024-09-17T05:09:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2f80042f0 con 0x7fc3000d0940 2024-09-17T05:09:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc307800640 1 -- 172.21.15.16:0/2745119557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3001f95c0 con 0x7fc3000d0940 2024-09-17T05:09:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80b2040 con 0x7fc3000d0940 2024-09-17T05:09:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc307800640 1 -- 172.21.15.16:0/2745119557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3000b3160 con 0x7fc3000d0940 2024-09-17T05:09:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc307800640 1 -- 172.21.15.16:0/2745119557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3000b2f50 con 0x7fc3000d0940 2024-09-17T05:09:56.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.901+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc2f80bf070 con 0x7fc3000d0940 2024-09-17T05:09:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.905+0000 7fc2fd600640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc2dc073200 0x7fc2dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:56.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.905+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7fc2f812e240 con 0x7fc3000d0940 2024-09-17T05:09:56.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.905+0000 7fc304e00640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc2dc073200 0x7fc2dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:56.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.905+0000 7fc304e00640 1 --2- 172.21.15.16:0/2745119557 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc2dc073200 0x7fc2dc075710 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009b90 tx=0x7fc2f00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:56.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:56.909+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc2f8016770 con 0x7fc3000d0940 2024-09-17T05:09:57.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.117+0000 7fc307800640 1 -- 172.21.15.16:0/2745119557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc3001f9350 con 0x7fc3000d0940 2024-09-17T05:09:57.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.117+0000 7fc2fd600640 1 -- 172.21.15.16:0/2745119557 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=-1 MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it v100) ==== 316+0+0 (secure 0 0 0) 0x7fc2f80f9010 con 0x7fc3000d0940 2024-09-17T05:09:57.117 INFO:teuthology.orchestra.run.smithi016.stderr:Error EPERM: MDS has one of two health warnings which could extend recovery: MDS_TRIM or MDS_CACHE_OVERSIZED. MDS failover is not recommended since it might cause unexpected file system unavailability. If you wish to proceed, pass --yes-i-really-mean-it 2024-09-17T05:09:57.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:09:57.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs --yes-i-really-mean-it 2024-09-17T05:09:57.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 -- 172.21.15.16:0/2264616290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 msgr2=0x7f34100cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 --2- 172.21.15.16:0/2264616290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34100cb980 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3408009bc0 tx=0x7f340801cae0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 -- 172.21.15.16:0/2264616290 shutdown_connections 2024-09-17T05:09:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 --2- 172.21.15.16:0/2264616290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34100cb980 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 -- 172.21.15.16:0/2264616290 >> 172.21.15.16:0/2264616290 conn(0x7f34100b2250 msgr2=0x7f34100d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 -- 172.21.15.16:0/2264616290 shutdown_connections 2024-09-17T05:09:57.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.285+0000 7f3417800640 1 -- 172.21.15.16:0/2264616290 wait complete. 2024-09-17T05:09:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 Processor -- start 2024-09-17T05:09:57.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 -- start start 2024-09-17T05:09:57.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3410073170 0x7f34100ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34100b4240 0x7f341014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34101533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f341016aec0 con 0x7f34100cb570 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f341016ad40 con 0x7f34100b4240 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3417800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f341016b040 con 0x7f3410073170 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3415800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34101533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3415800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34101533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59850/0 (socket says 172.21.15.16:59850) 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.289+0000 7f3415800640 1 -- 172.21.15.16:0/1927404344 learned_addr learned my addr 172.21.15.16:0/1927404344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3414e00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3410073170 0x7f34100ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f340fe00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34100b4240 0x7f341014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3415800640 1 -- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3410073170 msgr2=0x7f34100ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3415800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3410073170 0x7f34100ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3415800640 1 -- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34100b4240 msgr2=0x7f341014eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3415800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34100b4240 0x7f341014eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3415800640 1 -- 172.21.15.16:0/1927404344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34100ae420 con 0x7f34100cb570 2024-09-17T05:09:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3414e00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3410073170 0x7f34100ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f340fe00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34100b4240 0x7f341014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:09:57.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3415800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34101533b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f340801c550 tx=0x7f3408005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34080b2040 con 0x7f34100cb570 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34080bb070 con 0x7f34100cb570 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408004570 con 0x7f34100cb570 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34100ae700 con 0x7f34100cb570 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34100af1c0 con 0x7f34100cb570 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34100cb980 con 0x7f34100cb570 2024-09-17T05:09:57.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.293+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3408004710 con 0x7f34100cb570 2024-09-17T05:09:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.297+0000 7f340cc00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33f0073200 0x7f33f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.297+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 6130+0+0 (secure 0 0 0) 0x7f340812eec0 con 0x7f34100cb570 2024-09-17T05:09:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.297+0000 7f3414e00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33f0073200 0x7f33f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.297+0000 7f3414e00640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33f0073200 0x7f33f0075710 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3404001230 tx=0x7f3404009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:57.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.301+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3408016790 con 0x7f34100cb570 2024-09-17T05:09:57.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.509+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f34100c6b40 con 0x7f34100cb570 2024-09-17T05:09:57.511 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:09:57.509+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:09:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f340cc00640 1 -- 172.21.15.16:0/1927404344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 191+0+0 (secure 0 0 0) 0x7f34080f9c90 con 0x7f34100cb570 2024-09-17T05:09:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:09:57.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33f0073200 msgr2=0x7f33f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33f0073200 0x7f33f0075710 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3404001230 tx=0x7f3404009040 comp rx=0 tx=0).stop 2024-09-17T05:09:57.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 msgr2=0x7f34101533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34101533b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f340801c550 tx=0x7f3408005ae0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 shutdown_connections 2024-09-17T05:09:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33f0073200 0x7f33f0075710 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34100cb570 0x7f34101533b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34100b4240 0x7f341014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 --2- 172.21.15.16:0/1927404344 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3410073170 0x7f34100ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.513+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 >> 172.21.15.16:0/1927404344 conn(0x7f34100b2250 msgr2=0x7f34100cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.517+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 shutdown_connections 2024-09-17T05:09:57.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.517+0000 7f3417800640 1 -- 172.21.15.16:0/1927404344 wait complete. 2024-09-17T05:09:57.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:09:57.655 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:09:57.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 -- 172.21.15.16:0/1059822433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074e30 msgr2=0x7f2a0c0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 --2- 172.21.15.16:0/1059822433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074e30 0x7f2a0c0ce5d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009bc0 tx=0x7f29f401cac0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 -- 172.21.15.16:0/1059822433 shutdown_connections 2024-09-17T05:09:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 --2- 172.21.15.16:0/1059822433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074e30 0x7f2a0c0ce5d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 -- 172.21.15.16:0/1059822433 >> 172.21.15.16:0/1059822433 conn(0x7f2a0c0b2250 msgr2=0x7f2a0c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 -- 172.21.15.16:0/1059822433 shutdown_connections 2024-09-17T05:09:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.697+0000 7f2a12a00640 1 -- 172.21.15.16:0/1059822433 wait complete. 2024-09-17T05:09:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 Processor -- start 2024-09-17T05:09:57.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 -- start start 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074a20 0x7f2a0c0b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 0x7f2a0c14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a0c15f6f0 0x7f2a0c0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c16a060 con 0x7f2a0c074e30 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c169ee0 con 0x7f2a0c074a20 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c16a1e0 con 0x7f2a0c15f6f0 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 0x7f2a0c14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074a20 0x7f2a0c0b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 0x7f2a0c14dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59852/0 (socket says 172.21.15.16:59852) 2024-09-17T05:09:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 -- 172.21.15.16:0/3007412973 learned_addr learned my addr 172.21.15.16:0/3007412973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a10a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a0c15f6f0 0x7f2a0c0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 -- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a0c15f6f0 msgr2=0x7f2a0c0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a0c15f6f0 0x7f2a0c0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 -- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074a20 msgr2=0x7f2a0c0b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074a20 0x7f2a0c0b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 -- 172.21.15.16:0/3007412973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c0b3060 con 0x7f2a0c074e30 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a0be00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074a20 0x7f2a0c0b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a03400640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 0x7f2a0c14dda0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009b10 tx=0x7f29f40a5f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f40bb070 con 0x7f2a0c074e30 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29f4004e00 con 0x7f2a0c074e30 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c0b3340 con 0x7f2a0c074e30 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f40b2040 con 0x7f2a0c074e30 2024-09-17T05:09:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.701+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c0ad870 con 0x7f2a0c074e30 2024-09-17T05:09:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.705+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c15efe0 con 0x7f2a0c074e30 2024-09-17T05:09:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.705+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f29f40be070 con 0x7f2a0c074e30 2024-09-17T05:09:57.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.705+0000 7f2a08c00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f29e8073200 0x7f29e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:57.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.705+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6216+0+0 (secure 0 0 0) 0x7f29f412dc60 con 0x7f2a0c074e30 2024-09-17T05:09:57.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.705+0000 7f2a0be00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f29e8073200 0x7f29e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:57.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.709+0000 7f2a0be00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f29e8073200 0x7f29e8075710 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007fd0 tx=0x7f29fc009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:57.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.713+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f29f40166b0 con 0x7f2a0c074e30 2024-09-17T05:09:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:57.925+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2a0c153170 con 0x7f2a0c074e30 2024-09-17T05:09:58.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a08c00640 1 -- 172.21.15.16:0/3007412973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) ==== 104+0+0 (secure 0 0 0) 0x7f29f40f8390 con 0x7f2a0c074e30 2024-09-17T05:09:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f29e8073200 msgr2=0x7f29e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f29e8073200 0x7f29e8075710 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007fd0 tx=0x7f29fc009040 comp rx=0 tx=0).stop 2024-09-17T05:09:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 msgr2=0x7f2a0c14dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 0x7f2a0c14dda0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009b10 tx=0x7f29f40a5f90 comp rx=0 tx=0).stop 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 shutdown_connections 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f29e8073200 0x7f29e8075710 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2a0c15f6f0 0x7f2a0c0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a0c074e30 0x7f2a0c14dda0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 --2- 172.21.15.16:0/3007412973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a0c074a20 0x7f2a0c0b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 >> 172.21.15.16:0/3007412973 conn(0x7f2a0c0b2250 msgr2=0x7f2a0c15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 shutdown_connections 2024-09-17T05:09:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.521+0000 7f2a12a00640 1 -- 172.21.15.16:0/3007412973 wait complete. 2024-09-17T05:09:58.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:09:58.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 -- 172.21.15.16:0/2505838098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40ce400 msgr2=0x7f71e40b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 --2- 172.21.15.16:0/2505838098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40ce400 0x7f71e40b5270 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009900 tx=0x7f71cc01c8b0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 -- 172.21.15.16:0/2505838098 shutdown_connections 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 --2- 172.21.15.16:0/2505838098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40ce400 0x7f71e40b5270 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 --2- 172.21.15.16:0/2505838098 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71e40cdff0 0x7f71e40b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 -- 172.21.15.16:0/2505838098 >> 172.21.15.16:0/2505838098 conn(0x7f71e4073ea0 msgr2=0x7f71e40742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 -- 172.21.15.16:0/2505838098 shutdown_connections 2024-09-17T05:09:58.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.713+0000 7f71e9000640 1 -- 172.21.15.16:0/2505838098 wait complete. 2024-09-17T05:09:58.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 Processor -- start 2024-09-17T05:09:58.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 -- start start 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 0x7f71e4157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71e40ce400 0x7f71e41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71e415f1f0 0x7f71e40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4169170 con 0x7f71e40cdff0 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4168ff0 con 0x7f71e415f1f0 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f71e41692f0 con 0x7f71e40ce400 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 0x7f71e4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 0x7f71e4157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59862/0 (socket says 172.21.15.16:59862) 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 -- 172.21.15.16:0/2402265394 learned_addr learned my addr 172.21.15.16:0/2402265394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:58.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71dbe00640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71e40ce400 0x7f71e41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e2800640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71e415f1f0 0x7f71e40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 -- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71e40ce400 msgr2=0x7f71e41509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71e40ce400 0x7f71e41509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 -- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71e415f1f0 msgr2=0x7f71e40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71e415f1f0 0x7f71e40b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 -- 172.21.15.16:0/2402265394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e41f6fb0 con 0x7f71e40cdff0 2024-09-17T05:09:58.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e1e00640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 0x7f71e4157770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f71d400e7f0 tx=0x7f71d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d400cc60 con 0x7f71e40cdff0 2024-09-17T05:09:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71d400ce20 con 0x7f71e40cdff0 2024-09-17T05:09:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d4005280 con 0x7f71e40cdff0 2024-09-17T05:09:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e4157cb0 con 0x7f71e40cdff0 2024-09-17T05:09:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.717+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e41fabd0 con 0x7f71e40cdff0 2024-09-17T05:09:58.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.721+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e415f600 con 0x7f71e40cdff0 2024-09-17T05:09:58.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.721+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71d4005420 con 0x7f71e40cdff0 2024-09-17T05:09:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.721+0000 7f71da000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71c0073200 0x7f71c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.721+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 6216+0+0 (secure 0 0 0) 0x7f71d4094e50 con 0x7f71e40cdff0 2024-09-17T05:09:58.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.721+0000 7f71dbe00640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71c0073200 0x7f71c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:58.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.721+0000 7f71dbe00640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71c0073200 0x7f71c0075710 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0af000 tx=0x7f71cc01b0f0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:58.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.725+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f71d405fbd0 con 0x7f71e40cdff0 2024-09-17T05:09:58.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:58.901+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71e415a860 con 0x7f71e40cdff0 2024-09-17T05:09:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.521+0000 7f71da000640 1 -- 172.21.15.16:0/2402265394 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) ==== 181+0+0 (secure 0 0 0) 0x7f71d405f4d0 con 0x7f71e40cdff0 2024-09-17T05:09:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:09:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71c0073200 msgr2=0x7f71c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71c0073200 0x7f71c0075710 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0af000 tx=0x7f71cc01b0f0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 msgr2=0x7f71e4157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:59.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 0x7f71e4157770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f71d400e7f0 tx=0x7f71d400ecb0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 shutdown_connections 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f71c0073200 0x7f71c0075710 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71e415f1f0 0x7f71e40b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f71e40ce400 0x7f71e41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 --2- 172.21.15.16:0/2402265394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e40cdff0 0x7f71e4157770 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 >> 172.21.15.16:0/2402265394 conn(0x7f71e4073ea0 msgr2=0x7f71e4068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:59.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 shutdown_connections 2024-09-17T05:09:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.525+0000 7f71e9000640 1 -- 172.21.15.16:0/2402265394 wait complete. 2024-09-17T05:09:59.550 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:09:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 -- 172.21.15.16:0/845153070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5618074120 msgr2=0x7f5618074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 --2- 172.21.15.16:0/845153070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5618074120 0x7f5618074530 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5618073800 tx=0x7f561001cd50 comp rx=0 tx=0).stop 2024-09-17T05:09:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 -- 172.21.15.16:0/845153070 shutdown_connections 2024-09-17T05:09:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 --2- 172.21.15.16:0/845153070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5618074120 0x7f5618074530 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 --2- 172.21.15.16:0/845153070 >> v2:172.21.15.104:3300/0 conn(0x7f561806e6b0 0x7f5618073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 -- 172.21.15.16:0/845153070 >> 172.21.15.16:0/845153070 conn(0x7f56180d0050 msgr2=0x7f56180d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:09:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 -- 172.21.15.16:0/845153070 shutdown_connections 2024-09-17T05:09:59.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.713+0000 7f561d600640 1 -- 172.21.15.16:0/845153070 wait complete. 2024-09-17T05:09:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 Processor -- start 2024-09-17T05:09:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 -- start start 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5618074120 0x7f561806d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56180b4620 0x7f5618151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 0x7f5618153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5618168aa0 con 0x7f561809d7b0 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5618168920 con 0x7f56180b4620 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5618168c20 con 0x7f5618074120 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 0x7f5618153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 0x7f5618153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59872/0 (socket says 172.21.15.16:59872) 2024-09-17T05:09:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 -- 172.21.15.16:0/537431046 learned_addr learned my addr 172.21.15.16:0/537431046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f560e000640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56180b4620 0x7f5618151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5616a00640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5618074120 0x7f561806d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 -- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5618074120 msgr2=0x7f561806d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5618074120 0x7f561806d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 -- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56180b4620 msgr2=0x7f5618151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56180b4620 0x7f5618151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 -- 172.21.15.16:0/537431046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5618156290 con 0x7f561809d7b0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f5617400640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 0x7f5618153a90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5610009bd0 tx=0x7f5610005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56100bc070 con 0x7f561809d7b0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56100ad7f0 con 0x7f561809d7b0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5618158b50 con 0x7f561809d7b0 2024-09-17T05:09:59.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56100b2040 con 0x7f561809d7b0 2024-09-17T05:09:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.717+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5618159110 con 0x7f561809d7b0 2024-09-17T05:09:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.721+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5618073be0 con 0x7f561809d7b0 2024-09-17T05:09:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.721+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f56100bf070 con 0x7f561809d7b0 2024-09-17T05:09:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.721+0000 7f560f400640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f55f4073200 0x7f55f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:09:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.721+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5749+0+0 (secure 0 0 0) 0x7f561012e0b0 con 0x7f561809d7b0 2024-09-17T05:09:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.721+0000 7f5616a00640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f55f4073200 0x7f55f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:09:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.721+0000 7f5616a00640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f55f4073200 0x7f55f4075710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f561809e130 tx=0x7f5608009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:09:59.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.725+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5610016970 con 0x7f561809d7b0 2024-09-17T05:09:59.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:09:59.901+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5618159fd0 con 0x7f561809d7b0 2024-09-17T05:10:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.529+0000 7f560f400640 1 -- 172.21.15.16:0/537431046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) ==== 169+0+0 (secure 0 0 0) 0x7f56100f90a0 con 0x7f561809d7b0 2024-09-17T05:10:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:10:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f55f4073200 msgr2=0x7f55f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f55f4073200 0x7f55f4075710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f561809e130 tx=0x7f5608009040 comp rx=0 tx=0).stop 2024-09-17T05:10:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 msgr2=0x7f5618153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 0x7f5618153a90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5610009bd0 tx=0x7f5610005b10 comp rx=0 tx=0).stop 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 shutdown_connections 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f55f4073200 0x7f55f4075710 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561809d7b0 0x7f5618153a90 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56180b4620 0x7f5618151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 --2- 172.21.15.16:0/537431046 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5618074120 0x7f561806d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 >> 172.21.15.16:0/537431046 conn(0x7f56180d0050 msgr2=0x7f561815ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 shutdown_connections 2024-09-17T05:10:00.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.533+0000 7f561d600640 1 -- 172.21.15.16:0/537431046 wait complete. 2024-09-17T05:10:00.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:10:00.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 -- 172.21.15.16:0/3662718156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5784074e30 msgr2=0x7f57840ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 --2- 172.21.15.16:0/3662718156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5784074e30 0x7f57840ce5d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5784156030 tx=0x7f577001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 -- 172.21.15.16:0/3662718156 shutdown_connections 2024-09-17T05:10:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 --2- 172.21.15.16:0/3662718156 >> v2:172.21.15.104:3300/0 conn(0x7f578415f6f0 0x7f57840ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 --2- 172.21.15.16:0/3662718156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5784074e30 0x7f57840ce5d0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 -- 172.21.15.16:0/3662718156 >> 172.21.15.16:0/3662718156 conn(0x7f57840b2250 msgr2=0x7f57840d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 -- 172.21.15.16:0/3662718156 shutdown_connections 2024-09-17T05:10:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.717+0000 7f578b800640 1 -- 172.21.15.16:0/3662718156 wait complete. 2024-09-17T05:10:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 Processor -- start 2024-09-17T05:10:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 -- start start 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5784074a20 0x7f5784157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5784074e30 0x7f57841580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 0x7f57841f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5784168e20 con 0x7f578415f6f0 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5784168ca0 con 0x7f5784074a20 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5784168fa0 con 0x7f5784074e30 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 0x7f57841f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 0x7f57841f9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59886/0 (socket says 172.21.15.16:59886) 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 -- 172.21.15.16:0/230128673 learned_addr learned my addr 172.21.15.16:0/230128673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5783e00640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5784074e30 0x7f57841580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5788e00640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5784074a20 0x7f5784157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 -- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5784074e30 msgr2=0x7f57841580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5784074e30 0x7f57841580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 -- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5784074a20 msgr2=0x7f5784157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5784074a20 0x7f5784157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 -- 172.21.15.16:0/230128673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57840aad80 con 0x7f578415f6f0 2024-09-17T05:10:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5789800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 0x7f57841f9dc0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f577c00e940 tx=0x7f577c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c004d30 con 0x7f578415f6f0 2024-09-17T05:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f577c00cb20 con 0x7f578415f6f0 2024-09-17T05:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c005020 con 0x7f578415f6f0 2024-09-17T05:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57840ab060 con 0x7f578415f6f0 2024-09-17T05:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.721+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57840ab620 con 0x7f578415f6f0 2024-09-17T05:10:00.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.725+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57840aba00 con 0x7f578415f6f0 2024-09-17T05:10:00.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.725+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f577c0051c0 con 0x7f578415f6f0 2024-09-17T05:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.725+0000 7f5780c00640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5764073200 0x7f5764075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.725+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7f577c094960 con 0x7f578415f6f0 2024-09-17T05:10:00.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.725+0000 7f5788e00640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5764073200 0x7f5764075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:00.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.725+0000 7f5788e00640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5764073200 0x7f5764075710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f5778009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:00.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.733+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f57840aba00 con 0x7f578415f6f0 2024-09-17T05:10:00.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.913+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f578415efe0 con 0x7f578415f6f0 2024-09-17T05:10:00.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.917+0000 7f5780c00640 1 -- 172.21.15.16:0/230128673 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v68) ==== 71+0+0 (secure 0 0 0) 0x7f577c068020 con 0x7f578415f6f0 2024-09-17T05:10:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5764073200 msgr2=0x7f5764075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5764073200 0x7f5764075710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f5778009040 comp rx=0 tx=0).stop 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 msgr2=0x7f57841f9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 0x7f57841f9dc0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f577c00e940 tx=0x7f577c00ee00 comp rx=0 tx=0).stop 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 shutdown_connections 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5764073200 0x7f5764075710 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f578415f6f0 0x7f57841f9dc0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5784074e30 0x7f57841580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 --2- 172.21.15.16:0/230128673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5784074a20 0x7f5784157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 >> 172.21.15.16:0/230128673 conn(0x7f57840b2250 msgr2=0x7f57840ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 shutdown_connections 2024-09-17T05:10:00.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:00.921+0000 7f578b800640 1 -- 172.21.15.16:0/230128673 wait complete. 2024-09-17T05:10:00.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-09-17T05:10:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 -- 172.21.15.16:0/615098443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 msgr2=0x7fddb015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 --2- 172.21.15.16:0/615098443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb015bb60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fddb006e960 tx=0x7fdda001c770 comp rx=0 tx=0).stop 2024-09-17T05:10:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 -- 172.21.15.16:0/615098443 shutdown_connections 2024-09-17T05:10:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 --2- 172.21.15.16:0/615098443 >> v2:172.21.15.104:3300/0 conn(0x7fddb015c0a0 0x7fddb015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 --2- 172.21.15.16:0/615098443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb015bb60 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 -- 172.21.15.16:0/615098443 >> 172.21.15.16:0/615098443 conn(0x7fddb006f900 msgr2=0x7fddb00b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 -- 172.21.15.16:0/615098443 shutdown_connections 2024-09-17T05:10:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.101+0000 7fddb5000640 1 -- 172.21.15.16:0/615098443 wait complete. 2024-09-17T05:10:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 Processor -- start 2024-09-17T05:10:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 -- start start 2024-09-17T05:10:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb01fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddb015c0a0 0x7fddb01512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb015f150 0x7fddb01517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0169010 con 0x7fddb00b2f50 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0168e90 con 0x7fddb015f150 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fddb0169190 con 0x7fddb015c0a0 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb01fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb01fa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59910/0 (socket says 172.21.15.16:59910) 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 -- 172.21.15.16:0/3015561785 learned_addr learned my addr 172.21.15.16:0/3015561785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 -- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddb015c0a0 msgr2=0x7fddb01512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fdda7e00640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddb015c0a0 0x7fddb01512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddb015c0a0 0x7fddb01512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddae800640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb015f150 0x7fddb01517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 -- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb015f150 msgr2=0x7fddb01517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb015f150 0x7fddb01517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 -- 172.21.15.16:0/3015561785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb01f6cd0 con 0x7fddb00b2f50 2024-09-17T05:10:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddae800640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb015f150 0x7fddb01517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:01.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddade00640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb01fa8c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fdda001cc50 tx=0x7fdda0005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00b2040 con 0x7fddb00b2f50 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdda0004130 con 0x7fddb00b2f50 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb01f6fb0 con 0x7fddb00b2f50 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.105+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0003600 con 0x7fddb00b2f50 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.109+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb01f7570 con 0x7fddb00b2f50 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.109+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb015bb60 con 0x7fddb00b2f50 2024-09-17T05:10:01.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.109+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdda00bd030 con 0x7fddb00b2f50 2024-09-17T05:10:01.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.109+0000 7fdda6000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdd8c073200 0x7fdd8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.109+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7fdda012dbf0 con 0x7fddb00b2f50 2024-09-17T05:10:01.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.109+0000 7fdda7e00640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdd8c073200 0x7fdd8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:01.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.113+0000 7fdda7e00640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdd8c073200 0x7fdd8c075710 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fddb00cc8d0 tx=0x7fdd980091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:01.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.113+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdda00c4060 con 0x7fddb00b2f50 2024-09-17T05:10:01.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.353+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7fddb00cf3b0 con 0x7fddb00b2f50 2024-09-17T05:10:01.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:10:01.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:10:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.529+0000 7fdda6000640 1 -- 172.21.15.16:0/3015561785 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v454) ==== 148+0+0 (secure 0 0 0) 0x7fdda00163f0 con 0x7fddb00b2f50 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdd8c073200 msgr2=0x7fdd8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdd8c073200 0x7fdd8c075710 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fddb00cc8d0 tx=0x7fdd980091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 msgr2=0x7fddb01fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb01fa8c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fdda001cc50 tx=0x7fdda0005db0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 shutdown_connections 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fdd8c073200 0x7fdd8c075710 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddb015f150 0x7fddb01517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddb015c0a0 0x7fddb01512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 --2- 172.21.15.16:0/3015561785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddb00b2f50 0x7fddb01fa8c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 >> 172.21.15.16:0/3015561785 conn(0x7fddb006f900 msgr2=0x7fddb015c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 shutdown_connections 2024-09-17T05:10:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.533+0000 7fddb5000640 1 -- 172.21.15.16:0/3015561785 wait complete. 2024-09-17T05:10:01.556 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-09-17T05:10:01.556 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) When a client has a bug (which we will simulate) preventing it from releasing caps, 2024-09-17T05:10:01.556 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2024-09-17T05:10:01.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 -- 172.21.15.16:0/74304023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab8073240 msgr2=0x7f8ab8073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 --2- 172.21.15.16:0/74304023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab8073240 0x7f8ab8073650 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0009bc0 tx=0x7f8ab001cb10 comp rx=0 tx=0).stop 2024-09-17T05:10:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 -- 172.21.15.16:0/74304023 shutdown_connections 2024-09-17T05:10:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 --2- 172.21.15.16:0/74304023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab8073240 0x7f8ab8073650 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 --2- 172.21.15.16:0/74304023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ab815c270 0x7f8ab815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 -- 172.21.15.16:0/74304023 >> 172.21.15.16:0/74304023 conn(0x7f8ab806f900 msgr2=0x7f8ab80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 -- 172.21.15.16:0/74304023 shutdown_connections 2024-09-17T05:10:01.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.769+0000 7f8abfc00640 1 -- 172.21.15.16:0/74304023 wait complete. 2024-09-17T05:10:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 Processor -- start 2024-09-17T05:10:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 -- start start 2024-09-17T05:10:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ab8073240 0x7f8ab81f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ab80b2f50 0x7f8ab8163130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 0x7f8ab80b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8168d60 con 0x7f8ab815c270 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8168be0 con 0x7f8ab8073240 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abfc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab8168ee0 con 0x7f8ab80b2f50 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 0x7f8ab80b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 0x7f8ab80b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59926/0 (socket says 172.21.15.16:59926) 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.773+0000 7f8abdc00640 1 -- 172.21.15.16:0/1515415285 learned_addr learned my addr 172.21.15.16:0/1515415285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abdc00640 1 -- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ab80b2f50 msgr2=0x7f8ab8163130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abdc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ab80b2f50 0x7f8ab8163130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abdc00640 1 -- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ab8073240 msgr2=0x7f8ab81f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abdc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ab8073240 0x7f8ab81f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abdc00640 1 -- 172.21.15.16:0/1515415285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab81f9580 con 0x7f8ab815c270 2024-09-17T05:10:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abdc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 0x7f8ab80b0d00 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8ab001a740 tx=0x7f8ab0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00bb070 con 0x7f8ab815c270 2024-09-17T05:10:01.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ab0004d40 con 0x7f8ab815c270 2024-09-17T05:10:01.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00b2040 con 0x7f8ab815c270 2024-09-17T05:10:01.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab81f9860 con 0x7f8ab815c270 2024-09-17T05:10:01.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab81f9e70 con 0x7f8ab815c270 2024-09-17T05:10:01.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab8073650 con 0x7f8ab815c270 2024-09-17T05:10:01.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.777+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ab00042a0 con 0x7f8ab815c270 2024-09-17T05:10:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.781+0000 7f8ab5600640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a94073230 0x7f8a94075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:01.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.781+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7f8ab012de80 con 0x7f8ab815c270 2024-09-17T05:10:01.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.781+0000 7f8abd200640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a94073230 0x7f8a94075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:01.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.781+0000 7f8abd200640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a94073230 0x7f8a94075740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8001530 tx=0x7f8aa8007a40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:01.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.785+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ab00f1db0 con 0x7f8ab815c270 2024-09-17T05:10:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.965+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ab81f9310 con 0x7f8ab815c270 2024-09-17T05:10:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.965+0000 7f8ab5600640 1 -- 172.21.15.16:0/1515415285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+13305 (secure 0 0 0) 0x7f8ab0016790 con 0x7f8ab815c270 2024-09-17T05:10:01.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:01.968 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":141,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:00.526511+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.104:6837/1269584978":"2024-09-18T05:09:57.514365+0000","172.21.15.104:6836/1269584978":"2024-09-18T05:09:57.514365+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a94073230 msgr2=0x7f8a94075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a94073230 0x7f8a94075740 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8001530 tx=0x7f8aa8007a40 comp rx=0 tx=0).stop 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 msgr2=0x7f8ab80b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 0x7f8ab80b0d00 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8ab001a740 tx=0x7f8ab0003f60 comp rx=0 tx=0).stop 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 shutdown_connections 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8a94073230 0x7f8a94075740 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab815c270 0x7f8ab80b0d00 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8ab80b2f50 0x7f8ab8163130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 --2- 172.21.15.16:0/1515415285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ab8073240 0x7f8ab81f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 >> 172.21.15.16:0/1515415285 conn(0x7f8ab806f900 msgr2=0x7f8ab80b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 shutdown_connections 2024-09-17T05:10:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:01.969+0000 7f8abfc00640 1 -- 172.21.15.16:0/1515415285 wait complete. 2024-09-17T05:10:01.990 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T05:00:57.487536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T05:10:01.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 -- 172.21.15.16:0/3297097235 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 msgr2=0x7f111c0cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 --2- 172.21.15.16:0/3297097235 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 0x7f111c0cf140 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1108009900 tx=0x7f110801c820 comp rx=0 tx=0).stop 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 -- 172.21.15.16:0/3297097235 shutdown_connections 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 --2- 172.21.15.16:0/3297097235 >> v2:172.21.15.104:3300/0 conn(0x7f111c0cf680 0x7f111c0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 --2- 172.21.15.16:0/3297097235 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 0x7f111c0cf140 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 -- 172.21.15.16:0/3297097235 >> 172.21.15.16:0/3297097235 conn(0x7f111c0b42d0 msgr2=0x7f111c0b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 -- 172.21.15.16:0/3297097235 shutdown_connections 2024-09-17T05:10:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 -- 172.21.15.16:0/3297097235 wait complete. 2024-09-17T05:10:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 Processor -- start 2024-09-17T05:10:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.153+0000 7f1124600640 1 -- start start 2024-09-17T05:10:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 0x7f111c152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 0x7f111c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c159f70 0x7f111c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f111c169e20 con 0x7f111c0cf680 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f111c169ca0 con 0x7f111c159f70 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f111c169fa0 con 0x7f111c0d02d0 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 0x7f111c152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 0x7f111c152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59936/0 (socket says 172.21.15.16:59936) 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 -- 172.21.15.16:0/2778225838 learned_addr learned my addr 172.21.15.16:0/2778225838 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121200640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 0x7f111c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1122600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c159f70 0x7f111c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 -- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 msgr2=0x7f111c156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 0x7f111c156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 -- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c159f70 msgr2=0x7f111c1565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c159f70 0x7f111c1565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 -- 172.21.15.16:0/2778225838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c0cc6d0 con 0x7f111c0cf680 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1121c00640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 0x7f111c152320 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f111000c3f0 tx=0x7f111000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110017070 con 0x7f111c0cf680 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11100053c0 con 0x7f111c0cf680 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110003860 con 0x7f111c0cf680 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c1558d0 con 0x7f111c0cf680 2024-09-17T05:10:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c0cb410 con 0x7f111c0cf680 2024-09-17T05:10:02.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c0b2100 con 0x7f111c0cf680 2024-09-17T05:10:02.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.157+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f111001a070 con 0x7f111c0cf680 2024-09-17T05:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.161+0000 7f1119600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10fc073230 0x7f10fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.161+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7f11100933b0 con 0x7f111c0cf680 2024-09-17T05:10:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.161+0000 7f1121200640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10fc073230 0x7f10fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.161+0000 7f1121200640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10fc073230 0x7f10fc075740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11080af000 tx=0x7f110801b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:02.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.165+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f111c0cfa90 con 0x7f111c0cf680 2024-09-17T05:10:02.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.341+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f111c15b600 con 0x7f111c0cf680 2024-09-17T05:10:02.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.341+0000 7f1119600640 1 -- 172.21.15.16:0/2778225838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7f111005def0 con 0x7f111c0cf680 2024-09-17T05:10:02.343 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10fc073230 msgr2=0x7f10fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10fc073230 0x7f10fc075740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f11080af000 tx=0x7f110801b060 comp rx=0 tx=0).stop 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 msgr2=0x7f111c152320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 0x7f111c152320 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f111000c3f0 tx=0x7f111000c8b0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 shutdown_connections 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f10fc073230 0x7f10fc075740 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c159f70 0x7f111c1565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f111c0d02d0 0x7f111c156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 --2- 172.21.15.16:0/2778225838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f111c0cf680 0x7f111c152320 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 >> 172.21.15.16:0/2778225838 conn(0x7f111c0b42d0 msgr2=0x7f111c150320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 shutdown_connections 2024-09-17T05:10:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.345+0000 7f1124600640 1 -- 172.21.15.16:0/2778225838 wait complete. 2024-09-17T05:10:02.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-09-17T05:10:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 -- 172.21.15.16:0/3668270181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4071210 msgr2=0x7ff6f4071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/3668270181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4071210 0x7ff6f4071620 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009840 tx=0x7ff6dc01c820 comp rx=0 tx=0).stop 2024-09-17T05:10:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 -- 172.21.15.16:0/3668270181 shutdown_connections 2024-09-17T05:10:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/3668270181 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6f40cda00 0x7ff6f40cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/3668270181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f4071210 0x7ff6f4071620 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 -- 172.21.15.16:0/3668270181 >> 172.21.15.16:0/3668270181 conn(0x7ff6f40d10f0 msgr2=0x7ff6f40d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 -- 172.21.15.16:0/3668270181 shutdown_connections 2024-09-17T05:10:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.521+0000 7ff6f3e00640 1 -- 172.21.15.16:0/3668270181 wait complete. 2024-09-17T05:10:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 Processor -- start 2024-09-17T05:10:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 -- start start 2024-09-17T05:10:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6f4071210 0x7ff6f413b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 0x7ff6f4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6f415d530 0x7ff6f413c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4169dd0 con 0x7ff6f40cda00 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4169c50 con 0x7ff6f415d530 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f4169f50 con 0x7ff6f4071210 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 0x7ff6f4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 0x7ff6f4138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59946/0 (socket says 172.21.15.16:59946) 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6ebe00640 1 -- 172.21.15.16:0/4226452433 learned_addr learned my addr 172.21.15.16:0/4226452433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f3400640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6f415d530 0x7ff6f413c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.525+0000 7ff6f2a00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6f4071210 0x7ff6f413b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6ebe00640 1 -- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6f4071210 msgr2=0x7ff6f413b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6ebe00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6f4071210 0x7ff6f413b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6ebe00640 1 -- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6f415d530 msgr2=0x7ff6f413c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6ebe00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6f415d530 0x7ff6f413c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6ebe00640 1 -- 172.21.15.16:0/4226452433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f413a700 con 0x7ff6f40cda00 2024-09-17T05:10:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6ebe00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 0x7ff6f4138af0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc01aed0 tx=0x7ff6dc005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0b2040 con 0x7ff6f40cda00 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6dc0043e0 con 0x7ff6f40cda00 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f413a9c0 con 0x7ff6f40cda00 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0052d0 con 0x7ff6f40cda00 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f413af80 con 0x7ff6f40cda00 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f4071620 con 0x7ff6f40cda00 2024-09-17T05:10:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.529+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6dc005470 con 0x7ff6f40cda00 2024-09-17T05:10:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.533+0000 7ff6eaa00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6d0073230 0x7ff6d0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.533+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7ff6dc12da90 con 0x7ff6f40cda00 2024-09-17T05:10:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.533+0000 7ff6f2a00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6d0073230 0x7ff6d0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.533+0000 7ff6f2a00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6d0073230 0x7ff6d0075740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4001400 tx=0x7ff6e4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:02.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.537+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff6dc0bd060 con 0x7ff6f40cda00 2024-09-17T05:10:02.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:02.777+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7ff6f40d1680 con 0x7ff6f40cda00 2024-09-17T05:10:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.541+0000 7ff6eaa00640 1 -- 172.21.15.16:0/4226452433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v456) ==== 154+0+0 (secure 0 0 0) 0x7ff6dc016410 con 0x7ff6f40cda00 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6d0073230 msgr2=0x7ff6d0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6d0073230 0x7ff6d0075740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4001400 tx=0x7ff6e4009040 comp rx=0 tx=0).stop 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 msgr2=0x7ff6f4138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 0x7ff6f4138af0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc01aed0 tx=0x7ff6dc005db0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 shutdown_connections 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff6d0073230 0x7ff6d0075740 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6f415d530 0x7ff6f413c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6f40cda00 0x7ff6f4138af0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 --2- 172.21.15.16:0/4226452433 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff6f4071210 0x7ff6f413b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 >> 172.21.15.16:0/4226452433 conn(0x7ff6f40d10f0 msgr2=0x7ff6f41665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:03.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 shutdown_connections 2024-09-17T05:10:03.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.545+0000 7ff6f3e00640 1 -- 172.21.15.16:0/4226452433 wait complete. 2024-09-17T05:10:03.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T05:10:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 -- 172.21.15.16:0/1317068135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c073240 msgr2=0x7fc08c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 --2- 172.21.15.16:0/1317068135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c073240 0x7fc08c073650 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc084009bc0 tx=0x7fc08401cb10 comp rx=0 tx=0).stop 2024-09-17T05:10:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 -- 172.21.15.16:0/1317068135 shutdown_connections 2024-09-17T05:10:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 --2- 172.21.15.16:0/1317068135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c073240 0x7fc08c073650 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 --2- 172.21.15.16:0/1317068135 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc08c15c270 0x7fc08c15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 -- 172.21.15.16:0/1317068135 >> 172.21.15.16:0/1317068135 conn(0x7fc08c06f900 msgr2=0x7fc08c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 -- 172.21.15.16:0/1317068135 shutdown_connections 2024-09-17T05:10:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 -- 172.21.15.16:0/1317068135 wait complete. 2024-09-17T05:10:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 Processor -- start 2024-09-17T05:10:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.733+0000 7fc093200640 1 -- start start 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08c073240 0x7fc08c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc08c0b2f50 0x7fc08c0cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 0x7fc08c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c168d60 con 0x7fc08c15c270 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c168be0 con 0x7fc08c073240 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fc08c168ee0 con 0x7fc08c0b2f50 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 0x7fc08c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 0x7fc08c0ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59964/0 (socket says 172.21.15.16:59964) 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 -- 172.21.15.16:0/451506861 learned_addr learned my addr 172.21.15.16:0/451506861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc08be00640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08c073240 0x7fc08c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc083400640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc08c0b2f50 0x7fc08c0cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 -- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc08c0b2f50 msgr2=0x7fc08c0cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc08c0b2f50 0x7fc08c0cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 -- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08c073240 msgr2=0x7fc08c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08c073240 0x7fc08c150290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 -- 172.21.15.16:0/451506861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c1507d0 con 0x7fc08c15c270 2024-09-17T05:10:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc091200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 0x7fc08c0ce380 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc08401c580 tx=0x7fc084005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840b2040 con 0x7fc08c15c270 2024-09-17T05:10:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0840bb070 con 0x7fc08c15c270 2024-09-17T05:10:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c160d30 con 0x7fc08c15c270 2024-09-17T05:10:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084004590 con 0x7fc08c15c270 2024-09-17T05:10:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1612b0 con 0x7fc08c15c270 2024-09-17T05:10:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.737+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c164400 con 0x7fc08c15c270 2024-09-17T05:10:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.741+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc0840c0460 con 0x7fc08c15c270 2024-09-17T05:10:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.741+0000 7fc088c00640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc068073200 0x7fc068075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.741+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7fc08412e880 con 0x7fc08c15c270 2024-09-17T05:10:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.741+0000 7fc08be00640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc068073200 0x7fc068075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.741+0000 7fc08be00640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc068073200 0x7fc068075710 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc07c007fd0 tx=0x7fc07c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:03.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.745+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc084016790 con 0x7fc08c15c270 2024-09-17T05:10:03.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.917+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc08c15de70 con 0x7fc08c15c270 2024-09-17T05:10:03.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.917+0000 7fc088c00640 1 -- 172.21.15.16:0/451506861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v68) ==== 59+0+2294 (secure 0 0 0) 0x7fc0840f9440 con 0x7fc08c15c270 2024-09-17T05:10:03.920 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T05:10:03.920 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T05:10:03.920 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3040.250401 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 2933.842623 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3171.160392 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3007.053489 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3980.295161 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7687.080753 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4174.212861 2024-09-17T05:10:03.921 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4284.596318 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-17T05:10:03.922 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-17T05:10:03.923 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T05:10:03.923 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc068073200 msgr2=0x7fc068075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc068073200 0x7fc068075710 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc07c007fd0 tx=0x7fc07c009040 comp rx=0 tx=0).stop 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 msgr2=0x7fc08c0ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 0x7fc08c0ce380 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc08401c580 tx=0x7fc084005a60 comp rx=0 tx=0).stop 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 shutdown_connections 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fc068073200 0x7fc068075710 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08c15c270 0x7fc08c0ce380 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fc08c0b2f50 0x7fc08c0cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 --2- 172.21.15.16:0/451506861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc08c073240 0x7fc08c150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 >> 172.21.15.16:0/451506861 conn(0x7fc08c06f900 msgr2=0x7fc08c15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 shutdown_connections 2024-09-17T05:10:03.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:03.921+0000 7fc093200640 1 -- 172.21.15.16:0/451506861 wait complete. 2024-09-17T05:10:03.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T05:10:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 -- 172.21.15.16:0/225696870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 msgr2=0x7f6ddc160130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 --2- 172.21.15.16:0/225696870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc160130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009840 tx=0x7f6dc401c710 comp rx=0 tx=0).stop 2024-09-17T05:10:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 -- 172.21.15.16:0/225696870 shutdown_connections 2024-09-17T05:10:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 --2- 172.21.15.16:0/225696870 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc160130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 --2- 172.21.15.16:0/225696870 >> v2:172.21.15.104:3300/0 conn(0x7f6ddc0733e0 0x7f6ddc15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 -- 172.21.15.16:0/225696870 >> 172.21.15.16:0/225696870 conn(0x7f6ddc0b2250 msgr2=0x7f6ddc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 -- 172.21.15.16:0/225696870 shutdown_connections 2024-09-17T05:10:04.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.105+0000 7f6de3400640 1 -- 172.21.15.16:0/225696870 wait complete. 2024-09-17T05:10:04.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 Processor -- start 2024-09-17T05:10:04.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 -- start start 2024-09-17T05:10:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ddc0733e0 0x7f6ddc0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc14a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 0x7f6ddc155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc073970 con 0x7f6ddc160670 2024-09-17T05:10:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0737f0 con 0x7f6ddc0733e0 2024-09-17T05:10:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc16a2d0 con 0x7f6ddc15d680 2024-09-17T05:10:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 0x7f6ddc155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 0x7f6ddc155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59968/0 (socket says 172.21.15.16:59968) 2024-09-17T05:10:04.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 -- 172.21.15.16:0/2903164554 learned_addr learned my addr 172.21.15.16:0/2903164554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de0a00640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ddc0733e0 0x7f6ddc0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6dd3e00640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 -- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 msgr2=0x7f6ddc14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc14a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 -- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ddc0733e0 msgr2=0x7f6ddc0b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ddc0733e0 0x7f6ddc0b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 -- 172.21.15.16:0/2903164554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc15deb0 con 0x7f6ddc160670 2024-09-17T05:10:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6dd3e00640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de0a00640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ddc0733e0 0x7f6ddc0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de1400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 0x7f6ddc155230 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400cbb0 tx=0x7f6dd400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:04.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4016020 con 0x7f6ddc160670 2024-09-17T05:10:04.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dd400ab00 con 0x7f6ddc160670 2024-09-17T05:10:04.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc0b5ab0 con 0x7f6ddc160670 2024-09-17T05:10:04.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4012420 con 0x7f6ddc160670 2024-09-17T05:10:04.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.109+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc0b54f0 con 0x7f6ddc160670 2024-09-17T05:10:04.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.113+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc15d140 con 0x7f6ddc160670 2024-09-17T05:10:04.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.113+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6dd4019020 con 0x7f6ddc160670 2024-09-17T05:10:04.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.113+0000 7f6dd9600640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6db8073200 0x7f6db8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.113+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7f6dd40945c0 con 0x7f6ddc160670 2024-09-17T05:10:04.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.113+0000 7f6de0a00640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6db8073200 0x7f6db8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.113+0000 7f6de0a00640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6db8073200 0x7f6db8075710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009b90 tx=0x7f6dcc0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:04.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.117+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6dd4020020 con 0x7f6ddc160670 2024-09-17T05:10:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.293+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6ddc06d290 con 0x7f6ddc160670 2024-09-17T05:10:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6dd9600640 1 -- 172.21.15.16:0/2903164554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v69)=0 v69) ==== 122+0+0 (secure 0 0 0) 0x7f6dd405f6c0 con 0x7f6ddc160670 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6db8073200 msgr2=0x7f6db8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6db8073200 0x7f6db8075710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009b90 tx=0x7f6dcc0091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 msgr2=0x7f6ddc155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 0x7f6ddc155230 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400cbb0 tx=0x7f6dd400cca0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 shutdown_connections 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6db8073200 0x7f6db8075710 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc160670 0x7f6ddc155230 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6ddc15d680 0x7f6ddc14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 --2- 172.21.15.16:0/2903164554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ddc0733e0 0x7f6ddc0b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 >> 172.21.15.16:0/2903164554 conn(0x7f6ddc0b2250 msgr2=0x7f6ddc0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 shutdown_connections 2024-09-17T05:10:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.297+0000 7f6de3400640 1 -- 172.21.15.16:0/2903164554 wait complete. 2024-09-17T05:10:04.320 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:10:04.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:10:04.407 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:10:04.511 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:10:04.512 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:04.512 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:04.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:10:04.521 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:04.521 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:04.530 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:10:04.530 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:04.531 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:04.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:10:04.572 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:04.572 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:04.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:10:04.582 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T05:10:04.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:04.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 -- 172.21.15.16:0/292992863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28000b2f50 msgr2=0x7f280015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 --2- 172.21.15.16:0/292992863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28000b2f50 0x7f280015bb60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f280015cc70 tx=0x7f27f001c830 comp rx=0 tx=0).stop 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 -- 172.21.15.16:0/292992863 shutdown_connections 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 --2- 172.21.15.16:0/292992863 >> v2:172.21.15.104:3300/0 conn(0x7f280015f150 0x7f280015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 --2- 172.21.15.16:0/292992863 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f280015c0a0 0x7f280015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 --2- 172.21.15.16:0/292992863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28000b2f50 0x7f280015bb60 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 -- 172.21.15.16:0/292992863 >> 172.21.15.16:0/292992863 conn(0x7f280006f900 msgr2=0x7f28000b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 -- 172.21.15.16:0/292992863 shutdown_connections 2024-09-17T05:10:04.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.773+0000 7f2807800640 1 -- 172.21.15.16:0/292992863 wait complete. 2024-09-17T05:10:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 Processor -- start 2024-09-17T05:10:04.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 -- start start 2024-09-17T05:10:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000b2f50 0x7f2800143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f280015c0a0 0x7f28000cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 0x7f28000cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2800073ef0 con 0x7f280015f150 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2800073d70 con 0x7f28000b2f50 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2800074070 con 0x7f280015c0a0 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 0x7f28000cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 0x7f28000cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59974/0 (socket says 172.21.15.16:59974) 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 -- 172.21.15.16:0/2393001797 learned_addr learned my addr 172.21.15.16:0/2393001797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2804e00640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000b2f50 0x7f2800143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f27f7e00640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f280015c0a0 0x7f28000cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 -- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f280015c0a0 msgr2=0x7f28000cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f280015c0a0 0x7f28000cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 -- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000b2f50 msgr2=0x7f2800143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000b2f50 0x7f2800143d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 -- 172.21.15.16:0/2393001797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28001442b0 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2805800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 0x7f28000cf430 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d9d0 tx=0x7f27f800de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8017070 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27f8004b30 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28001447c0 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f800be60 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.777+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2800144d80 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.781+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280015e9b0 con 0x7f280015f150 2024-09-17T05:10:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.781+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27f8003980 con 0x7f280015f150 2024-09-17T05:10:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.781+0000 7f27fd600640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f27dc073200 0x7f27dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.781+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7f27f8094740 con 0x7f280015f150 2024-09-17T05:10:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.781+0000 7f2804e00640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f27dc073200 0x7f27dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.785+0000 7f2804e00640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f27dc073200 0x7f27dc075710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27f00af000 tx=0x7f27f001b070 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:04.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.785+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27f8020020 con 0x7f280015f150 2024-09-17T05:10:05.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:04.997+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2800144550 con 0x7f280015f150 2024-09-17T05:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f27fd600640 1 -- 172.21.15.16:0/2393001797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+3919 (secure 0 0 0) 0x7f27f805f1e0 con 0x7f280015f150 2024-09-17T05:10:05.001 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:05.001 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":103,"btime":"2024-09-17T05:09:58:517121+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6800,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/2199058015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2199058015},{"type":"v1","addr":"172.21.15.16:6834","nonce":2199058015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":6806,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3947900229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3947900229},{"type":"v1","addr":"172.21.15.104:6835","nonce":3947900229}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7642,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1736301452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1736301452},{"type":"v1","addr":"172.21.15.16:6835","nonce":1736301452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":7867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2161228574","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2161228574},{"type":"v1","addr":"172.21.15.104:6837","nonce":2161228574}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[]} 2024-09-17T05:10:05.001 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 103 2024-09-17T05:10:05.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f27dc073200 msgr2=0x7f27dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f27dc073200 0x7f27dc075710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27f00af000 tx=0x7f27f001b070 comp rx=0 tx=0).stop 2024-09-17T05:10:05.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 msgr2=0x7f28000cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 0x7f28000cf430 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d9d0 tx=0x7f27f800de90 comp rx=0 tx=0).stop 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 shutdown_connections 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f27dc073200 0x7f27dc075710 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280015f150 0x7f28000cf430 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f280015c0a0 0x7f28000cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 --2- 172.21.15.16:0/2393001797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000b2f50 0x7f2800143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 >> 172.21.15.16:0/2393001797 conn(0x7f280006f900 msgr2=0x7f28000cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 shutdown_connections 2024-09-17T05:10:05.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.001+0000 7f2807800640 1 -- 172.21.15.16:0/2393001797 wait complete. 2024-09-17T05:10:05.024 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T05:10:05.024 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T05:10:05.024 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:10:05.024 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:05.024 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T05:10:05.024 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T05:10:05.025 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:10:05.025 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:05.025 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T05:10:05.025 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T05:10:05.025 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:10:05.025 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:05.025 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T05:10:05.026 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T05:10:05.026 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:10:05.026 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:05.038 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:10:05.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T05:10:05.039 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:10:05.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T05:10:05.045 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:10:05.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T05:10:05.054 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:10:05.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T05:10:05.083 INFO:tasks.ceph.mds.c:Started 2024-09-17T05:10:05.084 DEBUG:teuthology.parallel:result is None 2024-09-17T05:10:05.084 INFO:tasks.ceph.mds.a:Started 2024-09-17T05:10:05.085 DEBUG:teuthology.parallel:result is None 2024-09-17T05:10:05.103 INFO:tasks.ceph.mds.d:Started 2024-09-17T05:10:05.103 DEBUG:teuthology.parallel:result is None 2024-09-17T05:10:05.108 INFO:tasks.ceph.mds.b:Started 2024-09-17T05:10:05.109 DEBUG:teuthology.parallel:result is None 2024-09-17T05:10:05.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T05:10:05.185 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-09-17T05:10:05.185 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:10:05.209 INFO:tasks.ceph.mds.d.smithi104.stdout:starting mds.d at 2024-09-17T05:10:05.209 INFO:tasks.ceph.mds.b.smithi104.stdout:starting mds.b at 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 -- 172.21.15.16:0/546410125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80cf680 msgr2=0x7ff9a80b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 --2- 172.21.15.16:0/546410125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80cf680 0x7ff9a80b2100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009cb0 tx=0x7ff9a001cce0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 -- 172.21.15.16:0/546410125 shutdown_connections 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 --2- 172.21.15.16:0/546410125 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80cf680 0x7ff9a80b2100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 --2- 172.21.15.16:0/546410125 >> v2:172.21.15.104:3300/0 conn(0x7ff9a80d02d0 0x7ff9a80cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 -- 172.21.15.16:0/546410125 >> 172.21.15.16:0/546410125 conn(0x7ff9a80b42d0 msgr2=0x7ff9a80b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 -- 172.21.15.16:0/546410125 shutdown_connections 2024-09-17T05:10:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 -- 172.21.15.16:0/546410125 wait complete. 2024-09-17T05:10:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 Processor -- start 2024-09-17T05:10:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.277+0000 7ff9ae800640 1 -- start start 2024-09-17T05:10:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a80cf680 0x7ff9a80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80d02d0 0x7ff9a80cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 0x7ff9a814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8169e20 con 0x7ff9a8159f70 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a8169ca0 con 0x7ff9a80cf680 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a8169fa0 con 0x7ff9a80d02d0 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 0x7ff9a814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 0x7ff9a814cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60012/0 (socket says 172.21.15.16:60012) 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 -- 172.21.15.16:0/4236510368 learned_addr learned my addr 172.21.15.16:0/4236510368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7400640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a80cf680 0x7ff9a80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff99fe00640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80d02d0 0x7ff9a80cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 -- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80d02d0 msgr2=0x7ff9a80cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80d02d0 0x7ff9a80cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 -- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a80cf680 msgr2=0x7ff9a80cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a80cf680 0x7ff9a80cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 -- 172.21.15.16:0/4236510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a80b56c0 con 0x7ff9a8159f70 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9a7e00640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 0x7ff9a814cea0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009bd0 tx=0x7ff9a0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00b2040 con 0x7ff9a8159f70 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a80b59a0 con 0x7ff9a8159f70 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a0004b40 con 0x7ff9a8159f70 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00ad5f0 con 0x7ff9a8159f70 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a80b7e20 con 0x7ff9a8159f70 2024-09-17T05:10:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.281+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a80b5430 con 0x7ff9a8159f70 2024-09-17T05:10:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.285+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff9a00be060 con 0x7ff9a8159f70 2024-09-17T05:10:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.285+0000 7ff99f400640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff984073200 0x7ff984075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.285+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5335+0+0 (secure 0 0 0) 0x7ff9a012e070 con 0x7ff9a8159f70 2024-09-17T05:10:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.285+0000 7ff9a7400640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff984073200 0x7ff984075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.285+0000 7ff9a7400640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff984073200 0x7ff984075710 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff9980017f0 tx=0x7ff998009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:05.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.289+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff9a0016990 con 0x7ff9a8159f70 2024-09-17T05:10:05.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.481+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff9a80d1370 con 0x7ff9a8159f70 2024-09-17T05:10:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.557+0000 7ff99f400640 1 -- 172.21.15.16:0/4236510368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v143) ==== 97+0+0 (secure 0 0 0) 0x7ff9a00f8ba0 con 0x7ff9a8159f70 2024-09-17T05:10:05.559 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff984073200 msgr2=0x7ff984075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff984073200 0x7ff984075710 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff9980017f0 tx=0x7ff998009040 comp rx=0 tx=0).stop 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 msgr2=0x7ff9a814cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 0x7ff9a814cea0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009bd0 tx=0x7ff9a0003f60 comp rx=0 tx=0).stop 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 shutdown_connections 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff984073200 0x7ff984075710 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9a8159f70 0x7ff9a814cea0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff9a80d02d0 0x7ff9a80cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 --2- 172.21.15.16:0/4236510368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a80cf680 0x7ff9a80cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 >> 172.21.15.16:0/4236510368 conn(0x7ff9a80b42d0 msgr2=0x7ff9a8150320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 shutdown_connections 2024-09-17T05:10:05.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.561+0000 7ff9ae800640 1 -- 172.21.15.16:0/4236510368 wait complete. 2024-09-17T05:10:05.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.741+0000 7f8b2f800640 1 -- 172.21.15.16:0/2608670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 msgr2=0x7f8b280b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.741+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2608670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b280b29f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8b280c5b80 tx=0x7f8b2001ca60 comp rx=0 tx=0).stop 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- 172.21.15.16:0/2608670197 shutdown_connections 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2608670197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b280b29f0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2608670197 >> v2:172.21.15.104:3300/0 conn(0x7f8b28160aa0 0x7f8b280b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- 172.21.15.16:0/2608670197 >> 172.21.15.16:0/2608670197 conn(0x7f8b280cf380 msgr2=0x7f8b280d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- 172.21.15.16:0/2608670197 shutdown_connections 2024-09-17T05:10:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- 172.21.15.16:0/2608670197 wait complete. 2024-09-17T05:10:05.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 Processor -- start 2024-09-17T05:10:05.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- start start 2024-09-17T05:10:05.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b28153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b2815c6f0 0x7f8b280b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b28154670 0x7f8b280b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2816a110 con 0x7f8b280b25e0 2024-09-17T05:10:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28169f90 con 0x7f8b28154670 2024-09-17T05:10:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.745+0000 7f8b2f800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2816a290 con 0x7f8b2815c6f0 2024-09-17T05:10:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b28153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b28153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60042/0 (socket says 172.21.15.16:60042) 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 -- 172.21.15.16:0/2563140192 learned_addr learned my addr 172.21.15.16:0/2563140192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2d800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b28154670 0x7f8b280b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b27e00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b2815c6f0 0x7f8b280b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 -- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b2815c6f0 msgr2=0x7f8b280b66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b2815c6f0 0x7f8b280b66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 -- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b28154670 msgr2=0x7f8b280b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b28154670 0x7f8b280b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 -- 172.21.15.16:0/2563140192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b280cc8b0 con 0x7f8b280b25e0 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b27e00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b2815c6f0 0x7f8b280b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2ce00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b28153a80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c400 tx=0x7f8b1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c017070 con 0x7f8b280b25e0 2024-09-17T05:10:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b1c003600 con 0x7f8b280b25e0 2024-09-17T05:10:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c003a50 con 0x7f8b280b25e0 2024-09-17T05:10:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b281f8740 con 0x7f8b280b25e0 2024-09-17T05:10:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b281f8d00 con 0x7f8b280b25e0 2024-09-17T05:10:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b280b29f0 con 0x7f8b280b25e0 2024-09-17T05:10:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.749+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b1c01a070 con 0x7f8b280b25e0 2024-09-17T05:10:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.753+0000 7f8b24c00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b08073230 0x7f8b08075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.753+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5249+0+0 (secure 0 0 0) 0x7f8b1c093330 con 0x7f8b280b25e0 2024-09-17T05:10:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.753+0000 7f8b27e00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b08073230 0x7f8b08075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.753+0000 7f8b27e00640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b08073230 0x7f8b08075740 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8b28160010 tx=0x7f8b140091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.757+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b1c05de20 con 0x7f8b280b25e0 2024-09-17T05:10:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8b281582a0 con 0x7f8b280b25e0 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b24c00640 1 -- 172.21.15.16:0/2563140192 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v40) ==== 80+0+5304 (secure 0 0 0) 0x7f8b1c062850 con 0x7f8b280b25e0 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlm2v1wCRAAGsgJD/dTaxUW5LgYxtG1Mg==", 2024-09-17T05:10:05.990 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmDMsEFxAAvIffhPw/oz3d2VzUB7HGoA==", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:10:05.991 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmKN5qERAA+HTfZGO8yay9rcb5BiVFJg==", 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.992 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmYgTiHhAAPnry5Shw+My57mWvfcZRRg==", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDUDOlmlje0CRAAms3hLKeLHNzxg/G1ZtCWng==", 2024-09-17T05:10:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDXDOlmEBCeHxAAbQcumxK1HfHXG9mBox1gYg==", 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDaDOlmTtORNRAAk+/Ow7AUyeBkESXtL1Butg==", 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDeDOlmKNkWEhAA9a6rc/BO0agTftdVKxmS6Q==", 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.995 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDkDOlms8nIGRAAhrAijDP62FLAeD7h8qpQRg==", 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.996 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDnDOlmbxKxMxAAg34u+dTRu/QVwPAR0/SVAQ==", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDrDOlmTacCDhAAjpxt8Z9SD3n93CSBb7/y/w==", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.997 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDuDOlm2EMJJBAA9k5ypsNfbVQZONnB1zPmsA==", 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-09-17T05:10:05.998 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg==", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow r", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmG4fLKxAA/+WulzKWAQRUEjHq2GvLDw==", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:05.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmnLLDJxAATMbjXLhsXX0QFnLBigAczg==", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmeaTTLxAAAzBEiXlzG/MrKET9QTJZAQ==", 2024-09-17T05:10:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmZu9MJhAAspGJdVGy1QBJjdcDOed0LQ==", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-09-17T05:10:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDSDOlmJRFxBBAAmPeuND9RtVMdAHM1pY9+8g==", 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDRDOlmnc95OxAA5WI1Uj+Jq90fAaHp1dKuSg==", 2024-09-17T05:10:06.002 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-17T05:10:06.003 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b08073230 msgr2=0x7f8b08075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b08073230 0x7f8b08075740 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8b28160010 tx=0x7f8b140091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 msgr2=0x7f8b28153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b28153a80 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00c400 tx=0x7f8b1c00c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 shutdown_connections 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8b08073230 0x7f8b08075740 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b28154670 0x7f8b280b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8b2815c6f0 0x7f8b280b66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 --2- 172.21.15.16:0/2563140192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b280b25e0 0x7f8b28153a80 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 >> 172.21.15.16:0/2563140192 conn(0x7f8b280cf380 msgr2=0x7f8b28157510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 shutdown_connections 2024-09-17T05:10:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:05.989+0000 7f8b2f800640 1 -- 172.21.15.16:0/2563140192 wait complete. 2024-09-17T05:10:06.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.169+0000 7f883c000640 1 -- 172.21.15.16:0/890375620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 msgr2=0x7f883415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.169+0000 7f883c000640 1 --2- 172.21.15.16:0/890375620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f883415c680 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f883415cc70 tx=0x7f882001c830 comp rx=0 tx=0).stop 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 -- 172.21.15.16:0/890375620 shutdown_connections 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 --2- 172.21.15.16:0/890375620 >> v2:172.21.15.104:3300/0 conn(0x7f8834073240 0x7f8834073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 --2- 172.21.15.16:0/890375620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f883415c680 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 -- 172.21.15.16:0/890375620 >> 172.21.15.16:0/890375620 conn(0x7f883406f900 msgr2=0x7f88340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 -- 172.21.15.16:0/890375620 shutdown_connections 2024-09-17T05:10:06.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 -- 172.21.15.16:0/890375620 wait complete. 2024-09-17T05:10:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 Processor -- start 2024-09-17T05:10:06.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 -- start start 2024-09-17T05:10:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.173+0000 7f883c000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8834073240 0x7f88341fba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88340b2f50 0x7f88340b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f88341fc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8834168c70 con 0x7f883415c270 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8834168af0 con 0x7f88340b2f50 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f8834168df0 con 0x7f8834073240 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f88341fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f88341fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60066/0 (socket says 172.21.15.16:60066) 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 -- 172.21.15.16:0/3535247638 learned_addr learned my addr 172.21.15.16:0/3535247638 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f8838c00640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88340b2f50 0x7f88340b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f8839600640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8834073240 0x7f88341fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 -- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8834073240 msgr2=0x7f88341fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8834073240 0x7f88341fba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 -- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88340b2f50 msgr2=0x7f88340b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88340b2f50 0x7f88340b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 -- 172.21.15.16:0/3535247638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88341fcfa0 con 0x7f883415c270 2024-09-17T05:10:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883a000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f88341fc150 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f882c00e910 tx=0x7f882c00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c004d30 con 0x7f883415c270 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f882c00cb20 con 0x7f883415c270 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88340b10b0 con 0x7f883415c270 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c005020 con 0x7f883415c270 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88340b1670 con 0x7f883415c270 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88340b0e20 con 0x7f883415c270 2024-09-17T05:10:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.177+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f882c0051c0 con 0x7f883415c270 2024-09-17T05:10:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.181+0000 7f8831600640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8814073200 0x7f8814075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.181+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5249+0+0 (secure 0 0 0) 0x7f882c0947b0 con 0x7f883415c270 2024-09-17T05:10:06.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.181+0000 7f8839600640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8814073200 0x7f8814075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.181+0000 7f8839600640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8814073200 0x7f8814075710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8828007fd0 tx=0x7f8828009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:06.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.185+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f882c016030 con 0x7f883415c270 2024-09-17T05:10:06.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.361+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8834073650 con 0x7f883415c270 2024-09-17T05:10:06.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.361+0000 7f8831600640 1 -- 172.21.15.16:0/3535247638 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v145) ==== 84+0+65 (secure 0 0 0) 0x7f882c05f330 con 0x7f883415c270 2024-09-17T05:10:06.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:06.364 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-17T05:10:06.364 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-17T05:10:06.365 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-17T05:10:06.365 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-17T05:10:06.365 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-17T05:10:06.365 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-17T05:10:06.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8814073200 msgr2=0x7f8814075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8814073200 0x7f8814075710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8828007fd0 tx=0x7f8828009040 comp rx=0 tx=0).stop 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 msgr2=0x7f88341fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f88341fc150 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f882c00e910 tx=0x7f882c00edd0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 shutdown_connections 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f8814073200 0x7f8814075710 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f883415c270 0x7f88341fc150 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88340b2f50 0x7f88340b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 --2- 172.21.15.16:0/3535247638 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f8834073240 0x7f88341fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 >> 172.21.15.16:0/3535247638 conn(0x7f883406f900 msgr2=0x7f883415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 shutdown_connections 2024-09-17T05:10:06.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.365+0000 7f883c000640 1 -- 172.21.15.16:0/3535247638 wait complete. 2024-09-17T05:10:06.386 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T05:10:06.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T05:10:06.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 -- 172.21.15.16:0/219281962 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 msgr2=0x7f5c1415efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 --2- 172.21.15.16:0/219281962 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 0x7f5c1415efa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c140b3080 tx=0x7f5bfc01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 -- 172.21.15.16:0/219281962 shutdown_connections 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 --2- 172.21.15.16:0/219281962 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 0x7f5c1415efa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 --2- 172.21.15.16:0/219281962 >> v2:172.21.15.104:3300/0 conn(0x7f5c140b4620 0x7f5c1406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 -- 172.21.15.16:0/219281962 >> 172.21.15.16:0/219281962 conn(0x7f5c140d0050 msgr2=0x7f5c140d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 -- 172.21.15.16:0/219281962 shutdown_connections 2024-09-17T05:10:06.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.549+0000 7f5c1a800640 1 -- 172.21.15.16:0/219281962 wait complete. 2024-09-17T05:10:06.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 Processor -- start 2024-09-17T05:10:06.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 -- start start 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 0x7f5c141f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c140b4620 0x7f5c1415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 0x7f5c1414d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14074b00 con 0x7f5c1415f4e0 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c14074c80 con 0x7f5c140b4620 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5c14074e00 con 0x7f5c1406e4e0 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 0x7f5c1414d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 0x7f5c1414d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60072/0 (socket says 172.21.15.16:60072) 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 -- 172.21.15.16:0/1606577905 learned_addr learned my addr 172.21.15.16:0/1606577905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c0be00640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c140b4620 0x7f5c1415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13400640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 0x7f5c141f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 -- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 msgr2=0x7f5c141f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 0x7f5c141f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 -- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c140b4620 msgr2=0x7f5c1415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c140b4620 0x7f5c1415d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 -- 172.21.15.16:0/1606577905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c141f7700 con 0x7f5c1415f4e0 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c0be00640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c140b4620 0x7f5c1415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c13e00640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 0x7f5c1414d400 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00cb10 tx=0x7f5c0c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c016020 con 0x7f5c1415f4e0 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c0c00ce50 con 0x7f5c1415f4e0 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c00ade0 con 0x7f5c1415f4e0 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1415de60 con 0x7f5c1415f4e0 2024-09-17T05:10:06.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.553+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1415ccf0 con 0x7f5c1415f4e0 2024-09-17T05:10:06.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.557+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1415efa0 con 0x7f5c1415f4e0 2024-09-17T05:10:06.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.557+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5c0c019020 con 0x7f5c1415f4e0 2024-09-17T05:10:06.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.557+0000 7f5c0b400640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5bf0073230 0x7f5bf0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.557+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5249+0+0 (secure 0 0 0) 0x7f5c0c094270 con 0x7f5c1415f4e0 2024-09-17T05:10:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.557+0000 7f5c13400640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5bf0073230 0x7f5bf0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:06.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.561+0000 7f5c13400640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5bf0073230 0x7f5bf0075740 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009600 tx=0x7f5c040091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:06.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.561+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5c0c020020 con 0x7f5c1415f4e0 2024-09-17T05:10:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:06.741+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5c1409da10 con 0x7f5c1415f4e0 2024-09-17T05:10:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.577+0000 7f5c0b400640 1 -- 172.21.15.16:0/1606577905 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v146) ==== 138+0+0 (secure 0 0 0) 0x7f5c141f7700 con 0x7f5c1415f4e0 2024-09-17T05:10:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5bf0073230 msgr2=0x7f5bf0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5bf0073230 0x7f5bf0075740 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009600 tx=0x7f5c040091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 msgr2=0x7f5c1414d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 0x7f5c1414d400 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00cb10 tx=0x7f5c0c00cc00 comp rx=0 tx=0).stop 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 shutdown_connections 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5bf0073230 0x7f5bf0075740 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1415f4e0 0x7f5c1414d400 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c140b4620 0x7f5c1415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 --2- 172.21.15.16:0/1606577905 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5c1406e4e0 0x7f5c141f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 >> 172.21.15.16:0/1606577905 conn(0x7f5c140d0050 msgr2=0x7f5c140d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 shutdown_connections 2024-09-17T05:10:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.581+0000 7f5c1a800640 1 -- 172.21.15.16:0/1606577905 wait complete. 2024-09-17T05:10:07.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T05:10:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 -- 172.21.15.16:0/4086694095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 msgr2=0x7f5a3415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 --2- 172.21.15.16:0/4086694095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a3415bb60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009900 tx=0x7f5a2401c850 comp rx=0 tx=0).stop 2024-09-17T05:10:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 -- 172.21.15.16:0/4086694095 shutdown_connections 2024-09-17T05:10:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 --2- 172.21.15.16:0/4086694095 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a34073240 0x7f5a34073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 --2- 172.21.15.16:0/4086694095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a3415bb60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 -- 172.21.15.16:0/4086694095 >> 172.21.15.16:0/4086694095 conn(0x7f5a3406f900 msgr2=0x7f5a340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 -- 172.21.15.16:0/4086694095 shutdown_connections 2024-09-17T05:10:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.777+0000 7f5a39e00640 1 -- 172.21.15.16:0/4086694095 wait complete. 2024-09-17T05:10:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 Processor -- start 2024-09-17T05:10:07.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 -- start start 2024-09-17T05:10:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a34073240 0x7f5a341fa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a341518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a3415c0a0 0x7f5a34151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34168c90 con 0x7f5a340b2f50 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34168b10 con 0x7f5a3415c0a0 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a39e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5a34168e10 con 0x7f5a34073240 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a341518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a341518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45782/0 (socket says 172.21.15.16:45782) 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 -- 172.21.15.16:0/639415120 learned_addr learned my addr 172.21.15.16:0/639415120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32a00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a34073240 0x7f5a341fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a33400640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a3415c0a0 0x7f5a34151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 -- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a34073240 msgr2=0x7f5a341fa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a34073240 0x7f5a341fa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 -- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a3415c0a0 msgr2=0x7f5a34151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a3415c0a0 0x7f5a34151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 -- 172.21.15.16:0/639415120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a341650c0 con 0x7f5a340b2f50 2024-09-17T05:10:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a32000640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a341518f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c3c0 tx=0x7f5a2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.781+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20017070 con 0x7f5a340b2f50 2024-09-17T05:10:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a200053c0 con 0x7f5a340b2f50 2024-09-17T05:10:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341653a0 con 0x7f5a340b2f50 2024-09-17T05:10:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20003860 con 0x7f5a340b2f50 2024-09-17T05:10:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a34165960 con 0x7f5a340b2f50 2024-09-17T05:10:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a34073650 con 0x7f5a340b2f50 2024-09-17T05:10:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a2001a070 con 0x7f5a340b2f50 2024-09-17T05:10:07.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a2b400640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a14073200 0x7f5a14075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5619+0+0 (secure 0 0 0) 0x7f5a200934f0 con 0x7f5a340b2f50 2024-09-17T05:10:07.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.785+0000 7f5a32a00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a14073200 0x7f5a14075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:07.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.789+0000 7f5a32a00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a14073200 0x7f5a14075710 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5a240af000 tx=0x7f5a2401b090 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:07.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.793+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a2005de70 con 0x7f5a340b2f50 2024-09-17T05:10:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:07.969+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5a340cdeb0 con 0x7f5a340b2f50 2024-09-17T05:10:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a2b400640 1 -- 172.21.15.16:0/639415120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v147) ==== 170+0+0 (secure 0 0 0) 0x7f5a200628a0 con 0x7f5a340b2f50 2024-09-17T05:10:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-17T05:10:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a14073200 msgr2=0x7f5a14075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a14073200 0x7f5a14075710 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5a240af000 tx=0x7f5a2401b090 comp rx=0 tx=0).stop 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 msgr2=0x7f5a341518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a341518f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c3c0 tx=0x7f5a2000c880 comp rx=0 tx=0).stop 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 shutdown_connections 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5a14073200 0x7f5a14075710 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5a3415c0a0 0x7f5a34151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a340b2f50 0x7f5a341518f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 --2- 172.21.15.16:0/639415120 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5a34073240 0x7f5a341fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 >> 172.21.15.16:0/639415120 conn(0x7f5a3406f900 msgr2=0x7f5a340cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 shutdown_connections 2024-09-17T05:10:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.585+0000 7f5a39e00640 1 -- 172.21.15.16:0/639415120 wait complete. 2024-09-17T05:10:08.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T05:10:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 -- 172.21.15.16:0/638453906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 msgr2=0x7fa13c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 --2- 172.21.15.16:0/638453906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c15efe0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009900 tx=0x7fa12c01c8e0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 -- 172.21.15.16:0/638453906 shutdown_connections 2024-09-17T05:10:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 --2- 172.21.15.16:0/638453906 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c074e30 0x7fa13c15f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 --2- 172.21.15.16:0/638453906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c15efe0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 -- 172.21.15.16:0/638453906 >> 172.21.15.16:0/638453906 conn(0x7fa13c0b2250 msgr2=0x7fa13c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 -- 172.21.15.16:0/638453906 shutdown_connections 2024-09-17T05:10:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 -- 172.21.15.16:0/638453906 wait complete. 2024-09-17T05:10:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 Processor -- start 2024-09-17T05:10:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.789+0000 7fa143600640 1 -- start start 2024-09-17T05:10:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa13c074e30 0x7fa13c14e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c15ba50 0x7fa13c14ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c168e80 con 0x7fa13c074a20 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c168d00 con 0x7fa13c074e30 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c169000 con 0x7fa13c15ba50 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c14cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45804/0 (socket says 172.21.15.16:45804) 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 -- 172.21.15.16:0/3900207251 learned_addr learned my addr 172.21.15.16:0/3900207251 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa133e00640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa13c074e30 0x7fa13c14e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa141600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c15ba50 0x7fa13c14ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 -- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c15ba50 msgr2=0x7fa13c14ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c15ba50 0x7fa13c14ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 -- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa13c074e30 msgr2=0x7fa13c14e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa13c074e30 0x7fa13c14e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 -- 172.21.15.16:0/3900207251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c14da30 con 0x7fa13c074a20 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa141600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c15ba50 0x7fa13c14ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa140c00640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c14cfe0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009850 tx=0x7fa12c005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0b2040 con 0x7fa13c074a20 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa12c004130 con 0x7fa13c074a20 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c14f110 con 0x7fa13c074a20 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c003600 con 0x7fa13c074a20 2024-09-17T05:10:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c156e50 con 0x7fa13c074a20 2024-09-17T05:10:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.793+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c15be60 con 0x7fa13c074a20 2024-09-17T05:10:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.797+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa12c0bf020 con 0x7fa13c074a20 2024-09-17T05:10:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.797+0000 7fa139600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa118073230 0x7fa118075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.797+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6001+0+0 (secure 0 0 0) 0x7fa12c12eae0 con 0x7fa13c074a20 2024-09-17T05:10:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.797+0000 7fa133e00640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa118073230 0x7fa118075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.797+0000 7fa133e00640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa118073230 0x7fa118075740 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa13c15d2f0 tx=0x7fa12400b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:08.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:08.801+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa12c0c6020 con 0x7fa13c074a20 2024-09-17T05:10:09.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.013+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa13c150170 con 0x7fa13c074a20 2024-09-17T05:10:09.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.585+0000 7fa139600640 1 -- 172.21.15.16:0/3900207251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v106) ==== 174+0+0 (secure 0 0 0) 0x7fa13c14da30 con 0x7fa13c074a20 2024-09-17T05:10:09.588 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 23 and data pool 24 2024-09-17T05:10:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa118073230 msgr2=0x7fa118075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:09.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa118073230 0x7fa118075740 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa13c15d2f0 tx=0x7fa12400b2e0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 msgr2=0x7fa13c14cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:09.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c14cfe0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009850 tx=0x7fa12c005e70 comp rx=0 tx=0).stop 2024-09-17T05:10:09.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 shutdown_connections 2024-09-17T05:10:09.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa118073230 0x7fa118075740 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa13c15ba50 0x7fa13c14ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa13c074e30 0x7fa13c14e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 --2- 172.21.15.16:0/3900207251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa13c074a20 0x7fa13c14cfe0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.589+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 >> 172.21.15.16:0/3900207251 conn(0x7fa13c0b2250 msgr2=0x7fa13c150970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:09.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.593+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 shutdown_connections 2024-09-17T05:10:09.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.593+0000 7fa143600640 1 -- 172.21.15.16:0/3900207251 wait complete. 2024-09-17T05:10:09.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:09.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 -- 172.21.15.16:0/3500084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc073240 msgr2=0x7fd1fc073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 --2- 172.21.15.16:0/3500084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc073240 0x7fd1fc073650 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc071ce0 tx=0x7fd1f401ccf0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 -- 172.21.15.16:0/3500084041 shutdown_connections 2024-09-17T05:10:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 --2- 172.21.15.16:0/3500084041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc073240 0x7fd1fc073650 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 --2- 172.21.15.16:0/3500084041 >> v2:172.21.15.104:3300/0 conn(0x7fd1fc0b2f50 0x7fd1fc15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 -- 172.21.15.16:0/3500084041 >> 172.21.15.16:0/3500084041 conn(0x7fd1fc06f900 msgr2=0x7fd1fc0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:09.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 -- 172.21.15.16:0/3500084041 shutdown_connections 2024-09-17T05:10:09.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 -- 172.21.15.16:0/3500084041 wait complete. 2024-09-17T05:10:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 Processor -- start 2024-09-17T05:10:09.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.809+0000 7fd201a00640 1 -- start start 2024-09-17T05:10:09.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1fc073240 0x7fd1fc0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:09.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1fc0b2f50 0x7fd1fc0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 0x7fd1fc0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc168d10 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc168b90 con 0x7fd1fc0b2f50 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc168e90 con 0x7fd1fc073240 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 0x7fd1fc0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 0x7fd1fc0cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45822/0 (socket says 172.21.15.16:45822) 2024-09-17T05:10:09.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 -- 172.21.15.16:0/901543135 learned_addr learned my addr 172.21.15.16:0/901543135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1faa00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1fc073240 0x7fd1fc0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1f3e00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1fc0b2f50 0x7fd1fc0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 -- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1fc073240 msgr2=0x7fd1fc0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1fc073240 0x7fd1fc0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 -- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1fc0b2f50 msgr2=0x7fd1fc0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1fc0b2f50 0x7fd1fc0b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 -- 172.21.15.16:0/901543135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc0b1090 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1fb400640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 0x7fd1fc0cae10 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fd1f401b3a0 tx=0x7fd1f4003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f40bc070 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1f40ad8e0 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f40b2040 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc0cf410 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc0cf9d0 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.813+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc073650 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.817+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd1f4004000 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.817+0000 7fd1f2a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd1d8073230 0x7fd1d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.817+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7fd1f412e380 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.817+0000 7fd1faa00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd1d8073230 0x7fd1d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:09.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.817+0000 7fd1faa00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd1d8073230 0x7fd1d8075740 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec005fd0 tx=0x7fd1ec005d10 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:09.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.821+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd1f40f2db0 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.997+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1fc145270 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:09.997+0000 7fd1f2a00640 1 -- 172.21.15.16:0/901543135 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16949 (secure 0 0 0) 0x7fd1fc0b1090 con 0x7fd1fc15c0a0 2024-09-17T05:10:09.999 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:09.999 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":148,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:09.017676+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:10:06.755147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-17T05:10:07.983444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd1d8073230 msgr2=0x7fd1d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd1d8073230 0x7fd1d8075740 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec005fd0 tx=0x7fd1ec005d10 comp rx=0 tx=0).stop 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 msgr2=0x7fd1fc0cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 0x7fd1fc0cae10 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fd1f401b3a0 tx=0x7fd1f4003f80 comp rx=0 tx=0).stop 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 shutdown_connections 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fd1d8073230 0x7fd1d8075740 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1fc15c0a0 0x7fd1fc0cae10 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1fc0b2f50 0x7fd1fc0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 --2- 172.21.15.16:0/901543135 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fd1fc073240 0x7fd1fc0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 >> 172.21.15.16:0/901543135 conn(0x7fd1fc06f900 msgr2=0x7fd1fc15d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 shutdown_connections 2024-09-17T05:10:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.001+0000 7fd201a00640 1 -- 172.21.15.16:0/901543135 wait complete. 2024-09-17T05:10:10.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 -- 172.21.15.16:0/1295469030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979815c6f0 msgr2=0x7f9798160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 --2- 172.21.15.16:0/1295469030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979815c6f0 0x7f9798160390 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f9788009900 tx=0x7f978801c7c0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 -- 172.21.15.16:0/1295469030 shutdown_connections 2024-09-17T05:10:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 --2- 172.21.15.16:0/1295469030 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97981608d0 0x7f97980d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 --2- 172.21.15.16:0/1295469030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f979815c6f0 0x7f9798160390 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 -- 172.21.15.16:0/1295469030 >> 172.21.15.16:0/1295469030 conn(0x7f97980cf380 msgr2=0x7f97980d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 -- 172.21.15.16:0/1295469030 shutdown_connections 2024-09-17T05:10:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 -- 172.21.15.16:0/1295469030 wait complete. 2024-09-17T05:10:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 Processor -- start 2024-09-17T05:10:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 -- start start 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97980d0b80 0x7f97980ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.181+0000 7f979ca00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f979815c6f0 0x7f9798147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 0x7f97981f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f97980b3350 con 0x7f97981608d0 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97980b31d0 con 0x7f97980d0b80 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f979816a530 con 0x7f979815c6f0 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 0x7f97981f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 0x7f97981f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45842/0 (socket says 172.21.15.16:45842) 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 -- 172.21.15.16:0/3581618021 learned_addr learned my addr 172.21.15.16:0/3581618021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f978d400640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f979815c6f0 0x7f9798147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9795e00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97980d0b80 0x7f97980ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 -- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f979815c6f0 msgr2=0x7f9798147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f979815c6f0 0x7f9798147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 -- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97980d0b80 msgr2=0x7f97980ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97980d0b80 0x7f97980ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 -- 172.21.15.16:0/3581618021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97980cab80 con 0x7f97981608d0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f9796800640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 0x7f97981f6f10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f979000c6b0 tx=0x7f979000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790003b80 con 0x7f97981608d0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9790003d20 con 0x7f97981608d0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97980cae60 con 0x7f97981608d0 2024-09-17T05:10:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790005330 con 0x7f97981608d0 2024-09-17T05:10:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97980cb420 con 0x7f97981608d0 2024-09-17T05:10:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9798160390 con 0x7f97981608d0 2024-09-17T05:10:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.185+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f97900054d0 con 0x7f97981608d0 2024-09-17T05:10:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.189+0000 7f978ea00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9774073230 0x7f9774075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.189+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f9790094110 con 0x7f97981608d0 2024-09-17T05:10:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.189+0000 7f9795e00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9774073230 0x7f9774075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.189+0000 7f9795e00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9774073230 0x7f9774075740 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f97880ae000 tx=0x7f97880b3040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.193+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9798160ce0 con 0x7f97981608d0 2024-09-17T05:10:10.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.369+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97981679c0 con 0x7f97981608d0 2024-09-17T05:10:10.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.369+0000 7f978ea00640 1 -- 172.21.15.16:0/3581618021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+16949 (secure 0 0 0) 0x7f979005e830 con 0x7f97981608d0 2024-09-17T05:10:10.370 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:10.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":148,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:09.017676+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:10:06.755147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-17T05:10:07.983444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9774073230 msgr2=0x7f9774075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9774073230 0x7f9774075740 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f97880ae000 tx=0x7f97880b3040 comp rx=0 tx=0).stop 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 msgr2=0x7f97981f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 0x7f97981f6f10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f979000c6b0 tx=0x7f979000cb70 comp rx=0 tx=0).stop 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 shutdown_connections 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9774073230 0x7f9774075740 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97981608d0 0x7f97981f6f10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f979815c6f0 0x7f9798147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 --2- 172.21.15.16:0/3581618021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f97980d0b80 0x7f97980ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 >> 172.21.15.16:0/3581618021 conn(0x7f97980cf380 msgr2=0x7f979815b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 shutdown_connections 2024-09-17T05:10:10.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.373+0000 7f979ca00640 1 -- 172.21.15.16:0/3581618021 wait complete. 2024-09-17T05:10:10.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T05:10:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 -- 172.21.15.16:0/390632142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5350074a20 msgr2=0x7f535015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 --2- 172.21.15.16:0/390632142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5350074a20 0x7f535015efe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5340009870 tx=0x7f534001c780 comp rx=0 tx=0).stop 2024-09-17T05:10:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 -- 172.21.15.16:0/390632142 shutdown_connections 2024-09-17T05:10:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 --2- 172.21.15.16:0/390632142 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5350074a20 0x7f535015efe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 -- 172.21.15.16:0/390632142 >> 172.21.15.16:0/390632142 conn(0x7f53500b2250 msgr2=0x7f53500d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 -- 172.21.15.16:0/390632142 shutdown_connections 2024-09-17T05:10:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 -- 172.21.15.16:0/390632142 wait complete. 2024-09-17T05:10:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 Processor -- start 2024-09-17T05:10:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.557+0000 7f5356200640 1 -- start start 2024-09-17T05:10:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 0x7f53500c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5350074e30 0x7f53500b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f535015f520 0x7f5350157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5350169d80 con 0x7f5350074a20 2024-09-17T05:10:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5350169c00 con 0x7f5350074e30 2024-09-17T05:10:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5350169f00 con 0x7f535015f520 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 0x7f53500c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 0x7f53500c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45848/0 (socket says 172.21.15.16:45848) 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 -- 172.21.15.16:0/3635210462 learned_addr learned my addr 172.21.15.16:0/3635210462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5347e00640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5350074e30 0x7f53500b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534fe00640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f535015f520 0x7f5350157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 -- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f535015f520 msgr2=0x7f5350157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f535015f520 0x7f5350157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 -- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5350074e30 msgr2=0x7f53500b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5350074e30 0x7f53500b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 -- 172.21.15.16:0/3635210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53500aff30 con 0x7f5350074a20 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f534f400640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 0x7f53500c6950 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f534001a3b0 tx=0x7f53400056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340004750 con 0x7f5350074a20 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53400048f0 con 0x7f5350074a20 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5340004d20 con 0x7f5350074a20 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53500b0210 con 0x7f5350074a20 2024-09-17T05:10:10.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53500b07d0 con 0x7f5350074a20 2024-09-17T05:10:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.561+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f535015efe0 con 0x7f5350074a20 2024-09-17T05:10:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.565+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f53400a8520 con 0x7f5350074a20 2024-09-17T05:10:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.565+0000 7f5347400640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f532c073200 0x7f532c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:10.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.565+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6101+0+0 (secure 0 0 0) 0x7f534012e610 con 0x7f5350074a20 2024-09-17T05:10:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.565+0000 7f5347e00640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f532c073200 0x7f532c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:10.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.565+0000 7f5347e00640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f532c073200 0x7f532c075710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f535014d800 tx=0x7f5338009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:10.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.569+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53400ac030 con 0x7f5350074a20 2024-09-17T05:10:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:10.785+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f535014f5d0 con 0x7f5350074a20 2024-09-17T05:10:11.574 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-09-17T05:10:11.573+0000 7f6a18400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:10:11.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.621+0000 7f5347400640 1 -- 172.21.15.16:0/3635210462 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v109) ==== 118+0+0 (secure 0 0 0) 0x7f5340016370 con 0x7f5350074a20 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f532c073200 msgr2=0x7f532c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f532c073200 0x7f532c075710 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f535014d800 tx=0x7f5338009040 comp rx=0 tx=0).stop 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 msgr2=0x7f53500c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 0x7f53500c6950 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f534001a3b0 tx=0x7f53400056d0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 shutdown_connections 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f532c073200 0x7f532c075710 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f535015f520 0x7f5350157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5350074e30 0x7f53500b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 --2- 172.21.15.16:0/3635210462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5350074a20 0x7f53500c6950 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 >> 172.21.15.16:0/3635210462 conn(0x7f53500b2250 msgr2=0x7f53501589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 shutdown_connections 2024-09-17T05:10:11.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.625+0000 7f5356200640 1 -- 172.21.15.16:0/3635210462 wait complete. 2024-09-17T05:10:11.651 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T05:10:11.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T05:10:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 -- 172.21.15.16:0/2698845732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074a20 msgr2=0x7fb2cc15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2698845732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074a20 0x7fb2cc15efe0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009900 tx=0x7fb2c001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 -- 172.21.15.16:0/2698845732 shutdown_connections 2024-09-17T05:10:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2698845732 >> v2:172.21.15.104:3300/0 conn(0x7fb2cc15f520 0x7fb2cc15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2698845732 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074a20 0x7fb2cc15efe0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 -- 172.21.15.16:0/2698845732 >> 172.21.15.16:0/2698845732 conn(0x7fb2cc0b2250 msgr2=0x7fb2cc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 -- 172.21.15.16:0/2698845732 shutdown_connections 2024-09-17T05:10:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.817+0000 7fb2d4200640 1 -- 172.21.15.16:0/2698845732 wait complete. 2024-09-17T05:10:11.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 Processor -- start 2024-09-17T05:10:11.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 -- start start 2024-09-17T05:10:11.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 0x7fb2cc15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074e30 0x7fb2cc155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2cc15f520 0x7fb2cc0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc168f30 con 0x7fb2cc074a20 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc168db0 con 0x7fb2cc15f520 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d4200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc1690b0 con 0x7fb2cc074e30 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 0x7fb2cc15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 0x7fb2cc15e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45854/0 (socket says 172.21.15.16:45854) 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.821+0000 7fb2d1800640 1 -- 172.21.15.16:0/2505153707 learned_addr learned my addr 172.21.15.16:0/2505153707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d0e00640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074e30 0x7fb2cc155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d2200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2cc15f520 0x7fb2cc0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d1800640 1 -- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074e30 msgr2=0x7fb2cc155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d1800640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074e30 0x7fb2cc155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d1800640 1 -- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2cc15f520 msgr2=0x7fb2cc0b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d1800640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2cc15f520 0x7fb2cc0b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d1800640 1 -- 172.21.15.16:0/2505153707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1fc640 con 0x7fb2cc074a20 2024-09-17T05:10:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d2200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2cc15f520 0x7fb2cc0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d1800640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 0x7fb2cc15e660 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00b0040 tx=0x7fb2c0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00b2040 con 0x7fb2cc074a20 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c0004320 con 0x7fb2cc074a20 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0005290 con 0x7fb2cc074a20 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc15eba0 con 0x7fb2cc074a20 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc1fc8e0 con 0x7fb2cc074a20 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc15efe0 con 0x7fb2cc074a20 2024-09-17T05:10:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.825+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb2c0005430 con 0x7fb2cc074a20 2024-09-17T05:10:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.829+0000 7fb2c9600640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb2ac073200 0x7fb2ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.829+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7fb2c012daf0 con 0x7fb2cc074a20 2024-09-17T05:10:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.829+0000 7fb2d0e00640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb2ac073200 0x7fb2ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:11.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.829+0000 7fb2d0e00640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb2ac073200 0x7fb2ac075710 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc14d8c0 tx=0x7fb2b8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:11.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:11.833+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb2c00164d0 con 0x7fb2cc074a20 2024-09-17T05:10:12.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.041+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb2cc15a830 con 0x7fb2cc074a20 2024-09-17T05:10:12.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2c9600640 1 -- 172.21.15.16:0/2505153707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v110) ==== 122+0+0 (secure 0 0 0) 0x7fb2c00f88e0 con 0x7fb2cc074a20 2024-09-17T05:10:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb2ac073200 msgr2=0x7fb2ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb2ac073200 0x7fb2ac075710 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc14d8c0 tx=0x7fb2b8009040 comp rx=0 tx=0).stop 2024-09-17T05:10:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 msgr2=0x7fb2cc15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:12.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 0x7fb2cc15e660 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00b0040 tx=0x7fb2c0005760 comp rx=0 tx=0).stop 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 shutdown_connections 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb2ac073200 0x7fb2ac075710 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2cc15f520 0x7fb2cc0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb2cc074e30 0x7fb2cc155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 --2- 172.21.15.16:0/2505153707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2cc074a20 0x7fb2cc15e660 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.624+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 >> 172.21.15.16:0/2505153707 conn(0x7fb2cc0b2250 msgr2=0x7fb2cc073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.628+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 shutdown_connections 2024-09-17T05:10:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.628+0000 7fb2d4200640 1 -- 172.21.15.16:0/2505153707 wait complete. 2024-09-17T05:10:12.650 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T05:10:12.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 -- 172.21.15.16:0/3995540740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994073240 msgr2=0x7f2994073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:12.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 --2- 172.21.15.16:0/3995540740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994073240 0x7f2994073650 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f298c009c80 tx=0x7f298c01cd10 comp rx=0 tx=0).stop 2024-09-17T05:10:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 -- 172.21.15.16:0/3995540740 shutdown_connections 2024-09-17T05:10:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 --2- 172.21.15.16:0/3995540740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994073240 0x7f2994073650 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 --2- 172.21.15.16:0/3995540740 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f299415c270 0x7f299415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 -- 172.21.15.16:0/3995540740 >> 172.21.15.16:0/3995540740 conn(0x7f299406f900 msgr2=0x7f29940b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 -- 172.21.15.16:0/3995540740 shutdown_connections 2024-09-17T05:10:12.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 -- 172.21.15.16:0/3995540740 wait complete. 2024-09-17T05:10:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 Processor -- start 2024-09-17T05:10:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.808+0000 7f299ba00640 1 -- start start 2024-09-17T05:10:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2994073240 0x7f299414e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 0x7f2994151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f299415c270 0x7f2994151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2994168e00 con 0x7f29940b2f50 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2994168c80 con 0x7f299415c270 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2994168f80 con 0x7f2994073240 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 0x7f2994151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 0x7f2994151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45872/0 (socket says 172.21.15.16:45872) 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 -- 172.21.15.16:0/1984845092 learned_addr learned my addr 172.21.15.16:0/1984845092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f2999000640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2994073240 0x7f299414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f2999a00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f299415c270 0x7f2994151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 -- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2994073240 msgr2=0x7f299414e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2994073240 0x7f299414e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 -- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f299415c270 msgr2=0x7f2994151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f299415c270 0x7f2994151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 -- 172.21.15.16:0/1984845092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2994165180 con 0x7f29940b2f50 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f298be00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 0x7f2994151860 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f297c00e740 tx=0x7f297c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c00ccc0 con 0x7f29940b2f50 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f297c00ce60 con 0x7f29940b2f50 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c005260 con 0x7f29940b2f50 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2994164bc0 con 0x7f29940b2f50 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2994153d70 con 0x7f29940b2f50 2024-09-17T05:10:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.812+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29941f9c50 con 0x7f29940b2f50 2024-09-17T05:10:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.816+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f297c005400 con 0x7f29940b2f50 2024-09-17T05:10:12.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.816+0000 7f2991600640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2970073200 0x7f2970075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.816+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f297c094bb0 con 0x7f29940b2f50 2024-09-17T05:10:12.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.816+0000 7f2999000640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2970073200 0x7f2970075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:12.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.816+0000 7f2999000640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2970073200 0x7f2970075710 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2984007fd0 tx=0x7f2984009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:12.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:12.820+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f297c05f3e0 con 0x7f29940b2f50 2024-09-17T05:10:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2994073650 con 0x7f29940b2f50 2024-09-17T05:10:13.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f2991600640 1 -- 172.21.15.16:0/1984845092 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5233 (secure 0 0 0) 0x7f297c063e10 con 0x7f29940b2f50 2024-09-17T05:10:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:13.038 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":110,"btime":"2024-09-17T05:10:12:626935+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:10:09.017636+0000","modified":"2024-09-17T05:10:12.046191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/291864941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":291864941},{"type":"v1","addr":"172.21.15.16:6835","nonce":291864941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":11}]} 2024-09-17T05:10:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 110 2024-09-17T05:10:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2970073200 msgr2=0x7f2970075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2970073200 0x7f2970075710 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2984007fd0 tx=0x7f2984009040 comp rx=0 tx=0).stop 2024-09-17T05:10:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 msgr2=0x7f2994151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 0x7f2994151860 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f297c00e740 tx=0x7f297c00ec00 comp rx=0 tx=0).stop 2024-09-17T05:10:13.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 shutdown_connections 2024-09-17T05:10:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2970073200 0x7f2970075710 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f299415c270 0x7f2994151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29940b2f50 0x7f2994151860 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 --2- 172.21.15.16:0/1984845092 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2994073240 0x7f299414e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 >> 172.21.15.16:0/1984845092 conn(0x7f299406f900 msgr2=0x7f299415d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 shutdown_connections 2024-09-17T05:10:13.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.036+0000 7f299ba00640 1 -- 172.21.15.16:0/1984845092 wait complete. 2024-09-17T05:10:13.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 -- 172.21.15.16:0/3391625429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c015cbb0 msgr2=0x7fb0c0073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 --2- 172.21.15.16:0/3391625429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c015cbb0 0x7fb0c0073610 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009900 tx=0x7fb0b001c8e0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 -- 172.21.15.16:0/3391625429 shutdown_connections 2024-09-17T05:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 --2- 172.21.15.16:0/3391625429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c015cbb0 0x7fb0c0073610 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 -- 172.21.15.16:0/3391625429 >> 172.21.15.16:0/3391625429 conn(0x7fb0c00d0860 msgr2=0x7fb0c00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 -- 172.21.15.16:0/3391625429 shutdown_connections 2024-09-17T05:10:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 -- 172.21.15.16:0/3391625429 wait complete. 2024-09-17T05:10:13.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 Processor -- start 2024-09-17T05:10:13.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.232+0000 7fb0c5400640 1 -- start start 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 0x7fb0c0143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c00cfe30 0x7fb0c0143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0c015cbb0 0x7fb0c01443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c016add0 con 0x7fb0c0073b50 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c016ac50 con 0x7fb0c00cfe30 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c016af50 con 0x7fb0c015cbb0 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 0x7fb0c0143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 0x7fb0c0143530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45878/0 (socket says 172.21.15.16:45878) 2024-09-17T05:10:13.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 -- 172.21.15.16:0/47226404 learned_addr learned my addr 172.21.15.16:0/47226404 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 -- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0c015cbb0 msgr2=0x7fb0c01443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0c015cbb0 0x7fb0c01443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 -- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c00cfe30 msgr2=0x7fb0c0143e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c00cfe30 0x7fb0c0143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 -- 172.21.15.16:0/47226404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c0155d10 con 0x7fb0c0073b50 2024-09-17T05:10:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0be800640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 0x7fb0c0143530 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00b0040 tx=0x7fb0b00a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00b2040 con 0x7fb0c0073b50 2024-09-17T05:10:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b00041b0 con 0x7fb0c0073b50 2024-09-17T05:10:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0003600 con 0x7fb0c0073b50 2024-09-17T05:10:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c01549d0 con 0x7fb0c0073b50 2024-09-17T05:10:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c014a840 con 0x7fb0c0073b50 2024-09-17T05:10:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.236+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c0073610 con 0x7fb0c0073b50 2024-09-17T05:10:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.240+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0b00bf020 con 0x7fb0c0073b50 2024-09-17T05:10:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.240+0000 7fb0b7400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb09c073230 0x7fb09c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.240+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7fb0b012e6a0 con 0x7fb0c0073b50 2024-09-17T05:10:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.240+0000 7fb0b5e00640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb09c073230 0x7fb09c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:13.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.240+0000 7fb0b5e00640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb09c073230 0x7fb09c075740 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb0c01537a0 tx=0x7fb0a8006bd0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.244+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb0b00c6020 con 0x7fb0c0073b50 2024-09-17T05:10:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.420+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0c00b0e70 con 0x7fb0c0073b50 2024-09-17T05:10:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.420+0000 7fb0b7400640 1 -- 172.21.15.16:0/47226404 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16936 (secure 0 0 0) 0x7fb0b00164d0 con 0x7fb0c0073b50 2024-09-17T05:10:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:13.423 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":149,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:10.590092+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:10:06.755147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-17T05:10:07.983444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb09c073230 msgr2=0x7fb09c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb09c073230 0x7fb09c075740 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb0c01537a0 tx=0x7fb0a8006bd0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 msgr2=0x7fb0c0143530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 0x7fb0c0143530 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00b0040 tx=0x7fb0b00a55a0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 shutdown_connections 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb09c073230 0x7fb09c075740 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb0c015cbb0 0x7fb0c01443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c00cfe30 0x7fb0c0143e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 --2- 172.21.15.16:0/47226404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0073b50 0x7fb0c0143530 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 >> 172.21.15.16:0/47226404 conn(0x7fb0c00d0860 msgr2=0x7fb0c00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 shutdown_connections 2024-09-17T05:10:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.424+0000 7fb0c5400640 1 -- 172.21.15.16:0/47226404 wait complete. 2024-09-17T05:10:13.446 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:10:13.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:13.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d3200640 1 -- 172.21.15.16:0/1533181679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b4003c20 msgr2=0x7f33b4004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d3200640 1 --2- 172.21.15.16:0/1533181679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33b4003c20 0x7f33b4004070 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f33c000c4a0 tx=0x7f33c001ced0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d2800640 1 -- 172.21.15.16:0/1533181679 reap_dead start 2024-09-17T05:10:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d3200640 1 -- 172.21.15.16:0/1533181679 shutdown_connections 2024-09-17T05:10:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d3200640 1 -- 172.21.15.16:0/1533181679 >> 172.21.15.16:0/1533181679 conn(0x7f33cc06f9c0 msgr2=0x7f33cc0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d3200640 1 -- 172.21.15.16:0/1533181679 shutdown_connections 2024-09-17T05:10:13.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.608+0000 7f33d3200640 1 -- 172.21.15.16:0/1533181679 wait complete. 2024-09-17T05:10:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 Processor -- start 2024-09-17T05:10:13.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 -- start start 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f33b4003c20 0x7f33cc1f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 0x7f33cc1446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33cc1f9ef0 0x7f33cc06d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1679c0 con 0x7f33cc164af0 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc167840 con 0x7f33cc1f9ef0 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f33cc167b40 con 0x7f33b4003c20 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 0x7f33cc1446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:13.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 0x7f33cc1446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45890/0 (socket says 172.21.15.16:45890) 2024-09-17T05:10:13.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 -- 172.21.15.16:0/595928952 learned_addr learned my addr 172.21.15.16:0/595928952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1e00640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f33b4003c20 0x7f33cc1f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d2800640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33cc1f9ef0 0x7f33cc06d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 -- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f33b4003c20 msgr2=0x7f33cc1f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f33b4003c20 0x7f33cc1f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 -- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33cc1f9ef0 msgr2=0x7f33cc06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33cc1f9ef0 0x7f33cc06d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 -- 172.21.15.16:0/595928952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc1f70c0 con 0x7f33cc164af0 2024-09-17T05:10:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d2800640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33cc1f9ef0 0x7f33cc06d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1e00640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f33b4003c20 0x7f33cc1f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d1400640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 0x7f33cc1446e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f33bc011ae0 tx=0x7f33bc009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc01e070 con 0x7f33cc164af0 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33bc00fcd0 con 0x7f33cc164af0 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc019670 con 0x7f33cc164af0 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.612+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc1f73a0 con 0x7f33cc164af0 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.616+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc144c20 con 0x7f33cc164af0 2024-09-17T05:10:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.616+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc1f78c0 con 0x7f33cc164af0 2024-09-17T05:10:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.616+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33bc021070 con 0x7f33cc164af0 2024-09-17T05:10:13.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.616+0000 7f33c6000640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33b0073230 0x7f33b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.616+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f33bc094cc0 con 0x7f33cc164af0 2024-09-17T05:10:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.616+0000 7f33d1e00640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33b0073230 0x7f33b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:13.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.620+0000 7f33d1e00640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33b0073230 0x7f33b0075740 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ac000 tx=0x7f33c0019d10 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.620+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33cc1f78c0 con 0x7f33cc164af0 2024-09-17T05:10:13.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.792+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33cc1f6e50 con 0x7f33b0073230 2024-09-17T05:10:13.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.796+0000 7f33c6000640 1 -- 172.21.15.16:0/595928952 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7f33cc1f6e50 con 0x7f33b0073230 2024-09-17T05:10:13.802 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:13.805 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33b0073230 msgr2=0x7f33b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33b0073230 0x7f33b0075740 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ac000 tx=0x7f33c0019d10 comp rx=0 tx=0).stop 2024-09-17T05:10:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 msgr2=0x7f33cc1446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:13.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 0x7f33cc1446e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f33bc011ae0 tx=0x7f33bc009ee0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 shutdown_connections 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f33b0073230 0x7f33b0075740 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33cc1f9ef0 0x7f33cc06d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc164af0 0x7f33cc1446e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 --2- 172.21.15.16:0/595928952 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f33b4003c20 0x7f33cc1f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 >> 172.21.15.16:0/595928952 conn(0x7f33cc06f9c0 msgr2=0x7f33cc0d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 shutdown_connections 2024-09-17T05:10:13.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:13.800+0000 7f33d3200640 1 -- 172.21.15.16:0/595928952 wait complete. 2024-09-17T05:10:13.824 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-17T05:10:13.612625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281976,"kb_used_data":15804,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692744,"statfs":{"total":773094113280,"available":772805369856,"internally_reserved":0,"allocated":16183296,"data_stored":8114104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002017"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637282+0000","last_change":"2024-09-17T05:10:09.631745+0000","last_active":"2024-09-17T05:10:10.637282+0000","last_peered":"2024-09-17T05:10:10.637282+0000","last_clean":"2024-09-17T05:10:10.637282+0000","last_became_active":"2024-09-17T05:10:09.630466+0000","last_became_peered":"2024-09-17T05:10:09.630466+0000","last_unstale":"2024-09-17T05:10:10.637282+0000","last_undegraded":"2024-09-17T05:10:10.637282+0000","last_fullsized":"2024-09-17T05:10:10.637282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:40.914328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632990+0000","last_change":"2024-09-17T05:10:09.610783+0000","last_active":"2024-09-17T05:10:10.632990+0000","last_peered":"2024-09-17T05:10:10.632990+0000","last_clean":"2024-09-17T05:10:10.632990+0000","last_became_active":"2024-09-17T05:10:09.610435+0000","last_became_peered":"2024-09-17T05:10:09.610435+0000","last_unstale":"2024-09-17T05:10:10.632990+0000","last_undegraded":"2024-09-17T05:10:10.632990+0000","last_fullsized":"2024-09-17T05:10:10.632990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:05.464420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633021+0000","last_change":"2024-09-17T05:10:09.608026+0000","last_active":"2024-09-17T05:10:10.633021+0000","last_peered":"2024-09-17T05:10:10.633021+0000","last_clean":"2024-09-17T05:10:10.633021+0000","last_became_active":"2024-09-17T05:10:09.607626+0000","last_became_peered":"2024-09-17T05:10:09.607626+0000","last_unstale":"2024-09-17T05:10:10.633021+0000","last_undegraded":"2024-09-17T05:10:10.633021+0000","last_fullsized":"2024-09-17T05:10:10.633021+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:02.303049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711949+0000","last_change":"2024-09-17T05:10:09.634104+0000","last_active":"2024-09-17T05:10:10.711949+0000","last_peered":"2024-09-17T05:10:10.711949+0000","last_clean":"2024-09-17T05:10:10.711949+0000","last_became_active":"2024-09-17T05:10:09.633693+0000","last_became_peered":"2024-09-17T05:10:09.633693+0000","last_unstale":"2024-09-17T05:10:10.711949+0000","last_undegraded":"2024-09-17T05:10:10.711949+0000","last_fullsized":"2024-09-17T05:10:10.711949+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:26.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710630+0000","last_change":"2024-09-17T05:10:09.625690+0000","last_active":"2024-09-17T05:10:10.710630+0000","last_peered":"2024-09-17T05:10:10.710630+0000","last_clean":"2024-09-17T05:10:10.710630+0000","last_became_active":"2024-09-17T05:10:09.625364+0000","last_became_peered":"2024-09-17T05:10:09.625364+0000","last_unstale":"2024-09-17T05:10:10.710630+0000","last_undegraded":"2024-09-17T05:10:10.710630+0000","last_fullsized":"2024-09-17T05:10:10.710630+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:43.854136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636801+0000","last_change":"2024-09-17T05:10:09.629500+0000","last_active":"2024-09-17T05:10:10.636801+0000","last_peered":"2024-09-17T05:10:10.636801+0000","last_clean":"2024-09-17T05:10:10.636801+0000","last_became_active":"2024-09-17T05:10:09.629194+0000","last_became_peered":"2024-09-17T05:10:09.629194+0000","last_unstale":"2024-09-17T05:10:10.636801+0000","last_undegraded":"2024-09-17T05:10:10.636801+0000","last_fullsized":"2024-09-17T05:10:10.636801+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:59.575299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709932+0000","last_change":"2024-09-17T05:10:09.616187+0000","last_active":"2024-09-17T05:10:10.709932+0000","last_peered":"2024-09-17T05:10:10.709932+0000","last_clean":"2024-09-17T05:10:10.709932+0000","last_became_active":"2024-09-17T05:10:09.615830+0000","last_became_peered":"2024-09-17T05:10:09.615830+0000","last_unstale":"2024-09-17T05:10:10.709932+0000","last_undegraded":"2024-09-17T05:10:10.709932+0000","last_fullsized":"2024-09-17T05:10:10.709932+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:15.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.593721+0000","last_change":"2024-09-17T05:10:09.617829+0000","last_active":"2024-09-17T05:10:10.593721+0000","last_peered":"2024-09-17T05:10:10.593721+0000","last_clean":"2024-09-17T05:10:10.593721+0000","last_became_active":"2024-09-17T05:10:09.615252+0000","last_became_peered":"2024-09-17T05:10:09.615252+0000","last_unstale":"2024-09-17T05:10:10.593721+0000","last_undegraded":"2024-09-17T05:10:10.593721+0000","last_fullsized":"2024-09-17T05:10:10.593721+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:59.023497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711158+0000","last_change":"2024-09-17T05:10:09.624167+0000","last_active":"2024-09-17T05:10:10.711158+0000","last_peered":"2024-09-17T05:10:10.711158+0000","last_clean":"2024-09-17T05:10:10.711158+0000","last_became_active":"2024-09-17T05:10:09.622484+0000","last_became_peered":"2024-09-17T05:10:09.622484+0000","last_unstale":"2024-09-17T05:10:10.711158+0000","last_undegraded":"2024-09-17T05:10:10.711158+0000","last_fullsized":"2024-09-17T05:10:10.711158+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:45.823474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710716+0000","last_change":"2024-09-17T05:10:09.623799+0000","last_active":"2024-09-17T05:10:10.710716+0000","last_peered":"2024-09-17T05:10:10.710716+0000","last_clean":"2024-09-17T05:10:10.710716+0000","last_became_active":"2024-09-17T05:10:09.622642+0000","last_became_peered":"2024-09-17T05:10:09.622642+0000","last_unstale":"2024-09-17T05:10:10.710716+0000","last_undegraded":"2024-09-17T05:10:10.710716+0000","last_fullsized":"2024-09-17T05:10:10.710716+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:38.082547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637328+0000","last_change":"2024-09-17T05:10:09.611715+0000","last_active":"2024-09-17T05:10:10.637328+0000","last_peered":"2024-09-17T05:10:10.637328+0000","last_clean":"2024-09-17T05:10:10.637328+0000","last_became_active":"2024-09-17T05:10:09.611362+0000","last_became_peered":"2024-09-17T05:10:09.611362+0000","last_unstale":"2024-09-17T05:10:10.637328+0000","last_undegraded":"2024-09-17T05:10:10.637328+0000","last_fullsized":"2024-09-17T05:10:10.637328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:19.643201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711851+0000","last_change":"2024-09-17T05:10:09.635446+0000","last_active":"2024-09-17T05:10:10.711851+0000","last_peered":"2024-09-17T05:10:10.711851+0000","last_clean":"2024-09-17T05:10:10.711851+0000","last_became_active":"2024-09-17T05:10:09.633274+0000","last_became_peered":"2024-09-17T05:10:09.633274+0000","last_unstale":"2024-09-17T05:10:10.711851+0000","last_undegraded":"2024-09-17T05:10:10.711851+0000","last_fullsized":"2024-09-17T05:10:10.711851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:26.751686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710279+0000","last_change":"2024-09-17T05:10:09.621827+0000","last_active":"2024-09-17T05:10:10.710279+0000","last_peered":"2024-09-17T05:10:10.710279+0000","last_clean":"2024-09-17T05:10:10.710279+0000","last_became_active":"2024-09-17T05:10:09.621284+0000","last_became_peered":"2024-09-17T05:10:09.621284+0000","last_unstale":"2024-09-17T05:10:10.710279+0000","last_undegraded":"2024-09-17T05:10:10.710279+0000","last_fullsized":"2024-09-17T05:10:10.710279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:26.751686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594385+0000","last_change":"2024-09-17T05:10:09.608271+0000","last_active":"2024-09-17T05:10:10.594385+0000","last_peered":"2024-09-17T05:10:10.594385+0000","last_clean":"2024-09-17T05:10:10.594385+0000","last_became_active":"2024-09-17T05:10:09.607944+0000","last_became_peered":"2024-09-17T05:10:09.607944+0000","last_unstale":"2024-09-17T05:10:10.594385+0000","last_undegraded":"2024-09-17T05:10:10.594385+0000","last_fullsized":"2024-09-17T05:10:10.594385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636735+0000","last_change":"2024-09-17T05:10:09.614855+0000","last_active":"2024-09-17T05:10:10.636735+0000","last_peered":"2024-09-17T05:10:10.636735+0000","last_clean":"2024-09-17T05:10:10.636735+0000","last_became_active":"2024-09-17T05:10:09.614545+0000","last_became_peered":"2024-09-17T05:10:09.614545+0000","last_unstale":"2024-09-17T05:10:10.636735+0000","last_undegraded":"2024-09-17T05:10:10.636735+0000","last_fullsized":"2024-09-17T05:10:10.636735+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:19.793741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636653+0000","last_change":"2024-09-17T05:10:09.612933+0000","last_active":"2024-09-17T05:10:10.636653+0000","last_peered":"2024-09-17T05:10:10.636653+0000","last_clean":"2024-09-17T05:10:10.636653+0000","last_became_active":"2024-09-17T05:10:09.612614+0000","last_became_peered":"2024-09-17T05:10:09.612614+0000","last_unstale":"2024-09-17T05:10:10.636653+0000","last_undegraded":"2024-09-17T05:10:10.636653+0000","last_fullsized":"2024-09-17T05:10:10.636653+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:26.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711089+0000","last_change":"2024-09-17T05:10:09.622665+0000","last_active":"2024-09-17T05:10:10.711089+0000","last_peered":"2024-09-17T05:10:10.711089+0000","last_clean":"2024-09-17T05:10:10.711089+0000","last_became_active":"2024-09-17T05:10:09.618310+0000","last_became_peered":"2024-09-17T05:10:09.618310+0000","last_unstale":"2024-09-17T05:10:10.711089+0000","last_undegraded":"2024-09-17T05:10:10.711089+0000","last_fullsized":"2024-09-17T05:10:10.711089+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:16.237008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595413+0000","last_change":"2024-09-17T05:10:09.616816+0000","last_active":"2024-09-17T05:10:10.595413+0000","last_peered":"2024-09-17T05:10:10.595413+0000","last_clean":"2024-09-17T05:10:10.595413+0000","last_became_active":"2024-09-17T05:10:09.616527+0000","last_became_peered":"2024-09-17T05:10:09.616527+0000","last_unstale":"2024-09-17T05:10:10.595413+0000","last_undegraded":"2024-09-17T05:10:10.595413+0000","last_fullsized":"2024-09-17T05:10:10.595413+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:16.357047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633033+0000","last_change":"2024-09-17T05:10:09.608833+0000","last_active":"2024-09-17T05:10:10.633033+0000","last_peered":"2024-09-17T05:10:10.633033+0000","last_clean":"2024-09-17T05:10:10.633033+0000","last_became_active":"2024-09-17T05:10:09.608498+0000","last_became_peered":"2024-09-17T05:10:09.608498+0000","last_unstale":"2024-09-17T05:10:10.633033+0000","last_undegraded":"2024-09-17T05:10:10.633033+0000","last_fullsized":"2024-09-17T05:10:10.633033+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:58.751360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595521+0000","last_change":"2024-09-17T05:10:09.605870+0000","last_active":"2024-09-17T05:10:10.595521+0000","last_peered":"2024-09-17T05:10:10.595521+0000","last_clean":"2024-09-17T05:10:10.595521+0000","last_became_active":"2024-09-17T05:10:09.603240+0000","last_became_peered":"2024-09-17T05:10:09.603240+0000","last_unstale":"2024-09-17T05:10:10.595521+0000","last_undegraded":"2024-09-17T05:10:10.595521+0000","last_fullsized":"2024-09-17T05:10:10.595521+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:38.437182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711231+0000","last_change":"2024-09-17T05:10:09.629435+0000","last_active":"2024-09-17T05:10:10.711231+0000","last_peered":"2024-09-17T05:10:10.711231+0000","last_clean":"2024-09-17T05:10:10.711231+0000","last_became_active":"2024-09-17T05:10:09.629038+0000","last_became_peered":"2024-09-17T05:10:09.629038+0000","last_unstale":"2024-09-17T05:10:10.711231+0000","last_undegraded":"2024-09-17T05:10:10.711231+0000","last_fullsized":"2024-09-17T05:10:10.711231+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:02.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637124+0000","last_change":"2024-09-17T05:10:09.611354+0000","last_active":"2024-09-17T05:10:10.637124+0000","last_peered":"2024-09-17T05:10:10.637124+0000","last_clean":"2024-09-17T05:10:10.637124+0000","last_became_active":"2024-09-17T05:10:09.609893+0000","last_became_peered":"2024-09-17T05:10:09.609893+0000","last_unstale":"2024-09-17T05:10:10.637124+0000","last_undegraded":"2024-09-17T05:10:10.637124+0000","last_fullsized":"2024-09-17T05:10:10.637124+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:38.946662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709685+0000","last_change":"2024-09-17T05:10:09.607675+0000","last_active":"2024-09-17T05:10:10.709685+0000","last_peered":"2024-09-17T05:10:10.709685+0000","last_clean":"2024-09-17T05:10:10.709685+0000","last_became_active":"2024-09-17T05:10:09.606067+0000","last_became_peered":"2024-09-17T05:10:09.606067+0000","last_unstale":"2024-09-17T05:10:10.709685+0000","last_undegraded":"2024-09-17T05:10:10.709685+0000","last_fullsized":"2024-09-17T05:10:10.709685+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:41.010567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632355+0000","last_change":"2024-09-17T05:10:09.610055+0000","last_active":"2024-09-17T05:10:10.632355+0000","last_peered":"2024-09-17T05:10:10.632355+0000","last_clean":"2024-09-17T05:10:10.632355+0000","last_became_active":"2024-09-17T05:10:09.609130+0000","last_became_peered":"2024-09-17T05:10:09.609130+0000","last_unstale":"2024-09-17T05:10:10.632355+0000","last_undegraded":"2024-09-17T05:10:10.632355+0000","last_fullsized":"2024-09-17T05:10:10.632355+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:07.302807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710337+0000","last_change":"2024-09-17T05:10:09.628366+0000","last_active":"2024-09-17T05:10:10.710337+0000","last_peered":"2024-09-17T05:10:10.710337+0000","last_clean":"2024-09-17T05:10:10.710337+0000","last_became_active":"2024-09-17T05:10:09.628033+0000","last_became_peered":"2024-09-17T05:10:09.628033+0000","last_unstale":"2024-09-17T05:10:10.710337+0000","last_undegraded":"2024-09-17T05:10:10.710337+0000","last_fullsized":"2024-09-17T05:10:10.710337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:19.397104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594500+0000","last_change":"2024-09-17T05:10:09.606716+0000","last_active":"2024-09-17T05:10:10.594500+0000","last_peered":"2024-09-17T05:10:10.594500+0000","last_clean":"2024-09-17T05:10:10.594500+0000","last_became_active":"2024-09-17T05:10:09.606349+0000","last_became_peered":"2024-09-17T05:10:09.606349+0000","last_unstale":"2024-09-17T05:10:10.594500+0000","last_undegraded":"2024-09-17T05:10:10.594500+0000","last_fullsized":"2024-09-17T05:10:10.594500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:54.333652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633273+0000","last_change":"2024-09-17T05:10:09.623285+0000","last_active":"2024-09-17T05:10:10.633273+0000","last_peered":"2024-09-17T05:10:10.633273+0000","last_clean":"2024-09-17T05:10:10.633273+0000","last_became_active":"2024-09-17T05:10:09.622926+0000","last_became_peered":"2024-09-17T05:10:09.622926+0000","last_unstale":"2024-09-17T05:10:10.633273+0000","last_undegraded":"2024-09-17T05:10:10.633273+0000","last_fullsized":"2024-09-17T05:10:10.633273+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:40.746210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709869+0000","last_change":"2024-09-17T05:10:09.627847+0000","last_active":"2024-09-17T05:10:10.709869+0000","last_peered":"2024-09-17T05:10:10.709869+0000","last_clean":"2024-09-17T05:10:10.709869+0000","last_became_active":"2024-09-17T05:10:09.627447+0000","last_became_peered":"2024-09-17T05:10:09.627447+0000","last_unstale":"2024-09-17T05:10:10.709869+0000","last_undegraded":"2024-09-17T05:10:10.709869+0000","last_fullsized":"2024-09-17T05:10:10.709869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:35.272095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594850+0000","last_change":"2024-09-17T05:10:09.620329+0000","last_active":"2024-09-17T05:10:10.594850+0000","last_peered":"2024-09-17T05:10:10.594850+0000","last_clean":"2024-09-17T05:10:10.594850+0000","last_became_active":"2024-09-17T05:10:09.619753+0000","last_became_peered":"2024-09-17T05:10:09.619753+0000","last_unstale":"2024-09-17T05:10:10.594850+0000","last_undegraded":"2024-09-17T05:10:10.594850+0000","last_fullsized":"2024-09-17T05:10:10.594850+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:59.023497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637059+0000","last_change":"2024-09-17T05:10:09.620194+0000","last_active":"2024-09-17T05:10:10.637059+0000","last_peered":"2024-09-17T05:10:10.637059+0000","last_clean":"2024-09-17T05:10:10.637059+0000","last_became_active":"2024-09-17T05:10:09.616836+0000","last_became_peered":"2024-09-17T05:10:09.616836+0000","last_unstale":"2024-09-17T05:10:10.637059+0000","last_undegraded":"2024-09-17T05:10:10.637059+0000","last_fullsized":"2024-09-17T05:10:10.637059+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:02.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711502+0000","last_change":"2024-09-17T05:10:09.614433+0000","last_active":"2024-09-17T05:10:10.711502+0000","last_peered":"2024-09-17T05:10:10.711502+0000","last_clean":"2024-09-17T05:10:10.711502+0000","last_became_active":"2024-09-17T05:10:09.612322+0000","last_became_peered":"2024-09-17T05:10:09.612322+0000","last_unstale":"2024-09-17T05:10:10.711502+0000","last_undegraded":"2024-09-17T05:10:10.711502+0000","last_fullsized":"2024-09-17T05:10:10.711502+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:06.224706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632397+0000","last_change":"2024-09-17T05:10:09.614277+0000","last_active":"2024-09-17T05:10:10.632397+0000","last_peered":"2024-09-17T05:10:10.632397+0000","last_clean":"2024-09-17T05:10:10.632397+0000","last_became_active":"2024-09-17T05:10:09.613559+0000","last_became_peered":"2024-09-17T05:10:09.613559+0000","last_unstale":"2024-09-17T05:10:10.632397+0000","last_undegraded":"2024-09-17T05:10:10.632397+0000","last_fullsized":"2024-09-17T05:10:10.632397+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:30.391166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632531+0000","last_change":"2024-09-17T05:10:09.613913+0000","last_active":"2024-09-17T05:10:10.632531+0000","last_peered":"2024-09-17T05:10:10.632531+0000","last_clean":"2024-09-17T05:10:10.632531+0000","last_became_active":"2024-09-17T05:10:09.613177+0000","last_became_peered":"2024-09-17T05:10:09.613177+0000","last_unstale":"2024-09-17T05:10:10.632531+0000","last_undegraded":"2024-09-17T05:10:10.632531+0000","last_fullsized":"2024-09-17T05:10:10.632531+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:13.998035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"148'4","reported_seq":19,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632956+0000","last_change":"2024-09-17T05:10:08.616110+0000","last_active":"2024-09-17T05:10:10.632956+0000","last_peered":"2024-09-17T05:10:10.632956+0000","last_clean":"2024-09-17T05:10:10.632956+0000","last_became_active":"2024-09-17T05:10:08.615796+0000","last_became_peered":"2024-09-17T05:10:08.615796+0000","last_unstale":"2024-09-17T05:10:10.632956+0000","last_undegraded":"2024-09-17T05:10:10.632956+0000","last_fullsized":"2024-09-17T05:10:10.632956+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:01.257841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709727+0000","last_change":"2024-09-17T05:10:09.619128+0000","last_active":"2024-09-17T05:10:10.709727+0000","last_peered":"2024-09-17T05:10:10.709727+0000","last_clean":"2024-09-17T05:10:10.709727+0000","last_became_active":"2024-09-17T05:10:09.618807+0000","last_became_peered":"2024-09-17T05:10:09.618807+0000","last_unstale":"2024-09-17T05:10:10.709727+0000","last_undegraded":"2024-09-17T05:10:10.709727+0000","last_fullsized":"2024-09-17T05:10:10.709727+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:07.178601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595215+0000","last_change":"2024-09-17T05:10:08.622137+0000","last_active":"2024-09-17T05:10:10.595215+0000","last_peered":"2024-09-17T05:10:10.595215+0000","last_clean":"2024-09-17T05:10:10.595215+0000","last_became_active":"2024-09-17T05:10:08.618489+0000","last_became_peered":"2024-09-17T05:10:08.618489+0000","last_unstale":"2024-09-17T05:10:10.595215+0000","last_undegraded":"2024-09-17T05:10:10.595215+0000","last_fullsized":"2024-09-17T05:10:10.595215+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:12.995112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710714+0000","last_change":"2024-09-17T05:10:09.620963+0000","last_active":"2024-09-17T05:10:10.710714+0000","last_peered":"2024-09-17T05:10:10.710714+0000","last_clean":"2024-09-17T05:10:10.710714+0000","last_became_active":"2024-09-17T05:10:09.620673+0000","last_became_peered":"2024-09-17T05:10:09.620673+0000","last_unstale":"2024-09-17T05:10:10.710714+0000","last_undegraded":"2024-09-17T05:10:10.710714+0000","last_fullsized":"2024-09-17T05:10:10.710714+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:26.454657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711806+0000","last_change":"2024-09-17T05:10:08.603988+0000","last_active":"2024-09-17T05:10:10.711806+0000","last_peered":"2024-09-17T05:10:10.711806+0000","last_clean":"2024-09-17T05:10:10.711806+0000","last_became_active":"2024-09-17T05:10:08.603638+0000","last_became_peered":"2024-09-17T05:10:08.603638+0000","last_unstale":"2024-09-17T05:10:10.711806+0000","last_undegraded":"2024-09-17T05:10:10.711806+0000","last_fullsized":"2024-09-17T05:10:10.711806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:34.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711765+0000","last_change":"2024-09-17T05:10:09.610728+0000","last_active":"2024-09-17T05:10:10.711765+0000","last_peered":"2024-09-17T05:10:10.711765+0000","last_clean":"2024-09-17T05:10:10.711765+0000","last_became_active":"2024-09-17T05:10:09.610362+0000","last_became_peered":"2024-09-17T05:10:09.610362+0000","last_unstale":"2024-09-17T05:10:10.711765+0000","last_undegraded":"2024-09-17T05:10:10.711765+0000","last_fullsized":"2024-09-17T05:10:10.711765+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:07.635433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710334+0000","last_change":"2024-09-17T05:10:08.614774+0000","last_active":"2024-09-17T05:10:10.710334+0000","last_peered":"2024-09-17T05:10:10.710334+0000","last_clean":"2024-09-17T05:10:10.710334+0000","last_became_active":"2024-09-17T05:10:08.614056+0000","last_became_peered":"2024-09-17T05:10:08.614056+0000","last_unstale":"2024-09-17T05:10:10.710334+0000","last_undegraded":"2024-09-17T05:10:10.710334+0000","last_fullsized":"2024-09-17T05:10:10.710334+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:13:10.358573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709950+0000","last_change":"2024-09-17T05:10:09.620716+0000","last_active":"2024-09-17T05:10:10.709950+0000","last_peered":"2024-09-17T05:10:10.709950+0000","last_clean":"2024-09-17T05:10:10.709950+0000","last_became_active":"2024-09-17T05:10:09.620373+0000","last_became_peered":"2024-09-17T05:10:09.620373+0000","last_unstale":"2024-09-17T05:10:10.709950+0000","last_undegraded":"2024-09-17T05:10:10.709950+0000","last_fullsized":"2024-09-17T05:10:10.709950+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:06.224706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711112+0000","last_change":"2024-09-17T05:10:08.622709+0000","last_active":"2024-09-17T05:10:10.711112+0000","last_peered":"2024-09-17T05:10:10.711112+0000","last_clean":"2024-09-17T05:10:10.711112+0000","last_became_active":"2024-09-17T05:10:08.622274+0000","last_became_peered":"2024-09-17T05:10:08.622274+0000","last_unstale":"2024-09-17T05:10:10.711112+0000","last_undegraded":"2024-09-17T05:10:10.711112+0000","last_fullsized":"2024-09-17T05:10:10.711112+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:29.690037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710324+0000","last_change":"2024-09-17T05:10:09.632375+0000","last_active":"2024-09-17T05:10:10.710324+0000","last_peered":"2024-09-17T05:10:10.710324+0000","last_clean":"2024-09-17T05:10:10.710324+0000","last_became_active":"2024-09-17T05:10:09.630863+0000","last_became_peered":"2024-09-17T05:10:09.630863+0000","last_unstale":"2024-09-17T05:10:10.710324+0000","last_undegraded":"2024-09-17T05:10:10.710324+0000","last_fullsized":"2024-09-17T05:10:10.710324+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:26.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594928+0000","last_change":"2024-09-17T05:10:08.618499+0000","last_active":"2024-09-17T05:10:10.594928+0000","last_peered":"2024-09-17T05:10:10.594928+0000","last_clean":"2024-09-17T05:10:10.594928+0000","last_became_active":"2024-09-17T05:10:08.617835+0000","last_became_peered":"2024-09-17T05:10:08.617835+0000","last_unstale":"2024-09-17T05:10:10.594928+0000","last_undegraded":"2024-09-17T05:10:10.594928+0000","last_fullsized":"2024-09-17T05:10:10.594928+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:34.802748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710782+0000","last_change":"2024-09-17T05:10:09.619523+0000","last_active":"2024-09-17T05:10:10.710782+0000","last_peered":"2024-09-17T05:10:10.710782+0000","last_clean":"2024-09-17T05:10:10.710782+0000","last_became_active":"2024-09-17T05:10:09.619178+0000","last_became_peered":"2024-09-17T05:10:09.619178+0000","last_unstale":"2024-09-17T05:10:10.710782+0000","last_undegraded":"2024-09-17T05:10:10.710782+0000","last_fullsized":"2024-09-17T05:10:10.710782+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:37.618349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594545+0000","last_change":"2024-09-17T05:10:08.620259+0000","last_active":"2024-09-17T05:10:10.594545+0000","last_peered":"2024-09-17T05:10:10.594545+0000","last_clean":"2024-09-17T05:10:10.594545+0000","last_became_active":"2024-09-17T05:10:08.619546+0000","last_became_peered":"2024-09-17T05:10:08.619546+0000","last_unstale":"2024-09-17T05:10:10.594545+0000","last_undegraded":"2024-09-17T05:10:10.594545+0000","last_fullsized":"2024-09-17T05:10:10.594545+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:00.078340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595464+0000","last_change":"2024-09-17T05:10:09.627317+0000","last_active":"2024-09-17T05:10:10.595464+0000","last_peered":"2024-09-17T05:10:10.595464+0000","last_clean":"2024-09-17T05:10:10.595464+0000","last_became_active":"2024-09-17T05:10:09.626997+0000","last_became_peered":"2024-09-17T05:10:09.626997+0000","last_unstale":"2024-09-17T05:10:10.595464+0000","last_undegraded":"2024-09-17T05:10:10.595464+0000","last_fullsized":"2024-09-17T05:10:10.595464+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:19.200502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594051+0000","last_change":"2024-09-17T05:10:08.609712+0000","last_active":"2024-09-17T05:10:10.594051+0000","last_peered":"2024-09-17T05:10:10.594051+0000","last_clean":"2024-09-17T05:10:10.594051+0000","last_became_active":"2024-09-17T05:10:08.608163+0000","last_became_peered":"2024-09-17T05:10:08.608163+0000","last_unstale":"2024-09-17T05:10:10.594051+0000","last_undegraded":"2024-09-17T05:10:10.594051+0000","last_fullsized":"2024-09-17T05:10:10.594051+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:02.449152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632598+0000","last_change":"2024-09-17T05:10:09.605344+0000","last_active":"2024-09-17T05:10:10.632598+0000","last_peered":"2024-09-17T05:10:10.632598+0000","last_clean":"2024-09-17T05:10:10.632598+0000","last_became_active":"2024-09-17T05:10:09.605036+0000","last_became_peered":"2024-09-17T05:10:09.605036+0000","last_unstale":"2024-09-17T05:10:10.632598+0000","last_undegraded":"2024-09-17T05:10:10.632598+0000","last_fullsized":"2024-09-17T05:10:10.632598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:35.805671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711153+0000","last_change":"2024-09-17T05:10:08.615422+0000","last_active":"2024-09-17T05:10:10.711153+0000","last_peered":"2024-09-17T05:10:10.711153+0000","last_clean":"2024-09-17T05:10:10.711153+0000","last_became_active":"2024-09-17T05:10:08.611354+0000","last_became_peered":"2024-09-17T05:10:08.611354+0000","last_unstale":"2024-09-17T05:10:10.711153+0000","last_undegraded":"2024-09-17T05:10:10.711153+0000","last_fullsized":"2024-09-17T05:10:10.711153+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:27.066794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710034+0000","last_change":"2024-09-17T05:10:09.630016+0000","last_active":"2024-09-17T05:10:10.710034+0000","last_peered":"2024-09-17T05:10:10.710034+0000","last_clean":"2024-09-17T05:10:10.710034+0000","last_became_active":"2024-09-17T05:10:09.629590+0000","last_became_peered":"2024-09-17T05:10:09.629590+0000","last_unstale":"2024-09-17T05:10:10.710034+0000","last_undegraded":"2024-09-17T05:10:10.710034+0000","last_fullsized":"2024-09-17T05:10:10.710034+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:26.897183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"148'3","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633132+0000","last_change":"2024-09-17T05:10:08.608985+0000","last_active":"2024-09-17T05:10:10.633132+0000","last_peered":"2024-09-17T05:10:10.633132+0000","last_clean":"2024-09-17T05:10:10.633132+0000","last_became_active":"2024-09-17T05:10:08.608684+0000","last_became_peered":"2024-09-17T05:10:08.608684+0000","last_unstale":"2024-09-17T05:10:10.633132+0000","last_undegraded":"2024-09-17T05:10:10.633132+0000","last_fullsized":"2024-09-17T05:10:10.633132+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:58.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710981+0000","last_change":"2024-09-17T05:10:09.619258+0000","last_active":"2024-09-17T05:10:10.710981+0000","last_peered":"2024-09-17T05:10:10.710981+0000","last_clean":"2024-09-17T05:10:10.710981+0000","last_became_active":"2024-09-17T05:10:09.618922+0000","last_became_peered":"2024-09-17T05:10:09.618922+0000","last_unstale":"2024-09-17T05:10:10.710981+0000","last_undegraded":"2024-09-17T05:10:10.710981+0000","last_fullsized":"2024-09-17T05:10:10.710981+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:10.939262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"148'6","reported_seq":21,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710127+0000","last_change":"2024-09-17T05:10:08.612149+0000","last_active":"2024-09-17T05:10:10.710127+0000","last_peered":"2024-09-17T05:10:10.710127+0000","last_clean":"2024-09-17T05:10:10.710127+0000","last_became_active":"2024-09-17T05:10:08.611792+0000","last_became_peered":"2024-09-17T05:10:08.611792+0000","last_unstale":"2024-09-17T05:10:10.710127+0000","last_undegraded":"2024-09-17T05:10:10.710127+0000","last_fullsized":"2024-09-17T05:10:10.710127+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:43.153997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.712107+0000","last_change":"2024-09-17T05:10:09.628351+0000","last_active":"2024-09-17T05:10:10.712107+0000","last_peered":"2024-09-17T05:10:10.712107+0000","last_clean":"2024-09-17T05:10:10.712107+0000","last_became_active":"2024-09-17T05:10:09.627984+0000","last_became_peered":"2024-09-17T05:10:09.627984+0000","last_unstale":"2024-09-17T05:10:10.712107+0000","last_undegraded":"2024-09-17T05:10:10.712107+0000","last_fullsized":"2024-09-17T05:10:10.712107+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:59.575299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594210+0000","last_change":"2024-09-17T05:10:08.612025+0000","last_active":"2024-09-17T05:10:10.594210+0000","last_peered":"2024-09-17T05:10:10.594210+0000","last_clean":"2024-09-17T05:10:10.594210+0000","last_became_active":"2024-09-17T05:10:08.607799+0000","last_became_peered":"2024-09-17T05:10:08.607799+0000","last_unstale":"2024-09-17T05:10:10.594210+0000","last_undegraded":"2024-09-17T05:10:10.594210+0000","last_fullsized":"2024-09-17T05:10:10.594210+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:19.401746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709837+0000","last_change":"2024-09-17T05:10:09.628294+0000","last_active":"2024-09-17T05:10:10.709837+0000","last_peered":"2024-09-17T05:10:10.709837+0000","last_clean":"2024-09-17T05:10:10.709837+0000","last_became_active":"2024-09-17T05:10:09.627713+0000","last_became_peered":"2024-09-17T05:10:09.627713+0000","last_unstale":"2024-09-17T05:10:10.709837+0000","last_undegraded":"2024-09-17T05:10:10.709837+0000","last_fullsized":"2024-09-17T05:10:10.709837+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:10.136192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710571+0000","last_change":"2024-09-17T05:10:08.610638+0000","last_active":"2024-09-17T05:10:10.710571+0000","last_peered":"2024-09-17T05:10:10.710571+0000","last_clean":"2024-09-17T05:10:10.710571+0000","last_became_active":"2024-09-17T05:10:08.610299+0000","last_became_peered":"2024-09-17T05:10:08.610299+0000","last_unstale":"2024-09-17T05:10:10.710571+0000","last_undegraded":"2024-09-17T05:10:10.710571+0000","last_fullsized":"2024-09-17T05:10:10.710571+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:07.190290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.712048+0000","last_change":"2024-09-17T05:10:09.623421+0000","last_active":"2024-09-17T05:10:10.712048+0000","last_peered":"2024-09-17T05:10:10.712048+0000","last_clean":"2024-09-17T05:10:10.712048+0000","last_became_active":"2024-09-17T05:10:09.623074+0000","last_became_peered":"2024-09-17T05:10:09.623074+0000","last_unstale":"2024-09-17T05:10:10.712048+0000","last_undegraded":"2024-09-17T05:10:10.712048+0000","last_fullsized":"2024-09-17T05:10:10.712048+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:19.793741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711569+0000","last_change":"2024-09-17T05:10:08.614128+0000","last_active":"2024-09-17T05:10:10.711569+0000","last_peered":"2024-09-17T05:10:10.711569+0000","last_clean":"2024-09-17T05:10:10.711569+0000","last_became_active":"2024-09-17T05:10:08.613663+0000","last_became_peered":"2024-09-17T05:10:08.613663+0000","last_unstale":"2024-09-17T05:10:10.711569+0000","last_undegraded":"2024-09-17T05:10:10.711569+0000","last_fullsized":"2024-09-17T05:10:10.711569+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:02.029837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594329+0000","last_change":"2024-09-17T05:10:09.619514+0000","last_active":"2024-09-17T05:10:10.594329+0000","last_peered":"2024-09-17T05:10:10.594329+0000","last_clean":"2024-09-17T05:10:10.594329+0000","last_became_active":"2024-09-17T05:10:09.619191+0000","last_became_peered":"2024-09-17T05:10:09.619191+0000","last_unstale":"2024-09-17T05:10:10.594329+0000","last_undegraded":"2024-09-17T05:10:10.594329+0000","last_fullsized":"2024-09-17T05:10:10.594329+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.054335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"148'8","reported_seq":23,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709489+0000","last_change":"2024-09-17T05:10:08.614136+0000","last_active":"2024-09-17T05:10:10.709489+0000","last_peered":"2024-09-17T05:10:10.709489+0000","last_clean":"2024-09-17T05:10:10.709489+0000","last_became_active":"2024-09-17T05:10:08.613818+0000","last_became_peered":"2024-09-17T05:10:08.613818+0000","last_unstale":"2024-09-17T05:10:10.709489+0000","last_undegraded":"2024-09-17T05:10:10.709489+0000","last_fullsized":"2024-09-17T05:10:10.709489+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:21.707300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.593973+0000","last_change":"2024-09-17T05:10:09.604347+0000","last_active":"2024-09-17T05:10:10.593973+0000","last_peered":"2024-09-17T05:10:10.593973+0000","last_clean":"2024-09-17T05:10:10.593973+0000","last_became_active":"2024-09-17T05:10:09.603715+0000","last_became_peered":"2024-09-17T05:10:09.603715+0000","last_unstale":"2024-09-17T05:10:10.593973+0000","last_undegraded":"2024-09-17T05:10:10.593973+0000","last_fullsized":"2024-09-17T05:10:10.593973+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.148642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594976+0000","last_change":"2024-09-17T05:10:08.618196+0000","last_active":"2024-09-17T05:10:10.594976+0000","last_peered":"2024-09-17T05:10:10.594976+0000","last_clean":"2024-09-17T05:10:10.594976+0000","last_became_active":"2024-09-17T05:10:08.617683+0000","last_became_peered":"2024-09-17T05:10:08.617683+0000","last_unstale":"2024-09-17T05:10:10.594976+0000","last_undegraded":"2024-09-17T05:10:10.594976+0000","last_fullsized":"2024-09-17T05:10:10.594976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:29.388243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636873+0000","last_change":"2024-09-17T05:10:09.626164+0000","last_active":"2024-09-17T05:10:10.636873+0000","last_peered":"2024-09-17T05:10:10.636873+0000","last_clean":"2024-09-17T05:10:10.636873+0000","last_became_active":"2024-09-17T05:10:09.621575+0000","last_became_peered":"2024-09-17T05:10:09.621575+0000","last_unstale":"2024-09-17T05:10:10.636873+0000","last_undegraded":"2024-09-17T05:10:10.636873+0000","last_fullsized":"2024-09-17T05:10:10.636873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:10.939262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710878+0000","last_change":"2024-09-17T05:10:08.613116+0000","last_active":"2024-09-17T05:10:10.710878+0000","last_peered":"2024-09-17T05:10:10.710878+0000","last_clean":"2024-09-17T05:10:10.710878+0000","last_became_active":"2024-09-17T05:10:08.612754+0000","last_became_peered":"2024-09-17T05:10:08.612754+0000","last_unstale":"2024-09-17T05:10:10.710878+0000","last_undegraded":"2024-09-17T05:10:10.710878+0000","last_fullsized":"2024-09-17T05:10:10.710878+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:26.869035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637357+0000","last_change":"2024-09-17T05:10:09.628661+0000","last_active":"2024-09-17T05:10:10.637357+0000","last_peered":"2024-09-17T05:10:10.637357+0000","last_clean":"2024-09-17T05:10:10.637357+0000","last_became_active":"2024-09-17T05:10:09.620876+0000","last_became_peered":"2024-09-17T05:10:09.620876+0000","last_unstale":"2024-09-17T05:10:10.637357+0000","last_undegraded":"2024-09-17T05:10:10.637357+0000","last_fullsized":"2024-09-17T05:10:10.637357+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:09.358197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594643+0000","last_change":"2024-09-17T05:10:08.613140+0000","last_active":"2024-09-17T05:10:10.594643+0000","last_peered":"2024-09-17T05:10:10.594643+0000","last_clean":"2024-09-17T05:10:10.594643+0000","last_became_active":"2024-09-17T05:10:08.612717+0000","last_became_peered":"2024-09-17T05:10:08.612717+0000","last_unstale":"2024-09-17T05:10:10.594643+0000","last_undegraded":"2024-09-17T05:10:10.594643+0000","last_fullsized":"2024-09-17T05:10:10.594643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:52.476600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636902+0000","last_change":"2024-09-17T05:10:09.619313+0000","last_active":"2024-09-17T05:10:10.636902+0000","last_peered":"2024-09-17T05:10:10.636902+0000","last_clean":"2024-09-17T05:10:10.636902+0000","last_became_active":"2024-09-17T05:10:09.618962+0000","last_became_peered":"2024-09-17T05:10:09.618962+0000","last_unstale":"2024-09-17T05:10:10.636902+0000","last_undegraded":"2024-09-17T05:10:10.636902+0000","last_fullsized":"2024-09-17T05:10:10.636902+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:41.452495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595164+0000","last_change":"2024-09-17T05:10:08.618485+0000","last_active":"2024-09-17T05:10:10.595164+0000","last_peered":"2024-09-17T05:10:10.595164+0000","last_clean":"2024-09-17T05:10:10.595164+0000","last_became_active":"2024-09-17T05:10:08.618140+0000","last_became_peered":"2024-09-17T05:10:08.618140+0000","last_unstale":"2024-09-17T05:10:10.595164+0000","last_undegraded":"2024-09-17T05:10:10.595164+0000","last_fullsized":"2024-09-17T05:10:10.595164+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:04.727144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710243+0000","last_change":"2024-09-17T05:10:09.625248+0000","last_active":"2024-09-17T05:10:10.710243+0000","last_peered":"2024-09-17T05:10:10.710243+0000","last_clean":"2024-09-17T05:10:10.710243+0000","last_became_active":"2024-09-17T05:10:09.624768+0000","last_became_peered":"2024-09-17T05:10:09.624768+0000","last_unstale":"2024-09-17T05:10:10.710243+0000","last_undegraded":"2024-09-17T05:10:10.710243+0000","last_fullsized":"2024-09-17T05:10:10.710243+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:07.635433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594189+0000","last_change":"2024-09-17T05:10:08.606682+0000","last_active":"2024-09-17T05:10:10.594189+0000","last_peered":"2024-09-17T05:10:10.594189+0000","last_clean":"2024-09-17T05:10:10.594189+0000","last_became_active":"2024-09-17T05:10:08.605794+0000","last_became_peered":"2024-09-17T05:10:08.605794+0000","last_unstale":"2024-09-17T05:10:10.594189+0000","last_undegraded":"2024-09-17T05:10:10.594189+0000","last_fullsized":"2024-09-17T05:10:10.594189+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:35.608634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594386+0000","last_change":"2024-09-17T05:10:09.608057+0000","last_active":"2024-09-17T05:10:10.594386+0000","last_peered":"2024-09-17T05:10:10.594386+0000","last_clean":"2024-09-17T05:10:10.594386+0000","last_became_active":"2024-09-17T05:10:09.607672+0000","last_became_peered":"2024-09-17T05:10:09.607672+0000","last_unstale":"2024-09-17T05:10:10.594386+0000","last_undegraded":"2024-09-17T05:10:10.594386+0000","last_fullsized":"2024-09-17T05:10:10.594386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:37.614737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632433+0000","last_change":"2024-09-17T05:10:08.621986+0000","last_active":"2024-09-17T05:10:10.632433+0000","last_peered":"2024-09-17T05:10:10.632433+0000","last_clean":"2024-09-17T05:10:10.632433+0000","last_became_active":"2024-09-17T05:10:08.621660+0000","last_became_peered":"2024-09-17T05:10:08.621660+0000","last_unstale":"2024-09-17T05:10:10.632433+0000","last_undegraded":"2024-09-17T05:10:10.632433+0000","last_fullsized":"2024-09-17T05:10:10.632433+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:45.331253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594559+0000","last_change":"2024-09-17T05:10:09.613692+0000","last_active":"2024-09-17T05:10:10.594559+0000","last_peered":"2024-09-17T05:10:10.594559+0000","last_clean":"2024-09-17T05:10:10.594559+0000","last_became_active":"2024-09-17T05:10:09.613348+0000","last_became_peered":"2024-09-17T05:10:09.613348+0000","last_unstale":"2024-09-17T05:10:10.594559+0000","last_undegraded":"2024-09-17T05:10:10.594559+0000","last_fullsized":"2024-09-17T05:10:10.594559+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:51.538160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709987+0000","last_change":"2024-09-17T05:10:08.609449+0000","last_active":"2024-09-17T05:10:10.709987+0000","last_peered":"2024-09-17T05:10:10.709987+0000","last_clean":"2024-09-17T05:10:10.709987+0000","last_became_active":"2024-09-17T05:10:08.609121+0000","last_became_peered":"2024-09-17T05:10:08.609121+0000","last_unstale":"2024-09-17T05:10:10.709987+0000","last_undegraded":"2024-09-17T05:10:10.709987+0000","last_fullsized":"2024-09-17T05:10:10.709987+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:07.620444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595493+0000","last_change":"2024-09-17T05:10:09.618340+0000","last_active":"2024-09-17T05:10:10.595493+0000","last_peered":"2024-09-17T05:10:10.595493+0000","last_clean":"2024-09-17T05:10:10.595493+0000","last_became_active":"2024-09-17T05:10:09.617963+0000","last_became_peered":"2024-09-17T05:10:09.617963+0000","last_unstale":"2024-09-17T05:10:10.595493+0000","last_undegraded":"2024-09-17T05:10:10.595493+0000","last_fullsized":"2024-09-17T05:10:10.595493+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:38.437182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632569+0000","last_change":"2024-09-17T05:10:08.612988+0000","last_active":"2024-09-17T05:10:10.632569+0000","last_peered":"2024-09-17T05:10:10.632569+0000","last_clean":"2024-09-17T05:10:10.632569+0000","last_became_active":"2024-09-17T05:10:08.612476+0000","last_became_peered":"2024-09-17T05:10:08.612476+0000","last_unstale":"2024-09-17T05:10:10.632569+0000","last_undegraded":"2024-09-17T05:10:10.632569+0000","last_fullsized":"2024-09-17T05:10:10.632569+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:04.272837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594799+0000","last_change":"2024-09-17T05:10:09.627603+0000","last_active":"2024-09-17T05:10:10.594799+0000","last_peered":"2024-09-17T05:10:10.594799+0000","last_clean":"2024-09-17T05:10:10.594799+0000","last_became_active":"2024-09-17T05:10:09.623999+0000","last_became_peered":"2024-09-17T05:10:09.623999+0000","last_unstale":"2024-09-17T05:10:10.594799+0000","last_undegraded":"2024-09-17T05:10:10.594799+0000","last_fullsized":"2024-09-17T05:10:10.594799+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:17.355866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595043+0000","last_change":"2024-09-17T05:10:08.615722+0000","last_active":"2024-09-17T05:10:10.595043+0000","last_peered":"2024-09-17T05:10:10.595043+0000","last_clean":"2024-09-17T05:10:10.595043+0000","last_became_active":"2024-09-17T05:10:08.613532+0000","last_became_peered":"2024-09-17T05:10:08.613532+0000","last_unstale":"2024-09-17T05:10:10.595043+0000","last_undegraded":"2024-09-17T05:10:10.595043+0000","last_fullsized":"2024-09-17T05:10:10.595043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:04.461498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594326+0000","last_change":"2024-09-17T05:10:09.605392+0000","last_active":"2024-09-17T05:10:10.594326+0000","last_peered":"2024-09-17T05:10:10.594326+0000","last_clean":"2024-09-17T05:10:10.594326+0000","last_became_active":"2024-09-17T05:10:09.605065+0000","last_became_peered":"2024-09-17T05:10:09.605065+0000","last_unstale":"2024-09-17T05:10:10.594326+0000","last_undegraded":"2024-09-17T05:10:10.594326+0000","last_fullsized":"2024-09-17T05:10:10.594326+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:43.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594644+0000","last_change":"2024-09-17T05:10:08.610723+0000","last_active":"2024-09-17T05:10:10.594644+0000","last_peered":"2024-09-17T05:10:10.594644+0000","last_clean":"2024-09-17T05:10:10.594644+0000","last_became_active":"2024-09-17T05:10:08.608307+0000","last_became_peered":"2024-09-17T05:10:08.608307+0000","last_unstale":"2024-09-17T05:10:10.594644+0000","last_undegraded":"2024-09-17T05:10:10.594644+0000","last_fullsized":"2024-09-17T05:10:10.594644+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:35.327369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595454+0000","last_change":"2024-09-17T05:10:09.604923+0000","last_active":"2024-09-17T05:10:10.595454+0000","last_peered":"2024-09-17T05:10:10.595454+0000","last_clean":"2024-09-17T05:10:10.595454+0000","last_became_active":"2024-09-17T05:10:09.604549+0000","last_became_peered":"2024-09-17T05:10:09.604549+0000","last_unstale":"2024-09-17T05:10:10.595454+0000","last_undegraded":"2024-09-17T05:10:10.595454+0000","last_fullsized":"2024-09-17T05:10:10.595454+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:16.357047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594868+0000","last_change":"2024-09-17T05:10:08.614232+0000","last_active":"2024-09-17T05:10:10.594868+0000","last_peered":"2024-09-17T05:10:10.594868+0000","last_clean":"2024-09-17T05:10:10.594868+0000","last_became_active":"2024-09-17T05:10:08.613385+0000","last_became_peered":"2024-09-17T05:10:08.613385+0000","last_unstale":"2024-09-17T05:10:10.594868+0000","last_undegraded":"2024-09-17T05:10:10.594868+0000","last_fullsized":"2024-09-17T05:10:10.594868+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:40.573069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711038+0000","last_change":"2024-09-17T05:10:09.626873+0000","last_active":"2024-09-17T05:10:10.711038+0000","last_peered":"2024-09-17T05:10:10.711038+0000","last_clean":"2024-09-17T05:10:10.711038+0000","last_became_active":"2024-09-17T05:10:09.626250+0000","last_became_peered":"2024-09-17T05:10:09.626250+0000","last_unstale":"2024-09-17T05:10:10.711038+0000","last_undegraded":"2024-09-17T05:10:10.711038+0000","last_fullsized":"2024-09-17T05:10:10.711038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:41.452495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595470+0000","last_change":"2024-09-17T05:10:08.614875+0000","last_active":"2024-09-17T05:10:10.595470+0000","last_peered":"2024-09-17T05:10:10.595470+0000","last_clean":"2024-09-17T05:10:10.595470+0000","last_became_active":"2024-09-17T05:10:08.614532+0000","last_became_peered":"2024-09-17T05:10:08.614532+0000","last_unstale":"2024-09-17T05:10:10.595470+0000","last_undegraded":"2024-09-17T05:10:10.595470+0000","last_fullsized":"2024-09-17T05:10:10.595470+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:06.299885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595361+0000","last_change":"2024-09-17T05:10:09.620911+0000","last_active":"2024-09-17T05:10:10.595361+0000","last_peered":"2024-09-17T05:10:10.595361+0000","last_clean":"2024-09-17T05:10:10.595361+0000","last_became_active":"2024-09-17T05:10:09.620435+0000","last_became_peered":"2024-09-17T05:10:09.620435+0000","last_unstale":"2024-09-17T05:10:10.595361+0000","last_undegraded":"2024-09-17T05:10:10.595361+0000","last_fullsized":"2024-09-17T05:10:10.595361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.148642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632465+0000","last_change":"2024-09-17T05:10:08.615803+0000","last_active":"2024-09-17T05:10:10.632465+0000","last_peered":"2024-09-17T05:10:10.632465+0000","last_clean":"2024-09-17T05:10:10.632465+0000","last_became_active":"2024-09-17T05:10:08.614659+0000","last_became_peered":"2024-09-17T05:10:08.614659+0000","last_unstale":"2024-09-17T05:10:10.632465+0000","last_undegraded":"2024-09-17T05:10:10.632465+0000","last_fullsized":"2024-09-17T05:10:10.632465+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:50.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595380+0000","last_change":"2024-09-17T05:10:09.632015+0000","last_active":"2024-09-17T05:10:10.595380+0000","last_peered":"2024-09-17T05:10:10.595380+0000","last_clean":"2024-09-17T05:10:10.595380+0000","last_became_active":"2024-09-17T05:10:09.631637+0000","last_became_peered":"2024-09-17T05:10:09.631637+0000","last_unstale":"2024-09-17T05:10:10.595380+0000","last_undegraded":"2024-09-17T05:10:10.595380+0000","last_fullsized":"2024-09-17T05:10:10.595380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:43.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710010+0000","last_change":"2024-09-17T05:10:08.620144+0000","last_active":"2024-09-17T05:10:10.710010+0000","last_peered":"2024-09-17T05:10:10.710010+0000","last_clean":"2024-09-17T05:10:10.710010+0000","last_became_active":"2024-09-17T05:10:08.619808+0000","last_became_peered":"2024-09-17T05:10:08.619808+0000","last_unstale":"2024-09-17T05:10:10.710010+0000","last_undegraded":"2024-09-17T05:10:10.710010+0000","last_fullsized":"2024-09-17T05:10:10.710010+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:23.628949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595476+0000","last_change":"2024-09-17T05:10:09.621104+0000","last_active":"2024-09-17T05:10:10.595476+0000","last_peered":"2024-09-17T05:10:10.595476+0000","last_clean":"2024-09-17T05:10:10.595476+0000","last_became_active":"2024-09-17T05:10:09.620786+0000","last_became_peered":"2024-09-17T05:10:09.620786+0000","last_unstale":"2024-09-17T05:10:10.595476+0000","last_undegraded":"2024-09-17T05:10:10.595476+0000","last_fullsized":"2024-09-17T05:10:10.595476+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:19.200502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632310+0000","last_change":"2024-09-17T05:10:08.608149+0000","last_active":"2024-09-17T05:10:10.632310+0000","last_peered":"2024-09-17T05:10:10.632310+0000","last_clean":"2024-09-17T05:10:10.632310+0000","last_became_active":"2024-09-17T05:10:08.607474+0000","last_became_peered":"2024-09-17T05:10:08.607474+0000","last_unstale":"2024-09-17T05:10:10.632310+0000","last_undegraded":"2024-09-17T05:10:10.632310+0000","last_fullsized":"2024-09-17T05:10:10.632310+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:15.069071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710693+0000","last_change":"2024-09-17T05:10:09.633606+0000","last_active":"2024-09-17T05:10:10.710693+0000","last_peered":"2024-09-17T05:10:10.710693+0000","last_clean":"2024-09-17T05:10:10.710693+0000","last_became_active":"2024-09-17T05:10:09.633172+0000","last_became_peered":"2024-09-17T05:10:09.633172+0000","last_unstale":"2024-09-17T05:10:10.710693+0000","last_undegraded":"2024-09-17T05:10:10.710693+0000","last_fullsized":"2024-09-17T05:10:10.710693+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:59.575299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709470+0000","last_change":"2024-09-17T05:10:08.609788+0000","last_active":"2024-09-17T05:10:10.709470+0000","last_peered":"2024-09-17T05:10:10.709470+0000","last_clean":"2024-09-17T05:10:10.709470+0000","last_became_active":"2024-09-17T05:10:08.608287+0000","last_became_peered":"2024-09-17T05:10:08.608287+0000","last_unstale":"2024-09-17T05:10:10.709470+0000","last_undegraded":"2024-09-17T05:10:10.709470+0000","last_fullsized":"2024-09-17T05:10:10.709470+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:07.766840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711712+0000","last_change":"2024-09-17T05:10:09.618662+0000","last_active":"2024-09-17T05:10:10.711712+0000","last_peered":"2024-09-17T05:10:10.711712+0000","last_clean":"2024-09-17T05:10:10.711712+0000","last_became_active":"2024-09-17T05:10:09.618300+0000","last_became_peered":"2024-09-17T05:10:09.618300+0000","last_unstale":"2024-09-17T05:10:10.711712+0000","last_undegraded":"2024-09-17T05:10:10.711712+0000","last_fullsized":"2024-09-17T05:10:10.711712+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:26.897183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710779+0000","last_change":"2024-09-17T05:10:08.621302+0000","last_active":"2024-09-17T05:10:10.710779+0000","last_peered":"2024-09-17T05:10:10.710779+0000","last_clean":"2024-09-17T05:10:10.710779+0000","last_became_active":"2024-09-17T05:10:08.616602+0000","last_became_peered":"2024-09-17T05:10:08.616602+0000","last_unstale":"2024-09-17T05:10:10.710779+0000","last_undegraded":"2024-09-17T05:10:10.710779+0000","last_fullsized":"2024-09-17T05:10:10.710779+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:09.535927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":358,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636518+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:10:10.636518+0000","last_peered":"2024-09-17T05:10:10.636518+0000","last_clean":"2024-09-17T05:10:10.636518+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:10.636518+0000","last_undegraded":"2024-09-17T05:10:10.636518+0000","last_fullsized":"2024-09-17T05:10:10.636518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1880064,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1880064,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":866927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":866939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1884160,"data_stored":867669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2465792,"data_stored":1457363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":866905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2461696,"data_stored":1451267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:13.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 -- 172.21.15.16:0/675499716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 msgr2=0x7ff74c070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 --2- 172.21.15.16:0/675499716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c070ba0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff74c074560 tx=0x7ff74401cac0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 -- 172.21.15.16:0/675499716 shutdown_connections 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 --2- 172.21.15.16:0/675499716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c070ba0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 --2- 172.21.15.16:0/675499716 >> v2:172.21.15.104:3300/0 conn(0x7ff74c0d1aa0 0x7ff74c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 -- 172.21.15.16:0/675499716 >> 172.21.15.16:0/675499716 conn(0x7ff74c0b4160 msgr2=0x7ff74c0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 -- 172.21.15.16:0/675499716 shutdown_connections 2024-09-17T05:10:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.028+0000 7ff74be00640 1 -- 172.21.15.16:0/675499716 wait complete. 2024-09-17T05:10:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 Processor -- start 2024-09-17T05:10:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 -- start start 2024-09-17T05:10:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c0d0830 0x7ff74c151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff74c0d1aa0 0x7ff74c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c1699c0 con 0x7ff74c15d5c0 2024-09-17T05:10:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff74c169840 con 0x7ff74c0d0830 2024-09-17T05:10:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74be00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ff74c169b40 con 0x7ff74c0d1aa0 2024-09-17T05:10:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45898/0 (socket says 172.21.15.16:45898) 2024-09-17T05:10:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 -- 172.21.15.16:0/85089314 learned_addr learned my addr 172.21.15.16:0/85089314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74aa00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c0d0830 0x7ff74c151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff743e00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff74c0d1aa0 0x7ff74c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 -- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff74c0d1aa0 msgr2=0x7ff74c151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff74c0d1aa0 0x7ff74c151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 -- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c0d0830 msgr2=0x7ff74c151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c0d0830 0x7ff74c151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 -- 172.21.15.16:0/85089314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74c15c210 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff743e00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff74c0d1aa0 0x7ff74c151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.032+0000 7ff74b400640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c157570 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ff74401c530 tx=0x7ff744005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7440b2040 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7440bb070 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744004550 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74c0b61b0 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74c152a10 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff74c1524a0 con 0x7ff74c15d5c0 2024-09-17T05:10:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7440046f0 con 0x7ff74c15d5c0 2024-09-17T05:10:14.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff742a00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff728073230 0x7ff728075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7ff74412ed20 con 0x7ff74c15d5c0 2024-09-17T05:10:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.036+0000 7ff74aa00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff728073230 0x7ff728075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.040+0000 7ff74aa00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff728073230 0x7ff728075740 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00bb50 tx=0x7ff73c00b2e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.040+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff744016770 con 0x7ff74c15d5c0 2024-09-17T05:10:14.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.212+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff74c0b3290 con 0x7ff728073230 2024-09-17T05:10:14.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.220+0000 7ff742a00640 1 -- 172.21.15.16:0/85089314 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7ff74c0b3290 con 0x7ff728073230 2024-09-17T05:10:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff728073230 msgr2=0x7ff728075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff728073230 0x7ff728075740 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00bb50 tx=0x7ff73c00b2e0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 msgr2=0x7ff74c157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c157570 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7ff74401c530 tx=0x7ff744005a60 comp rx=0 tx=0).stop 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 shutdown_connections 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ff728073230 0x7ff728075740 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff74c15d5c0 0x7ff74c157570 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ff74c0d1aa0 0x7ff74c151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 --2- 172.21.15.16:0/85089314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff74c0d0830 0x7ff74c151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 >> 172.21.15.16:0/85089314 conn(0x7ff74c0b4160 msgr2=0x7ff74c1545f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 shutdown_connections 2024-09-17T05:10:14.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.224+0000 7ff74be00640 1 -- 172.21.15.16:0/85089314 wait complete. 2024-09-17T05:10:14.246 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-17T05:10:13.612625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281976,"kb_used_data":15804,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692744,"statfs":{"total":773094113280,"available":772805369856,"internally_reserved":0,"allocated":16183296,"data_stored":8114104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002017"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637282+0000","last_change":"2024-09-17T05:10:09.631745+0000","last_active":"2024-09-17T05:10:10.637282+0000","last_peered":"2024-09-17T05:10:10.637282+0000","last_clean":"2024-09-17T05:10:10.637282+0000","last_became_active":"2024-09-17T05:10:09.630466+0000","last_became_peered":"2024-09-17T05:10:09.630466+0000","last_unstale":"2024-09-17T05:10:10.637282+0000","last_undegraded":"2024-09-17T05:10:10.637282+0000","last_fullsized":"2024-09-17T05:10:10.637282+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:40.914328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632990+0000","last_change":"2024-09-17T05:10:09.610783+0000","last_active":"2024-09-17T05:10:10.632990+0000","last_peered":"2024-09-17T05:10:10.632990+0000","last_clean":"2024-09-17T05:10:10.632990+0000","last_became_active":"2024-09-17T05:10:09.610435+0000","last_became_peered":"2024-09-17T05:10:09.610435+0000","last_unstale":"2024-09-17T05:10:10.632990+0000","last_undegraded":"2024-09-17T05:10:10.632990+0000","last_fullsized":"2024-09-17T05:10:10.632990+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:05.464420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633021+0000","last_change":"2024-09-17T05:10:09.608026+0000","last_active":"2024-09-17T05:10:10.633021+0000","last_peered":"2024-09-17T05:10:10.633021+0000","last_clean":"2024-09-17T05:10:10.633021+0000","last_became_active":"2024-09-17T05:10:09.607626+0000","last_became_peered":"2024-09-17T05:10:09.607626+0000","last_unstale":"2024-09-17T05:10:10.633021+0000","last_undegraded":"2024-09-17T05:10:10.633021+0000","last_fullsized":"2024-09-17T05:10:10.633021+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:02.303049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711949+0000","last_change":"2024-09-17T05:10:09.634104+0000","last_active":"2024-09-17T05:10:10.711949+0000","last_peered":"2024-09-17T05:10:10.711949+0000","last_clean":"2024-09-17T05:10:10.711949+0000","last_became_active":"2024-09-17T05:10:09.633693+0000","last_became_peered":"2024-09-17T05:10:09.633693+0000","last_unstale":"2024-09-17T05:10:10.711949+0000","last_undegraded":"2024-09-17T05:10:10.711949+0000","last_fullsized":"2024-09-17T05:10:10.711949+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:26.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710630+0000","last_change":"2024-09-17T05:10:09.625690+0000","last_active":"2024-09-17T05:10:10.710630+0000","last_peered":"2024-09-17T05:10:10.710630+0000","last_clean":"2024-09-17T05:10:10.710630+0000","last_became_active":"2024-09-17T05:10:09.625364+0000","last_became_peered":"2024-09-17T05:10:09.625364+0000","last_unstale":"2024-09-17T05:10:10.710630+0000","last_undegraded":"2024-09-17T05:10:10.710630+0000","last_fullsized":"2024-09-17T05:10:10.710630+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:43.854136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636801+0000","last_change":"2024-09-17T05:10:09.629500+0000","last_active":"2024-09-17T05:10:10.636801+0000","last_peered":"2024-09-17T05:10:10.636801+0000","last_clean":"2024-09-17T05:10:10.636801+0000","last_became_active":"2024-09-17T05:10:09.629194+0000","last_became_peered":"2024-09-17T05:10:09.629194+0000","last_unstale":"2024-09-17T05:10:10.636801+0000","last_undegraded":"2024-09-17T05:10:10.636801+0000","last_fullsized":"2024-09-17T05:10:10.636801+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:59.575299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709932+0000","last_change":"2024-09-17T05:10:09.616187+0000","last_active":"2024-09-17T05:10:10.709932+0000","last_peered":"2024-09-17T05:10:10.709932+0000","last_clean":"2024-09-17T05:10:10.709932+0000","last_became_active":"2024-09-17T05:10:09.615830+0000","last_became_peered":"2024-09-17T05:10:09.615830+0000","last_unstale":"2024-09-17T05:10:10.709932+0000","last_undegraded":"2024-09-17T05:10:10.709932+0000","last_fullsized":"2024-09-17T05:10:10.709932+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:15.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.593721+0000","last_change":"2024-09-17T05:10:09.617829+0000","last_active":"2024-09-17T05:10:10.593721+0000","last_peered":"2024-09-17T05:10:10.593721+0000","last_clean":"2024-09-17T05:10:10.593721+0000","last_became_active":"2024-09-17T05:10:09.615252+0000","last_became_peered":"2024-09-17T05:10:09.615252+0000","last_unstale":"2024-09-17T05:10:10.593721+0000","last_undegraded":"2024-09-17T05:10:10.593721+0000","last_fullsized":"2024-09-17T05:10:10.593721+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:59.023497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711158+0000","last_change":"2024-09-17T05:10:09.624167+0000","last_active":"2024-09-17T05:10:10.711158+0000","last_peered":"2024-09-17T05:10:10.711158+0000","last_clean":"2024-09-17T05:10:10.711158+0000","last_became_active":"2024-09-17T05:10:09.622484+0000","last_became_peered":"2024-09-17T05:10:09.622484+0000","last_unstale":"2024-09-17T05:10:10.711158+0000","last_undegraded":"2024-09-17T05:10:10.711158+0000","last_fullsized":"2024-09-17T05:10:10.711158+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:45.823474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710716+0000","last_change":"2024-09-17T05:10:09.623799+0000","last_active":"2024-09-17T05:10:10.710716+0000","last_peered":"2024-09-17T05:10:10.710716+0000","last_clean":"2024-09-17T05:10:10.710716+0000","last_became_active":"2024-09-17T05:10:09.622642+0000","last_became_peered":"2024-09-17T05:10:09.622642+0000","last_unstale":"2024-09-17T05:10:10.710716+0000","last_undegraded":"2024-09-17T05:10:10.710716+0000","last_fullsized":"2024-09-17T05:10:10.710716+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:38.082547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637328+0000","last_change":"2024-09-17T05:10:09.611715+0000","last_active":"2024-09-17T05:10:10.637328+0000","last_peered":"2024-09-17T05:10:10.637328+0000","last_clean":"2024-09-17T05:10:10.637328+0000","last_became_active":"2024-09-17T05:10:09.611362+0000","last_became_peered":"2024-09-17T05:10:09.611362+0000","last_unstale":"2024-09-17T05:10:10.637328+0000","last_undegraded":"2024-09-17T05:10:10.637328+0000","last_fullsized":"2024-09-17T05:10:10.637328+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:19.643201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711851+0000","last_change":"2024-09-17T05:10:09.635446+0000","last_active":"2024-09-17T05:10:10.711851+0000","last_peered":"2024-09-17T05:10:10.711851+0000","last_clean":"2024-09-17T05:10:10.711851+0000","last_became_active":"2024-09-17T05:10:09.633274+0000","last_became_peered":"2024-09-17T05:10:09.633274+0000","last_unstale":"2024-09-17T05:10:10.711851+0000","last_undegraded":"2024-09-17T05:10:10.711851+0000","last_fullsized":"2024-09-17T05:10:10.711851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:26.751686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710279+0000","last_change":"2024-09-17T05:10:09.621827+0000","last_active":"2024-09-17T05:10:10.710279+0000","last_peered":"2024-09-17T05:10:10.710279+0000","last_clean":"2024-09-17T05:10:10.710279+0000","last_became_active":"2024-09-17T05:10:09.621284+0000","last_became_peered":"2024-09-17T05:10:09.621284+0000","last_unstale":"2024-09-17T05:10:10.710279+0000","last_undegraded":"2024-09-17T05:10:10.710279+0000","last_fullsized":"2024-09-17T05:10:10.710279+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:26.751686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594385+0000","last_change":"2024-09-17T05:10:09.608271+0000","last_active":"2024-09-17T05:10:10.594385+0000","last_peered":"2024-09-17T05:10:10.594385+0000","last_clean":"2024-09-17T05:10:10.594385+0000","last_became_active":"2024-09-17T05:10:09.607944+0000","last_became_peered":"2024-09-17T05:10:09.607944+0000","last_unstale":"2024-09-17T05:10:10.594385+0000","last_undegraded":"2024-09-17T05:10:10.594385+0000","last_fullsized":"2024-09-17T05:10:10.594385+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:27.431500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636735+0000","last_change":"2024-09-17T05:10:09.614855+0000","last_active":"2024-09-17T05:10:10.636735+0000","last_peered":"2024-09-17T05:10:10.636735+0000","last_clean":"2024-09-17T05:10:10.636735+0000","last_became_active":"2024-09-17T05:10:09.614545+0000","last_became_peered":"2024-09-17T05:10:09.614545+0000","last_unstale":"2024-09-17T05:10:10.636735+0000","last_undegraded":"2024-09-17T05:10:10.636735+0000","last_fullsized":"2024-09-17T05:10:10.636735+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:19.793741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636653+0000","last_change":"2024-09-17T05:10:09.612933+0000","last_active":"2024-09-17T05:10:10.636653+0000","last_peered":"2024-09-17T05:10:10.636653+0000","last_clean":"2024-09-17T05:10:10.636653+0000","last_became_active":"2024-09-17T05:10:09.612614+0000","last_became_peered":"2024-09-17T05:10:09.612614+0000","last_unstale":"2024-09-17T05:10:10.636653+0000","last_undegraded":"2024-09-17T05:10:10.636653+0000","last_fullsized":"2024-09-17T05:10:10.636653+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:26.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711089+0000","last_change":"2024-09-17T05:10:09.622665+0000","last_active":"2024-09-17T05:10:10.711089+0000","last_peered":"2024-09-17T05:10:10.711089+0000","last_clean":"2024-09-17T05:10:10.711089+0000","last_became_active":"2024-09-17T05:10:09.618310+0000","last_became_peered":"2024-09-17T05:10:09.618310+0000","last_unstale":"2024-09-17T05:10:10.711089+0000","last_undegraded":"2024-09-17T05:10:10.711089+0000","last_fullsized":"2024-09-17T05:10:10.711089+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:16.237008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595413+0000","last_change":"2024-09-17T05:10:09.616816+0000","last_active":"2024-09-17T05:10:10.595413+0000","last_peered":"2024-09-17T05:10:10.595413+0000","last_clean":"2024-09-17T05:10:10.595413+0000","last_became_active":"2024-09-17T05:10:09.616527+0000","last_became_peered":"2024-09-17T05:10:09.616527+0000","last_unstale":"2024-09-17T05:10:10.595413+0000","last_undegraded":"2024-09-17T05:10:10.595413+0000","last_fullsized":"2024-09-17T05:10:10.595413+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:16.357047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633033+0000","last_change":"2024-09-17T05:10:09.608833+0000","last_active":"2024-09-17T05:10:10.633033+0000","last_peered":"2024-09-17T05:10:10.633033+0000","last_clean":"2024-09-17T05:10:10.633033+0000","last_became_active":"2024-09-17T05:10:09.608498+0000","last_became_peered":"2024-09-17T05:10:09.608498+0000","last_unstale":"2024-09-17T05:10:10.633033+0000","last_undegraded":"2024-09-17T05:10:10.633033+0000","last_fullsized":"2024-09-17T05:10:10.633033+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:58.751360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595521+0000","last_change":"2024-09-17T05:10:09.605870+0000","last_active":"2024-09-17T05:10:10.595521+0000","last_peered":"2024-09-17T05:10:10.595521+0000","last_clean":"2024-09-17T05:10:10.595521+0000","last_became_active":"2024-09-17T05:10:09.603240+0000","last_became_peered":"2024-09-17T05:10:09.603240+0000","last_unstale":"2024-09-17T05:10:10.595521+0000","last_undegraded":"2024-09-17T05:10:10.595521+0000","last_fullsized":"2024-09-17T05:10:10.595521+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:38.437182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711231+0000","last_change":"2024-09-17T05:10:09.629435+0000","last_active":"2024-09-17T05:10:10.711231+0000","last_peered":"2024-09-17T05:10:10.711231+0000","last_clean":"2024-09-17T05:10:10.711231+0000","last_became_active":"2024-09-17T05:10:09.629038+0000","last_became_peered":"2024-09-17T05:10:09.629038+0000","last_unstale":"2024-09-17T05:10:10.711231+0000","last_undegraded":"2024-09-17T05:10:10.711231+0000","last_fullsized":"2024-09-17T05:10:10.711231+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:02.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637124+0000","last_change":"2024-09-17T05:10:09.611354+0000","last_active":"2024-09-17T05:10:10.637124+0000","last_peered":"2024-09-17T05:10:10.637124+0000","last_clean":"2024-09-17T05:10:10.637124+0000","last_became_active":"2024-09-17T05:10:09.609893+0000","last_became_peered":"2024-09-17T05:10:09.609893+0000","last_unstale":"2024-09-17T05:10:10.637124+0000","last_undegraded":"2024-09-17T05:10:10.637124+0000","last_fullsized":"2024-09-17T05:10:10.637124+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:38.946662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709685+0000","last_change":"2024-09-17T05:10:09.607675+0000","last_active":"2024-09-17T05:10:10.709685+0000","last_peered":"2024-09-17T05:10:10.709685+0000","last_clean":"2024-09-17T05:10:10.709685+0000","last_became_active":"2024-09-17T05:10:09.606067+0000","last_became_peered":"2024-09-17T05:10:09.606067+0000","last_unstale":"2024-09-17T05:10:10.709685+0000","last_undegraded":"2024-09-17T05:10:10.709685+0000","last_fullsized":"2024-09-17T05:10:10.709685+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:20:41.010567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632355+0000","last_change":"2024-09-17T05:10:09.610055+0000","last_active":"2024-09-17T05:10:10.632355+0000","last_peered":"2024-09-17T05:10:10.632355+0000","last_clean":"2024-09-17T05:10:10.632355+0000","last_became_active":"2024-09-17T05:10:09.609130+0000","last_became_peered":"2024-09-17T05:10:09.609130+0000","last_unstale":"2024-09-17T05:10:10.632355+0000","last_undegraded":"2024-09-17T05:10:10.632355+0000","last_fullsized":"2024-09-17T05:10:10.632355+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:07.302807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710337+0000","last_change":"2024-09-17T05:10:09.628366+0000","last_active":"2024-09-17T05:10:10.710337+0000","last_peered":"2024-09-17T05:10:10.710337+0000","last_clean":"2024-09-17T05:10:10.710337+0000","last_became_active":"2024-09-17T05:10:09.628033+0000","last_became_peered":"2024-09-17T05:10:09.628033+0000","last_unstale":"2024-09-17T05:10:10.710337+0000","last_undegraded":"2024-09-17T05:10:10.710337+0000","last_fullsized":"2024-09-17T05:10:10.710337+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:45:19.397104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594500+0000","last_change":"2024-09-17T05:10:09.606716+0000","last_active":"2024-09-17T05:10:10.594500+0000","last_peered":"2024-09-17T05:10:10.594500+0000","last_clean":"2024-09-17T05:10:10.594500+0000","last_became_active":"2024-09-17T05:10:09.606349+0000","last_became_peered":"2024-09-17T05:10:09.606349+0000","last_unstale":"2024-09-17T05:10:10.594500+0000","last_undegraded":"2024-09-17T05:10:10.594500+0000","last_fullsized":"2024-09-17T05:10:10.594500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:49:54.333652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633273+0000","last_change":"2024-09-17T05:10:09.623285+0000","last_active":"2024-09-17T05:10:10.633273+0000","last_peered":"2024-09-17T05:10:10.633273+0000","last_clean":"2024-09-17T05:10:10.633273+0000","last_became_active":"2024-09-17T05:10:09.622926+0000","last_became_peered":"2024-09-17T05:10:09.622926+0000","last_unstale":"2024-09-17T05:10:10.633273+0000","last_undegraded":"2024-09-17T05:10:10.633273+0000","last_fullsized":"2024-09-17T05:10:10.633273+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:40.746210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709869+0000","last_change":"2024-09-17T05:10:09.627847+0000","last_active":"2024-09-17T05:10:10.709869+0000","last_peered":"2024-09-17T05:10:10.709869+0000","last_clean":"2024-09-17T05:10:10.709869+0000","last_became_active":"2024-09-17T05:10:09.627447+0000","last_became_peered":"2024-09-17T05:10:09.627447+0000","last_unstale":"2024-09-17T05:10:10.709869+0000","last_undegraded":"2024-09-17T05:10:10.709869+0000","last_fullsized":"2024-09-17T05:10:10.709869+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:35.272095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594850+0000","last_change":"2024-09-17T05:10:09.620329+0000","last_active":"2024-09-17T05:10:10.594850+0000","last_peered":"2024-09-17T05:10:10.594850+0000","last_clean":"2024-09-17T05:10:10.594850+0000","last_became_active":"2024-09-17T05:10:09.619753+0000","last_became_peered":"2024-09-17T05:10:09.619753+0000","last_unstale":"2024-09-17T05:10:10.594850+0000","last_undegraded":"2024-09-17T05:10:10.594850+0000","last_fullsized":"2024-09-17T05:10:10.594850+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:59.023497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637059+0000","last_change":"2024-09-17T05:10:09.620194+0000","last_active":"2024-09-17T05:10:10.637059+0000","last_peered":"2024-09-17T05:10:10.637059+0000","last_clean":"2024-09-17T05:10:10.637059+0000","last_became_active":"2024-09-17T05:10:09.616836+0000","last_became_peered":"2024-09-17T05:10:09.616836+0000","last_unstale":"2024-09-17T05:10:10.637059+0000","last_undegraded":"2024-09-17T05:10:10.637059+0000","last_fullsized":"2024-09-17T05:10:10.637059+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:02.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711502+0000","last_change":"2024-09-17T05:10:09.614433+0000","last_active":"2024-09-17T05:10:10.711502+0000","last_peered":"2024-09-17T05:10:10.711502+0000","last_clean":"2024-09-17T05:10:10.711502+0000","last_became_active":"2024-09-17T05:10:09.612322+0000","last_became_peered":"2024-09-17T05:10:09.612322+0000","last_unstale":"2024-09-17T05:10:10.711502+0000","last_undegraded":"2024-09-17T05:10:10.711502+0000","last_fullsized":"2024-09-17T05:10:10.711502+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:06.224706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632397+0000","last_change":"2024-09-17T05:10:09.614277+0000","last_active":"2024-09-17T05:10:10.632397+0000","last_peered":"2024-09-17T05:10:10.632397+0000","last_clean":"2024-09-17T05:10:10.632397+0000","last_became_active":"2024-09-17T05:10:09.613559+0000","last_became_peered":"2024-09-17T05:10:09.613559+0000","last_unstale":"2024-09-17T05:10:10.632397+0000","last_undegraded":"2024-09-17T05:10:10.632397+0000","last_fullsized":"2024-09-17T05:10:10.632397+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:30.391166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632531+0000","last_change":"2024-09-17T05:10:09.613913+0000","last_active":"2024-09-17T05:10:10.632531+0000","last_peered":"2024-09-17T05:10:10.632531+0000","last_clean":"2024-09-17T05:10:10.632531+0000","last_became_active":"2024-09-17T05:10:09.613177+0000","last_became_peered":"2024-09-17T05:10:09.613177+0000","last_unstale":"2024-09-17T05:10:10.632531+0000","last_undegraded":"2024-09-17T05:10:10.632531+0000","last_fullsized":"2024-09-17T05:10:10.632531+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:13.998035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"148'4","reported_seq":19,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632956+0000","last_change":"2024-09-17T05:10:08.616110+0000","last_active":"2024-09-17T05:10:10.632956+0000","last_peered":"2024-09-17T05:10:10.632956+0000","last_clean":"2024-09-17T05:10:10.632956+0000","last_became_active":"2024-09-17T05:10:08.615796+0000","last_became_peered":"2024-09-17T05:10:08.615796+0000","last_unstale":"2024-09-17T05:10:10.632956+0000","last_undegraded":"2024-09-17T05:10:10.632956+0000","last_fullsized":"2024-09-17T05:10:10.632956+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:01.257841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709727+0000","last_change":"2024-09-17T05:10:09.619128+0000","last_active":"2024-09-17T05:10:10.709727+0000","last_peered":"2024-09-17T05:10:10.709727+0000","last_clean":"2024-09-17T05:10:10.709727+0000","last_became_active":"2024-09-17T05:10:09.618807+0000","last_became_peered":"2024-09-17T05:10:09.618807+0000","last_unstale":"2024-09-17T05:10:10.709727+0000","last_undegraded":"2024-09-17T05:10:10.709727+0000","last_fullsized":"2024-09-17T05:10:10.709727+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:07.178601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595215+0000","last_change":"2024-09-17T05:10:08.622137+0000","last_active":"2024-09-17T05:10:10.595215+0000","last_peered":"2024-09-17T05:10:10.595215+0000","last_clean":"2024-09-17T05:10:10.595215+0000","last_became_active":"2024-09-17T05:10:08.618489+0000","last_became_peered":"2024-09-17T05:10:08.618489+0000","last_unstale":"2024-09-17T05:10:10.595215+0000","last_undegraded":"2024-09-17T05:10:10.595215+0000","last_fullsized":"2024-09-17T05:10:10.595215+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:37:12.995112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710714+0000","last_change":"2024-09-17T05:10:09.620963+0000","last_active":"2024-09-17T05:10:10.710714+0000","last_peered":"2024-09-17T05:10:10.710714+0000","last_clean":"2024-09-17T05:10:10.710714+0000","last_became_active":"2024-09-17T05:10:09.620673+0000","last_became_peered":"2024-09-17T05:10:09.620673+0000","last_unstale":"2024-09-17T05:10:10.710714+0000","last_undegraded":"2024-09-17T05:10:10.710714+0000","last_fullsized":"2024-09-17T05:10:10.710714+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:26.454657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711806+0000","last_change":"2024-09-17T05:10:08.603988+0000","last_active":"2024-09-17T05:10:10.711806+0000","last_peered":"2024-09-17T05:10:10.711806+0000","last_clean":"2024-09-17T05:10:10.711806+0000","last_became_active":"2024-09-17T05:10:08.603638+0000","last_became_peered":"2024-09-17T05:10:08.603638+0000","last_unstale":"2024-09-17T05:10:10.711806+0000","last_undegraded":"2024-09-17T05:10:10.711806+0000","last_fullsized":"2024-09-17T05:10:10.711806+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:48:34.999504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711765+0000","last_change":"2024-09-17T05:10:09.610728+0000","last_active":"2024-09-17T05:10:10.711765+0000","last_peered":"2024-09-17T05:10:10.711765+0000","last_clean":"2024-09-17T05:10:10.711765+0000","last_became_active":"2024-09-17T05:10:09.610362+0000","last_became_peered":"2024-09-17T05:10:09.610362+0000","last_unstale":"2024-09-17T05:10:10.711765+0000","last_undegraded":"2024-09-17T05:10:10.711765+0000","last_fullsized":"2024-09-17T05:10:10.711765+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:07.635433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710334+0000","last_change":"2024-09-17T05:10:08.614774+0000","last_active":"2024-09-17T05:10:10.710334+0000","last_peered":"2024-09-17T05:10:10.710334+0000","last_clean":"2024-09-17T05:10:10.710334+0000","last_became_active":"2024-09-17T05:10:08.614056+0000","last_became_peered":"2024-09-17T05:10:08.614056+0000","last_unstale":"2024-09-17T05:10:10.710334+0000","last_undegraded":"2024-09-17T05:10:10.710334+0000","last_fullsized":"2024-09-17T05:10:10.710334+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:13:10.358573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709950+0000","last_change":"2024-09-17T05:10:09.620716+0000","last_active":"2024-09-17T05:10:10.709950+0000","last_peered":"2024-09-17T05:10:10.709950+0000","last_clean":"2024-09-17T05:10:10.709950+0000","last_became_active":"2024-09-17T05:10:09.620373+0000","last_became_peered":"2024-09-17T05:10:09.620373+0000","last_unstale":"2024-09-17T05:10:10.709950+0000","last_undegraded":"2024-09-17T05:10:10.709950+0000","last_fullsized":"2024-09-17T05:10:10.709950+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:06.224706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711112+0000","last_change":"2024-09-17T05:10:08.622709+0000","last_active":"2024-09-17T05:10:10.711112+0000","last_peered":"2024-09-17T05:10:10.711112+0000","last_clean":"2024-09-17T05:10:10.711112+0000","last_became_active":"2024-09-17T05:10:08.622274+0000","last_became_peered":"2024-09-17T05:10:08.622274+0000","last_unstale":"2024-09-17T05:10:10.711112+0000","last_undegraded":"2024-09-17T05:10:10.711112+0000","last_fullsized":"2024-09-17T05:10:10.711112+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:50:29.690037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710324+0000","last_change":"2024-09-17T05:10:09.632375+0000","last_active":"2024-09-17T05:10:10.710324+0000","last_peered":"2024-09-17T05:10:10.710324+0000","last_clean":"2024-09-17T05:10:10.710324+0000","last_became_active":"2024-09-17T05:10:09.630863+0000","last_became_peered":"2024-09-17T05:10:09.630863+0000","last_unstale":"2024-09-17T05:10:10.710324+0000","last_undegraded":"2024-09-17T05:10:10.710324+0000","last_fullsized":"2024-09-17T05:10:10.710324+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:26.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594928+0000","last_change":"2024-09-17T05:10:08.618499+0000","last_active":"2024-09-17T05:10:10.594928+0000","last_peered":"2024-09-17T05:10:10.594928+0000","last_clean":"2024-09-17T05:10:10.594928+0000","last_became_active":"2024-09-17T05:10:08.617835+0000","last_became_peered":"2024-09-17T05:10:08.617835+0000","last_unstale":"2024-09-17T05:10:10.594928+0000","last_undegraded":"2024-09-17T05:10:10.594928+0000","last_fullsized":"2024-09-17T05:10:10.594928+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:34.802748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710782+0000","last_change":"2024-09-17T05:10:09.619523+0000","last_active":"2024-09-17T05:10:10.710782+0000","last_peered":"2024-09-17T05:10:10.710782+0000","last_clean":"2024-09-17T05:10:10.710782+0000","last_became_active":"2024-09-17T05:10:09.619178+0000","last_became_peered":"2024-09-17T05:10:09.619178+0000","last_unstale":"2024-09-17T05:10:10.710782+0000","last_undegraded":"2024-09-17T05:10:10.710782+0000","last_fullsized":"2024-09-17T05:10:10.710782+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:28:37.618349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594545+0000","last_change":"2024-09-17T05:10:08.620259+0000","last_active":"2024-09-17T05:10:10.594545+0000","last_peered":"2024-09-17T05:10:10.594545+0000","last_clean":"2024-09-17T05:10:10.594545+0000","last_became_active":"2024-09-17T05:10:08.619546+0000","last_became_peered":"2024-09-17T05:10:08.619546+0000","last_unstale":"2024-09-17T05:10:10.594545+0000","last_undegraded":"2024-09-17T05:10:10.594545+0000","last_fullsized":"2024-09-17T05:10:10.594545+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:00.078340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595464+0000","last_change":"2024-09-17T05:10:09.627317+0000","last_active":"2024-09-17T05:10:10.595464+0000","last_peered":"2024-09-17T05:10:10.595464+0000","last_clean":"2024-09-17T05:10:10.595464+0000","last_became_active":"2024-09-17T05:10:09.626997+0000","last_became_peered":"2024-09-17T05:10:09.626997+0000","last_unstale":"2024-09-17T05:10:10.595464+0000","last_undegraded":"2024-09-17T05:10:10.595464+0000","last_fullsized":"2024-09-17T05:10:10.595464+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:19.200502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594051+0000","last_change":"2024-09-17T05:10:08.609712+0000","last_active":"2024-09-17T05:10:10.594051+0000","last_peered":"2024-09-17T05:10:10.594051+0000","last_clean":"2024-09-17T05:10:10.594051+0000","last_became_active":"2024-09-17T05:10:08.608163+0000","last_became_peered":"2024-09-17T05:10:08.608163+0000","last_unstale":"2024-09-17T05:10:10.594051+0000","last_undegraded":"2024-09-17T05:10:10.594051+0000","last_fullsized":"2024-09-17T05:10:10.594051+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:25:02.449152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632598+0000","last_change":"2024-09-17T05:10:09.605344+0000","last_active":"2024-09-17T05:10:10.632598+0000","last_peered":"2024-09-17T05:10:10.632598+0000","last_clean":"2024-09-17T05:10:10.632598+0000","last_became_active":"2024-09-17T05:10:09.605036+0000","last_became_peered":"2024-09-17T05:10:09.605036+0000","last_unstale":"2024-09-17T05:10:10.632598+0000","last_undegraded":"2024-09-17T05:10:10.632598+0000","last_fullsized":"2024-09-17T05:10:10.632598+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:08:35.805671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711153+0000","last_change":"2024-09-17T05:10:08.615422+0000","last_active":"2024-09-17T05:10:10.711153+0000","last_peered":"2024-09-17T05:10:10.711153+0000","last_clean":"2024-09-17T05:10:10.711153+0000","last_became_active":"2024-09-17T05:10:08.611354+0000","last_became_peered":"2024-09-17T05:10:08.611354+0000","last_unstale":"2024-09-17T05:10:10.711153+0000","last_undegraded":"2024-09-17T05:10:10.711153+0000","last_fullsized":"2024-09-17T05:10:10.711153+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:02:27.066794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710034+0000","last_change":"2024-09-17T05:10:09.630016+0000","last_active":"2024-09-17T05:10:10.710034+0000","last_peered":"2024-09-17T05:10:10.710034+0000","last_clean":"2024-09-17T05:10:10.710034+0000","last_became_active":"2024-09-17T05:10:09.629590+0000","last_became_peered":"2024-09-17T05:10:09.629590+0000","last_unstale":"2024-09-17T05:10:10.710034+0000","last_undegraded":"2024-09-17T05:10:10.710034+0000","last_fullsized":"2024-09-17T05:10:10.710034+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:26.897183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"148'3","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.633132+0000","last_change":"2024-09-17T05:10:08.608985+0000","last_active":"2024-09-17T05:10:10.633132+0000","last_peered":"2024-09-17T05:10:10.633132+0000","last_clean":"2024-09-17T05:10:10.633132+0000","last_became_active":"2024-09-17T05:10:08.608684+0000","last_became_peered":"2024-09-17T05:10:08.608684+0000","last_unstale":"2024-09-17T05:10:10.633132+0000","last_undegraded":"2024-09-17T05:10:10.633132+0000","last_fullsized":"2024-09-17T05:10:10.633132+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:58.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710981+0000","last_change":"2024-09-17T05:10:09.619258+0000","last_active":"2024-09-17T05:10:10.710981+0000","last_peered":"2024-09-17T05:10:10.710981+0000","last_clean":"2024-09-17T05:10:10.710981+0000","last_became_active":"2024-09-17T05:10:09.618922+0000","last_became_peered":"2024-09-17T05:10:09.618922+0000","last_unstale":"2024-09-17T05:10:10.710981+0000","last_undegraded":"2024-09-17T05:10:10.710981+0000","last_fullsized":"2024-09-17T05:10:10.710981+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:10.939262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"148'6","reported_seq":21,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710127+0000","last_change":"2024-09-17T05:10:08.612149+0000","last_active":"2024-09-17T05:10:10.710127+0000","last_peered":"2024-09-17T05:10:10.710127+0000","last_clean":"2024-09-17T05:10:10.710127+0000","last_became_active":"2024-09-17T05:10:08.611792+0000","last_became_peered":"2024-09-17T05:10:08.611792+0000","last_unstale":"2024-09-17T05:10:10.710127+0000","last_undegraded":"2024-09-17T05:10:10.710127+0000","last_fullsized":"2024-09-17T05:10:10.710127+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:43.153997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.712107+0000","last_change":"2024-09-17T05:10:09.628351+0000","last_active":"2024-09-17T05:10:10.712107+0000","last_peered":"2024-09-17T05:10:10.712107+0000","last_clean":"2024-09-17T05:10:10.712107+0000","last_became_active":"2024-09-17T05:10:09.627984+0000","last_became_peered":"2024-09-17T05:10:09.627984+0000","last_unstale":"2024-09-17T05:10:10.712107+0000","last_undegraded":"2024-09-17T05:10:10.712107+0000","last_fullsized":"2024-09-17T05:10:10.712107+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:59.575299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594210+0000","last_change":"2024-09-17T05:10:08.612025+0000","last_active":"2024-09-17T05:10:10.594210+0000","last_peered":"2024-09-17T05:10:10.594210+0000","last_clean":"2024-09-17T05:10:10.594210+0000","last_became_active":"2024-09-17T05:10:08.607799+0000","last_became_peered":"2024-09-17T05:10:08.607799+0000","last_unstale":"2024-09-17T05:10:10.594210+0000","last_undegraded":"2024-09-17T05:10:10.594210+0000","last_fullsized":"2024-09-17T05:10:10.594210+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:19.401746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709837+0000","last_change":"2024-09-17T05:10:09.628294+0000","last_active":"2024-09-17T05:10:10.709837+0000","last_peered":"2024-09-17T05:10:10.709837+0000","last_clean":"2024-09-17T05:10:10.709837+0000","last_became_active":"2024-09-17T05:10:09.627713+0000","last_became_peered":"2024-09-17T05:10:09.627713+0000","last_unstale":"2024-09-17T05:10:10.709837+0000","last_undegraded":"2024-09-17T05:10:10.709837+0000","last_fullsized":"2024-09-17T05:10:10.709837+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:10.136192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710571+0000","last_change":"2024-09-17T05:10:08.610638+0000","last_active":"2024-09-17T05:10:10.710571+0000","last_peered":"2024-09-17T05:10:10.710571+0000","last_clean":"2024-09-17T05:10:10.710571+0000","last_became_active":"2024-09-17T05:10:08.610299+0000","last_became_peered":"2024-09-17T05:10:08.610299+0000","last_unstale":"2024-09-17T05:10:10.710571+0000","last_undegraded":"2024-09-17T05:10:10.710571+0000","last_fullsized":"2024-09-17T05:10:10.710571+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:07.190290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.712048+0000","last_change":"2024-09-17T05:10:09.623421+0000","last_active":"2024-09-17T05:10:10.712048+0000","last_peered":"2024-09-17T05:10:10.712048+0000","last_clean":"2024-09-17T05:10:10.712048+0000","last_became_active":"2024-09-17T05:10:09.623074+0000","last_became_peered":"2024-09-17T05:10:09.623074+0000","last_unstale":"2024-09-17T05:10:10.712048+0000","last_undegraded":"2024-09-17T05:10:10.712048+0000","last_fullsized":"2024-09-17T05:10:10.712048+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:19.793741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711569+0000","last_change":"2024-09-17T05:10:08.614128+0000","last_active":"2024-09-17T05:10:10.711569+0000","last_peered":"2024-09-17T05:10:10.711569+0000","last_clean":"2024-09-17T05:10:10.711569+0000","last_became_active":"2024-09-17T05:10:08.613663+0000","last_became_peered":"2024-09-17T05:10:08.613663+0000","last_unstale":"2024-09-17T05:10:10.711569+0000","last_undegraded":"2024-09-17T05:10:10.711569+0000","last_fullsized":"2024-09-17T05:10:10.711569+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:08:02.029837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594329+0000","last_change":"2024-09-17T05:10:09.619514+0000","last_active":"2024-09-17T05:10:10.594329+0000","last_peered":"2024-09-17T05:10:10.594329+0000","last_clean":"2024-09-17T05:10:10.594329+0000","last_became_active":"2024-09-17T05:10:09.619191+0000","last_became_peered":"2024-09-17T05:10:09.619191+0000","last_unstale":"2024-09-17T05:10:10.594329+0000","last_undegraded":"2024-09-17T05:10:10.594329+0000","last_fullsized":"2024-09-17T05:10:10.594329+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:19.054335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"148'8","reported_seq":23,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709489+0000","last_change":"2024-09-17T05:10:08.614136+0000","last_active":"2024-09-17T05:10:10.709489+0000","last_peered":"2024-09-17T05:10:10.709489+0000","last_clean":"2024-09-17T05:10:10.709489+0000","last_became_active":"2024-09-17T05:10:08.613818+0000","last_became_peered":"2024-09-17T05:10:08.613818+0000","last_unstale":"2024-09-17T05:10:10.709489+0000","last_undegraded":"2024-09-17T05:10:10.709489+0000","last_fullsized":"2024-09-17T05:10:10.709489+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:21.707300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.593973+0000","last_change":"2024-09-17T05:10:09.604347+0000","last_active":"2024-09-17T05:10:10.593973+0000","last_peered":"2024-09-17T05:10:10.593973+0000","last_clean":"2024-09-17T05:10:10.593973+0000","last_became_active":"2024-09-17T05:10:09.603715+0000","last_became_peered":"2024-09-17T05:10:09.603715+0000","last_unstale":"2024-09-17T05:10:10.593973+0000","last_undegraded":"2024-09-17T05:10:10.593973+0000","last_fullsized":"2024-09-17T05:10:10.593973+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.148642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594976+0000","last_change":"2024-09-17T05:10:08.618196+0000","last_active":"2024-09-17T05:10:10.594976+0000","last_peered":"2024-09-17T05:10:10.594976+0000","last_clean":"2024-09-17T05:10:10.594976+0000","last_became_active":"2024-09-17T05:10:08.617683+0000","last_became_peered":"2024-09-17T05:10:08.617683+0000","last_unstale":"2024-09-17T05:10:10.594976+0000","last_undegraded":"2024-09-17T05:10:10.594976+0000","last_fullsized":"2024-09-17T05:10:10.594976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:29.388243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636873+0000","last_change":"2024-09-17T05:10:09.626164+0000","last_active":"2024-09-17T05:10:10.636873+0000","last_peered":"2024-09-17T05:10:10.636873+0000","last_clean":"2024-09-17T05:10:10.636873+0000","last_became_active":"2024-09-17T05:10:09.621575+0000","last_became_peered":"2024-09-17T05:10:09.621575+0000","last_unstale":"2024-09-17T05:10:10.636873+0000","last_undegraded":"2024-09-17T05:10:10.636873+0000","last_fullsized":"2024-09-17T05:10:10.636873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:39:10.939262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710878+0000","last_change":"2024-09-17T05:10:08.613116+0000","last_active":"2024-09-17T05:10:10.710878+0000","last_peered":"2024-09-17T05:10:10.710878+0000","last_clean":"2024-09-17T05:10:10.710878+0000","last_became_active":"2024-09-17T05:10:08.612754+0000","last_became_peered":"2024-09-17T05:10:08.612754+0000","last_unstale":"2024-09-17T05:10:10.710878+0000","last_undegraded":"2024-09-17T05:10:10.710878+0000","last_fullsized":"2024-09-17T05:10:10.710878+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:26.869035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.637357+0000","last_change":"2024-09-17T05:10:09.628661+0000","last_active":"2024-09-17T05:10:10.637357+0000","last_peered":"2024-09-17T05:10:10.637357+0000","last_clean":"2024-09-17T05:10:10.637357+0000","last_became_active":"2024-09-17T05:10:09.620876+0000","last_became_peered":"2024-09-17T05:10:09.620876+0000","last_unstale":"2024-09-17T05:10:10.637357+0000","last_undegraded":"2024-09-17T05:10:10.637357+0000","last_fullsized":"2024-09-17T05:10:10.637357+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:28:09.358197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"148'1","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594643+0000","last_change":"2024-09-17T05:10:08.613140+0000","last_active":"2024-09-17T05:10:10.594643+0000","last_peered":"2024-09-17T05:10:10.594643+0000","last_clean":"2024-09-17T05:10:10.594643+0000","last_became_active":"2024-09-17T05:10:08.612717+0000","last_became_peered":"2024-09-17T05:10:08.612717+0000","last_unstale":"2024-09-17T05:10:10.594643+0000","last_undegraded":"2024-09-17T05:10:10.594643+0000","last_fullsized":"2024-09-17T05:10:10.594643+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:52.476600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636902+0000","last_change":"2024-09-17T05:10:09.619313+0000","last_active":"2024-09-17T05:10:10.636902+0000","last_peered":"2024-09-17T05:10:10.636902+0000","last_clean":"2024-09-17T05:10:10.636902+0000","last_became_active":"2024-09-17T05:10:09.618962+0000","last_became_peered":"2024-09-17T05:10:09.618962+0000","last_unstale":"2024-09-17T05:10:10.636902+0000","last_undegraded":"2024-09-17T05:10:10.636902+0000","last_fullsized":"2024-09-17T05:10:10.636902+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:41.452495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595164+0000","last_change":"2024-09-17T05:10:08.618485+0000","last_active":"2024-09-17T05:10:10.595164+0000","last_peered":"2024-09-17T05:10:10.595164+0000","last_clean":"2024-09-17T05:10:10.595164+0000","last_became_active":"2024-09-17T05:10:08.618140+0000","last_became_peered":"2024-09-17T05:10:08.618140+0000","last_unstale":"2024-09-17T05:10:10.595164+0000","last_undegraded":"2024-09-17T05:10:10.595164+0000","last_fullsized":"2024-09-17T05:10:10.595164+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:04.727144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710243+0000","last_change":"2024-09-17T05:10:09.625248+0000","last_active":"2024-09-17T05:10:10.710243+0000","last_peered":"2024-09-17T05:10:10.710243+0000","last_clean":"2024-09-17T05:10:10.710243+0000","last_became_active":"2024-09-17T05:10:09.624768+0000","last_became_peered":"2024-09-17T05:10:09.624768+0000","last_unstale":"2024-09-17T05:10:10.710243+0000","last_undegraded":"2024-09-17T05:10:10.710243+0000","last_fullsized":"2024-09-17T05:10:10.710243+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:07.635433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594189+0000","last_change":"2024-09-17T05:10:08.606682+0000","last_active":"2024-09-17T05:10:10.594189+0000","last_peered":"2024-09-17T05:10:10.594189+0000","last_clean":"2024-09-17T05:10:10.594189+0000","last_became_active":"2024-09-17T05:10:08.605794+0000","last_became_peered":"2024-09-17T05:10:08.605794+0000","last_unstale":"2024-09-17T05:10:10.594189+0000","last_undegraded":"2024-09-17T05:10:10.594189+0000","last_fullsized":"2024-09-17T05:10:10.594189+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:35.608634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594386+0000","last_change":"2024-09-17T05:10:09.608057+0000","last_active":"2024-09-17T05:10:10.594386+0000","last_peered":"2024-09-17T05:10:10.594386+0000","last_clean":"2024-09-17T05:10:10.594386+0000","last_became_active":"2024-09-17T05:10:09.607672+0000","last_became_peered":"2024-09-17T05:10:09.607672+0000","last_unstale":"2024-09-17T05:10:10.594386+0000","last_undegraded":"2024-09-17T05:10:10.594386+0000","last_fullsized":"2024-09-17T05:10:10.594386+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:23:37.614737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632433+0000","last_change":"2024-09-17T05:10:08.621986+0000","last_active":"2024-09-17T05:10:10.632433+0000","last_peered":"2024-09-17T05:10:10.632433+0000","last_clean":"2024-09-17T05:10:10.632433+0000","last_became_active":"2024-09-17T05:10:08.621660+0000","last_became_peered":"2024-09-17T05:10:08.621660+0000","last_unstale":"2024-09-17T05:10:10.632433+0000","last_undegraded":"2024-09-17T05:10:10.632433+0000","last_fullsized":"2024-09-17T05:10:10.632433+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:45.331253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594559+0000","last_change":"2024-09-17T05:10:09.613692+0000","last_active":"2024-09-17T05:10:10.594559+0000","last_peered":"2024-09-17T05:10:10.594559+0000","last_clean":"2024-09-17T05:10:10.594559+0000","last_became_active":"2024-09-17T05:10:09.613348+0000","last_became_peered":"2024-09-17T05:10:09.613348+0000","last_unstale":"2024-09-17T05:10:10.594559+0000","last_undegraded":"2024-09-17T05:10:10.594559+0000","last_fullsized":"2024-09-17T05:10:10.594559+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:51.538160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709987+0000","last_change":"2024-09-17T05:10:08.609449+0000","last_active":"2024-09-17T05:10:10.709987+0000","last_peered":"2024-09-17T05:10:10.709987+0000","last_clean":"2024-09-17T05:10:10.709987+0000","last_became_active":"2024-09-17T05:10:08.609121+0000","last_became_peered":"2024-09-17T05:10:08.609121+0000","last_unstale":"2024-09-17T05:10:10.709987+0000","last_undegraded":"2024-09-17T05:10:10.709987+0000","last_fullsized":"2024-09-17T05:10:10.709987+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:07.620444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595493+0000","last_change":"2024-09-17T05:10:09.618340+0000","last_active":"2024-09-17T05:10:10.595493+0000","last_peered":"2024-09-17T05:10:10.595493+0000","last_clean":"2024-09-17T05:10:10.595493+0000","last_became_active":"2024-09-17T05:10:09.617963+0000","last_became_peered":"2024-09-17T05:10:09.617963+0000","last_unstale":"2024-09-17T05:10:10.595493+0000","last_undegraded":"2024-09-17T05:10:10.595493+0000","last_fullsized":"2024-09-17T05:10:10.595493+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:38.437182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632569+0000","last_change":"2024-09-17T05:10:08.612988+0000","last_active":"2024-09-17T05:10:10.632569+0000","last_peered":"2024-09-17T05:10:10.632569+0000","last_clean":"2024-09-17T05:10:10.632569+0000","last_became_active":"2024-09-17T05:10:08.612476+0000","last_became_peered":"2024-09-17T05:10:08.612476+0000","last_unstale":"2024-09-17T05:10:10.632569+0000","last_undegraded":"2024-09-17T05:10:10.632569+0000","last_fullsized":"2024-09-17T05:10:10.632569+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:56:04.272837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594799+0000","last_change":"2024-09-17T05:10:09.627603+0000","last_active":"2024-09-17T05:10:10.594799+0000","last_peered":"2024-09-17T05:10:10.594799+0000","last_clean":"2024-09-17T05:10:10.594799+0000","last_became_active":"2024-09-17T05:10:09.623999+0000","last_became_peered":"2024-09-17T05:10:09.623999+0000","last_unstale":"2024-09-17T05:10:10.594799+0000","last_undegraded":"2024-09-17T05:10:10.594799+0000","last_fullsized":"2024-09-17T05:10:10.594799+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:26:17.355866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"148'2","reported_seq":17,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595043+0000","last_change":"2024-09-17T05:10:08.615722+0000","last_active":"2024-09-17T05:10:10.595043+0000","last_peered":"2024-09-17T05:10:10.595043+0000","last_clean":"2024-09-17T05:10:10.595043+0000","last_became_active":"2024-09-17T05:10:08.613532+0000","last_became_peered":"2024-09-17T05:10:08.613532+0000","last_unstale":"2024-09-17T05:10:10.595043+0000","last_undegraded":"2024-09-17T05:10:10.595043+0000","last_fullsized":"2024-09-17T05:10:10.595043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:04.461498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594326+0000","last_change":"2024-09-17T05:10:09.605392+0000","last_active":"2024-09-17T05:10:10.594326+0000","last_peered":"2024-09-17T05:10:10.594326+0000","last_clean":"2024-09-17T05:10:10.594326+0000","last_became_active":"2024-09-17T05:10:09.605065+0000","last_became_peered":"2024-09-17T05:10:09.605065+0000","last_unstale":"2024-09-17T05:10:10.594326+0000","last_undegraded":"2024-09-17T05:10:10.594326+0000","last_fullsized":"2024-09-17T05:10:10.594326+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:43.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594644+0000","last_change":"2024-09-17T05:10:08.610723+0000","last_active":"2024-09-17T05:10:10.594644+0000","last_peered":"2024-09-17T05:10:10.594644+0000","last_clean":"2024-09-17T05:10:10.594644+0000","last_became_active":"2024-09-17T05:10:08.608307+0000","last_became_peered":"2024-09-17T05:10:08.608307+0000","last_unstale":"2024-09-17T05:10:10.594644+0000","last_undegraded":"2024-09-17T05:10:10.594644+0000","last_fullsized":"2024-09-17T05:10:10.594644+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:18:35.327369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595454+0000","last_change":"2024-09-17T05:10:09.604923+0000","last_active":"2024-09-17T05:10:10.595454+0000","last_peered":"2024-09-17T05:10:10.595454+0000","last_clean":"2024-09-17T05:10:10.595454+0000","last_became_active":"2024-09-17T05:10:09.604549+0000","last_became_peered":"2024-09-17T05:10:09.604549+0000","last_unstale":"2024-09-17T05:10:10.595454+0000","last_undegraded":"2024-09-17T05:10:10.595454+0000","last_fullsized":"2024-09-17T05:10:10.595454+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:16.357047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.594868+0000","last_change":"2024-09-17T05:10:08.614232+0000","last_active":"2024-09-17T05:10:10.594868+0000","last_peered":"2024-09-17T05:10:10.594868+0000","last_clean":"2024-09-17T05:10:10.594868+0000","last_became_active":"2024-09-17T05:10:08.613385+0000","last_became_peered":"2024-09-17T05:10:08.613385+0000","last_unstale":"2024-09-17T05:10:10.594868+0000","last_undegraded":"2024-09-17T05:10:10.594868+0000","last_fullsized":"2024-09-17T05:10:10.594868+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:40.573069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711038+0000","last_change":"2024-09-17T05:10:09.626873+0000","last_active":"2024-09-17T05:10:10.711038+0000","last_peered":"2024-09-17T05:10:10.711038+0000","last_clean":"2024-09-17T05:10:10.711038+0000","last_became_active":"2024-09-17T05:10:09.626250+0000","last_became_peered":"2024-09-17T05:10:09.626250+0000","last_unstale":"2024-09-17T05:10:10.711038+0000","last_undegraded":"2024-09-17T05:10:10.711038+0000","last_fullsized":"2024-09-17T05:10:10.711038+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:41.452495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595470+0000","last_change":"2024-09-17T05:10:08.614875+0000","last_active":"2024-09-17T05:10:10.595470+0000","last_peered":"2024-09-17T05:10:10.595470+0000","last_clean":"2024-09-17T05:10:10.595470+0000","last_became_active":"2024-09-17T05:10:08.614532+0000","last_became_peered":"2024-09-17T05:10:08.614532+0000","last_unstale":"2024-09-17T05:10:10.595470+0000","last_undegraded":"2024-09-17T05:10:10.595470+0000","last_fullsized":"2024-09-17T05:10:10.595470+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:06.299885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595361+0000","last_change":"2024-09-17T05:10:09.620911+0000","last_active":"2024-09-17T05:10:10.595361+0000","last_peered":"2024-09-17T05:10:10.595361+0000","last_clean":"2024-09-17T05:10:10.595361+0000","last_became_active":"2024-09-17T05:10:09.620435+0000","last_became_peered":"2024-09-17T05:10:09.620435+0000","last_unstale":"2024-09-17T05:10:10.595361+0000","last_undegraded":"2024-09-17T05:10:10.595361+0000","last_fullsized":"2024-09-17T05:10:10.595361+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:02.148642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632465+0000","last_change":"2024-09-17T05:10:08.615803+0000","last_active":"2024-09-17T05:10:10.632465+0000","last_peered":"2024-09-17T05:10:10.632465+0000","last_clean":"2024-09-17T05:10:10.632465+0000","last_became_active":"2024-09-17T05:10:08.614659+0000","last_became_peered":"2024-09-17T05:10:08.614659+0000","last_unstale":"2024-09-17T05:10:10.632465+0000","last_undegraded":"2024-09-17T05:10:10.632465+0000","last_fullsized":"2024-09-17T05:10:10.632465+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:18:50.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595380+0000","last_change":"2024-09-17T05:10:09.632015+0000","last_active":"2024-09-17T05:10:10.595380+0000","last_peered":"2024-09-17T05:10:10.595380+0000","last_clean":"2024-09-17T05:10:10.595380+0000","last_became_active":"2024-09-17T05:10:09.631637+0000","last_became_peered":"2024-09-17T05:10:09.631637+0000","last_unstale":"2024-09-17T05:10:10.595380+0000","last_undegraded":"2024-09-17T05:10:10.595380+0000","last_fullsized":"2024-09-17T05:10:10.595380+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:43.362155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710010+0000","last_change":"2024-09-17T05:10:08.620144+0000","last_active":"2024-09-17T05:10:10.710010+0000","last_peered":"2024-09-17T05:10:10.710010+0000","last_clean":"2024-09-17T05:10:10.710010+0000","last_became_active":"2024-09-17T05:10:08.619808+0000","last_became_peered":"2024-09-17T05:10:08.619808+0000","last_unstale":"2024-09-17T05:10:10.710010+0000","last_undegraded":"2024-09-17T05:10:10.710010+0000","last_fullsized":"2024-09-17T05:10:10.710010+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:23.628949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.595476+0000","last_change":"2024-09-17T05:10:09.621104+0000","last_active":"2024-09-17T05:10:10.595476+0000","last_peered":"2024-09-17T05:10:10.595476+0000","last_clean":"2024-09-17T05:10:10.595476+0000","last_became_active":"2024-09-17T05:10:09.620786+0000","last_became_peered":"2024-09-17T05:10:09.620786+0000","last_unstale":"2024-09-17T05:10:10.595476+0000","last_undegraded":"2024-09-17T05:10:10.595476+0000","last_fullsized":"2024-09-17T05:10:10.595476+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:19.200502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.632310+0000","last_change":"2024-09-17T05:10:08.608149+0000","last_active":"2024-09-17T05:10:10.632310+0000","last_peered":"2024-09-17T05:10:10.632310+0000","last_clean":"2024-09-17T05:10:10.632310+0000","last_became_active":"2024-09-17T05:10:08.607474+0000","last_became_peered":"2024-09-17T05:10:08.607474+0000","last_unstale":"2024-09-17T05:10:10.632310+0000","last_undegraded":"2024-09-17T05:10:10.632310+0000","last_fullsized":"2024-09-17T05:10:10.632310+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:15.069071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710693+0000","last_change":"2024-09-17T05:10:09.633606+0000","last_active":"2024-09-17T05:10:10.710693+0000","last_peered":"2024-09-17T05:10:10.710693+0000","last_clean":"2024-09-17T05:10:10.710693+0000","last_became_active":"2024-09-17T05:10:09.633172+0000","last_became_peered":"2024-09-17T05:10:09.633172+0000","last_unstale":"2024-09-17T05:10:10.710693+0000","last_undegraded":"2024-09-17T05:10:10.710693+0000","last_fullsized":"2024-09-17T05:10:10.710693+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:44:59.575299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.709470+0000","last_change":"2024-09-17T05:10:08.609788+0000","last_active":"2024-09-17T05:10:10.709470+0000","last_peered":"2024-09-17T05:10:10.709470+0000","last_clean":"2024-09-17T05:10:10.709470+0000","last_became_active":"2024-09-17T05:10:08.608287+0000","last_became_peered":"2024-09-17T05:10:08.608287+0000","last_unstale":"2024-09-17T05:10:10.709470+0000","last_undegraded":"2024-09-17T05:10:10.709470+0000","last_fullsized":"2024-09-17T05:10:10.709470+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:12:07.766840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.711712+0000","last_change":"2024-09-17T05:10:09.618662+0000","last_active":"2024-09-17T05:10:10.711712+0000","last_peered":"2024-09-17T05:10:10.711712+0000","last_clean":"2024-09-17T05:10:10.711712+0000","last_became_active":"2024-09-17T05:10:09.618300+0000","last_became_peered":"2024-09-17T05:10:09.618300+0000","last_unstale":"2024-09-17T05:10:10.711712+0000","last_undegraded":"2024-09-17T05:10:10.711712+0000","last_fullsized":"2024-09-17T05:10:10.711712+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:08.581101+0000","last_clean_scrub_stamp":"2024-09-17T05:10:08.581101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T16:00:26.897183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.710779+0000","last_change":"2024-09-17T05:10:08.621302+0000","last_active":"2024-09-17T05:10:10.710779+0000","last_peered":"2024-09-17T05:10:10.710779+0000","last_clean":"2024-09-17T05:10:10.710779+0000","last_became_active":"2024-09-17T05:10:08.616602+0000","last_became_peered":"2024-09-17T05:10:08.616602+0000","last_unstale":"2024-09-17T05:10:10.710779+0000","last_undegraded":"2024-09-17T05:10:10.710779+0000","last_fullsized":"2024-09-17T05:10:10.710779+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:10:07.578179+0000","last_clean_scrub_stamp":"2024-09-17T05:10:07.578179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:16:09.535927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":358,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-17T05:10:10.636518+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:10:10.636518+0000","last_peered":"2024-09-17T05:10:10.636518+0000","last_clean":"2024-09-17T05:10:10.636518+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:10.636518+0000","last_undegraded":"2024-09-17T05:10:10.636518+0000","last_fullsized":"2024-09-17T05:10:10.636518+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1880064,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1880064,"data_stored":868517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":866927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":866939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1884160,"data_stored":867669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":2465792,"data_stored":1457363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1867776,"data_stored":866905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2461696,"data_stored":1451267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:14.250 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:10:14.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T05:10:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 -- 172.21.15.16:0/3726819408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9708073240 msgr2=0x7f9708073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 --2- 172.21.15.16:0/3726819408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9708073240 0x7f9708073650 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9708071ce0 tx=0x7f970001ccf0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 -- 172.21.15.16:0/3726819408 shutdown_connections 2024-09-17T05:10:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 --2- 172.21.15.16:0/3726819408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9708073240 0x7f9708073650 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 --2- 172.21.15.16:0/3726819408 >> v2:172.21.15.104:3300/0 conn(0x7f97080b2f50 0x7f970815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 -- 172.21.15.16:0/3726819408 >> 172.21.15.16:0/3726819408 conn(0x7f970806f900 msgr2=0x7f97080b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:14.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 -- 172.21.15.16:0/3726819408 shutdown_connections 2024-09-17T05:10:14.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.452+0000 7f970d600640 1 -- 172.21.15.16:0/3726819408 wait complete. 2024-09-17T05:10:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 Processor -- start 2024-09-17T05:10:14.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 -- start start 2024-09-17T05:10:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9708073240 0x7f97080c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97080b2f50 0x7f97080cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 0x7f970813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9708168c70 con 0x7f970815c0a0 2024-09-17T05:10:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9708168af0 con 0x7f9708073240 2024-09-17T05:10:14.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f970d600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9708168df0 con 0x7f97080b2f50 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f9707400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 0x7f970813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f9707400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 0x7f970813c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45922/0 (socket says 172.21.15.16:45922) 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.456+0000 7f9707400640 1 -- 172.21.15.16:0/3886816872 learned_addr learned my addr 172.21.15.16:0/3886816872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9706a00640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9708073240 0x7f97080c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f96ffe00640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97080b2f50 0x7f97080cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9707400640 1 -- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97080b2f50 msgr2=0x7f97080cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9707400640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97080b2f50 0x7f97080cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9707400640 1 -- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9708073240 msgr2=0x7f97080c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9707400640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9708073240 0x7f97080c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9707400640 1 -- 172.21.15.16:0/3886816872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080cc6d0 con 0x7f970815c0a0 2024-09-17T05:10:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f96ffe00640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97080b2f50 0x7f97080cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f9707400640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 0x7f970813c1c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9700098900 tx=0x7f9700005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000b2040 con 0x7f970815c0a0 2024-09-17T05:10:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97000bb070 con 0x7f970815c0a0 2024-09-17T05:10:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700004550 con 0x7f970815c0a0 2024-09-17T05:10:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080cb3d0 con 0x7f970815c0a0 2024-09-17T05:10:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97080cb990 con 0x7f970815c0a0 2024-09-17T05:10:14.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97080c9a60 con 0x7f970815c0a0 2024-09-17T05:10:14.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.460+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f97000046f0 con 0x7f970815c0a0 2024-09-17T05:10:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.464+0000 7f96fea00640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96e4073200 0x7f96e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.464+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f970012eee0 con 0x7f970815c0a0 2024-09-17T05:10:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.464+0000 7f9706a00640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96e4073200 0x7f96e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.464+0000 7f9706a00640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96e4073200 0x7f96e4075710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f96f80058d0 tx=0x7f96f8005860 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:14.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.468+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9700016970 con 0x7f970815c0a0 2024-09-17T05:10:14.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.704+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9708073650 con 0x7f970815c0a0 2024-09-17T05:10:14.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f96fea00640 1 -- 172.21.15.16:0/3886816872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v41) ==== 190+0+147 (secure 0 0 0) 0x7f97000f9e00 con 0x7f970815c0a0 2024-09-17T05:10:14.709 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-17T05:10:14.709 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDSDOlmY8qQIxAAFIyHsGecbESRSGdFg1zcQg== 2024-09-17T05:10:14.709 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2024-09-17T05:10:14.709 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2024-09-17T05:10:14.709 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T05:10:14.709 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96e4073200 msgr2=0x7f96e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96e4073200 0x7f96e4075710 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f96f80058d0 tx=0x7f96f8005860 comp rx=0 tx=0).stop 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 msgr2=0x7f970813c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 0x7f970813c1c0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9700098900 tx=0x7f9700005c90 comp rx=0 tx=0).stop 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 shutdown_connections 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96e4073200 0x7f96e4075710 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f970815c0a0 0x7f970813c1c0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f97080b2f50 0x7f97080cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 --2- 172.21.15.16:0/3886816872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9708073240 0x7f97080c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 >> 172.21.15.16:0/3886816872 conn(0x7f970806f900 msgr2=0x7f970815e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 shutdown_connections 2024-09-17T05:10:14.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.708+0000 7f970d600640 1 -- 172.21.15.16:0/3886816872 wait complete. 2024-09-17T05:10:14.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 -- 172.21.15.16:0/1595657593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 msgr2=0x7fec6400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 --2- 172.21.15.16:0/1595657593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec6400b9d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fec800ce430 tx=0x7fec7801caf0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 -- 172.21.15.16:0/1595657593 shutdown_connections 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 --2- 172.21.15.16:0/1595657593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec6400b9d0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 -- 172.21.15.16:0/1595657593 >> 172.21.15.16:0/1595657593 conn(0x7fec800d10c0 msgr2=0x7fec800d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 -- 172.21.15.16:0/1595657593 shutdown_connections 2024-09-17T05:10:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.888+0000 7fec86800640 1 -- 172.21.15.16:0/1595657593 wait complete. 2024-09-17T05:10:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 Processor -- start 2024-09-17T05:10:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 -- start start 2024-09-17T05:10:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec80155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec801ef440 0x7fec80153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fec801f5ad0 0x7fec801f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec8015c060 con 0x7fec64005360 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fec8015bee0 con 0x7fec801ef440 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fec800b40d0 con 0x7fec801f5ad0 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec80155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec80155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45936/0 (socket says 172.21.15.16:45936) 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 -- 172.21.15.16:0/248384358 learned_addr learned my addr 172.21.15.16:0/248384358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec76a00640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec801ef440 0x7fec80153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7fe00640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fec801f5ad0 0x7fec801f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 -- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fec801f5ad0 msgr2=0x7fec801f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fec801f5ad0 0x7fec801f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 -- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec801ef440 msgr2=0x7fec80153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec801ef440 0x7fec80153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 -- 172.21.15.16:0/248384358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec801f6420 con 0x7fec64005360 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec7f400640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec80155ff0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fec700126c0 tx=0x7fec70012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70010b40 con 0x7fec64005360 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec70010ce0 con 0x7fec64005360 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec700136b0 con 0x7fec64005360 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec801f55f0 con 0x7fec64005360 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.892+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec801f9350 con 0x7fec64005360 2024-09-17T05:10:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.896+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec800b45c0 con 0x7fec64005360 2024-09-17T05:10:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.896+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fec70013850 con 0x7fec64005360 2024-09-17T05:10:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.896+0000 7fec77e00640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fec5c073200 0x7fec5c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.896+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7fec70098100 con 0x7fec64005360 2024-09-17T05:10:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.896+0000 7fec76a00640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fec5c073200 0x7fec5c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.900+0000 7fec76a00640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fec5c073200 0x7fec5c075710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fec8006a6e0 tx=0x7fec680091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:14.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:14.900+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fec70022020 con 0x7fec64005360 2024-09-17T05:10:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.112+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec800b6090 con 0x7fec64005360 2024-09-17T05:10:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.112+0000 7fec77e00640 1 -- 172.21.15.16:0/248384358 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5233 (secure 0 0 0) 0x7fec70062930 con 0x7fec64005360 2024-09-17T05:10:15.115 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:15.115 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":110,"btime":"2024-09-17T05:10:12:626935+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:10:09.017636+0000","modified":"2024-09-17T05:10:12.046191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/291864941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":291864941},{"type":"v1","addr":"172.21.15.16:6835","nonce":291864941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":11}]} 2024-09-17T05:10:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 110 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fec5c073200 msgr2=0x7fec5c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fec5c073200 0x7fec5c075710 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fec8006a6e0 tx=0x7fec680091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 msgr2=0x7fec80155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec80155ff0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fec700126c0 tx=0x7fec70012b80 comp rx=0 tx=0).stop 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 shutdown_connections 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fec5c073200 0x7fec5c075710 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fec801f5ad0 0x7fec801f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec801ef440 0x7fec80153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 --2- 172.21.15.16:0/248384358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec64005360 0x7fec80155ff0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 >> 172.21.15.16:0/248384358 conn(0x7fec800d10c0 msgr2=0x7fec80154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 shutdown_connections 2024-09-17T05:10:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.116+0000 7fec86800640 1 -- 172.21.15.16:0/248384358 wait complete. 2024-09-17T05:10:15.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T05:10:09.017636+0000', 'modified': '2024-09-17T05:10:12.046191+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8526': {'gid': 8526, 'name': 'a', 'rank': 0, 'incarnation': 106, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/291864941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 291864941}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 291864941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8526, 'qdb_cluster': [8526]} 2024-09-17T05:10:15.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T05:10:15.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T05:10:15.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 -- 172.21.15.16:0/4195854362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 msgr2=0x7f4e080ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/4195854362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e080ce5d0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4e080ce2c0 tx=0x7f4df401c820 comp rx=0 tx=0).stop 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 -- 172.21.15.16:0/4195854362 shutdown_connections 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/4195854362 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4e0815f6f0 0x7f4e080ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/4195854362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e080ce5d0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 -- 172.21.15.16:0/4195854362 >> 172.21.15.16:0/4195854362 conn(0x7f4e080b2250 msgr2=0x7f4e080d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 -- 172.21.15.16:0/4195854362 shutdown_connections 2024-09-17T05:10:15.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.700+0000 7f4e0fc00640 1 -- 172.21.15.16:0/4195854362 wait complete. 2024-09-17T05:10:15.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 Processor -- start 2024-09-17T05:10:15.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 -- start start 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4e08074a20 0x7f4e080b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e081fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e0815f6f0 0x7f4e080ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08168ed0 con 0x7f4e08074e30 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08168d50 con 0x7f4e0815f6f0 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4e08169050 con 0x7f4e08074a20 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e081fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e081fcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45942/0 (socket says 172.21.15.16:45942) 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 -- 172.21.15.16:0/638165909 learned_addr learned my addr 172.21.15.16:0/638165909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0d200640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4e08074a20 0x7f4e080b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e0815f6f0 0x7f4e080ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 -- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4e08074a20 msgr2=0x7f4e080b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4e08074a20 0x7f4e080b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 -- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e0815f6f0 msgr2=0x7f4e080ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e0815f6f0 0x7f4e080ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 -- 172.21.15.16:0/638165909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e081fd480 con 0x7f4e08074e30 2024-09-17T05:10:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e07e00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e081fcf40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c10 tx=0x7f4df40a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e04c00640 1 -- 172.21.15.16:0/638165909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df40041c0 con 0x7f4e08074e30 2024-09-17T05:10:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e04c00640 1 -- 172.21.15.16:0/638165909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4df4004450 con 0x7f4e08074e30 2024-09-17T05:10:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e04c00640 1 -- 172.21.15.16:0/638165909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4005280 con 0x7f4e08074e30 2024-09-17T05:10:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e08144120 con 0x7f4e08074e30 2024-09-17T05:10:15.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.704+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e081446e0 con 0x7f4e08074e30 2024-09-17T05:10:15.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.708+0000 7f4e04c00640 1 -- 172.21.15.16:0/638165909 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4df40c1030 con 0x7f4e08074e30 2024-09-17T05:10:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.708+0000 7f4e04c00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4de8073200 0x7f4de8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.708+0000 7f4e04c00640 1 -- 172.21.15.16:0/638165909 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f4df412ecc0 con 0x7f4e08074e30 2024-09-17T05:10:15.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.708+0000 7f4e171b0000 1 Processor -- start 2024-09-17T05:10:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 -- start start 2024-09-17T05:10:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e0d200640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4de8073200 0x7f4de8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e0d200640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4de8073200 0x7f4de8075710 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc007fd0 tx=0x7f4dfc007ec0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51bdc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51bdcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51bdd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df51ab1410 con 0x55df51a8b950 2024-09-17T05:10:15.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df519eeb40 con 0x55df5169b460 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55df519ddf70 con 0x55df51a74ca0 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51bdcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51bdcbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45948/0 (socket says 172.21.15.16:45948) 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 -- 172.21.15.16:0/4070767853 learned_addr learned my addr 172.21.15.16:0/4070767853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e0d200640 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51bdc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51bdd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 -- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 msgr2=0x55df51bdd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51bdd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 -- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 msgr2=0x55df51bdc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51bdc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 -- 172.21.15.16:0/4070767853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df519a55b0 con 0x55df51a8b950 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51bdd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:15.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e07e00640 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51bdcbe0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4df40f9290 tx=0x7f4df40f5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af26520b3d129c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4df9600640 1 -- 172.21.15.16:0/4070767853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df40fdfc0 con 0x55df51a8b950 2024-09-17T05:10:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4df9600640 1 -- 172.21.15.16:0/4070767853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4df4102e70 con 0x55df51a8b950 2024-09-17T05:10:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4df9600640 1 -- 172.21.15.16:0/4070767853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df41029e0 con 0x55df51a8b950 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 -- 172.21.15.16:0/4070767853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 msgr2=0x55df51bdcbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.712+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51bdcbe0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4df40f9290 tx=0x7f4df40f5560 comp rx=0 tx=0).stop 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- 172.21.15.16:0/4070767853 shutdown_connections 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51bdd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51bdcbe0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4070767853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51bdc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- 172.21.15.16:0/4070767853 >> 172.21.15.16:0/4070767853 conn(0x55df51b61d90 msgr2=0x55df51b91690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- 172.21.15.16:0/4070767853 shutdown_connections 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- 172.21.15.16:0/4070767853 wait complete. 2024-09-17T05:10:15.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- start start 2024-09-17T05:10:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 Processor -- start 2024-09-17T05:10:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 0x55df51be3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a74ca0 0x55df51be3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a8b950 0x55df51be43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df519785f0 con 0x55df5169b460 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df519e1d80 con 0x55df51a74ca0 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55df51984520 con 0x55df51a8b950 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e171b0000 10 client.0 _ll_register_callbacks cb 0x55df51988830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 0x55df51be3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 0x55df51be3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45950/0 (socket says 172.21.15.16:45950) 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 -- 172.21.15.16:0/1890714816 learned_addr learned my addr 172.21.15.16:0/1890714816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a8b950 0x55df51be43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e07e00640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a74ca0 0x55df51be3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 -- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a8b950 msgr2=0x55df51be43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a8b950 0x55df51be43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 -- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a74ca0 msgr2=0x55df51be3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a74ca0 0x55df51be3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e0d200640 1 -- 172.21.15.16:0/1890714816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5197c190 con 0x55df5169b460 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.716+0000 7f4e07e00640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a74ca0 0x55df51be3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e0d200640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 0x55df51be3920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0146d0 tx=0x7f4dfc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfe8f38c4e00148 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 1 -- 172.21.15.16:0/1890714816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc025020 con 0x55df5169b460 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55df51976110 con 0x55df5169b460 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 1 -- 172.21.15.16:0/1890714816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f4dfc0156d0 con 0x55df5169b460 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 1 -- 172.21.15.16:0/1890714816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f4dfc02f050 con 0x55df5169b460 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 10 client.8655 fetch_fsmap learned FSMap version 110 2024-09-17T05:10:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55df51a62f10 con 0x55df5169b460 2024-09-17T05:10:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 1 -- 172.21.15.16:0/1890714816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55df51976110 con 0x55df5169b460 2024-09-17T05:10:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 1 -- 172.21.15.16:0/1890714816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 110) ==== 3502+0+0 (secure 0 0 0) 0x7f4dfc032020 con 0x55df5169b460 2024-09-17T05:10:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 10 client.8655 fetch_fsmap finished waiting for FSMap version 110 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 10 client.8655 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 20 client.8655 populate_metadata read hostname 'smithi016' 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df516807d0 0x55df51bedd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 4 client.8655 mds_command: new command op to 8526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 --> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df519f0a90 con 0x55df516807d0 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e07e00640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df516807d0 0x55df51bedd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e07e00640 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df516807d0 0x55df51bedd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56466623f69f5180 server_cookie=bdf19ba80a81083c in_seq=0 out_seq=0 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 10 client.8655 ms_handle_connect on v2:172.21.15.16:6833/291864941 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 1 -- 172.21.15.16:0/1890714816 <== mds.0 v2:172.21.15.16:6833/291864941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55df519f0a90 con 0x55df516807d0 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 10 client.8655 handle_command_reply: tid=0 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 client.8655 shutdown 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 msgr2=0x55df51be3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 0x55df51be3920 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc0146d0 tx=0x7f4dfc000bd0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 shutdown_connections 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df516807d0 0x55df51bedd00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a8b950 0x55df51be43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a74ca0 0x55df51be3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 --2- 172.21.15.16:0/1890714816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df5169b460 0x55df51be3920 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4dee000640 0 client.8655 ms_handle_reset on v2:172.21.15.16:6833/291864941 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 >> 172.21.15.16:0/1890714816 conn(0x55df51b61d90 msgr2=0x55df51b98440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 shutdown_connections 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 1 -- 172.21.15.16:0/1890714816 wait complete. 2024-09-17T05:10:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.720+0000 7f4e171b0000 20 client.8655 trim_cache size 0 max 16384 2024-09-17T05:10:15.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.740+0000 7f4e171b0000 1 Processor -- start 2024-09-17T05:10:15.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.740+0000 7f4e171b0000 1 -- start start 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51c4e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51c4eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51c4f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08205a70 con 0x55df51a8b950 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08204ae0 con 0x55df5169b460 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f4e081f9d60 con 0x55df51a74ca0 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51c4f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51c4f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45956/0 (socket says 172.21.15.16:45956) 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 -- 172.21.15.16:0/4137997989 learned_addr learned my addr 172.21.15.16:0/4137997989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0d200640 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51c4e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e07e00640 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51c4eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 -- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 msgr2=0x55df51c4eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51c4eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 -- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 msgr2=0x55df51c4e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51c4e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 -- 172.21.15.16:0/4137997989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df51a39db0 con 0x55df51a8b950 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e07e00640 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51c4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51c4f0a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009950 tx=0x7f4e00009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e0e5caca9c125b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4df8c00640 1 -- 172.21.15.16:0/4137997989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e00004660 con 0x55df51a8b950 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4df8c00640 1 -- 172.21.15.16:0/4137997989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e00004800 con 0x55df51a8b950 2024-09-17T05:10:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4df8c00640 1 -- 172.21.15.16:0/4137997989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e000116a0 con 0x55df51a8b950 2024-09-17T05:10:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- 172.21.15.16:0/4137997989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 msgr2=0x55df51c4f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51c4f0a0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4e00009950 tx=0x7f4e00009e10 comp rx=0 tx=0).stop 2024-09-17T05:10:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- 172.21.15.16:0/4137997989 shutdown_connections 2024-09-17T05:10:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a8b950 0x55df51c4f0a0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df51a74ca0 0x55df51c4eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 --2- 172.21.15.16:0/4137997989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df5169b460 0x55df51c4e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- 172.21.15.16:0/4137997989 >> 172.21.15.16:0/4137997989 conn(0x55df51b61d90 msgr2=0x55df51bda620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- 172.21.15.16:0/4137997989 shutdown_connections 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- 172.21.15.16:0/4137997989 wait complete. 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 -- start start 2024-09-17T05:10:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.744+0000 7f4e171b0000 1 Processor -- start 2024-09-17T05:10:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df5169b460 0x55df51c52390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 0x55df51c528d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a8b950 0x55df51c52e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df51a9d750 con 0x55df51a74ca0 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55df519eeb40 con 0x55df51a8b950 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x55df519e1090 con 0x55df5169b460 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 10 client.0 _ll_register_callbacks cb 0x7f4dfc0014b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 0x55df51c528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e0d200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df5169b460 0x55df51c52390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 0x55df51c528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45970/0 (socket says 172.21.15.16:45970) 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 -- 172.21.15.16:0/3122287889 learned_addr learned my addr 172.21.15.16:0/3122287889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a8b950 0x55df51c52e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 -- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df5169b460 msgr2=0x55df51c52390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df5169b460 0x55df51c52390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 -- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a8b950 msgr2=0x55df51c52e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a8b950 0x55df51c52e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 -- 172.21.15.16:0/3122287889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df519a4ab0 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e0dc00640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a8b950 0x55df51c52e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e07e00640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 0x55df51c528d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4df4004660 tx=0x7f4df40f9010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168c42574a7e4c93 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4dee000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4dee000640 1 -- 172.21.15.16:0/3122287889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df41029e0 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4dee000640 1 -- 172.21.15.16:0/3122287889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f4df4102b80 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e00004800 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4dee000640 1 -- 172.21.15.16:0/3122287889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) ==== 24+0+0 (secure 0 0 0) 0x7f4df4119050 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 10 client.8661 fetch_fsmap learned FSMap version 110 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) -- 0x55df51983070 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4dee000640 1 -- 172.21.15.16:0/3122287889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e00004800 con 0x55df51a74ca0 2024-09-17T05:10:15.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4dee000640 1 -- 172.21.15.16:0/3122287889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 110) ==== 3502+0+0 (secure 0 0 0) 0x7f4df40f87f0 con 0x55df51a74ca0 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 10 client.8661 fetch_fsmap finished waiting for FSMap version 110 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 10 client.8661 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 20 client.8661 populate_metadata read hostname 'smithi016' 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df5166e830 0x55df51c0e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 4 client.8661 mds_command: new command op to 8526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 --> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] -- command(tid 0: {"prefix": "status"}) -- 0x55df519ec540 con 0x55df5166e830 2024-09-17T05:10:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.748+0000 7f4e0d200640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df5166e830 0x55df51c0e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0d200640 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df5166e830 0x55df51c0e6a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c09f41232c87073 server_cookie=5e3a1fe37b200983 in_seq=0 out_seq=0 2024-09-17T05:10:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4dee000640 10 client.8661 ms_handle_connect on v2:172.21.15.16:6833/291864941 2024-09-17T05:10:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4dee000640 1 -- 172.21.15.16:0/3122287889 <== mds.0 v2:172.21.15.16:6833/291864941 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55df519ec540 con 0x55df5166e830 2024-09-17T05:10:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4dee000640 10 client.8661 handle_command_reply: tid=0 2024-09-17T05:10:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 client.8661 shutdown 2024-09-17T05:10:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 msgr2=0x55df51c528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 0x55df51c528d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4df4004660 tx=0x7f4df40f9010 comp rx=0 tx=0).stop 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 shutdown_connections 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:6833/291864941,v1:172.21.15.16:6835/291864941] conn(0x55df5166e830 0x55df51c0e6a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df51a8b950 0x55df51c52e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df51a74ca0 0x55df51c528d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 --2- 172.21.15.16:0/3122287889 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x55df5169b460 0x55df51c52390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4dee000640 0 client.8661 ms_handle_reset on v2:172.21.15.16:6833/291864941 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 >> 172.21.15.16:0/3122287889 conn(0x55df51b61d90 msgr2=0x55df51bdafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 shutdown_connections 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 1 -- 172.21.15.16:0/3122287889 wait complete. 2024-09-17T05:10:15.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e171b0000 20 client.8661 trim_cache size 0 max 16384 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "23ee29f8-0841-48bc-a01a-8eb97d561cfc", 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 8526, 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1646784620000004, 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 110, 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 149, 2024-09-17T05:10:15.757 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 149, 2024-09-17T05:10:15.758 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 10.564120864 2024-09-17T05:10:15.758 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4de8073200 msgr2=0x7f4de8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4de8073200 0x7f4de8075710 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc007fd0 tx=0x7f4dfc007ec0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 msgr2=0x7f4e081fcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e081fcf40 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c10 tx=0x7f4df40a67b0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 shutdown_connections 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f4de8073200 0x7f4de8075710 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e0815f6f0 0x7f4e080ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08074e30 0x7f4e081fcf40 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 --2- 172.21.15.16:0/638165909 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f4e08074a20 0x7f4e080b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 >> 172.21.15.16:0/638165909 conn(0x7f4e080b2250 msgr2=0x7f4e0815cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 shutdown_connections 2024-09-17T05:10:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.752+0000 7f4e0fc00640 1 -- 172.21.15.16:0/638165909 wait complete. 2024-09-17T05:10:15.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:10:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 -- 172.21.15.16:0/2600055726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e340b5190 msgr2=0x7f6e340731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/2600055726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e340b5190 0x7f6e340731c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009bc0 tx=0x7f6e2c01cac0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 -- 172.21.15.16:0/2600055726 shutdown_connections 2024-09-17T05:10:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/2600055726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e340b5190 0x7f6e340731c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 -- 172.21.15.16:0/2600055726 >> 172.21.15.16:0/2600055726 conn(0x7f6e3406f900 msgr2=0x7f6e340b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 -- 172.21.15.16:0/2600055726 shutdown_connections 2024-09-17T05:10:15.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 -- 172.21.15.16:0/2600055726 wait complete. 2024-09-17T05:10:15.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 Processor -- start 2024-09-17T05:10:15.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.932+0000 7f6e3ae00640 1 -- start start 2024-09-17T05:10:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e34072db0 0x7f6e340d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 0x7f6e34150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3415bbc0 0x7f6e34150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34169f00 con 0x7f6e340b3960 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e34169d80 con 0x7f6e3415bbc0 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3416a080 con 0x7f6e34072db0 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 0x7f6e34150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 0x7f6e34150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45978/0 (socket says 172.21.15.16:45978) 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 -- 172.21.15.16:0/649517315 learned_addr learned my addr 172.21.15.16:0/649517315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e33e00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e34072db0 0x7f6e340d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e38e00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3415bbc0 0x7f6e34150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 -- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e34072db0 msgr2=0x7f6e340d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e34072db0 0x7f6e340d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 -- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3415bbc0 msgr2=0x7f6e34150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3415bbc0 0x7f6e34150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 -- 172.21.15.16:0/649517315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e340d0c60 con 0x7f6e340b3960 2024-09-17T05:10:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e2be00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 0x7f6e34150a30 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00c6e0 tx=0x7f6e1c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c016020 con 0x7f6e340b3960 2024-09-17T05:10:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e1c00a930 con 0x7f6e340b3960 2024-09-17T05:10:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00ad80 con 0x7f6e340b3960 2024-09-17T05:10:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3414a830 con 0x7f6e340b3960 2024-09-17T05:10:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341596e0 con 0x7f6e340b3960 2024-09-17T05:10:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.936+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e34159150 con 0x7f6e340b3960 2024-09-17T05:10:15.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.940+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6e1c019020 con 0x7f6e340b3960 2024-09-17T05:10:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.940+0000 7f6e30c00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e10073230 0x7f6e10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.940+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f6e1c094490 con 0x7f6e340b3960 2024-09-17T05:10:15.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.940+0000 7f6e33e00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e10073230 0x7f6e10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:15.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.940+0000 7f6e33e00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e10073230 0x7f6e10075740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c10 tx=0x7f6e24009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:15.944+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6e1c020020 con 0x7f6e340b3960 2024-09-17T05:10:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6e340731c0 con 0x7f6e340b3960 2024-09-17T05:10:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e30c00640 1 -- 172.21.15.16:0/649517315 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) ==== 53+0+73 (secure 0 0 0) 0x7f6e1c05ec30 con 0x7f6e340b3960 2024-09-17T05:10:16.162 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e10073230 msgr2=0x7f6e10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e10073230 0x7f6e10075740 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000c10 tx=0x7f6e24009040 comp rx=0 tx=0).stop 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 msgr2=0x7f6e34150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 0x7f6e34150a30 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00c6e0 tx=0x7f6e1c00cba0 comp rx=0 tx=0).stop 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 shutdown_connections 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f6e10073230 0x7f6e10075740 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e3415bbc0 0x7f6e34150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e340b3960 0x7f6e34150a30 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:16.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 --2- 172.21.15.16:0/649517315 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f6e34072db0 0x7f6e340d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 >> 172.21.15.16:0/649517315 conn(0x7f6e3406f900 msgr2=0x7f6e3414ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 shutdown_connections 2024-09-17T05:10:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16.160+0000 7f6e3ae00640 1 -- 172.21.15.16:0/649517315 wait complete. 2024-09-17T05:10:16.184 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:10:16.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:10:16.185 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:10:16.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.185 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:16.195 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 41877sec preferred_lft 41877sec 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 02:42:e5:22:25:ec brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:16.196 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:10:16.198 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:10:16.198 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:10:16.198 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:16.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.199 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:10:16.258 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-17T05:10:16.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.260 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-17T05:10:16.313 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-17T05:10:16.314 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:10:16.314 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-17T05:10:16.314 INFO:teuthology.orchestra.run.smithi016.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:10:16.314 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-17T05:10:16.314 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:10:16.314 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:10:16.314 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-17T05:10:16.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-17T05:10:16.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-17T05:10:16.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-17T05:10:16.315 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-17T05:10:16.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:10:16.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:10:16.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.452 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:10:16.461 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:10:16.461 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:10:16.462 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:10:16.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-17T05:10:16.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:10:16.528 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T05:10:16.585 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T05:10:16.585 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:10:16.585 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:10:16.636 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:10:16.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:10:16.741 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:10:16.741 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:10:16.741 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:10:16.759 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:10:16.760 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:10:16.760 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:10:16.760 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:10:16.760 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:invalid new device string format 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:10:16.770 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:10:16.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:10:16.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:16.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T05:10:16.795 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:10:16.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:10:16.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:10:16.899 INFO:teuthology.orchestra.run.smithi016.stdout:client7148 2024-09-17T05:10:16.900 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:16.900 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:16.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T05:10:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 -- 172.21.15.16:0/349560614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 msgr2=0x7f3044073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 --2- 172.21.15.16:0/349560614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 0x7f3044073a80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f303c009bf0 tx=0x7f303c01cbe0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 -- 172.21.15.16:0/349560614 shutdown_connections 2024-09-17T05:10:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 --2- 172.21.15.16:0/349560614 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 0x7f3044073a80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 -- 172.21.15.16:0/349560614 >> 172.21.15.16:0/349560614 conn(0x7f304406f900 msgr2=0x7f30440b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 -- 172.21.15.16:0/349560614 shutdown_connections 2024-09-17T05:10:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.108+0000 7f3049a00640 1 -- 172.21.15.16:0/349560614 wait complete. 2024-09-17T05:10:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 Processor -- start 2024-09-17T05:10:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 -- start start 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 0x7f3044154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304415f690 0x7f3044154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 0x7f30440c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3044169c40 con 0x7f30440b2f50 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3044169ac0 con 0x7f304415f690 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3049a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3044169dc0 con 0x7f304415faa0 2024-09-17T05:10:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 0x7f3044154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 0x7f3044154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59000/0 (socket says 172.21.15.16:59000) 2024-09-17T05:10:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 -- 172.21.15.16:0/3414793280 learned_addr learned my addr 172.21.15.16:0/3414793280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3043400640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 0x7f30440c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f303be00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304415f690 0x7f3044154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 -- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 msgr2=0x7f30440c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 0x7f30440c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 -- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304415f690 msgr2=0x7f3044154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304415f690 0x7f3044154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 -- 172.21.15.16:0/3414793280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30440c6870 con 0x7f30440b2f50 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f3042a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 0x7f3044154220 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f303400c400 tx=0x7f303400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.112+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034017070 con 0x7f30440b2f50 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3034005400 con 0x7f30440b2f50 2024-09-17T05:10:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3034003860 con 0x7f30440b2f50 2024-09-17T05:10:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30440c6b50 con 0x7f30440b2f50 2024-09-17T05:10:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30440c7110 con 0x7f30440b2f50 2024-09-17T05:10:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f304415ef80 con 0x7f30440b2f50 2024-09-17T05:10:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f303401a070 con 0x7f30440b2f50 2024-09-17T05:10:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f303aa00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3020073200 0x7f3020075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f30340937b0 con 0x7f30440b2f50 2024-09-17T05:10:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.116+0000 7f303be00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3020073200 0x7f3020075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.120+0000 7f303be00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3020073200 0x7f3020075710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f302c007f30 tx=0x7f302c007ec0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:17.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.120+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f303405dfe0 con 0x7f30440b2f50 2024-09-17T05:10:17.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.332+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f304406e2f0 con 0x7f30440b2f50 2024-09-17T05:10:17.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.332+0000 7f303aa00640 1 -- 172.21.15.16:0/3414793280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) ==== 53+0+73 (secure 0 0 0) 0x7f3034062a10 con 0x7f30440b2f50 2024-09-17T05:10:17.336 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T05:10:17.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3020073200 msgr2=0x7f3020075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:17.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3020073200 0x7f3020075710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f302c007f30 tx=0x7f302c007ec0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 msgr2=0x7f3044154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 0x7f3044154220 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f303400c400 tx=0x7f303400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 shutdown_connections 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3020073200 0x7f3020075710 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f304415faa0 0x7f30440c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f304415f690 0x7f3044154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 --2- 172.21.15.16:0/3414793280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30440b2f50 0x7f3044154220 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 >> 172.21.15.16:0/3414793280 conn(0x7f304406f900 msgr2=0x7f304406d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 shutdown_connections 2024-09-17T05:10:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:17.336+0000 7f3049a00640 1 -- 172.21.15.16:0/3414793280 wait complete. 2024-09-17T05:10:17.358 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T05:10:17.358 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T05:10:17.359 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:10:17.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.359 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-09-17T05:10:17.369 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T05:10:17.369 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T05:10:17.369 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T05:10:17.369 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:17.370 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 41870sec preferred_lft 41870sec 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::ec4:7aff:fe8f:ce89/64 scope link 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 02:42:b3:11:9c:6d brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T05:10:17.371 INFO:teuthology.orchestra.run.smithi104.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:10:17.372 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T05:10:17.373 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-09-17T05:10:17.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.374 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T05:10:17.440 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-09-17T05:10:17.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.442 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T05:10:17.497 INFO:teuthology.orchestra.run.smithi104.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:10:17.498 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:10:17.498 INFO:teuthology.orchestra.run.smithi104.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T05:10:17.498 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T05:10:17.498 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T05:10:17.498 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-09-17T05:10:17.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T05:10:17.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T05:10:17.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T05:10:17.498 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-09-17T05:10:17.592 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:10:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T05:10:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:10:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:10:17.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.647 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:10:17.655 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:10:17.656 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:10:17.656 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:10:17.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.657 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-09-17T05:10:17.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:10:17.728 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T05:10:17.781 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T05:10:17.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T05:10:17.781 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:10:17.835 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-17T05:10:17.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.836 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:10:17.936 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:10:17.937 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:10:17.937 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:10:17.955 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:10:17.955 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:10:17.955 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:10:17.955 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:10:17.955 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:10:17.964 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-17T05:10:17.964 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-17T05:10:17.964 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-09-17T05:10:17.964 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.104:3300,172.21.15.104:3301" 2024-09-17T05:10:17.964 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-17T05:10:17.964 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790" 2024-09-17T05:10:17.965 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.16:6789,172.21.15.104:6789,172.21.15.104:6790:/ 2024-09-17T05:10:17.965 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=23ee29f8-0841-48bc-a01a-8eb97d561cfc" will pass to kernel 2024-09-17T05:10:17.968 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-17T05:10:17.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:17.968 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T05:10:17.987 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-17T05:10:18.079 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:10:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T05:10:18.082 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-17T05:10:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T05:10:18.089 INFO:teuthology.orchestra.run.smithi104.stdout:client8077 2024-09-17T05:10:18.089 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:18.089 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:18.099 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T05:10:18.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:10:18.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T05:10:18.271 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T05:10:18.272 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:10:18.272 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:18.272 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:18.281 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:18.282 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:18.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:10:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 -- 172.21.15.16:0/2834017938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4070630 msgr2=0x7f3fb40cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2834017938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4070630 0x7f3fb40cb760 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009840 tx=0x7f3f9c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 -- 172.21.15.16:0/2834017938 shutdown_connections 2024-09-17T05:10:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2834017938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb4070630 0x7f3fb40cb760 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 -- 172.21.15.16:0/2834017938 >> 172.21.15.16:0/2834017938 conn(0x7f3fb40cd980 msgr2=0x7f3fb40cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 -- 172.21.15.16:0/2834017938 shutdown_connections 2024-09-17T05:10:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.548+0000 7f3fb9400640 1 -- 172.21.15.16:0/2834017938 wait complete. 2024-09-17T05:10:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 Processor -- start 2024-09-17T05:10:18.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 -- start start 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3fb406ce90 0x7f3fb40c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 0x7f3fb413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb40cf780 0x7f3fb40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb416aef0 con 0x7f3fb40cb350 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb416ad70 con 0x7f3fb40cf780 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb9400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb416b070 con 0x7f3fb406ce90 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fa9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 0x7f3fb413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fa9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 0x7f3fb413fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59010/0 (socket says 172.21.15.16:59010) 2024-09-17T05:10:18.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fa9e00640 1 -- 172.21.15.16:0/2419295842 learned_addr learned my addr 172.21.15.16:0/2419295842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb3200640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb40cf780 0x7f3fb40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:18.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.552+0000 7f3fb2800640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3fb406ce90 0x7f3fb40c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fa9e00640 1 -- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3fb406ce90 msgr2=0x7f3fb40c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fa9e00640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3fb406ce90 0x7f3fb40c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fa9e00640 1 -- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb40cf780 msgr2=0x7f3fb40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fa9e00640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb40cf780 0x7f3fb40cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fa9e00640 1 -- 172.21.15.16:0/2419295842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb40b3860 con 0x7f3fb40cb350 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fa9e00640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 0x7f3fb413fc50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009790 tx=0x7f3f9c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c0b2040 con 0x7f3fb40cb350 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f9c004470 con 0x7f3fb40cb350 2024-09-17T05:10:18.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c005220 con 0x7f3fb40cb350 2024-09-17T05:10:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb40b3b40 con 0x7f3fb40cb350 2024-09-17T05:10:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb40cf260 con 0x7f3fb40cb350 2024-09-17T05:10:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb40cb760 con 0x7f3fb40cb350 2024-09-17T05:10:18.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.556+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3f9c0bd060 con 0x7f3fb40cb350 2024-09-17T05:10:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.560+0000 7f3fab400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f90073200 0x7f3f90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.560+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f3f9c12dc80 con 0x7f3fb40cb350 2024-09-17T05:10:18.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.560+0000 7f3fb2800640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f90073200 0x7f3f90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.560+0000 7f3fb2800640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f90073200 0x7f3f90075710 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4007fd0 tx=0x7f3fa4009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:18.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.564+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3f9c016410 con 0x7f3fb40cb350 2024-09-17T05:10:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.748+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3fb406a180 con 0x7f3fb40cb350 2024-09-17T05:10:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.748+0000 7f3fab400640 1 -- 172.21.15.16:0/2419295842 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7f3f9c0f84b0 con 0x7f3fb40cb350 2024-09-17T05:10:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:10:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.748+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f90073200 msgr2=0x7f3f90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.748+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f90073200 0x7f3f90075710 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4007fd0 tx=0x7f3fa4009040 comp rx=0 tx=0).stop 2024-09-17T05:10:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.748+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 msgr2=0x7f3fb413fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.748+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 0x7f3fb413fc50 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009790 tx=0x7f3f9c0a56b0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 shutdown_connections 2024-09-17T05:10:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3f90073200 0x7f3f90075710 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fb40cf780 0x7f3fb40cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fb40cb350 0x7f3fb413fc50 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 --2- 172.21.15.16:0/2419295842 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3fb406ce90 0x7f3fb40c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 >> 172.21.15.16:0/2419295842 conn(0x7f3fb40cd980 msgr2=0x7f3fb406f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 shutdown_connections 2024-09-17T05:10:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:18.752+0000 7f3fb9400640 1 -- 172.21.15.16:0/2419295842 wait complete. 2024-09-17T05:10:18.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-17T05:10:18.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:18.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-17T05:10:18.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:10:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:18.799 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T05:10:18.854 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:10:18.854 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:18.854 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:18.864 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:18.865 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:18.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T05:10:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 -- 172.21.15.16:0/3476321607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b54004030 msgr2=0x7f1b54004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 --2- 172.21.15.16:0/3476321607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b54004030 0x7f1b54004810 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c2e0 tx=0x7f1b5801f700 comp rx=0 tx=0).stop 2024-09-17T05:10:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b71400640 1 -- 172.21.15.16:0/3476321607 reap_dead start 2024-09-17T05:10:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 -- 172.21.15.16:0/3476321607 shutdown_connections 2024-09-17T05:10:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 -- 172.21.15.16:0/3476321607 >> 172.21.15.16:0/3476321607 conn(0x7f1b6c0b2dd0 msgr2=0x7f1b6c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 -- 172.21.15.16:0/3476321607 shutdown_connections 2024-09-17T05:10:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 -- 172.21.15.16:0/3476321607 wait complete. 2024-09-17T05:10:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 Processor -- start 2024-09-17T05:10:19.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.076+0000 7f1b73400640 1 -- start start 2024-09-17T05:10:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 0x7f1b6c145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 0x7f1b6c1f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b6c156080 0x7f1b6c0b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0cd620 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0cd4a0 con 0x7f1b54004030 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c06e8f0 con 0x7f1b6c156080 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 0x7f1b6c1f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b70a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 0x7f1b6c145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b70a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 0x7f1b6c145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.16:53922/0 (socket says 172.21.15.16:53922) 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 0x7f1b6c1f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59032/0 (socket says 172.21.15.16:59032) 2024-09-17T05:10:19.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b70a00640 1 -- 172.21.15.16:0/1286626468 learned_addr learned my addr 172.21.15.16:0/1286626468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b71400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b6c156080 0x7f1b6c0b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 -- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b6c156080 msgr2=0x7f1b6c0b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b6c156080 0x7f1b6c0b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 -- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 msgr2=0x7f1b6c145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 0x7f1b6c145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 -- 172.21.15.16:0/1286626468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c1f71f0 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b70a00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 0x7f1b6c145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:10:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b6be00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 0x7f1b6c1f68c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1b580ad000 tx=0x7f1b58006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580b3030 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b580c0070 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c1f74d0 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58003600 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c0ca390 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.080+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c06ee50 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.084+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1b580037c0 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.084+0000 7f1b68c00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b4c073230 0x7f1b4c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.084+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f1b58132450 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.084+0000 7f1b70a00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b4c073230 0x7f1b4c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.084+0000 7f1b70a00640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b4c073230 0x7f1b4c075740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1b60012510 tx=0x7f1b600091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:19.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.088+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1b580194b0 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1b6c074350 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b68c00640 1 -- 172.21.15.16:0/1286626468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v149) ==== 80+0+0 (secure 0 0 0) 0x7f1b580fcbf0 con 0x7f1b6c0c9ca0 2024-09-17T05:10:19.278 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2024-09-17T05:10:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b4c073230 msgr2=0x7f1b4c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b4c073230 0x7f1b4c075740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1b60012510 tx=0x7f1b600091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 msgr2=0x7f1b6c1f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 0x7f1b6c1f68c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1b580ad000 tx=0x7f1b58006ce0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 shutdown_connections 2024-09-17T05:10:19.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b4c073230 0x7f1b4c075740 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b6c156080 0x7f1b6c0b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b6c0c9ca0 0x7f1b6c1f68c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 --2- 172.21.15.16:0/1286626468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b54004030 0x7f1b6c145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 >> 172.21.15.16:0/1286626468 conn(0x7f1b6c0b2dd0 msgr2=0x7f1b6c0c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 shutdown_connections 2024-09-17T05:10:19.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.276+0000 7f1b73400640 1 -- 172.21.15.16:0/1286626468 wait complete. 2024-09-17T05:10:19.301 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-17T05:10:19.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:19.301 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-17T05:10:19.327 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:10:19.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:19.328 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T05:10:19.377 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:10:19.378 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:19.378 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:19.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:10:19.387 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T05:10:19.387 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:19.387 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:19.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:10:19.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 -- 172.21.15.16:0/1295354385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330073240 msgr2=0x7f2330073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 --2- 172.21.15.16:0/1295354385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330073240 0x7f2330073650 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f2330071ce0 tx=0x7f232801ccf0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 -- 172.21.15.16:0/1295354385 shutdown_connections 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 --2- 172.21.15.16:0/1295354385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330073240 0x7f2330073650 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 --2- 172.21.15.16:0/1295354385 >> v2:172.21.15.104:3300/0 conn(0x7f23300b2f50 0x7f233015bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 -- 172.21.15.16:0/1295354385 >> 172.21.15.16:0/1295354385 conn(0x7f233006f900 msgr2=0x7f23300b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 -- 172.21.15.16:0/1295354385 shutdown_connections 2024-09-17T05:10:19.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.584+0000 7f2337000640 1 -- 172.21.15.16:0/1295354385 wait complete. 2024-09-17T05:10:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 Processor -- start 2024-09-17T05:10:19.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 -- start start 2024-09-17T05:10:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2330073240 0x7f233015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23300b2f50 0x7f233015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 0x7f23301f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2330168d10 con 0x7f233015c0a0 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2330168b90 con 0x7f23300b2f50 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2330168e90 con 0x7f2330073240 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 0x7f23301f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 0x7f23301f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59056/0 (socket says 172.21.15.16:59056) 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 -- 172.21.15.16:0/1458289145 learned_addr learned my addr 172.21.15.16:0/1458289145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2327e00640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23300b2f50 0x7f233015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f232fe00640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2330073240 0x7f233015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 -- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2330073240 msgr2=0x7f233015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2330073240 0x7f233015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 -- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23300b2f50 msgr2=0x7f233015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23300b2f50 0x7f233015b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 -- 172.21.15.16:0/1458289145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23300cbf80 con 0x7f233015c0a0 2024-09-17T05:10:19.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2335000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 0x7f23301f8110 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2328019900 tx=0x7f23280092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23280bb070 con 0x7f233015c0a0 2024-09-17T05:10:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2328004d40 con 0x7f233015c0a0 2024-09-17T05:10:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23280b2040 con 0x7f233015c0a0 2024-09-17T05:10:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2330141870 con 0x7f233015c0a0 2024-09-17T05:10:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.588+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23300c88f0 con 0x7f233015c0a0 2024-09-17T05:10:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.592+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2330073650 con 0x7f233015c0a0 2024-09-17T05:10:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.592+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f23280ad850 con 0x7f233015c0a0 2024-09-17T05:10:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.592+0000 7f232cc00640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f230c073200 0x7f230c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.592+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f232812e1a0 con 0x7f233015c0a0 2024-09-17T05:10:19.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.592+0000 7f232fe00640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f230c073200 0x7f230c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.592+0000 7f232fe00640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f230c073200 0x7f230c075710 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2320009b40 tx=0x7f23200091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:19.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.596+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23280f1db0 con 0x7f233015c0a0 2024-09-17T05:10:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23300ce630 con 0x7f233015c0a0 2024-09-17T05:10:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f232cc00640 1 -- 172.21.15.16:0/1458289145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5233 (secure 0 0 0) 0x7f23300cbf80 con 0x7f233015c0a0 2024-09-17T05:10:19.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:19.809 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":110,"btime":"2024-09-17T05:10:12:626935+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:10:09.017636+0000","modified":"2024-09-17T05:10:12.046191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/291864941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":291864941},{"type":"v1","addr":"172.21.15.16:6835","nonce":291864941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":11}]} 2024-09-17T05:10:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 110 2024-09-17T05:10:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f230c073200 msgr2=0x7f230c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f230c073200 0x7f230c075710 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2320009b40 tx=0x7f23200091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 msgr2=0x7f23301f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 0x7f23301f8110 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f2328019900 tx=0x7f23280092c0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 shutdown_connections 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f230c073200 0x7f230c075710 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f233015c0a0 0x7f23301f8110 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23300b2f50 0x7f233015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 --2- 172.21.15.16:0/1458289145 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2330073240 0x7f233015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 >> 172.21.15.16:0/1458289145 conn(0x7f233006f900 msgr2=0x7f2330151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 shutdown_connections 2024-09-17T05:10:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.808+0000 7f2337000640 1 -- 172.21.15.16:0/1458289145 wait complete. 2024-09-17T05:10:19.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 -- 172.21.15.16:0/2891684203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15c0a0 msgr2=0x7fa96c15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 --2- 172.21.15.16:0/2891684203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15c0a0 0x7fa96c15c4b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa96c15cc70 tx=0x7fa95801c830 comp rx=0 tx=0).stop 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 -- 172.21.15.16:0/2891684203 shutdown_connections 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 --2- 172.21.15.16:0/2891684203 >> v2:172.21.15.104:3300/0 conn(0x7fa96c15f150 0x7fa96c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 --2- 172.21.15.16:0/2891684203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c15c0a0 0x7fa96c15c4b0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 --2- 172.21.15.16:0/2891684203 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa96c0b2f50 0x7fa96c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 -- 172.21.15.16:0/2891684203 >> 172.21.15.16:0/2891684203 conn(0x7fa96c06f900 msgr2=0x7fa96c0b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 -- 172.21.15.16:0/2891684203 shutdown_connections 2024-09-17T05:10:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.988+0000 7fa971c00640 1 -- 172.21.15.16:0/2891684203 wait complete. 2024-09-17T05:10:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 Processor -- start 2024-09-17T05:10:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 -- start start 2024-09-17T05:10:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 0x7fa96c1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa96c15c0a0 0x7fa96c1f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa96c15f150 0x7fa96c1fa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c06e430 con 0x7fa96c0b2f50 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c06e2b0 con 0x7fa96c15f150 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa971c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c06e5b0 con 0x7fa96c15c0a0 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 0x7fa96c1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 0x7fa96c1501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59066/0 (socket says 172.21.15.16:59066) 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 -- 172.21.15.16:0/3350123972 learned_addr learned my addr 172.21.15.16:0/3350123972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96a000640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa96c15c0a0 0x7fa96c1f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96b400640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa96c15f150 0x7fa96c1fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 -- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa96c15c0a0 msgr2=0x7fa96c1f9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa96c15c0a0 0x7fa96c1f9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 -- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa96c15f150 msgr2=0x7fa96c1fa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa96c15f150 0x7fa96c1fa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 -- 172.21.15.16:0/3350123972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c151570 con 0x7fa96c0b2f50 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.992+0000 7fa96aa00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 0x7fa96c1501e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00e740 tx=0x7fa95c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00ccc0 con 0x7fa96c0b2f50 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c151850 con 0x7fa96c0b2f50 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa95c00ce60 con 0x7fa96c0b2f50 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c005280 con 0x7fa96c0b2f50 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c160460 con 0x7fa96c0b2f50 2024-09-17T05:10:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c160280 con 0x7fa96c0b2f50 2024-09-17T05:10:19.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:19.996+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa95c005420 con 0x7fa96c0b2f50 2024-09-17T05:10:20.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.000+0000 7fa962a00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa94c073200 0x7fa94c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.000+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7fa95c094cd0 con 0x7fa96c0b2f50 2024-09-17T05:10:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.000+0000 7fa96a000640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa94c073200 0x7fa94c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.000+0000 7fa96a000640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa94c073200 0x7fa94c075710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa9580af000 tx=0x7fa9580a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.004+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa96c160280 con 0x7fa96c0b2f50 2024-09-17T05:10:20.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.208+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa96c15c4b0 con 0x7fa96c0b2f50 2024-09-17T05:10:20.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.208+0000 7fa962a00640 1 -- 172.21.15.16:0/3350123972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5233 (secure 0 0 0) 0x7fa95c05f480 con 0x7fa96c0b2f50 2024-09-17T05:10:20.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:20.212 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":110,"btime":"2024-09-17T05:10:12:626935+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:10:09.017636+0000","modified":"2024-09-17T05:10:12.046191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/291864941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":291864941},{"type":"v1","addr":"172.21.15.16:6835","nonce":291864941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":11}]} 2024-09-17T05:10:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 110 2024-09-17T05:10:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa94c073200 msgr2=0x7fa94c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa94c073200 0x7fa94c075710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa9580af000 tx=0x7fa9580a7040 comp rx=0 tx=0).stop 2024-09-17T05:10:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 msgr2=0x7fa96c1501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 0x7fa96c1501e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00e740 tx=0x7fa95c00ec00 comp rx=0 tx=0).stop 2024-09-17T05:10:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 shutdown_connections 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa94c073200 0x7fa94c075710 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa96c15f150 0x7fa96c1fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa96c15c0a0 0x7fa96c1f9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 --2- 172.21.15.16:0/3350123972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96c0b2f50 0x7fa96c1501e0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 >> 172.21.15.16:0/3350123972 conn(0x7fa96c06f900 msgr2=0x7fa96c165ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 shutdown_connections 2024-09-17T05:10:20.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.212+0000 7fa971c00640 1 -- 172.21.15.16:0/3350123972 wait complete. 2024-09-17T05:10:20.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 -- 172.21.15.16:0/1679204163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840b2f50 msgr2=0x7fcd8415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/1679204163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840b2f50 0x7fcd8415bb60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fcd8406e960 tx=0x7fcd7401c830 comp rx=0 tx=0).stop 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 -- 172.21.15.16:0/1679204163 shutdown_connections 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/1679204163 >> v2:172.21.15.104:3300/0 conn(0x7fcd8415c0a0 0x7fcd8415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/1679204163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd840b2f50 0x7fcd8415bb60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 -- 172.21.15.16:0/1679204163 >> 172.21.15.16:0/1679204163 conn(0x7fcd8406f900 msgr2=0x7fcd840b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 -- 172.21.15.16:0/1679204163 shutdown_connections 2024-09-17T05:10:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 -- 172.21.15.16:0/1679204163 wait complete. 2024-09-17T05:10:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 Processor -- start 2024-09-17T05:10:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.392+0000 7fcd8ae00640 1 -- start start 2024-09-17T05:10:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd840b2f50 0x7fcd84158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd8415c0a0 0x7fcd84150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 0x7fcd84164690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd84169090 con 0x7fcd8415f150 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd84168f10 con 0x7fcd8415c0a0 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fcd84169210 con 0x7fcd840b2f50 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 0x7fcd84164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 0x7fcd84164690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59080/0 (socket says 172.21.15.16:59080) 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 -- 172.21.15.16:0/381433409 learned_addr learned my addr 172.21.15.16:0/381433409 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd7be00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd8415c0a0 0x7fcd84150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd83e00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd840b2f50 0x7fcd84158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 -- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd840b2f50 msgr2=0x7fcd84158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd840b2f50 0x7fcd84158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 -- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd8415c0a0 msgr2=0x7fcd84150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd8415c0a0 0x7fcd84150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 -- 172.21.15.16:0/381433409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd84150d40 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd88e00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 0x7fcd84164690 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c760 tx=0x7fcd7c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c003b80 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd7c003d20 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd840d07e0 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c00aab0 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd840cb960 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.396+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd840d0550 con 0x7fcd8415f150 2024-09-17T05:10:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.400+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd7c00ac50 con 0x7fcd8415f150 2024-09-17T05:10:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.400+0000 7fcd80c00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd60073200 0x7fcd60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.400+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7fcd7c094010 con 0x7fcd8415f150 2024-09-17T05:10:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.400+0000 7fcd83e00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd60073200 0x7fcd60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.400+0000 7fcd83e00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd60073200 0x7fcd60075710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcd740af000 tx=0x7fcd740a7040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.404+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd7c05e850 con 0x7fcd8415f150 2024-09-17T05:10:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.576+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd8415bb60 con 0x7fcd8415f150 2024-09-17T05:10:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd80c00640 1 -- 172.21.15.16:0/381433409 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16936 (secure 0 0 0) 0x7fcd7c063280 con 0x7fcd8415f150 2024-09-17T05:10:20.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:20.581 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":149,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:10.590092+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:10:06.755147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-17T05:10:07.983444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd60073200 msgr2=0x7fcd60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd60073200 0x7fcd60075710 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcd740af000 tx=0x7fcd740a7040 comp rx=0 tx=0).stop 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 msgr2=0x7fcd84164690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 0x7fcd84164690 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c760 tx=0x7fcd7c00cc20 comp rx=0 tx=0).stop 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 shutdown_connections 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fcd60073200 0x7fcd60075710 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd8415f150 0x7fcd84164690 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd8415c0a0 0x7fcd84150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 --2- 172.21.15.16:0/381433409 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fcd840b2f50 0x7fcd84158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 >> 172.21.15.16:0/381433409 conn(0x7fcd8406f900 msgr2=0x7fcd841514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 shutdown_connections 2024-09-17T05:10:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.580+0000 7fcd8ae00640 1 -- 172.21.15.16:0/381433409 wait complete. 2024-09-17T05:10:20.604 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:10:20.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 -- 172.21.15.16:0/2478669196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 msgr2=0x7f47d415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2478669196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 0x7f47d415cab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009870 tx=0x7f47c401c860 comp rx=0 tx=0).stop 2024-09-17T05:10:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 -- 172.21.15.16:0/2478669196 shutdown_connections 2024-09-17T05:10:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2478669196 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 0x7f47d415cab0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 -- 172.21.15.16:0/2478669196 >> 172.21.15.16:0/2478669196 conn(0x7f47d40d2270 msgr2=0x7f47d40d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 -- 172.21.15.16:0/2478669196 shutdown_connections 2024-09-17T05:10:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.760+0000 7f47d8c00640 1 -- 172.21.15.16:0/2478669196 wait complete. 2024-09-17T05:10:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 Processor -- start 2024-09-17T05:10:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 -- start start 2024-09-17T05:10:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47d40c9440 0x7f47d406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 0x7f47d41f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 0x7f47d41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f47d416b0a0 con 0x7f47d40d1580 2024-09-17T05:10:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47d416af20 con 0x7f47d40c9440 2024-09-17T05:10:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d8c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f47d416b220 con 0x7f47d40cc8c0 2024-09-17T05:10:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.764+0000 7f47d2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 0x7f47d41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 0x7f47d41562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59086/0 (socket says 172.21.15.16:59086) 2024-09-17T05:10:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 -- 172.21.15.16:0/2939473062 learned_addr learned my addr 172.21.15.16:0/2939473062 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d1e00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47d40c9440 0x7f47d406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47cbe00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 0x7f47d41f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 -- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 msgr2=0x7f47d41f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 0x7f47d41f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 -- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47d40c9440 msgr2=0x7f47d406dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47d40c9440 0x7f47d406dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 -- 172.21.15.16:0/2939473062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d406e760 con 0x7f47d40d1580 2024-09-17T05:10:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d1e00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47d40c9440 0x7f47d406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T05:10:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d2800640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 0x7f47d41562c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00cb10 tx=0x7f47cc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc016020 con 0x7f47d40d1580 2024-09-17T05:10:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47cc00ce50 con 0x7f47d40d1580 2024-09-17T05:10:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00ae00 con 0x7f47d40d1580 2024-09-17T05:10:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d406ea40 con 0x7f47d40d1580 2024-09-17T05:10:20.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d406f000 con 0x7f47d40d1580 2024-09-17T05:10:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d415cab0 con 0x7f47d40d1580 2024-09-17T05:10:20.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.768+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f47cc019020 con 0x7f47d40d1580 2024-09-17T05:10:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.772+0000 7f47ca000640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f47b0073200 0x7f47b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.772+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f47cc0943a0 con 0x7f47d40d1580 2024-09-17T05:10:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.772+0000 7f47d1e00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f47b0073200 0x7f47b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.772+0000 7f47d1e00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f47b0073200 0x7f47b0075710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f47c40af000 tx=0x7f47c401b0a0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.776+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f47cc020020 con 0x7f47d40d1580 2024-09-17T05:10:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47d406e4f0 con 0x7f47d40d1580 2024-09-17T05:10:20.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47ca000640 1 -- 172.21.15.16:0/2939473062 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5233 (secure 0 0 0) 0x7f47cc05ec70 con 0x7f47d40d1580 2024-09-17T05:10:20.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:20.986 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":110,"btime":"2024-09-17T05:10:12:626935+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:10:09.017636+0000","modified":"2024-09-17T05:10:12.046191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/291864941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":291864941},{"type":"v1","addr":"172.21.15.16:6835","nonce":291864941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":11}]} 2024-09-17T05:10:20.986 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 110 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f47b0073200 msgr2=0x7f47b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f47b0073200 0x7f47b0075710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f47c40af000 tx=0x7f47c401b0a0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 msgr2=0x7f47d41562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 0x7f47d41562c0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00cb10 tx=0x7f47cc00cc00 comp rx=0 tx=0).stop 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 shutdown_connections 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f47b0073200 0x7f47b0075710 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40d1580 0x7f47d41562c0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f47d40cc8c0 0x7f47d41f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 --2- 172.21.15.16:0/2939473062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47d40c9440 0x7f47d406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 >> 172.21.15.16:0/2939473062 conn(0x7f47d40d2270 msgr2=0x7f47d40cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:20.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 shutdown_connections 2024-09-17T05:10:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:20.984+0000 7f47d8c00640 1 -- 172.21.15.16:0/2939473062 wait complete. 2024-09-17T05:10:21.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 -- 172.21.15.16:0/2395442225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6100073ec0 msgr2=0x7f61000742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 --2- 172.21.15.16:0/2395442225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6100073ec0 0x7f61000742d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6100157010 tx=0x7f60f001c830 comp rx=0 tx=0).stop 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 -- 172.21.15.16:0/2395442225 shutdown_connections 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 --2- 172.21.15.16:0/2395442225 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f610006ce90 0x7f610006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 --2- 172.21.15.16:0/2395442225 >> v2:172.21.15.104:3300/0 conn(0x7f610015ca90 0x7f610015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 --2- 172.21.15.16:0/2395442225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6100073ec0 0x7f61000742d0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 -- 172.21.15.16:0/2395442225 >> 172.21.15.16:0/2395442225 conn(0x7f61000d10c0 msgr2=0x7f61000d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:21.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 -- 172.21.15.16:0/2395442225 shutdown_connections 2024-09-17T05:10:21.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.164+0000 7f6105e00640 1 -- 172.21.15.16:0/2395442225 wait complete. 2024-09-17T05:10:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 Processor -- start 2024-09-17T05:10:21.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 -- start start 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 0x7f610014cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6100073ec0 0x7f610014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f610015ca90 0x7f610015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61000d0b00 con 0x7f610006ce90 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61000d0980 con 0x7f6100073ec0 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f61000d0c80 con 0x7f610015ca90 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 0x7f610014cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 0x7f610014cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59108/0 (socket says 172.21.15.16:59108) 2024-09-17T05:10:21.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 -- 172.21.15.16:0/3671126806 learned_addr learned my addr 172.21.15.16:0/3671126806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60f6000640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6100073ec0 0x7f610014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60ff400640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f610015ca90 0x7f610015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 -- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f610015ca90 msgr2=0x7f610015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f610015ca90 0x7f610015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 -- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6100073ec0 msgr2=0x7f610014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6100073ec0 0x7f610014fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 -- 172.21.15.16:0/3671126806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6100150150 con 0x7f610006ce90 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60fea00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 0x7f610014cc00 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f60f001aee0 tx=0x7f60f0005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f00b2040 con 0x7f610006ce90 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60f0004230 con 0x7f610006ce90 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0005070 con 0x7f610006ce90 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610014eb30 con 0x7f610006ce90 2024-09-17T05:10:21.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.168+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610014f0f0 con 0x7f610006ce90 2024-09-17T05:10:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.172+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6100166e20 con 0x7f610006ce90 2024-09-17T05:10:21.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.172+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f60f0005230 con 0x7f610006ce90 2024-09-17T05:10:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.172+0000 7f60f7e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60dc073200 0x7f60dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.172+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f60f012e1b0 con 0x7f610006ce90 2024-09-17T05:10:21.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.172+0000 7f60f6000640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60dc073200 0x7f60dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.172+0000 7f60f6000640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60dc073200 0x7f60dc075710 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f610015a2d0 tx=0x7f60e80091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:21.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.176+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f60f00164b0 con 0x7f610006ce90 2024-09-17T05:10:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.384+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6100166b40 con 0x7f610006ce90 2024-09-17T05:10:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.384+0000 7f60f7e00640 1 -- 172.21.15.16:0/3671126806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5233 (secure 0 0 0) 0x7f60f00f9030 con 0x7f610006ce90 2024-09-17T05:10:21.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:21.387 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":110,"btime":"2024-09-17T05:10:12:626935+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T05:10:09.017636+0000","modified":"2024-09-17T05:10:12.046191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"a","rank":0,"incarnation":106,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/291864941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":291864941},{"type":"v1","addr":"172.21.15.16:6835","nonce":291864941}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8526,"qdb_cluster":[8526]},"id":11}]} 2024-09-17T05:10:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 110 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60dc073200 msgr2=0x7f60dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60dc073200 0x7f60dc075710 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f610015a2d0 tx=0x7f60e80091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 msgr2=0x7f610014cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 0x7f610014cc00 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f60f001aee0 tx=0x7f60f0005de0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 shutdown_connections 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f60dc073200 0x7f60dc075710 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f610015ca90 0x7f610015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6100073ec0 0x7f610014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 --2- 172.21.15.16:0/3671126806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f610006ce90 0x7f610014cc00 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 >> 172.21.15.16:0/3671126806 conn(0x7f61000d10c0 msgr2=0x7f610014d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 shutdown_connections 2024-09-17T05:10:21.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.388+0000 7f6105e00640 1 -- 172.21.15.16:0/3671126806 wait complete. 2024-09-17T05:10:21.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.564+0000 7f2f07200640 1 -- 172.21.15.16:0/1343562020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 msgr2=0x7f2f000d0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.564+0000 7f2f07200640 1 --2- 172.21.15.16:0/1343562020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f000d0f90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2f000cfdd0 tx=0x7f2ef801cc60 comp rx=0 tx=0).stop 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 -- 172.21.15.16:0/1343562020 shutdown_connections 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 --2- 172.21.15.16:0/1343562020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f000d0f90 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 --2- 172.21.15.16:0/1343562020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f001608d0 0x7f2f000d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 --2- 172.21.15.16:0/1343562020 >> v2:172.21.15.104:3300/0 conn(0x7f2f0015c6f0 0x7f2f00160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 -- 172.21.15.16:0/1343562020 >> 172.21.15.16:0/1343562020 conn(0x7f2f000cf380 msgr2=0x7f2f000d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 -- 172.21.15.16:0/1343562020 shutdown_connections 2024-09-17T05:10:21.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 -- 172.21.15.16:0/1343562020 wait complete. 2024-09-17T05:10:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 Processor -- start 2024-09-17T05:10:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.568+0000 7f2f07200640 1 -- start start 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f00153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f0015c6f0 0x7f2f0006eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f001608d0 0x7f2f001598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b2c40 con 0x7f2f000d0b80 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000b2ac0 con 0x7f2f0015c6f0 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2f000b2dc0 con 0x7f2f001608d0 2024-09-17T05:10:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f00153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f00153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59128/0 (socket says 172.21.15.16:59128) 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 -- 172.21.15.16:0/1073525006 learned_addr learned my addr 172.21.15.16:0/1073525006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2ef7e00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f0015c6f0 0x7f2f0006eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f05200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f001608d0 0x7f2f001598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 -- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f001608d0 msgr2=0x7f2f001598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f001608d0 0x7f2f001598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 -- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f0015c6f0 msgr2=0x7f2f0006eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f0015c6f0 0x7f2f0006eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 -- 172.21.15.16:0/1073525006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0015a340 con 0x7f2f000d0b80 2024-09-17T05:10:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2effe00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f00153060 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e7c0 tx=0x7f2ef000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000ccc0 con 0x7f2f000d0b80 2024-09-17T05:10:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef000ee70 con 0x7f2f000d0b80 2024-09-17T05:10:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0015a620 con 0x7f2f000d0b80 2024-09-17T05:10:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0010690 con 0x7f2f000d0b80 2024-09-17T05:10:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0015abe0 con 0x7f2f000d0b80 2024-09-17T05:10:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f000b3300 con 0x7f2f000d0b80 2024-09-17T05:10:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.572+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ef0010830 con 0x7f2f000d0b80 2024-09-17T05:10:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.576+0000 7f2efcc00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2edc073200 0x7f2edc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.576+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f2ef0094e60 con 0x7f2f000d0b80 2024-09-17T05:10:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.576+0000 7f2ef7e00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2edc073200 0x7f2edc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.576+0000 7f2ef7e00640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2edc073200 0x7f2edc075710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8005d10 tx=0x7f2ee8005c80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:21.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.580+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ef005fc50 con 0x7f2f000d0b80 2024-09-17T05:10:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.756+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f00159380 con 0x7f2f000d0b80 2024-09-17T05:10:21.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.756+0000 7f2efcc00640 1 -- 172.21.15.16:0/1073525006 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16936 (secure 0 0 0) 0x7f2ef005f550 con 0x7f2f000d0b80 2024-09-17T05:10:21.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:21.760 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":149,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:10.590092+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-09-17T05:10:06.755147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-09-17T05:10:07.983444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2edc073200 msgr2=0x7f2edc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2edc073200 0x7f2edc075710 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8005d10 tx=0x7f2ee8005c80 comp rx=0 tx=0).stop 2024-09-17T05:10:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 msgr2=0x7f2f00153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f00153060 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000e7c0 tx=0x7f2ef000ec80 comp rx=0 tx=0).stop 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 shutdown_connections 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2edc073200 0x7f2edc075710 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2f001608d0 0x7f2f001598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f0015c6f0 0x7f2f0006eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 --2- 172.21.15.16:0/1073525006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f000d0b80 0x7f2f00153060 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 >> 172.21.15.16:0/1073525006 conn(0x7f2f000cf380 msgr2=0x7f2f001552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 shutdown_connections 2024-09-17T05:10:21.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.760+0000 7f2f07200640 1 -- 172.21.15.16:0/1073525006 wait complete. 2024-09-17T05:10:21.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T05:10:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5edb800640 1 -- 172.21.15.16:0/2306048757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ebc0061e0 msgr2=0x7f5ebc00cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5edb800640 1 --2- 172.21.15.16:0/2306048757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ebc0061e0 0x7f5ebc00cae0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ce3f0 tx=0x7f5ecc01cba0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5ed9800640 1 -- 172.21.15.16:0/2306048757 reap_dead start 2024-09-17T05:10:21.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5edb800640 1 -- 172.21.15.16:0/2306048757 shutdown_connections 2024-09-17T05:10:21.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5edb800640 1 -- 172.21.15.16:0/2306048757 >> 172.21.15.16:0/2306048757 conn(0x7f5ed40b2250 msgr2=0x7f5ed40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:21.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5edb800640 1 -- 172.21.15.16:0/2306048757 shutdown_connections 2024-09-17T05:10:21.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.944+0000 7f5edb800640 1 -- 172.21.15.16:0/2306048757 wait complete. 2024-09-17T05:10:21.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 Processor -- start 2024-09-17T05:10:21.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 -- start start 2024-09-17T05:10:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ebc0061e0 0x7f5ed4151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 0x7f5ed40b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed40aece0 0x7f5ed40af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40cf620 con 0x7f5ed40b45d0 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40cf4a0 con 0x7f5ed40aece0 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed40cf7a0 con 0x7f5ebc0061e0 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 0x7f5ed40b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 0x7f5ed40b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59144/0 (socket says 172.21.15.16:59144) 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 -- 172.21.15.16:0/3642907622 learned_addr learned my addr 172.21.15.16:0/3642907622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed8e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ebc0061e0 0x7f5ed4151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed9800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed40aece0 0x7f5ed40af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 -- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ebc0061e0 msgr2=0x7f5ed4151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ebc0061e0 0x7f5ed4151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 -- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed40aece0 msgr2=0x7f5ed40af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:21.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed40aece0 0x7f5ed40af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:21.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 -- 172.21.15.16:0/3642907622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed40af630 con 0x7f5ed40b45d0 2024-09-17T05:10:21.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed8e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ebc0061e0 0x7f5ed4151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:21.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed3e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 0x7f5ed40b49e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0011ab0 tx=0x7f5ec0011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001e070 con 0x7f5ed40b45d0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec000fc70 con 0x7f5ed40b45d0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0019670 con 0x7f5ed40b45d0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.948+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41fbaf0 con 0x7f5ed40b45d0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.952+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41fc0b0 con 0x7f5ed40b45d0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.952+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed4073a70 con 0x7f5ed40b45d0 2024-09-17T05:10:21.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.952+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ec0021070 con 0x7f5ed40b45d0 2024-09-17T05:10:21.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.952+0000 7f5ed0c00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:21.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.952+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f5ec0094cf0 con 0x7f5ed40b45d0 2024-09-17T05:10:21.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.952+0000 7f5ed8e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:21.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.956+0000 7f5ed8e00640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8001250 tx=0x7f5ec8006730 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:21.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:21.956+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ed40af630 con 0x7f5ed40b45d0 2024-09-17T05:10:22.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.172+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5ed415b5d0 con 0x7f5ed40b45d0 2024-09-17T05:10:22.176 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-09-17T05:10:22.176+0000 7fdc8f200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T05:10:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.176+0000 7f5ed0c00640 1 -- 172.21.15.16:0/3642907622 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) ==== 161+0+0 (secure 0 0 0) 0x7f5ec000fe60 con 0x7f5ed40b45d0 2024-09-17T05:10:22.179 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T05:10:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 msgr2=0x7f5eb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8001250 tx=0x7f5ec8006730 comp rx=0 tx=0).stop 2024-09-17T05:10:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 msgr2=0x7f5ed40b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:22.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 0x7f5ed40b49e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0011ab0 tx=0x7f5ec0011f70 comp rx=0 tx=0).stop 2024-09-17T05:10:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 shutdown_connections 2024-09-17T05:10:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5eb4073200 0x7f5eb4075710 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ed40aece0 0x7f5ed40af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b45d0 0x7f5ed40b49e0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 --2- 172.21.15.16:0/3642907622 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5ebc0061e0 0x7f5ed4151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 >> 172.21.15.16:0/3642907622 conn(0x7f5ed40b2250 msgr2=0x7f5ed406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 shutdown_connections 2024-09-17T05:10:22.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.180+0000 7f5edb800640 1 -- 172.21.15.16:0/3642907622 wait complete. 2024-09-17T05:10:22.188 INFO:tasks.ceph.mds.a.smithi016.stderr: -2184> 2024-09-17T05:10:11.573+0000 7f6a18400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T05:10:22.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T05:10:22.309 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 -- 172.21.15.16:0/1698140284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200d02d0 msgr2=0x7fad200cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 --2- 172.21.15.16:0/1698140284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200d02d0 0x7fad200cf140 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fad08009930 tx=0x7fad0801c3c0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 -- 172.21.15.16:0/1698140284 shutdown_connections 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 --2- 172.21.15.16:0/1698140284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad200d02d0 0x7fad200cf140 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 -- 172.21.15.16:0/1698140284 >> 172.21.15.16:0/1698140284 conn(0x7fad200b42d0 msgr2=0x7fad200b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 -- 172.21.15.16:0/1698140284 shutdown_connections 2024-09-17T05:10:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.368+0000 7fad26000640 1 -- 172.21.15.16:0/1698140284 wait complete. 2024-09-17T05:10:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 Processor -- start 2024-09-17T05:10:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 -- start start 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad200cf680 0x7fad2013c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad200d02d0 0x7fad201fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 0x7fad201fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad2016c3e0 con 0x7fad20159f70 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad2016c260 con 0x7fad200cf680 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fad2016c560 con 0x7fad200d02d0 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 0x7fad201fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 0x7fad201fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59176/0 (socket says 172.21.15.16:59176) 2024-09-17T05:10:22.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 -- 172.21.15.16:0/2340983515 learned_addr learned my addr 172.21.15.16:0/2340983515 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad16a00640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad200d02d0 0x7fad201fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1f400640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad200cf680 0x7fad2013c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 -- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad200d02d0 msgr2=0x7fad201fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad200d02d0 0x7fad201fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 -- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad200cf680 msgr2=0x7fad2013c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad200cf680 0x7fad2013c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 -- 172.21.15.16:0/2340983515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2013ca50 con 0x7fad20159f70 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1f400640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad200cf680 0x7fad2013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad1fe00640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 0x7fad201fd860 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fad1800cad0 tx=0x7fad1800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18017070 con 0x7fad20159f70 2024-09-17T05:10:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad1800a7c0 con 0x7fad20159f70 2024-09-17T05:10:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1800ac10 con 0x7fad20159f70 2024-09-17T05:10:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad20153110 con 0x7fad20159f70 2024-09-17T05:10:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.372+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201536d0 con 0x7fad20159f70 2024-09-17T05:10:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.376+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad201538d0 con 0x7fad20159f70 2024-09-17T05:10:22.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.376+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad1801a070 con 0x7fad20159f70 2024-09-17T05:10:22.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.376+0000 7fad17e00640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7facfc073200 0x7facfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:22.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.376+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6187+0+0 (secure 0 0 0) 0x7fad18093d60 con 0x7fad20159f70 2024-09-17T05:10:22.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.376+0000 7fad1f400640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7facfc073200 0x7facfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.376+0000 7fad1f400640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7facfc073200 0x7facfc075710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f70 tx=0x7fad10009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.380+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad2013ca50 con 0x7fad20159f70 2024-09-17T05:10:22.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:22.588+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fad200cf140 con 0x7fad20159f70 2024-09-17T05:10:23.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad17e00640 1 -- 172.21.15.16:0/2340983515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v112) ==== 104+0+0 (secure 0 0 0) 0x7fad200cfa90 con 0x7fad20159f70 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7facfc073200 msgr2=0x7facfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7facfc073200 0x7facfc075710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fad10000f70 tx=0x7fad10009040 comp rx=0 tx=0).stop 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 msgr2=0x7fad201fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 0x7fad201fd860 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fad1800cad0 tx=0x7fad1800cf90 comp rx=0 tx=0).stop 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 shutdown_connections 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7facfc073200 0x7facfc075710 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad20159f70 0x7fad201fd860 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad200d02d0 0x7fad201fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 --2- 172.21.15.16:0/2340983515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad200cf680 0x7fad2013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 >> 172.21.15.16:0/2340983515 conn(0x7fad200b42d0 msgr2=0x7fad200b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 shutdown_connections 2024-09-17T05:10:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.180+0000 7fad26000640 1 -- 172.21.15.16:0/2340983515 wait complete. 2024-09-17T05:10:23.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T05:10:23.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 -- 172.21.15.16:0/387951869 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38480cfa10 msgr2=0x7f384806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 --2- 172.21.15.16:0/387951869 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38480cfa10 0x7f384806eeb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f383c009840 tx=0x7f383c01c740 comp rx=0 tx=0).stop 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 -- 172.21.15.16:0/387951869 shutdown_connections 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 --2- 172.21.15.16:0/387951869 >> v2:172.21.15.104:3300/0 conn(0x7f3848156a90 0x7f38481601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 --2- 172.21.15.16:0/387951869 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f38480cfa10 0x7f384806eeb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 -- 172.21.15.16:0/387951869 >> 172.21.15.16:0/387951869 conn(0x7f38480d0050 msgr2=0x7f38480d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 -- 172.21.15.16:0/387951869 shutdown_connections 2024-09-17T05:10:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.368+0000 7f384f600640 1 -- 172.21.15.16:0/387951869 wait complete. 2024-09-17T05:10:23.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 Processor -- start 2024-09-17T05:10:23.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 -- start start 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480cfa10 0x7f384814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3848156a90 0x7f3848157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 0x7f384815e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38480cf270 con 0x7f38481606e0 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f38481570b0 con 0x7f38480cfa10 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f38480cf3f0 con 0x7f3848156a90 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 0x7f384815e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 0x7f384815e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59184/0 (socket says 172.21.15.16:59184) 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 -- 172.21.15.16:0/2611995110 learned_addr learned my addr 172.21.15.16:0/2611995110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:23.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f3847e00640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3848156a90 0x7f3848157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384cc00640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480cfa10 0x7f384814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 -- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3848156a90 msgr2=0x7f3848157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3848156a90 0x7f3848157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 -- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480cfa10 msgr2=0x7f384814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480cfa10 0x7f384814f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 -- 172.21.15.16:0/2611995110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3848157310 con 0x7f38481606e0 2024-09-17T05:10:23.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384d600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 0x7f384815e4c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f384000cae0 tx=0x7f384000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384000a9f0 con 0x7f38481606e0 2024-09-17T05:10:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3840017070 con 0x7f38481606e0 2024-09-17T05:10:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38481f6b50 con 0x7f38481606e0 2024-09-17T05:10:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384000ce20 con 0x7f38481606e0 2024-09-17T05:10:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.372+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38481f70f0 con 0x7f38481606e0 2024-09-17T05:10:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.376+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38481f7a90 con 0x7f38481606e0 2024-09-17T05:10:23.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.376+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f384001c460 con 0x7f38481606e0 2024-09-17T05:10:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.376+0000 7f3844c00640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3828073200 0x7f3828075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.376+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6187+0+0 (secure 0 0 0) 0x7f38400943b0 con 0x7f38481606e0 2024-09-17T05:10:23.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.376+0000 7f384cc00640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3828073200 0x7f3828075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:23.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.380+0000 7f384cc00640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3828073200 0x7f3828075710 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f383c0ae000 tx=0x7f383c01af80 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:23.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.380+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3848074f50 con 0x7f38481606e0 2024-09-17T05:10:23.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:23.556+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f38480adfc0 con 0x7f38481606e0 2024-09-17T05:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.188+0000 7f3844c00640 1 -- 172.21.15.16:0/2611995110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7f384005eb10 con 0x7f38481606e0 2024-09-17T05:10:24.189 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3828073200 msgr2=0x7f3828075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3828073200 0x7f3828075710 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f383c0ae000 tx=0x7f383c01af80 comp rx=0 tx=0).stop 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 msgr2=0x7f384815e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 0x7f384815e4c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f384000cae0 tx=0x7f384000cbd0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 shutdown_connections 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3828073200 0x7f3828075710 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38481606e0 0x7f384815e4c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3848156a90 0x7f3848157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 --2- 172.21.15.16:0/2611995110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38480cfa10 0x7f384814f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 >> 172.21.15.16:0/2611995110 conn(0x7f38480d0050 msgr2=0x7f38480b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 shutdown_connections 2024-09-17T05:10:24.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.192+0000 7f384f600640 1 -- 172.21.15.16:0/2611995110 wait complete. 2024-09-17T05:10:24.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T05:10:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf83400640 1 -- 172.21.15.16:0/2365511306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 msgr2=0x7fbf64004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:24.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf83400640 1 --2- 172.21.15.16:0/2365511306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf64004070 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fbf70006fa0 tx=0x7fbf7001ceb0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf81400640 1 -- 172.21.15.16:0/2365511306 reap_dead start 2024-09-17T05:10:24.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf83400640 1 -- 172.21.15.16:0/2365511306 shutdown_connections 2024-09-17T05:10:24.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf83400640 1 -- 172.21.15.16:0/2365511306 >> 172.21.15.16:0/2365511306 conn(0x7fbf7c0d0050 msgr2=0x7fbf7c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:24.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf83400640 1 -- 172.21.15.16:0/2365511306 shutdown_connections 2024-09-17T05:10:24.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.384+0000 7fbf83400640 1 -- 172.21.15.16:0/2365511306 wait complete. 2024-09-17T05:10:24.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 Processor -- start 2024-09-17T05:10:24.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 -- start start 2024-09-17T05:10:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf7c1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbf7c1489c0 0x7fbf7c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf7c0703d0 0x7fbf7c1f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c1674f0 con 0x7fbf64003c20 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c167370 con 0x7fbf7c0703d0 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf83400640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c167670 con 0x7fbf7c1489c0 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf7c1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf7c1483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59210/0 (socket says 172.21.15.16:59210) 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 -- 172.21.15.16:0/556859423 learned_addr learned my addr 172.21.15.16:0/556859423 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf7be00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbf7c1489c0 0x7fbf7c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf81400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf7c0703d0 0x7fbf7c1f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 -- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbf7c1489c0 msgr2=0x7fbf7c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbf7c1489c0 0x7fbf7c06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 -- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf7c0703d0 msgr2=0x7fbf7c1f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf7c0703d0 0x7fbf7c1f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf80a00640 1 -- 172.21.15.16:0/556859423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c0707e0 con 0x7fbf64003c20 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf81400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf7c0703d0 0x7fbf7c1f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.388+0000 7fbf7be00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbf7c1489c0 0x7fbf7c06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf80a00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf7c1483d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbf7001d390 tx=0x7fbf70006cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf70006440 con 0x7fbf64003c20 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf70003600 con 0x7fbf64003c20 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c1f67e0 con 0x7fbf64003c20 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf700039d0 con 0x7fbf64003c20 2024-09-17T05:10:24.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c14e090 con 0x7fbf64003c20 2024-09-17T05:10:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c073be0 con 0x7fbf64003c20 2024-09-17T05:10:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf70003b70 con 0x7fbf64003c20 2024-09-17T05:10:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf78c00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf5c073200 0x7fbf5c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.392+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5749+0+0 (secure 0 0 0) 0x7fbf7012efe0 con 0x7fbf64003c20 2024-09-17T05:10:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.396+0000 7fbf7be00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf5c073200 0x7fbf5c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.396+0000 7fbf7be00640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf5c073200 0x7fbf5c075710 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c070fa0 tx=0x7fbf68006ec0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.396+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf7c0707e0 con 0x7fbf64003c20 2024-09-17T05:10:24.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:24.572+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbf7c0d0500 con 0x7fbf64003c20 2024-09-17T05:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.188+0000 7fbf78c00640 1 -- 172.21.15.16:0/556859423 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7fbf70016c90 con 0x7fbf64003c20 2024-09-17T05:10:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-17T05:10:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.196+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf5c073200 msgr2=0x7fbf5c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.196+0000 7fbf83400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf5c073200 0x7fbf5c075710 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c070fa0 tx=0x7fbf68006ec0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.196+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 msgr2=0x7fbf7c1483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.196+0000 7fbf83400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf7c1483d0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fbf7001d390 tx=0x7fbf70006cd0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 shutdown_connections 2024-09-17T05:10:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fbf5c073200 0x7fbf5c075710 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf7c0703d0 0x7fbf7c1f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fbf7c1489c0 0x7fbf7c06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 --2- 172.21.15.16:0/556859423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf64003c20 0x7fbf7c1483d0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 >> 172.21.15.16:0/556859423 conn(0x7fbf7c0d0050 msgr2=0x7fbf7c0add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 shutdown_connections 2024-09-17T05:10:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.200+0000 7fbf83400640 1 -- 172.21.15.16:0/556859423 wait complete. 2024-09-17T05:10:25.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dd4a00640 1 -- 172.21.15.16:0/259840123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4003fe0 msgr2=0x7f3db4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/259840123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3db4003fe0 0x7f3db4004810 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db801cf30 comp rx=0 tx=0).stop 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dce800640 1 -- 172.21.15.16:0/259840123 reap_dead start 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dd4a00640 1 -- 172.21.15.16:0/259840123 shutdown_connections 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dd4a00640 1 -- 172.21.15.16:0/259840123 >> 172.21.15.16:0/259840123 conn(0x7f3dd00b2250 msgr2=0x7f3dd00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dd4a00640 1 -- 172.21.15.16:0/259840123 shutdown_connections 2024-09-17T05:10:25.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.392+0000 7f3dd4a00640 1 -- 172.21.15.16:0/259840123 wait complete. 2024-09-17T05:10:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 Processor -- start 2024-09-17T05:10:25.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 -- start start 2024-09-17T05:10:25.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3db4003fe0 0x7f3dd0153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd01f6db0 0x7f3dd0159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 0x7f3dd0159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0166850 con 0x7f3dd0154780 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01666d0 con 0x7f3dd01f6db0 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dd4a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd01669d0 con 0x7f3db4003fe0 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 0x7f3dd0159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dce800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 0x7f3dd0159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59220/0 (socket says 172.21.15.16:59220) 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.396+0000 7f3dce800640 1 -- 172.21.15.16:0/124906601 learned_addr learned my addr 172.21.15.16:0/124906601 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dc5400640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd01f6db0 0x7f3dd0159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dcde00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3db4003fe0 0x7f3dd0153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dce800640 1 -- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3db4003fe0 msgr2=0x7f3dd0153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dce800640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3db4003fe0 0x7f3dd0153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dce800640 1 -- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd01f6db0 msgr2=0x7f3dd0159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dce800640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd01f6db0 0x7f3dd0159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dce800640 1 -- 172.21.15.16:0/124906601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd00b5680 con 0x7f3dd0154780 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dc5400640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd01f6db0 0x7f3dd0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dce800640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 0x7f3dd0159790 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00cac20 tx=0x7f3dc800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8004b30 con 0x7f3dd0154780 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc8004cd0 con 0x7f3dd0154780 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003be0 con 0x7f3dd0154780 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd0157710 con 0x7f3dd0154780 2024-09-17T05:10:25.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd0154500 con 0x7f3dd0154780 2024-09-17T05:10:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd00cb0a0 con 0x7f3dd0154780 2024-09-17T05:10:25.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.400+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3dc8004030 con 0x7f3dd0154780 2024-09-17T05:10:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.404+0000 7f3dc6a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3dac073230 0x7f3dac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.404+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f3dc80948b0 con 0x7f3dd0154780 2024-09-17T05:10:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.404+0000 7f3dcde00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3dac073230 0x7f3dac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.404+0000 7f3dcde00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3dac073230 0x7f3dac075740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009b00 tx=0x7f3dc00091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:25.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.408+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3dc805fa30 con 0x7f3dd0154780 2024-09-17T05:10:25.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.588+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f3dd0156010 con 0x7f3dd0154780 2024-09-17T05:10:25.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dc6a00640 1 -- 172.21.15.16:0/124906601 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v70) ==== 71+0+0 (secure 0 0 0) 0x7f3dc805f330 con 0x7f3dd0154780 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3dac073230 msgr2=0x7f3dac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3dac073230 0x7f3dac075740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009b00 tx=0x7f3dc00091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 msgr2=0x7f3dd0159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 0x7f3dd0159790 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00cac20 tx=0x7f3dc800f3c0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 shutdown_connections 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f3dac073230 0x7f3dac075740 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd0154780 0x7f3dd0159790 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dd01f6db0 0x7f3dd0159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 --2- 172.21.15.16:0/124906601 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3db4003fe0 0x7f3dd0153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 >> 172.21.15.16:0/124906601 conn(0x7f3dd00b2250 msgr2=0x7f3dd0156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 shutdown_connections 2024-09-17T05:10:25.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.596+0000 7f3dd4a00640 1 -- 172.21.15.16:0/124906601 wait complete. 2024-09-17T05:10:25.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-09-17T05:10:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 -- 172.21.15.16:0/2246172191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c073d20 msgr2=0x7f756c0cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 --2- 172.21.15.16:0/2246172191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c073d20 0x7f756c0cee20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7554009840 tx=0x7f755401c820 comp rx=0 tx=0).stop 2024-09-17T05:10:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 -- 172.21.15.16:0/2246172191 shutdown_connections 2024-09-17T05:10:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 --2- 172.21.15.16:0/2246172191 >> v2:172.21.15.104:3300/0 conn(0x7f756c0cf360 0x7f756c0cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 --2- 172.21.15.16:0/2246172191 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c073d20 0x7f756c0cee20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 -- 172.21.15.16:0/2246172191 >> 172.21.15.16:0/2246172191 conn(0x7f756c0d0860 msgr2=0x7f756c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 -- 172.21.15.16:0/2246172191 shutdown_connections 2024-09-17T05:10:25.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.776+0000 7f7570c00640 1 -- 172.21.15.16:0/2246172191 wait complete. 2024-09-17T05:10:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 Processor -- start 2024-09-17T05:10:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 -- start start 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 0x7f756c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f756c15cbb0 0x7f756c144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c15af30 0x7f756c145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f756c169fa0 con 0x7f756c0cf360 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f756c169e20 con 0x7f756c15cbb0 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7570c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f756c16a120 con 0x7f756c15af30 2024-09-17T05:10:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 0x7f756c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 0x7f756c154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59228/0 (socket says 172.21.15.16:59228) 2024-09-17T05:10:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 -- 172.21.15.16:0/2060726913 learned_addr learned my addr 172.21.15.16:0/2060726913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f756a800640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c15af30 0x7f756c145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7561400640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f756c15cbb0 0x7f756c144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 -- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c15af30 msgr2=0x7f756c145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c15af30 0x7f756c145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 -- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f756c15cbb0 msgr2=0x7f756c144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f756c15cbb0 0x7f756c144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7569e00640 1 -- 172.21.15.16:0/2060726913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c155330 con 0x7f756c0cf360 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f7561400640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f756c15cbb0 0x7f756c144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.780+0000 7f756a800640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c15af30 0x7f756c145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7569e00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 0x7f756c154df0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c430 tx=0x7f755c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c011790 con 0x7f756c0cf360 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f755c017070 con 0x7f756c0cf360 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c1450f0 con 0x7f756c0cf360 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c011cc0 con 0x7f756c0cf360 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c15a8b0 con 0x7f756c0cf360 2024-09-17T05:10:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f756c0cee20 con 0x7f756c0cf360 2024-09-17T05:10:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.784+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f755c01c460 con 0x7f756c0cf360 2024-09-17T05:10:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.788+0000 7f7562a00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7548073230 0x7f7548075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.788+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f755c093cf0 con 0x7f756c0cf360 2024-09-17T05:10:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.788+0000 7f7561400640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7548073230 0x7f7548075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.788+0000 7f7561400640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7548073230 0x7f7548075740 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f75540af000 tx=0x7f755401b060 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:25.792+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f755c021020 con 0x7f756c0cf360 2024-09-17T05:10:26.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.028+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f756c0b0ae0 con 0x7f756c0cf360 2024-09-17T05:10:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.200+0000 7f7562a00640 1 -- 172.21.15.16:0/2060726913 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v478) ==== 151+0+0 (secure 0 0 0) 0x7f755c05ede0 con 0x7f756c0cf360 2024-09-17T05:10:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7548073230 msgr2=0x7f7548075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7548073230 0x7f7548075740 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f75540af000 tx=0x7f755401b060 comp rx=0 tx=0).stop 2024-09-17T05:10:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 msgr2=0x7f756c154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 0x7f756c154df0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f755c00c430 tx=0x7f755c00c8f0 comp rx=0 tx=0).stop 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 shutdown_connections 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f7548073230 0x7f7548075740 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f756c15af30 0x7f756c145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f756c15cbb0 0x7f756c144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 --2- 172.21.15.16:0/2060726913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f756c0cf360 0x7f756c154df0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 >> 172.21.15.16:0/2060726913 conn(0x7f756c0d0860 msgr2=0x7f756c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 shutdown_connections 2024-09-17T05:10:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:26.204+0000 7f7570c00640 1 -- 172.21.15.16:0/2060726913 wait complete. 2024-09-17T05:10:26.225 INFO:tasks.cephfs_test_runner:When a client has a bug (which we will simulate) preventing it from releasing caps, ... skipped 'Require FUSE client to inject client release failure' 2024-09-17T05:10:26.225 INFO:tasks.cephfs_test_runner: 2024-09-17T05:10:26.226 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T05:10:26.226 INFO:tasks.cephfs_test_runner:Ran 10 tests in 520.198s 2024-09-17T05:10:26.226 INFO:tasks.cephfs_test_runner: 2024-09-17T05:10:26.226 INFO:tasks.cephfs_test_runner:OK (skipped=4) 2024-09-17T05:10:26.226 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T05:10:26.235 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-17T05:10:26.287 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-17T05:10:26.287 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:26.287 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:26.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:10:26.296 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-17T05:10:26.296 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:26.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353336k,nr_inodes=4088334,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3482 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275600k,nr_inodes=818900,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275604k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T05:10:26.349 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:26.349 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:26.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:10:26.358 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-17T05:10:26.358 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T05:10:26.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350236k,nr_inodes=4087559,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4590 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T05:10:26.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:26.409 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:10:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T05:10:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T05:10:26.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:26.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T05:10:26.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:26.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T05:10:26.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:26.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-17T05:10:27.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:27.095 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-17T05:10:27.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:27.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-17T05:10:27.559 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-17T05:10:27.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:10:27.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:27.562 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:10:27.572 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T05:10:27.572 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T05:10:27.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:27.573 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T05:10:27.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:27.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T05:10:28.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:28.116 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-09-17T05:10:28.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:28.280 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-09-17T05:10:28.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T05:10:28.290 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-09-17T05:10:28.738 INFO:teuthology.orchestra.run.smithi104.stderr:Cannot find device "ceph-brx" 2024-09-17T05:10:28.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:10:28.741 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T05:10:28.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T05:10:28.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 -- 172.21.15.16:0/2029869179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 msgr2=0x7f210806f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 --2- 172.21.15.16:0/2029869179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f210806f3c0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f2108073750 tx=0x7f20f401c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:28.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 -- 172.21.15.16:0/2029869179 shutdown_connections 2024-09-17T05:10:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 --2- 172.21.15.16:0/2029869179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f210806f3c0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 --2- 172.21.15.16:0/2029869179 >> v2:172.21.15.104:3300/0 conn(0x7f2108071380 0x7f2108071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 -- 172.21.15.16:0/2029869179 >> 172.21.15.16:0/2029869179 conn(0x7f2108073ca0 msgr2=0x7f21080740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 -- 172.21.15.16:0/2029869179 shutdown_connections 2024-09-17T05:10:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.924+0000 7f210e200640 1 -- 172.21.15.16:0/2029869179 wait complete. 2024-09-17T05:10:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 Processor -- start 2024-09-17T05:10:28.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 -- start start 2024-09-17T05:10:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f2108156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f210806ffb0 0x7f21081517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:28.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2108071380 0x7f21081fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2108168c10 con 0x7f210806efb0 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2108168a90 con 0x7f210806ffb0 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210e200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f2108168d90 con 0x7f2108071380 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f2108156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f2108156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53740/0 (socket says 172.21.15.16:53740) 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 -- 172.21.15.16:0/1022772689 learned_addr learned my addr 172.21.15.16:0/1022772689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f2107e00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f210806ffb0 0x7f21081517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210d800640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2108071380 0x7f21081fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 -- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2108071380 msgr2=0x7f21081fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2108071380 0x7f21081fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:28.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 -- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f210806ffb0 msgr2=0x7f21081517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f210806ffb0 0x7f21081517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f210ce00640 1 -- 172.21.15.16:0/1022772689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081fc210 con 0x7f210806efb0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.928+0000 7f2107e00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f210806ffb0 0x7f21081517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f210ce00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f2108156910 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00e7f0 tx=0x7f20fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00ccc0 con 0x7f210806efb0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20fc00ce80 con 0x7f210806efb0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081571a0 con 0x7f210806efb0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc005280 con 0x7f210806efb0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081fcf10 con 0x7f210806efb0 2024-09-17T05:10:28.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210806f3c0 con 0x7f210806efb0 2024-09-17T05:10:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20fc005420 con 0x7f210806efb0 2024-09-17T05:10:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f2104c00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20e8073200 0x7f20e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.932+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f20fc094b80 con 0x7f210806efb0 2024-09-17T05:10:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.936+0000 7f2107e00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20e8073200 0x7f20e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:28.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.936+0000 7f2107e00640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20e8073200 0x7f20e8075710 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20f40af000 tx=0x7f20f401b100 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:28.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:28.940+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f20fc05fc70 con 0x7f210806efb0 2024-09-17T05:10:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.108+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21081576c0 con 0x7f20e8073200 2024-09-17T05:10:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f2104c00640 1 -- 172.21.15.16:0/1022772689 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13487 (secure 0 0 0) 0x7f21081576c0 con 0x7f20e8073200 2024-09-17T05:10:29.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20e8073200 msgr2=0x7f20e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20e8073200 0x7f20e8075710 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20f40af000 tx=0x7f20f401b100 comp rx=0 tx=0).stop 2024-09-17T05:10:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 msgr2=0x7f2108156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f2108156910 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00e7f0 tx=0x7f20fc00ecb0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 shutdown_connections 2024-09-17T05:10:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f20e8073200 0x7f20e8075710 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f2108071380 0x7f21081fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f210806ffb0 0x7f21081517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 --2- 172.21.15.16:0/1022772689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210806efb0 0x7f2108156910 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 >> 172.21.15.16:0/1022772689 conn(0x7f2108073ca0 msgr2=0x7f2108154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 shutdown_connections 2024-09-17T05:10:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.112+0000 7f210e200640 1 -- 172.21.15.16:0/1022772689 wait complete. 2024-09-17T05:10:29.136 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-17T05:10:27.616490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282264,"kb_used_data":16104,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692456,"statfs":{"total":773094113280,"available":772805074944,"internally_reserved":0,"allocated":16490496,"data_stored":8266576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2669,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001807"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":364,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T05:10:25.197252+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:10:25.197252+0000","last_peered":"2024-09-17T05:10:25.197252+0000","last_clean":"2024-09-17T05:10:25.197252+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:25.197252+0000","last_undegraded":"2024-09-17T05:10:25.197252+0000","last_fullsized":"2024-09-17T05:10:25.197252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509559,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33064,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33056,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:29.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 -- 172.21.15.16:0/1333188927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda28004d50 msgr2=0x7fda28005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 --2- 172.21.15.16:0/1333188927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda28004d50 0x7fda28005160 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009ee0 tx=0x7fda3c01d630 comp rx=0 tx=0).stop 2024-09-17T05:10:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda43e00640 1 -- 172.21.15.16:0/1333188927 reap_dead start 2024-09-17T05:10:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 -- 172.21.15.16:0/1333188927 shutdown_connections 2024-09-17T05:10:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 -- 172.21.15.16:0/1333188927 >> 172.21.15.16:0/1333188927 conn(0x7fda4406f900 msgr2=0x7fda440b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 -- 172.21.15.16:0/1333188927 shutdown_connections 2024-09-17T05:10:29.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 -- 172.21.15.16:0/1333188927 wait complete. 2024-09-17T05:10:29.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 Processor -- start 2024-09-17T05:10:29.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.296+0000 7fda4a200640 1 -- start start 2024-09-17T05:10:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda28004d50 0x7fda4413e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 0x7fda441e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda441544d0 0x7fda441548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda440b4600 con 0x7fda441e47c0 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fda440b4480 con 0x7fda28004d50 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fda4406ce90 con 0x7fda441544d0 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 0x7fda441e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 0x7fda441e4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53768/0 (socket says 172.21.15.16:53768) 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 -- 172.21.15.16:0/3099781788 learned_addr learned my addr 172.21.15.16:0/3099781788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda43e00640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda441544d0 0x7fda441548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda43400640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda28004d50 0x7fda4413e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 -- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda441544d0 msgr2=0x7fda441548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda441544d0 0x7fda441548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 -- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda28004d50 msgr2=0x7fda4413e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda28004d50 0x7fda4413e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 -- 172.21.15.16:0/3099781788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda441e5110 con 0x7fda441e47c0 2024-09-17T05:10:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda43400640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda28004d50 0x7fda4413e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:29.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3be00640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 0x7fda441e4bd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001060 tx=0x7fda2c0127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c010890 con 0x7fda441e47c0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda2c010a30 con 0x7fda441e47c0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c013450 con 0x7fda441e47c0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda44154e20 con 0x7fda441e47c0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441e5560 con 0x7fda441e47c0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.300+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4406d3f0 con 0x7fda441e47c0 2024-09-17T05:10:29.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.304+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda2c0135f0 con 0x7fda441e47c0 2024-09-17T05:10:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.304+0000 7fda3b400640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda20073200 0x7fda20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.304+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fda2c097fc0 con 0x7fda441e47c0 2024-09-17T05:10:29.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.304+0000 7fda43400640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda20073200 0x7fda20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.304+0000 7fda43400640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda20073200 0x7fda20075710 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fda3400abd0 tx=0x7fda34011040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:29.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.308+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda44154e20 con 0x7fda441e47c0 2024-09-17T05:10:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda441e52f0 con 0x7fda20073200 2024-09-17T05:10:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda3b400640 1 -- 172.21.15.16:0/3099781788 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13487 (secure 0 0 0) 0x7fda441e52f0 con 0x7fda20073200 2024-09-17T05:10:29.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:29.482 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda20073200 msgr2=0x7fda20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda20073200 0x7fda20075710 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fda3400abd0 tx=0x7fda34011040 comp rx=0 tx=0).stop 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 msgr2=0x7fda441e4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 0x7fda441e4bd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001060 tx=0x7fda2c0127b0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 shutdown_connections 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fda20073200 0x7fda20075710 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fda441544d0 0x7fda441548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda441e47c0 0x7fda441e4bd0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 --2- 172.21.15.16:0/3099781788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda28004d50 0x7fda4413e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 >> 172.21.15.16:0/3099781788 conn(0x7fda4406f900 msgr2=0x7fda4415d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 shutdown_connections 2024-09-17T05:10:29.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.480+0000 7fda4a200640 1 -- 172.21.15.16:0/3099781788 wait complete. 2024-09-17T05:10:29.504 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-17T05:10:27.616490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282264,"kb_used_data":16104,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692456,"statfs":{"total":773094113280,"available":772805074944,"internally_reserved":0,"allocated":16490496,"data_stored":8266576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2669,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001807"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":364,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T05:10:25.197252+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:10:25.197252+0000","last_peered":"2024-09-17T05:10:25.197252+0000","last_clean":"2024-09-17T05:10:25.197252+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:25.197252+0000","last_undegraded":"2024-09-17T05:10:25.197252+0000","last_fullsized":"2024-09-17T05:10:25.197252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509559,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33064,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33056,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:29.505 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T05:10:29.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 -- 172.21.15.16:0/2486593358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6806e170 msgr2=0x7fad6806e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 --2- 172.21.15.16:0/2486593358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6806e170 0x7fad6806e580 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009900 tx=0x7fad5c01c850 comp rx=0 tx=0).stop 2024-09-17T05:10:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 -- 172.21.15.16:0/2486593358 shutdown_connections 2024-09-17T05:10:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 --2- 172.21.15.16:0/2486593358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6806e170 0x7fad6806e580 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 -- 172.21.15.16:0/2486593358 >> 172.21.15.16:0/2486593358 conn(0x7fad680d2270 msgr2=0x7fad680d2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:29.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 -- 172.21.15.16:0/2486593358 shutdown_connections 2024-09-17T05:10:29.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.660+0000 7fad70600640 1 -- 172.21.15.16:0/2486593358 wait complete. 2024-09-17T05:10:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 Processor -- start 2024-09-17T05:10:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 -- start start 2024-09-17T05:10:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad6806e170 0x7fad68160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b2010 0x7fad681605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 0x7fad68154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad6816d0f0 con 0x7fad6815f8c0 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fad68162700 con 0x7fad680b2010 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad70600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fad6816d270 con 0x7fad6806e170 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad6e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 0x7fad68154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad6e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 0x7fad68154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53782/0 (socket says 172.21.15.16:53782) 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad6e600640 1 -- 172.21.15.16:0/2470427295 learned_addr learned my addr 172.21.15.16:0/2470427295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad6dc00640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad6806e170 0x7fad68160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.664+0000 7fad6d200640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b2010 0x7fad681605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad6e600640 1 -- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad6806e170 msgr2=0x7fad68160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad6e600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad6806e170 0x7fad68160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad6e600640 1 -- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b2010 msgr2=0x7fad681605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad6e600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b2010 0x7fad681605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad6e600640 1 -- 172.21.15.16:0/2470427295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad68153880 con 0x7fad6815f8c0 2024-09-17T05:10:29.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad6e600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 0x7fad68154dd0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c760 tx=0x7fad6000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60003db0 con 0x7fad6815f8c0 2024-09-17T05:10:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6000ce10 con 0x7fad6815f8c0 2024-09-17T05:10:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad600052d0 con 0x7fad6815f8c0 2024-09-17T05:10:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad681535a0 con 0x7fad6815f8c0 2024-09-17T05:10:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad681545d0 con 0x7fad6815f8c0 2024-09-17T05:10:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6806e580 con 0x7fad6815f8c0 2024-09-17T05:10:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad60005470 con 0x7fad6815f8c0 2024-09-17T05:10:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.668+0000 7fad65600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad48073200 0x7fad48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.672+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fad60093d30 con 0x7fad6815f8c0 2024-09-17T05:10:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.672+0000 7fad6dc00640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad48073200 0x7fad48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.672+0000 7fad6dc00640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad48073200 0x7fad48075710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0af000 tx=0x7fad5c01b090 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.676+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad6806eac0 con 0x7fad6815f8c0 2024-09-17T05:10:29.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.848+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad68153330 con 0x7fad48073200 2024-09-17T05:10:29.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.848+0000 7fad65600640 1 -- 172.21.15.16:0/2470427295 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13487 (secure 0 0 0) 0x7fad68153330 con 0x7fad48073200 2024-09-17T05:10:29.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:29.853 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad48073200 msgr2=0x7fad48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad48073200 0x7fad48075710 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0af000 tx=0x7fad5c01b090 comp rx=0 tx=0).stop 2024-09-17T05:10:29.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 msgr2=0x7fad68154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 0x7fad68154dd0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c760 tx=0x7fad6000cc20 comp rx=0 tx=0).stop 2024-09-17T05:10:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 shutdown_connections 2024-09-17T05:10:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fad48073200 0x7fad48075710 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6815f8c0 0x7fad68154dd0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad680b2010 0x7fad681605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 --2- 172.21.15.16:0/2470427295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fad6806e170 0x7fad68160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 >> 172.21.15.16:0/2470427295 conn(0x7fad680d2270 msgr2=0x7fad680cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 shutdown_connections 2024-09-17T05:10:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:29.852+0000 7fad70600640 1 -- 172.21.15.16:0/2470427295 wait complete. 2024-09-17T05:10:29.873 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-17T05:10:29.616959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282264,"kb_used_data":16104,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692456,"statfs":{"total":773094113280,"available":772805074944,"internally_reserved":0,"allocated":16490496,"data_stored":8266576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2669,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.435580"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":364,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T05:10:25.197252+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:10:25.197252+0000","last_peered":"2024-09-17T05:10:25.197252+0000","last_clean":"2024-09-17T05:10:25.197252+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:25.197252+0000","last_undegraded":"2024-09-17T05:10:25.197252+0000","last_fullsized":"2024-09-17T05:10:25.197252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509559,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33064,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509557,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33056,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33640,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338200,"statfs":{"total":96636764160,"available":96602316800,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:29.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T05:10:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095f200640 1 -- 172.21.15.16:0/1151423041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 msgr2=0x7f093c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095f200640 1 --2- 172.21.15.16:0/1151423041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f093c004810 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f09580b2970 tx=0x7f094001d1f0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095d200640 1 -- 172.21.15.16:0/1151423041 reap_dead start 2024-09-17T05:10:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095f200640 1 -- 172.21.15.16:0/1151423041 shutdown_connections 2024-09-17T05:10:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095f200640 1 -- 172.21.15.16:0/1151423041 >> 172.21.15.16:0/1151423041 conn(0x7f095806f900 msgr2=0x7f09580b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095f200640 1 -- 172.21.15.16:0/1151423041 shutdown_connections 2024-09-17T05:10:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.032+0000 7f095f200640 1 -- 172.21.15.16:0/1151423041 wait complete. 2024-09-17T05:10:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 Processor -- start 2024-09-17T05:10:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 -- start start 2024-09-17T05:10:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f095806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f095806e2e0 0x7f095814a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0958166a10 0x7f095814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0958167c30 con 0x7f093c004030 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0958167ab0 con 0x7f0958166a10 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095f200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0958167db0 con 0x7f095806e2e0 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f0957e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f095806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f0957e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f095806dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53802/0 (socket says 172.21.15.16:53802) 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f0957e00640 1 -- 172.21.15.16:0/2934277236 learned_addr learned my addr 172.21.15.16:0/2934277236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f094fe00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f095806e2e0 0x7f095814a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.036+0000 7f095d200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0958166a10 0x7f095814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0957e00640 1 -- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f095806e2e0 msgr2=0x7f095814a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0957e00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f095806e2e0 0x7f095814a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0957e00640 1 -- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0958166a10 msgr2=0x7f095814aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0957e00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0958166a10 0x7f095814aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0957e00640 1 -- 172.21.15.16:0/2934277236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09580d04d0 con 0x7f093c004030 2024-09-17T05:10:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0957e00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f095806dda0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0948004ef0 tx=0x7f0948001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948015020 con 0x7f093c004030 2024-09-17T05:10:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0948011960 con 0x7f093c004030 2024-09-17T05:10:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0958164eb0 con 0x7f093c004030 2024-09-17T05:10:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0948011d30 con 0x7f093c004030 2024-09-17T05:10:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0958165470 con 0x7f093c004030 2024-09-17T05:10:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0958073650 con 0x7f093c004030 2024-09-17T05:10:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0948018020 con 0x7f093c004030 2024-09-17T05:10:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.040+0000 7f0954c00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0934073230 0x7f0934075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.044+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f0948094870 con 0x7f093c004030 2024-09-17T05:10:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.044+0000 7f094fe00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0934073230 0x7f0934075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.044+0000 7f094fe00640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0934073230 0x7f0934075740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f09400ad000 tx=0x7f0940019f90 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.048+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f094801f030 con 0x7f093c004030 2024-09-17T05:10:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.220+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f095815d0a0 con 0x7f093c004030 2024-09-17T05:10:30.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.220+0000 7f0954c00640 1 -- 172.21.15.16:0/2934277236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13301 (secure 0 0 0) 0x7f094805f9f0 con 0x7f093c004030 2024-09-17T05:10:30.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:30.224 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":152,"fsid":"23ee29f8-0841-48bc-a01a-8eb97d561cfc","created":"2024-09-17T05:00:42.161181+0000","modified":"2024-09-17T05:10:25.191967+0000","last_up_change":"2024-09-17T05:00:57.415317+0000","last_in_change":"2024-09-17T05:00:46.688430+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T05:00:57.487536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0b146e62-e5a9-4933-a5f7-95e1c4231514","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6825","nonce":3154417598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6827","nonce":3154417598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6831","nonce":3154417598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3154417598},{"type":"v1","addr":"172.21.15.16:6829","nonce":3154417598}]},"public_addr":"172.21.15.16:6825/3154417598","cluster_addr":"172.21.15.16:6827/3154417598","heartbeat_back_addr":"172.21.15.16:6831/3154417598","heartbeat_front_addr":"172.21.15.16:6829/3154417598","state":["exists","up"]},{"osd":1,"uuid":"e77bd243-cf7e-422c-8f72-6b796df25a4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6809","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6811","nonce":40873108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6813","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6815","nonce":40873108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6823","nonce":40873108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":40873108},{"type":"v1","addr":"172.21.15.16:6819","nonce":40873108}]},"public_addr":"172.21.15.16:6811/40873108","cluster_addr":"172.21.15.16:6815/40873108","heartbeat_back_addr":"172.21.15.16:6823/40873108","heartbeat_front_addr":"172.21.15.16:6819/40873108","state":["exists","up"]},{"osd":2,"uuid":"ac3ac825-4660-4f82-91bf-9fb14fc1765c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6810","nonce":3210692914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6814","nonce":3210692914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6822","nonce":3210692914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3210692914},{"type":"v1","addr":"172.21.15.16:6818","nonce":3210692914}]},"public_addr":"172.21.15.16:6810/3210692914","cluster_addr":"172.21.15.16:6814/3210692914","heartbeat_back_addr":"172.21.15.16:6822/3210692914","heartbeat_front_addr":"172.21.15.16:6818/3210692914","state":["exists","up"]},{"osd":3,"uuid":"465db35d-f042-4b72-bec6-79f3208dd66a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6801","nonce":3613434049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6803","nonce":3613434049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6807","nonce":3613434049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3613434049},{"type":"v1","addr":"172.21.15.16:6805","nonce":3613434049}]},"public_addr":"172.21.15.16:6801/3613434049","cluster_addr":"172.21.15.16:6803/3613434049","heartbeat_back_addr":"172.21.15.16:6807/3613434049","heartbeat_front_addr":"172.21.15.16:6805/3613434049","state":["exists","up"]},{"osd":4,"uuid":"e3bbd5f6-7043-4a0b-95b6-4b576d2e1994","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6811","nonce":2719508844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6813","nonce":2719508844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6817","nonce":2719508844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":2719508844},{"type":"v1","addr":"172.21.15.104:6815","nonce":2719508844}]},"public_addr":"172.21.15.104:6811/2719508844","cluster_addr":"172.21.15.104:6813/2719508844","heartbeat_back_addr":"172.21.15.104:6817/2719508844","heartbeat_front_addr":"172.21.15.104:6815/2719508844","state":["exists","up"]},{"osd":5,"uuid":"967db2c5-8b0e-4551-8a02-77c8628fda1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6803","nonce":464547405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6805","nonce":464547405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6809","nonce":464547405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":464547405},{"type":"v1","addr":"172.21.15.104:6807","nonce":464547405}]},"public_addr":"172.21.15.104:6803/464547405","cluster_addr":"172.21.15.104:6805/464547405","heartbeat_back_addr":"172.21.15.104:6809/464547405","heartbeat_front_addr":"172.21.15.104:6807/464547405","state":["exists","up"]},{"osd":6,"uuid":"5f89fe76-ba31-4115-b1d1-ec3ffa44fdc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6827","nonce":3199393218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6829","nonce":3199393218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6833","nonce":3199393218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3199393218},{"type":"v1","addr":"172.21.15.104:6831","nonce":3199393218}]},"public_addr":"172.21.15.104:6827/3199393218","cluster_addr":"172.21.15.104:6829/3199393218","heartbeat_back_addr":"172.21.15.104:6833/3199393218","heartbeat_front_addr":"172.21.15.104:6831/3199393218","state":["exists","up"]},{"osd":7,"uuid":"53764c31-8173-4f18-8367-b1f777813351","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6819","nonce":2403343602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6821","nonce":2403343602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6825","nonce":2403343602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2403343602},{"type":"v1","addr":"172.21.15.104:6823","nonce":2403343602}]},"public_addr":"172.21.15.104:6819/2403343602","cluster_addr":"172.21.15.104:6821/2403343602","heartbeat_back_addr":"172.21.15.104:6825/2403343602","heartbeat_front_addr":"172.21.15.104:6823/2403343602","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.048417+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.320902+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.470936+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.720581+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.027116+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.445719+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:54.075384+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T05:00:53.690588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/291864941":"2024-09-18T05:10:22.180007+0000","172.21.15.16:6833/291864941":"2024-09-18T05:10:22.180007+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0934073230 msgr2=0x7f0934075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0934073230 0x7f0934075740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f09400ad000 tx=0x7f0940019f90 comp rx=0 tx=0).stop 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 msgr2=0x7f095806dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f095806dda0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f0948004ef0 tx=0x7f0948001640 comp rx=0 tx=0).stop 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 shutdown_connections 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0934073230 0x7f0934075740 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0958166a10 0x7f095814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f095806e2e0 0x7f095814a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 --2- 172.21.15.16:0/2934277236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f093c004030 0x7f095806dda0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 >> 172.21.15.16:0/2934277236 conn(0x7f095806f900 msgr2=0x7f095815d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 shutdown_connections 2024-09-17T05:10:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:30.224+0000 7f095f200640 1 -- 172.21.15.16:0/2934277236 wait complete. 2024-09-17T05:10:31.247 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T05:10:31.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 -- 172.21.15.16:0/4201385319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 msgr2=0x7fa2e8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 --2- 172.21.15.16:0/4201385319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa2e8003eb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c2e0 tx=0x7fa2f401cec0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 -- 172.21.15.16:0/4201385319 shutdown_connections 2024-09-17T05:10:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 --2- 172.21.15.16:0/4201385319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa2e8003eb0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 -- 172.21.15.16:0/4201385319 >> 172.21.15.16:0/4201385319 conn(0x7fa3040d2180 msgr2=0x7fa3040d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 -- 172.21.15.16:0/4201385319 shutdown_connections 2024-09-17T05:10:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 -- 172.21.15.16:0/4201385319 wait complete. 2024-09-17T05:10:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 Processor -- start 2024-09-17T05:10:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.424+0000 7fa309200640 1 -- start start 2024-09-17T05:10:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa3040709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3040c5ff0 0x7fa30415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa304154fc0 0x7fa3041553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa304160a50 con 0x7fa2e8003a80 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa3041608d0 con 0x7fa304154fc0 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fa304160bd0 con 0x7fa3040c5ff0 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa3040709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa3040709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53818/0 (socket says 172.21.15.16:53818) 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 -- 172.21.15.16:0/776662136 learned_addr learned my addr 172.21.15.16:0/776662136 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa2fbe00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3040c5ff0 0x7fa30415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303e00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa304154fc0 0x7fa3041553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 -- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3040c5ff0 msgr2=0x7fa30415a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fa3040c5ff0 0x7fa30415a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 -- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa304154fc0 msgr2=0x7fa3041553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa304154fc0 0x7fa3041553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 -- 172.21.15.16:0/776662136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa304155cf0 con 0x7fa2e8003a80 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303e00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa304154fc0 0x7fa3041553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa303400640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa3040709f0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4000c10 tx=0x7fa2f4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4006cd0 con 0x7fa2e8003a80 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2f4001520 con 0x7fa2e8003a80 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3041599e0 con 0x7fa2e8003a80 2024-09-17T05:10:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40a5dc0 con 0x7fa2e8003a80 2024-09-17T05:10:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa304159fa0 con 0x7fa2e8003a80 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.428+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa304070f30 con 0x7fa2e8003a80 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa2f4006140 con 0x7fa2e8003a80 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2e0073200 0x7fa2e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fa2f412df30 con 0x7fa2e8003a80 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7fa2e00796f0 0x7fa2e007bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 --> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa2e007c250 con 0x7fa2e00796f0 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fa2f412e320 con 0x7fa2e8003a80 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fbe00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2e0073200 0x7fa2e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa303e00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7fa2e00796f0 0x7fa2e007bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fbe00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2e0073200 0x7fa2e0075710 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa3040c8b50 tx=0x7fa2ec006b20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa303e00640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7fa2e00796f0 0x7fa2e007bb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.432+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== osd.0 v2:172.21.15.16:6824/3154417598 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa2e007c250 con 0x7fa2e00796f0 2024-09-17T05:10:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 --> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa3040b1ad0 con 0x7fa2e00796f0 2024-09-17T05:10:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa2fb400640 1 -- 172.21.15.16:0/776662136 <== osd.0 v2:172.21.15.16:6824/3154417598 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa3040b1ad0 con 0x7fa2e00796f0 2024-09-17T05:10:31.449 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:31.449 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:31.449 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7fa2e00796f0 msgr2=0x7fa2e007bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:6824/3154417598,v1:172.21.15.16:6825/3154417598] conn(0x7fa2e00796f0 0x7fa2e007bb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2e0073200 msgr2=0x7fa2e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fa2e0073200 0x7fa2e0075710 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa3040c8b50 tx=0x7fa2ec006b20 comp rx=0 tx=0).stop 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 msgr2=0x7fa3040709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 --2- 172.21.15.16:0/776662136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2e8003a80 0x7fa3040709f0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4000c10 tx=0x7fa2f4001120 comp rx=0 tx=0).stop 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa303e00640 1 -- 172.21.15.16:0/776662136 reap_dead start 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 shutdown_connections 2024-09-17T05:10:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 >> 172.21.15.16:0/776662136 conn(0x7fa3040d2180 msgr2=0x7fa30415c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 shutdown_connections 2024-09-17T05:10:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.448+0000 7fa309200640 1 -- 172.21.15.16:0/776662136 wait complete. 2024-09-17T05:10:31.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T05:10:31.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:10:31.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:10:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 -- 172.21.15.16:0/2121622173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 msgr2=0x7f9efc0ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 --2- 172.21.15.16:0/2121622173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc0ce5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f9efc156030 tx=0x7f9ee801c800 comp rx=0 tx=0).stop 2024-09-17T05:10:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 -- 172.21.15.16:0/2121622173 shutdown_connections 2024-09-17T05:10:31.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 --2- 172.21.15.16:0/2121622173 >> v2:172.21.15.104:3300/0 conn(0x7f9efc15f6f0 0x7f9efc0ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 --2- 172.21.15.16:0/2121622173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc0ce5d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 -- 172.21.15.16:0/2121622173 >> 172.21.15.16:0/2121622173 conn(0x7f9efc0b2250 msgr2=0x7f9efc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 -- 172.21.15.16:0/2121622173 shutdown_connections 2024-09-17T05:10:31.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.624+0000 7f9f04200640 1 -- 172.21.15.16:0/2121622173 wait complete. 2024-09-17T05:10:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 Processor -- start 2024-09-17T05:10:31.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 -- start start 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9efc074a20 0x7f9efc0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc15f6f0 0x7f9efc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc168e50 con 0x7f9efc074e30 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc168cd0 con 0x7f9efc15f6f0 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f04200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9efc168fd0 con 0x7f9efc074a20 2024-09-17T05:10:31.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f00e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f00e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53830/0 (socket says 172.21.15.16:53830) 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.628+0000 7f9f00e00640 1 -- 172.21.15.16:0/583496426 learned_addr learned my addr 172.21.15.16:0/583496426 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f02200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc15f6f0 0x7f9efc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f01800640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9efc074a20 0x7f9efc0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f00e00640 1 -- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9efc074a20 msgr2=0x7f9efc0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f00e00640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9efc074a20 0x7f9efc0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f00e00640 1 -- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc15f6f0 msgr2=0x7f9efc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f00e00640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc15f6f0 0x7f9efc15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f00e00640 1 -- 172.21.15.16:0/583496426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc0ca410 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f00e00640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc15a760 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80b0040 tx=0x7f9ee80a6640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8004100 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ee80bf070 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc165c90 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8003600 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc163fb0 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9efc15efe0 con 0x7f9efc074e30 2024-09-17T05:10:31.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.632+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ee80037a0 con 0x7f9efc074e30 2024-09-17T05:10:31.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.636+0000 7f9ef9600640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9edc073200 0x7f9edc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:31.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.636+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f9ee812eb40 con 0x7f9efc074e30 2024-09-17T05:10:31.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.636+0000 7f9f01800640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9edc073200 0x7f9edc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:31.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.636+0000 7f9f01800640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9edc073200 0x7f9edc075710 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0007fd0 tx=0x7f9ef0009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:31.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.640+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ee80c8020 con 0x7f9efc074e30 2024-09-17T05:10:31.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.812+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9efc165a20 con 0x7f9edc073200 2024-09-17T05:10:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.812+0000 7f9ef9600640 1 -- 172.21.15.16:0/583496426 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9efc165a20 con 0x7f9edc073200 2024-09-17T05:10:31.816 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T05:10:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9edc073200 msgr2=0x7f9edc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9edc073200 0x7f9edc075710 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0007fd0 tx=0x7f9ef0009040 comp rx=0 tx=0).stop 2024-09-17T05:10:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 msgr2=0x7f9efc15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc15a760 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80b0040 tx=0x7f9ee80a6640 comp rx=0 tx=0).stop 2024-09-17T05:10:31.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 shutdown_connections 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9edc073200 0x7f9edc075710 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9efc15f6f0 0x7f9efc15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc074e30 0x7f9efc15a760 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 --2- 172.21.15.16:0/583496426 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f9efc074a20 0x7f9efc0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 >> 172.21.15.16:0/583496426 conn(0x7f9efc0b2250 msgr2=0x7f9efc0cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 shutdown_connections 2024-09-17T05:10:31.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.816+0000 7f9f04200640 1 -- 172.21.15.16:0/583496426 wait complete. 2024-09-17T05:10:31.838 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T05:10:31.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 -- 172.21.15.16:0/3173154914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693006e0d0 msgr2=0x7f6930073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 --2- 172.21.15.16:0/3173154914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693006e0d0 0x7f6930073bc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f693015f6d0 tx=0x7f692001c870 comp rx=0 tx=0).stop 2024-09-17T05:10:32.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 -- 172.21.15.16:0/3173154914 shutdown_connections 2024-09-17T05:10:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 --2- 172.21.15.16:0/3173154914 >> v2:172.21.15.104:3300/0 conn(0x7f69300b4a20 0x7f69300b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 --2- 172.21.15.16:0/3173154914 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f693006e4e0 0x7f6930074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 --2- 172.21.15.16:0/3173154914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f693006e0d0 0x7f6930073bc0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 -- 172.21.15.16:0/3173154914 >> 172.21.15.16:0/3173154914 conn(0x7f69300d1170 msgr2=0x7f69300734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 -- 172.21.15.16:0/3173154914 shutdown_connections 2024-09-17T05:10:32.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:31.996+0000 7f6935c00640 1 -- 172.21.15.16:0/3173154914 wait complete. 2024-09-17T05:10:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 Processor -- start 2024-09-17T05:10:32.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 -- start start 2024-09-17T05:10:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f693006e0d0 0x7f6930160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f693006e4e0 0x7f69301540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69300b4a20 0x7f69301646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f69300b4650 con 0x7f69300b4a20 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69300b44d0 con 0x7f693006e0d0 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f69300ce5a0 con 0x7f693006e4e0 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69300b4a20 0x7f69301646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69300b4a20 0x7f69301646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53848/0 (socket says 172.21.15.16:53848) 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.000+0000 7f6935200640 1 -- 172.21.15.16:0/1934683020 learned_addr learned my addr 172.21.15.16:0/1934683020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6927e00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f693006e4e0 0x7f69301540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f692fe00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f693006e0d0 0x7f6930160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935200640 1 -- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f693006e4e0 msgr2=0x7f69301540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935200640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f693006e4e0 0x7f69301540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935200640 1 -- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f693006e0d0 msgr2=0x7f6930160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935200640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f693006e0d0 0x7f6930160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935200640 1 -- 172.21.15.16:0/1934683020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6930153380 con 0x7f69300b4a20 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935200640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69300b4a20 0x7f69301646b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f692800d970 tx=0x7f692800de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928017070 con 0x7f69300b4a20 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f692800be00 con 0x7f69300b4a20 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69280106c0 con 0x7f69300b4a20 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6930154610 con 0x7f69300b4a20 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69301538d0 con 0x7f69300b4a20 2024-09-17T05:10:32.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69300b4e30 con 0x7f69300b4a20 2024-09-17T05:10:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.004+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6928010860 con 0x7f69300b4a20 2024-09-17T05:10:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692cc00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f690c073200 0x7f690c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f6928094de0 con 0x7f69300b4a20 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692fe00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f690c073200 0x7f690c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692cc00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7f690c0796f0 0x7f690c07bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 --> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f690c07c250 con 0x7f690c0796f0 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f6928095190 con 0x7f69300b4a20 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f6927e00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7f690c0796f0 0x7f690c07bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692fe00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f690c073200 0x7f690c075710 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f69200af000 tx=0x7f692001b0b0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f6927e00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7f690c0796f0 0x7f690c07bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.008+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== osd.1 v2:172.21.15.16:6809/40873108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f690c07c250 con 0x7f690c0796f0 2024-09-17T05:10:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.020+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 --> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f693015e870 con 0x7f690c0796f0 2024-09-17T05:10:32.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.020+0000 7f692cc00640 1 -- 172.21.15.16:0/1934683020 <== osd.1 v2:172.21.15.16:6809/40873108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f693015e870 con 0x7f690c0796f0 2024-09-17T05:10:32.024 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:32.024 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:32.024 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7f690c0796f0 msgr2=0x7f690c07bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:6809/40873108,v1:172.21.15.16:6811/40873108] conn(0x7f690c0796f0 0x7f690c07bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f690c073200 msgr2=0x7f690c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f690c073200 0x7f690c075710 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f69200af000 tx=0x7f692001b0b0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69300b4a20 msgr2=0x7f69301646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 --2- 172.21.15.16:0/1934683020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69300b4a20 0x7f69301646b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f692800d970 tx=0x7f692800de30 comp rx=0 tx=0).stop 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935200640 1 -- 172.21.15.16:0/1934683020 reap_dead start 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 shutdown_connections 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 >> 172.21.15.16:0/1934683020 conn(0x7f69300d1170 msgr2=0x7f69300b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 shutdown_connections 2024-09-17T05:10:32.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.024+0000 7f6935c00640 1 -- 172.21.15.16:0/1934683020 wait complete. 2024-09-17T05:10:32.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T05:10:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 -- 172.21.15.16:0/1804235584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c40d0940 msgr2=0x7f53c41590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 --2- 172.21.15.16:0/1804235584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c40d0940 0x7f53c41590a0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009900 tx=0x7f53b401c700 comp rx=0 tx=0).stop 2024-09-17T05:10:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 -- 172.21.15.16:0/1804235584 shutdown_connections 2024-09-17T05:10:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 --2- 172.21.15.16:0/1804235584 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40ca6b0 0x7f53c40caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 --2- 172.21.15.16:0/1804235584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c40d0940 0x7f53c41590a0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 -- 172.21.15.16:0/1804235584 >> 172.21.15.16:0/1804235584 conn(0x7f53c40b2250 msgr2=0x7f53c40d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 -- 172.21.15.16:0/1804235584 shutdown_connections 2024-09-17T05:10:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.200+0000 7f53caa00640 1 -- 172.21.15.16:0/1804235584 wait complete. 2024-09-17T05:10:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 Processor -- start 2024-09-17T05:10:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 -- start start 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c40ca6b0 0x7f53c4157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40d0940 0x7f53c40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 0x7f53c406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4168e70 con 0x7f53c415f110 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4168cf0 con 0x7f53c40ca6b0 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f53c4168ff0 con 0x7f53c40d0940 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 0x7f53c406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 0x7f53c406ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53870/0 (socket says 172.21.15.16:53870) 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 -- 172.21.15.16:0/4190115595 learned_addr learned my addr 172.21.15.16:0/4190115595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53bbe00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40d0940 0x7f53c40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c3e00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c40ca6b0 0x7f53c4157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 -- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40d0940 msgr2=0x7f53c40c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40d0940 0x7f53c40c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 -- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c40ca6b0 msgr2=0x7f53c4157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c40ca6b0 0x7f53c4157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 -- 172.21.15.16:0/4190115595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c4157da0 con 0x7f53c415f110 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c3e00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c40ca6b0 0x7f53c4157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53bbe00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40d0940 0x7f53c40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c8a00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 0x7f53c406ce90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00b9c0 tx=0x7f53bc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc019070 con 0x7f53c415f110 2024-09-17T05:10:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53bc00cc90 con 0x7f53c415f110 2024-09-17T05:10:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0146a0 con 0x7f53c415f110 2024-09-17T05:10:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c40b32f0 con 0x7f53c415f110 2024-09-17T05:10:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.204+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c40b38b0 con 0x7f53c415f110 2024-09-17T05:10:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.208+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c40cc650 con 0x7f53c415f110 2024-09-17T05:10:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.208+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f53bc01c070 con 0x7f53c415f110 2024-09-17T05:10:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.208+0000 7f53c0c00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53a0073200 0x7f53a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.208+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f53bc093f60 con 0x7f53c415f110 2024-09-17T05:10:32.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.212+0000 7f53c3e00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53a0073200 0x7f53a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.212+0000 7f53c3e00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53a0073200 0x7f53a0075710 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f53b40af000 tx=0x7f53b401af40 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.216+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53bc05f050 con 0x7f53c415f110 2024-09-17T05:10:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f53c40caac0 con 0x7f53a0073200 2024-09-17T05:10:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53c0c00640 1 -- 172.21.15.16:0/4190115595 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f53c40caac0 con 0x7f53a0073200 2024-09-17T05:10:32.394 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T05:10:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53a0073200 msgr2=0x7f53a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53a0073200 0x7f53a0075710 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f53b40af000 tx=0x7f53b401af40 comp rx=0 tx=0).stop 2024-09-17T05:10:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 msgr2=0x7f53c406ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 0x7f53c406ce90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00b9c0 tx=0x7f53bc00be80 comp rx=0 tx=0).stop 2024-09-17T05:10:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 shutdown_connections 2024-09-17T05:10:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f53a0073200 0x7f53a0075710 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c415f110 0x7f53c406ce90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f53c40d0940 0x7f53c40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 --2- 172.21.15.16:0/4190115595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53c40ca6b0 0x7f53c4157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.392+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 >> 172.21.15.16:0/4190115595 conn(0x7f53c40b2250 msgr2=0x7f53c41581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.396+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 shutdown_connections 2024-09-17T05:10:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.396+0000 7f53caa00640 1 -- 172.21.15.16:0/4190115595 wait complete. 2024-09-17T05:10:32.417 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T05:10:32.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 -- 172.21.15.16:0/3837069983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc003ce0 msgr2=0x7fddbc004130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 --2- 172.21.15.16:0/3837069983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddbc003ce0 0x7fddbc004130 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fddd80ce910 tx=0x7fddc801ce60 comp rx=0 tx=0).stop 2024-09-17T05:10:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fddd7e00640 1 -- 172.21.15.16:0/3837069983 reap_dead start 2024-09-17T05:10:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 -- 172.21.15.16:0/3837069983 shutdown_connections 2024-09-17T05:10:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 -- 172.21.15.16:0/3837069983 >> 172.21.15.16:0/3837069983 conn(0x7fddd80d2180 msgr2=0x7fddd80d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 -- 172.21.15.16:0/3837069983 shutdown_connections 2024-09-17T05:10:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 -- 172.21.15.16:0/3837069983 wait complete. 2024-09-17T05:10:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 Processor -- start 2024-09-17T05:10:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.580+0000 7fdddd000640 1 -- start start 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddbc003ce0 0x7fddd806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd81fdb90 0x7fddd80b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddd81fcfc0 0x7fddd81fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8168eb0 con 0x7fddd81fdb90 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8168d30 con 0x7fddbc003ce0 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fddd8169030 con 0x7fddd81fcfc0 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd81fdb90 0x7fddd80b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd81fdb90 0x7fddd80b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53888/0 (socket says 172.21.15.16:53888) 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 -- 172.21.15.16:0/3013919630 learned_addr learned my addr 172.21.15.16:0/3013919630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:32.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddd7400640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddbc003ce0 0x7fddd806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddd7e00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddd81fcfc0 0x7fddd81fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 -- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddd81fcfc0 msgr2=0x7fddd81fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddd81fcfc0 0x7fddd81fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 -- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddbc003ce0 msgr2=0x7fddd806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fddbc003ce0 0x7fddd806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 -- 172.21.15.16:0/3013919630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81fd910 con 0x7fddd81fdb90 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddd7e00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fddd81fcfc0 0x7fddd81fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcea00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd81fdb90 0x7fddd80b0ca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fddc00119b0 tx=0x7fddc0011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc000fcf0 con 0x7fddd81fdb90 2024-09-17T05:10:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddc0004030 con 0x7fddd81fdb90 2024-09-17T05:10:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd8167330 con 0x7fddd81fdb90 2024-09-17T05:10:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0004460 con 0x7fddd81fdb90 2024-09-17T05:10:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81678f0 con 0x7fddd81fdb90 2024-09-17T05:10:32.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.584+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddd80b1e70 con 0x7fddd81fdb90 2024-09-17T05:10:32.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fddc0003600 con 0x7fddd81fdb90 2024-09-17T05:10:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddb4073230 0x7fddb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fddc0095a10 con 0x7fddd81fdb90 2024-09-17T05:10:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fddb4079720 0x7fddb407bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 --> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddc0016e50 con 0x7fddb4079720 2024-09-17T05:10:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fddc0095dc0 con 0x7fddd81fdb90 2024-09-17T05:10:32.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddd7400640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddb4073230 0x7fddb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddd7e00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fddb4079720 0x7fddb407bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddd7e00640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fddb4079720 0x7fddb407bbb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddd7400640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddb4073230 0x7fddb4075740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ac000 tx=0x7fddc8019ca0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.588+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== osd.2 v2:172.21.15.16:6808/3210692914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddc0016e50 con 0x7fddb4079720 2024-09-17T05:10:32.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 --> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fddd806fee0 con 0x7fddb4079720 2024-09-17T05:10:32.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fddcfe00640 1 -- 172.21.15.16:0/3013919630 <== osd.2 v2:172.21.15.16:6808/3210692914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fddd806fee0 con 0x7fddb4079720 2024-09-17T05:10:32.606 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:32.606 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:32.606 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fddb4079720 msgr2=0x7fddb407bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:6808/3210692914,v1:172.21.15.16:6810/3210692914] conn(0x7fddb4079720 0x7fddb407bbb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddb4073230 msgr2=0x7fddb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fddb4073230 0x7fddb4075740 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ac000 tx=0x7fddc8019ca0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd81fdb90 msgr2=0x7fddd80b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 --2- 172.21.15.16:0/3013919630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddd81fdb90 0x7fddd80b0ca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fddc00119b0 tx=0x7fddc0011e70 comp rx=0 tx=0).stop 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fddd7e00640 1 -- 172.21.15.16:0/3013919630 reap_dead start 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 shutdown_connections 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 >> 172.21.15.16:0/3013919630 conn(0x7fddd80d2180 msgr2=0x7fddd8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 shutdown_connections 2024-09-17T05:10:32.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.604+0000 7fdddd000640 1 -- 172.21.15.16:0/3013919630 wait complete. 2024-09-17T05:10:32.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T05:10:32.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.780+0000 7f25ab000640 1 -- 172.21.15.16:0/3930814862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f900 msgr2=0x7f25a406e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.780+0000 7f25ab000640 1 --2- 172.21.15.16:0/3930814862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f900 0x7f25a406e4e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f25a40b3cc0 tx=0x7f259c01cb50 comp rx=0 tx=0).stop 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- 172.21.15.16:0/3930814862 shutdown_connections 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 --2- 172.21.15.16:0/3930814862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f900 0x7f25a406e4e0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 --2- 172.21.15.16:0/3930814862 >> v2:172.21.15.104:3300/0 conn(0x7f25a415f4f0 0x7f25a406dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- 172.21.15.16:0/3930814862 >> 172.21.15.16:0/3930814862 conn(0x7f25a40d0f10 msgr2=0x7f25a40d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- 172.21.15.16:0/3930814862 shutdown_connections 2024-09-17T05:10:32.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- 172.21.15.16:0/3930814862 wait complete. 2024-09-17T05:10:32.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 Processor -- start 2024-09-17T05:10:32.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- start start 2024-09-17T05:10:32.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f25a4070340 0x7f25a41f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 0x7f25a41fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a415f900 0x7f25a41fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4168810 con 0x7f25a415f4f0 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4168690 con 0x7f25a415f900 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.784+0000 7f25ab000640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f25a4168990 con 0x7f25a4070340 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 0x7f25a41fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 0x7f25a41fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53900/0 (socket says 172.21.15.16:53900) 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 -- 172.21.15.16:0/2053971991 learned_addr learned my addr 172.21.15.16:0/2053971991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25aa600640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a415f900 0x7f25a41fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 -- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f25a4070340 msgr2=0x7f25a41f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9c00640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f25a4070340 0x7f25a41f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f25a4070340 0x7f25a41f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 -- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a415f900 msgr2=0x7f25a41fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a415f900 0x7f25a41fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 -- 172.21.15.16:0/2053971991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a40b53a0 con 0x7f25a415f4f0 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25aa600640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a415f900 0x7f25a41fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9c00640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f25a4070340 0x7f25a41f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a9200640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 0x7f25a41fbff0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f259000e7c0 tx=0x7f259000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259000cc60 con 0x7f25a415f4f0 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f259000ce00 con 0x7f25a415f4f0 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590005280 con 0x7f25a415f4f0 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a40b63e0 con 0x7f25a415f4f0 2024-09-17T05:10:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a40b5900 con 0x7f25a415f4f0 2024-09-17T05:10:32.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a415ede0 con 0x7f25a415f4f0 2024-09-17T05:10:32.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.788+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2590005420 con 0x7f25a415f4f0 2024-09-17T05:10:32.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.792+0000 7f25a1600640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2584073230 0x7f2584075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.792+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f2590094af0 con 0x7f25a415f4f0 2024-09-17T05:10:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.792+0000 7f25a9c00640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2584073230 0x7f2584075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.792+0000 7f25a9c00640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2584073230 0x7f2584075740 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2598001630 tx=0x7f2598009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:32.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.796+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f25a40b53a0 con 0x7f25a415f4f0 2024-09-17T05:10:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.972+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f25a40cd790 con 0x7f2584073230 2024-09-17T05:10:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.972+0000 7f25a1600640 1 -- 172.21.15.16:0/2053971991 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f25a40cd790 con 0x7f2584073230 2024-09-17T05:10:32.975 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T05:10:32.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2584073230 msgr2=0x7f2584075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2584073230 0x7f2584075740 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2598001630 tx=0x7f2598009040 comp rx=0 tx=0).stop 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 msgr2=0x7f25a41fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 0x7f25a41fbff0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f259000e7c0 tx=0x7f259000ec80 comp rx=0 tx=0).stop 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 shutdown_connections 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f2584073230 0x7f2584075740 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a415f900 0x7f25a41fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a415f4f0 0x7f25a41fbff0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 --2- 172.21.15.16:0/2053971991 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f25a4070340 0x7f25a41f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 >> 172.21.15.16:0/2053971991 conn(0x7f25a40d0f10 msgr2=0x7f25a41519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 shutdown_connections 2024-09-17T05:10:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:32.976+0000 7f25ab000640 1 -- 172.21.15.16:0/2053971991 wait complete. 2024-09-17T05:10:32.996 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T05:10:32.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 -- 172.21.15.16:0/3710145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3006e4e0 msgr2=0x7fce3015efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 --2- 172.21.15.16:0/3710145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3006e4e0 0x7fce3015efa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fce30073800 tx=0x7fce1801c7d0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 -- 172.21.15.16:0/3710145076 shutdown_connections 2024-09-17T05:10:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 --2- 172.21.15.16:0/3710145076 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce3015f4e0 0x7fce3015f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 --2- 172.21.15.16:0/3710145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3006e4e0 0x7fce3015efa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 --2- 172.21.15.16:0/3710145076 >> v2:172.21.15.16:3300/0 conn(0x7fce300b4620 0x7fce3006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 -- 172.21.15.16:0/3710145076 >> 172.21.15.16:0/3710145076 conn(0x7fce300d0050 msgr2=0x7fce300d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 -- 172.21.15.16:0/3710145076 shutdown_connections 2024-09-17T05:10:33.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.156+0000 7fce36c00640 1 -- 172.21.15.16:0/3710145076 wait complete. 2024-09-17T05:10:33.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 Processor -- start 2024-09-17T05:10:33.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 -- start start 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce3006e4e0 0x7fce301519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce300b4620 0x7fce30155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3015f4e0 0x7fce301f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce3006d6a0 con 0x7fce300b4620 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce3006d820 con 0x7fce3015f4e0 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce36c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fce3006d9a0 con 0x7fce3006e4e0 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce300b4620 0x7fce30155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce300b4620 0x7fce30155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53922/0 (socket says 172.21.15.16:53922) 2024-09-17T05:10:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 -- 172.21.15.16:0/2592477077 learned_addr learned my addr 172.21.15.16:0/2592477077 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce2fe00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce3006e4e0 0x7fce301519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce34c00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3015f4e0 0x7fce301f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 -- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce3006e4e0 msgr2=0x7fce301519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce3006e4e0 0x7fce301519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 -- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3015f4e0 msgr2=0x7fce301f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce3015f4e0 0x7fce301f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce27400640 1 -- 172.21.15.16:0/2592477077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce30168140 con 0x7fce300b4620 2024-09-17T05:10:33.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.160+0000 7fce2fe00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fce3006e4e0 0x7fce301519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce27400640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce300b4620 0x7fce30155400 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fce18000c10 tx=0x7fce18005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce180b2040 con 0x7fce300b4620 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce18004320 con 0x7fce300b4620 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce30155940 con 0x7fce300b4620 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18005270 con 0x7fce300b4620 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce300cf070 con 0x7fce300b4620 2024-09-17T05:10:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce3006dfa0 con 0x7fce300b4620 2024-09-17T05:10:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce18005410 con 0x7fce300b4620 2024-09-17T05:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce0c073230 0x7fce0c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fce1812da90 con 0x7fce300b4620 2024-09-17T05:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.164+0000 7fce2cc00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7fce0c079720 0x7fce0c07bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 --> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce0c07c280 con 0x7fce0c079720 2024-09-17T05:10:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fce1812de40 con 0x7fce300b4620 2024-09-17T05:10:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce34c00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7fce0c079720 0x7fce0c07bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce2fe00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce0c073230 0x7fce0c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce34c00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7fce0c079720 0x7fce0c07bbb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce2fe00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce0c073230 0x7fce0c075740 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fce20009b90 tx=0x7fce20009260 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.168+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== osd.3 v2:172.21.15.16:6800/3613434049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce0c07c280 con 0x7fce0c079720 2024-09-17T05:10:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 --> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fce3015bc30 con 0x7fce0c079720 2024-09-17T05:10:33.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce2cc00640 1 -- 172.21.15.16:0/2592477077 <== osd.3 v2:172.21.15.16:6800/3613434049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fce3015bc30 con 0x7fce0c079720 2024-09-17T05:10:33.184 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:33.184 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7fce0c079720 msgr2=0x7fce0c07bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:6800/3613434049,v1:172.21.15.16:6801/3613434049] conn(0x7fce0c079720 0x7fce0c07bbb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce0c073230 msgr2=0x7fce0c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fce0c073230 0x7fce0c075740 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fce20009b90 tx=0x7fce20009260 comp rx=0 tx=0).stop 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce300b4620 msgr2=0x7fce30155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 --2- 172.21.15.16:0/2592477077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce300b4620 0x7fce30155400 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fce18000c10 tx=0x7fce18005d40 comp rx=0 tx=0).stop 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce34c00640 1 -- 172.21.15.16:0/2592477077 reap_dead start 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 shutdown_connections 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.180+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 >> 172.21.15.16:0/2592477077 conn(0x7fce300d0050 msgr2=0x7fce30070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.184+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 shutdown_connections 2024-09-17T05:10:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.184+0000 7fce36c00640 1 -- 172.21.15.16:0/2592477077 wait complete. 2024-09-17T05:10:33.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T05:10:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 -- 172.21.15.16:0/3598474714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a015f6f0 msgr2=0x7f96a00ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 --2- 172.21.15.16:0/3598474714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a015f6f0 0x7f96a00ceb10 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9698009c80 tx=0x7f969801cda0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 -- 172.21.15.16:0/3598474714 shutdown_connections 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 --2- 172.21.15.16:0/3598474714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a015f6f0 0x7f96a00ceb10 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 --2- 172.21.15.16:0/3598474714 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96a0074e30 0x7f96a00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 -- 172.21.15.16:0/3598474714 >> 172.21.15.16:0/3598474714 conn(0x7f96a00b2250 msgr2=0x7f96a00d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 -- 172.21.15.16:0/3598474714 shutdown_connections 2024-09-17T05:10:33.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.360+0000 7f96a8200640 1 -- 172.21.15.16:0/3598474714 wait complete. 2024-09-17T05:10:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 Processor -- start 2024-09-17T05:10:33.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 -- start start 2024-09-17T05:10:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 0x7f96a0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96a0074e30 0x7f96a01f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96a015f6f0 0x7f96a015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0168e80 con 0x7f96a0074a20 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0168d00 con 0x7f96a015f6f0 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a8200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f96a0169000 con 0x7f96a0074e30 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 0x7f96a0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 0x7f96a0154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53940/0 (socket says 172.21.15.16:53940) 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 -- 172.21.15.16:0/2992331185 learned_addr learned my addr 172.21.15.16:0/2992331185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a6200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96a015f6f0 0x7f96a015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a4e00640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96a0074e30 0x7f96a01f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 -- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96a0074e30 msgr2=0x7f96a01f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96a0074e30 0x7f96a01f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 -- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96a015f6f0 msgr2=0x7f96a015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96a015f6f0 0x7f96a015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a5800640 1 -- 172.21.15.16:0/2992331185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a0154590 con 0x7f96a0074a20 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.364+0000 7f96a6200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96a015f6f0 0x7f96a015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f96a5800640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 0x7f96a0154050 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f969400c3d0 tx=0x7f969400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694017070 con 0x7f96a0074a20 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96940053e0 con 0x7f96a0074a20 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694003880 con 0x7f96a0074a20 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a01fc720 con 0x7f96a0074a20 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a01fcce0 con 0x7f96a0074a20 2024-09-17T05:10:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a015efe0 con 0x7f96a0074a20 2024-09-17T05:10:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f969401a070 con 0x7f96a0074a20 2024-09-17T05:10:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f969d600640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9680073200 0x7f9680075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.368+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f9694093490 con 0x7f96a0074a20 2024-09-17T05:10:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.372+0000 7f96a4e00640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9680073200 0x7f9680075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.372+0000 7f96a4e00640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9680073200 0x7f9680075710 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b1930 tx=0x7f968c0091e0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.376+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f969405e600 con 0x7f96a0074a20 2024-09-17T05:10:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.552+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f96a01fcee0 con 0x7f9680073200 2024-09-17T05:10:33.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.552+0000 7f969d600640 1 -- 172.21.15.16:0/2992331185 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f96a01fcee0 con 0x7f9680073200 2024-09-17T05:10:33.555 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9680073200 msgr2=0x7f9680075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9680073200 0x7f9680075710 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b1930 tx=0x7f968c0091e0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 msgr2=0x7f96a0154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 0x7f96a0154050 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f969400c3d0 tx=0x7f969400c890 comp rx=0 tx=0).stop 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 shutdown_connections 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9680073200 0x7f9680075710 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96a015f6f0 0x7f96a015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96a0074e30 0x7f96a01f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 --2- 172.21.15.16:0/2992331185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a0074a20 0x7f96a0154050 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 >> 172.21.15.16:0/2992331185 conn(0x7f96a00b2250 msgr2=0x7f96a00b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 shutdown_connections 2024-09-17T05:10:33.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.556+0000 7f96a8200640 1 -- 172.21.15.16:0/2992331185 wait complete. 2024-09-17T05:10:33.577 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T05:10:33.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.736+0000 7f5885600640 1 -- 172.21.15.16:0/1609472739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 msgr2=0x7f588015ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.736+0000 7f5885600640 1 --2- 172.21.15.16:0/1609472739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f588015ccc0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5880075a50 tx=0x7f587801cd50 comp rx=0 tx=0).stop 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- 172.21.15.16:0/1609472739 shutdown_connections 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 --2- 172.21.15.16:0/1609472739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f588015ccc0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 --2- 172.21.15.16:0/1609472739 >> v2:172.21.15.104:3300/0 conn(0x7f58800d1dc0 0x7f58800cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 --2- 172.21.15.16:0/1609472739 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58800cfdf0 0x7f58800d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- 172.21.15.16:0/1609472739 >> 172.21.15.16:0/1609472739 conn(0x7f58800715e0 msgr2=0x7f58800719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- 172.21.15.16:0/1609472739 shutdown_connections 2024-09-17T05:10:33.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- 172.21.15.16:0/1609472739 wait complete. 2024-09-17T05:10:33.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 Processor -- start 2024-09-17T05:10:33.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- start start 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f58800c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58800cfdf0 0x7f58801f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58800d1dc0 0x7f58800c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5880072f80 con 0x7f58800cf170 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5880072e00 con 0x7f58800d1dc0 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.740+0000 7f5885600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5880073100 con 0x7f58800cfdf0 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f58800c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f58800c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53954/0 (socket says 172.21.15.16:53954) 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 -- 172.21.15.16:0/535585986 learned_addr learned my addr 172.21.15.16:0/535585986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 -- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58800cfdf0 msgr2=0x7f58801f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f58800cfdf0 0x7f58801f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 -- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58800d1dc0 msgr2=0x7f58800c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587f400640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58800d1dc0 0x7f58800c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58800d1dc0 0x7f58800c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 -- 172.21.15.16:0/535585986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5880151910 con 0x7f58800cf170 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587f400640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58800d1dc0 0x7f58800c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f587ea00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f58800c8790 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f587000e740 tx=0x7f587000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587000ccc0 con 0x7f58800cf170 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f587000ce60 con 0x7f58800cf170 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870005280 con 0x7f58800cf170 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5880151bf0 con 0x7f58800cf170 2024-09-17T05:10:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801f8af0 con 0x7f58800cf170 2024-09-17T05:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58801f9550 con 0x7f58800cf170 2024-09-17T05:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.744+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5870005420 con 0x7f58800cf170 2024-09-17T05:10:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5876a00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f585c073230 0x7f585c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f5870094ae0 con 0x7f58800cf170 2024-09-17T05:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5876a00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f585c079720 0x7f585c07bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 --> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5870016ec0 con 0x7f585c079720 2024-09-17T05:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f58700992a0 con 0x7f58800cf170 2024-09-17T05:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5877e00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f585c073230 0x7f585c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f587f400640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f585c079720 0x7f585c07bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5877e00640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f585c073230 0x7f585c075740 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f588015cde0 tx=0x7f5868005cf0 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f587f400640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f585c079720 0x7f585c07bbb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.748+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== osd.4 v2:172.21.15.104:6810/2719508844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5870016ec0 con 0x7f585c079720 2024-09-17T05:10:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 --> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f58801f8600 con 0x7f585c079720 2024-09-17T05:10:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5876a00640 1 -- 172.21.15.16:0/535585986 <== osd.4 v2:172.21.15.104:6810/2719508844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f58801f8600 con 0x7f585c079720 2024-09-17T05:10:33.765 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:33.766 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f585c079720 msgr2=0x7f585c07bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6810/2719508844,v1:172.21.15.104:6811/2719508844] conn(0x7f585c079720 0x7f585c07bbb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f585c073230 msgr2=0x7f585c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f585c073230 0x7f585c075740 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f588015cde0 tx=0x7f5868005cf0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 msgr2=0x7f58800c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 --2- 172.21.15.16:0/535585986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58800cf170 0x7f58800c8790 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f587000e740 tx=0x7f587000ec00 comp rx=0 tx=0).stop 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f587f400640 1 -- 172.21.15.16:0/535585986 reap_dead start 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 shutdown_connections 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 >> 172.21.15.16:0/535585986 conn(0x7f58800715e0 msgr2=0x7f588015dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 shutdown_connections 2024-09-17T05:10:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.764+0000 7f5885600640 1 -- 172.21.15.16:0/535585986 wait complete. 2024-09-17T05:10:33.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T05:10:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3808200640 1 -- 172.21.15.16:0/805000066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 msgr2=0x7f37e8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3808200640 1 --2- 172.21.15.16:0/805000066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f37e8004c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f38000d0c90 tx=0x7f37ec01f6e0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3806200640 1 -- 172.21.15.16:0/805000066 reap_dead start 2024-09-17T05:10:33.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3808200640 1 -- 172.21.15.16:0/805000066 shutdown_connections 2024-09-17T05:10:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3808200640 1 -- 172.21.15.16:0/805000066 >> 172.21.15.16:0/805000066 conn(0x7f38000b2250 msgr2=0x7f38000d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3808200640 1 -- 172.21.15.16:0/805000066 shutdown_connections 2024-09-17T05:10:33.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.944+0000 7f3808200640 1 -- 172.21.15.16:0/805000066 wait complete. 2024-09-17T05:10:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 Processor -- start 2024-09-17T05:10:33.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 -- start start 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f38000c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38001f65d0 0x7f3800150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3800156080 0x7f38000b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f380015c6a0 con 0x7f37e8004810 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f380006e790 con 0x7f38001f65d0 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3808200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f380015c820 con 0x7f3800156080 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f38000c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f38000c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53986/0 (socket says 172.21.15.16:53986) 2024-09-17T05:10:33.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 -- 172.21.15.16:0/985536945 learned_addr learned my addr 172.21.15.16:0/985536945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3804e00640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38001f65d0 0x7f3800150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 -- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3800156080 msgr2=0x7f38000b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3800156080 0x7f38000b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 -- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38001f65d0 msgr2=0x7f3800150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38001f65d0 0x7f3800150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 -- 172.21.15.16:0/985536945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38000b2ff0 con 0x7f37e8004810 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3804e00640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38001f65d0 0x7f3800150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f3805800640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f38000c9e00 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f37f40119b0 tx=0x7f37f4011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f400fc70 con 0x7f37e8004810 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37f4004030 con 0x7f37e8004810 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.948+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4004460 con 0x7f37e8004810 2024-09-17T05:10:33.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38000b32d0 con 0x7f37e8004810 2024-09-17T05:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38000b3890 con 0x7f37e8004810 2024-09-17T05:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380015f280 con 0x7f37e8004810 2024-09-17T05:10:33.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37f4003600 con 0x7f37e8004810 2024-09-17T05:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f37fd600640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37e0073200 0x7f37e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:33.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f37f4094e20 con 0x7f37e8004810 2024-09-17T05:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.952+0000 7f3804e00640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37e0073200 0x7f37e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:33.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.956+0000 7f3804e00640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37e0073200 0x7f37e0075710 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38000d0c90 tx=0x7f37ec01c520 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:33.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:33.956+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f37f405ff10 con 0x7f37e8004810 2024-09-17T05:10:34.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.132+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f38000cf7a0 con 0x7f37e0073200 2024-09-17T05:10:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.132+0000 7f37fd600640 1 -- 172.21.15.16:0/985536945 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f38000cf7a0 con 0x7f37e0073200 2024-09-17T05:10:34.134 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T05:10:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.132+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37e0073200 msgr2=0x7f37e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.132+0000 7f3808200640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37e0073200 0x7f37e0075710 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38000d0c90 tx=0x7f37ec01c520 comp rx=0 tx=0).stop 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.132+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 msgr2=0x7f38000c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.132+0000 7f3808200640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f38000c9e00 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f37f40119b0 tx=0x7f37f4011e70 comp rx=0 tx=0).stop 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 shutdown_connections 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f37e0073200 0x7f37e0075710 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f3800156080 0x7f38000b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38001f65d0 0x7f3800150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 --2- 172.21.15.16:0/985536945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37e8004810 0x7f38000c9e00 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 >> 172.21.15.16:0/985536945 conn(0x7f38000b2250 msgr2=0x7f38000cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 shutdown_connections 2024-09-17T05:10:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.136+0000 7f3808200640 1 -- 172.21.15.16:0/985536945 wait complete. 2024-09-17T05:10:34.156 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T05:10:34.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 -- 172.21.15.16:0/1046894120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c0b4620 msgr2=0x7f5e2c06dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 --2- 172.21.15.16:0/1046894120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c0b4620 0x7f5e2c06dfa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009840 tx=0x7f5e1c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 -- 172.21.15.16:0/1046894120 shutdown_connections 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 --2- 172.21.15.16:0/1046894120 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e2c074bf0 0x7f5e2c15f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 --2- 172.21.15.16:0/1046894120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c0b4620 0x7f5e2c06dfa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 -- 172.21.15.16:0/1046894120 >> 172.21.15.16:0/1046894120 conn(0x7f5e2c0d0050 msgr2=0x7f5e2c0d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 -- 172.21.15.16:0/1046894120 shutdown_connections 2024-09-17T05:10:34.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.316+0000 7f5e33800640 1 -- 172.21.15.16:0/1046894120 wait complete. 2024-09-17T05:10:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 Processor -- start 2024-09-17T05:10:34.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 -- start start 2024-09-17T05:10:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2c06e4e0 0x7f5e2c09cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c074bf0 0x7f5e2c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e2c0b4620 0x7f5e2c1f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c152840 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c15fe00 con 0x7f5e2c074bf0 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e33800640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2c168920 con 0x7f5e2c0b4620 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e30e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2c06e4e0 0x7f5e2c09cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e30e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2c06e4e0 0x7f5e2c09cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54002/0 (socket says 172.21.15.16:54002) 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e30e00640 1 -- 172.21.15.16:0/672278023 learned_addr learned my addr 172.21.15.16:0/672278023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e23e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c074bf0 0x7f5e2c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.320+0000 7f5e31800640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e2c0b4620 0x7f5e2c1f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e30e00640 1 -- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e2c0b4620 msgr2=0x7f5e2c1f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e30e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f5e2c0b4620 0x7f5e2c1f9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e30e00640 1 -- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c074bf0 msgr2=0x7f5e2c152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e30e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c074bf0 0x7f5e2c152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e30e00640 1 -- 172.21.15.16:0/672278023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c153980 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e23e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e2c074bf0 0x7f5e2c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e30e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2c06e4e0 0x7f5e2c09cf10 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009790 tx=0x7f5e1c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0b2040 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c15cc10 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e1c004330 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c003800 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c15d1d0 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e2c153c60 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e1c0bd060 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.324+0000 7f5e29600640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e08073200 0x7f5e08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f5e1c12d910 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e29600640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f5e080796f0 0x7f5e0807bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 --> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e1c0b0e70 con 0x7f5e080796f0 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f5e1c1322a0 con 0x7f5e2c06e4e0 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e23e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e08073200 0x7f5e08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e31800640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f5e080796f0 0x7f5e0807bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e23e00640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e08073200 0x7f5e08075710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c14f850 tx=0x7f5e14009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e31800640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f5e080796f0 0x7f5e0807bb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.328+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== osd.5 v2:172.21.15.104:6802/464547405 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e1c0b0e70 con 0x7f5e080796f0 2024-09-17T05:10:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.340+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 --> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5e2c06dfa0 con 0x7f5e080796f0 2024-09-17T05:10:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.340+0000 7f5e29600640 1 -- 172.21.15.16:0/672278023 <== osd.5 v2:172.21.15.104:6802/464547405 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5e2c06dfa0 con 0x7f5e080796f0 2024-09-17T05:10:34.344 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:34.344 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f5e080796f0 msgr2=0x7f5e0807bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6802/464547405,v1:172.21.15.104:6803/464547405] conn(0x7f5e080796f0 0x7f5e0807bb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e08073200 msgr2=0x7f5e08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f5e08073200 0x7f5e08075710 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c14f850 tx=0x7f5e14009040 comp rx=0 tx=0).stop 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2c06e4e0 msgr2=0x7f5e2c09cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 --2- 172.21.15.16:0/672278023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e2c06e4e0 0x7f5e2c09cf10 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009790 tx=0x7f5e1c005db0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e31800640 1 -- 172.21.15.16:0/672278023 reap_dead start 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 shutdown_connections 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 >> 172.21.15.16:0/672278023 conn(0x7f5e2c0d0050 msgr2=0x7f5e2c151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 shutdown_connections 2024-09-17T05:10:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.344+0000 7f5e33800640 1 -- 172.21.15.16:0/672278023 wait complete. 2024-09-17T05:10:34.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 -- 172.21.15.16:0/1617942088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 msgr2=0x7f1b5015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 --2- 172.21.15.16:0/1617942088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b5015efe0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f1b500ce2c0 tx=0x7f1b4001cda0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 -- 172.21.15.16:0/1617942088 shutdown_connections 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 --2- 172.21.15.16:0/1617942088 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b5015f520 0x7f1b5015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 --2- 172.21.15.16:0/1617942088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b5015efe0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 -- 172.21.15.16:0/1617942088 >> 172.21.15.16:0/1617942088 conn(0x7f1b500b2250 msgr2=0x7f1b500d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 -- 172.21.15.16:0/1617942088 shutdown_connections 2024-09-17T05:10:34.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.520+0000 7f1b56a00640 1 -- 172.21.15.16:0/1617942088 wait complete. 2024-09-17T05:10:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 Processor -- start 2024-09-17T05:10:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 -- start start 2024-09-17T05:10:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b500c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b50074e30 0x7f1b501f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b5015f520 0x7f1b500adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50168f30 con 0x7f1b50074a20 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50168db0 con 0x7f1b50074e30 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f1b501690b0 con 0x7f1b5015f520 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b500c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b500c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54020/0 (socket says 172.21.15.16:54020) 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 -- 172.21.15.16:0/921976664 learned_addr learned my addr 172.21.15.16:0/921976664 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b47400640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b50074e30 0x7f1b501f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b54a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b5015f520 0x7f1b500adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 -- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b5015f520 msgr2=0x7f1b500adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b5015f520 0x7f1b500adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 -- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b50074e30 msgr2=0x7f1b501f9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b50074e30 0x7f1b501f9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 -- 172.21.15.16:0/921976664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b501f9cd0 con 0x7f1b50074a20 2024-09-17T05:10:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4fe00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b500c8160 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1b4001a9d0 tx=0x7f1b400092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400bb070 con 0x7f1b50074a20 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b40004c50 con 0x7f1b50074a20 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b500b3080 con 0x7f1b50074a20 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b400b2040 con 0x7f1b50074a20 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.524+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b500ad9f0 con 0x7f1b50074a20 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.528+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5015efe0 con 0x7f1b50074a20 2024-09-17T05:10:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.528+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1b400be070 con 0x7f1b50074a20 2024-09-17T05:10:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.528+0000 7f1b4cc00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b2c0731f0 0x7f1b2c075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.528+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f1b4012db20 con 0x7f1b50074a20 2024-09-17T05:10:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.528+0000 7f1b47400640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b2c0731f0 0x7f1b2c075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.532+0000 7f1b47400640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b2c0731f0 0x7f1b2c075700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1b500b4880 tx=0x7f1b38009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.532+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1b501f9cd0 con 0x7f1b50074a20 2024-09-17T05:10:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f1b500af5f0 con 0x7f1b2c0731f0 2024-09-17T05:10:34.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b4cc00640 1 -- 172.21.15.16:0/921976664 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1b38001490 con 0x7f1b2c0731f0 2024-09-17T05:10:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b2c0731f0 msgr2=0x7f1b2c075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b2c0731f0 0x7f1b2c075700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1b500b4880 tx=0x7f1b38009040 comp rx=0 tx=0).stop 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 msgr2=0x7f1b500c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b500c8160 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1b4001a9d0 tx=0x7f1b400092c0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 shutdown_connections 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f1b2c0731f0 0x7f1b2c075700 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f1b5015f520 0x7f1b500adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b50074e30 0x7f1b501f9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 --2- 172.21.15.16:0/921976664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50074a20 0x7f1b500c8160 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 >> 172.21.15.16:0/921976664 conn(0x7f1b500b2250 msgr2=0x7f1b5015ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 shutdown_connections 2024-09-17T05:10:34.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.708+0000 7f1b56a00640 1 -- 172.21.15.16:0/921976664 wait complete. 2024-09-17T05:10:34.732 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T05:10:34.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 -- 172.21.15.16:0/558779862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c0ce400 msgr2=0x7f0e3c070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 --2- 172.21.15.16:0/558779862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c070d00 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0702e0 tx=0x7f0e3c071a50 comp rx=0 tx=0).stop 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 -- 172.21.15.16:0/558779862 shutdown_connections 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 --2- 172.21.15.16:0/558779862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c070d00 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 -- 172.21.15.16:0/558779862 >> 172.21.15.16:0/558779862 conn(0x7f0e3c073ea0 msgr2=0x7f0e3c0742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 -- 172.21.15.16:0/558779862 shutdown_connections 2024-09-17T05:10:34.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.892+0000 7f0e40e00640 1 -- 172.21.15.16:0/558779862 wait complete. 2024-09-17T05:10:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 Processor -- start 2024-09-17T05:10:34.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 -- start start 2024-09-17T05:10:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0e3c0cdff0 0x7f0e3c148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c14a260 0x7f0e3c14af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c169f20 con 0x7f0e3c0ce400 2024-09-17T05:10:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c169da0 con 0x7f0e3c14a260 2024-09-17T05:10:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c16a0a0 con 0x7f0e3c0cdff0 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54030/0 (socket says 172.21.15.16:54030) 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 -- 172.21.15.16:0/1579381817 learned_addr learned my addr 172.21.15.16:0/1579381817 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e39e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0e3c0cdff0 0x7f0e3c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e3a800640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c14a260 0x7f0e3c14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 -- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0e3c0cdff0 msgr2=0x7f0e3c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0e3c0cdff0 0x7f0e3c148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 -- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c14a260 msgr2=0x7f0e3c14af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e3c14a260 0x7f0e3c14af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 -- 172.21.15.16:0/1579381817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c14df50 con 0x7f0e3c0ce400 2024-09-17T05:10:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e39e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f0e3c0cdff0 0x7f0e3c148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e31400640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c14da10 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c400 tx=0x7f0e2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24011790 con 0x7f0e3c0ce400 2024-09-17T05:10:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e24017070 con 0x7f0e3c0ce400 2024-09-17T05:10:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24011ca0 con 0x7f0e3c0ce400 2024-09-17T05:10:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c1491c0 con 0x7f0e3c0ce400 2024-09-17T05:10:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.896+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c149780 con 0x7f0e3c0ce400 2024-09-17T05:10:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e3c0b4d30 con 0x7f0e3c0ce400 2024-09-17T05:10:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0e2401c460 con 0x7f0e3c0ce400 2024-09-17T05:10:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e32a00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0e18073230 0x7f0e18075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f0e24094780 con 0x7f0e3c0ce400 2024-09-17T05:10:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e32a00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f0e18079720 0x7f0e1807bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 --> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e24025e50 con 0x7f0e18079720 2024-09-17T05:10:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f0e24094b30 con 0x7f0e3c0ce400 2024-09-17T05:10:34.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e39e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0e18073230 0x7f0e18075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.900+0000 7f0e3a800640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f0e18079720 0x7f0e1807bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:34.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.904+0000 7f0e3a800640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f0e18079720 0x7f0e1807bbb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.904+0000 7f0e39e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0e18073230 0x7f0e18075740 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c000f70 tx=0x7f0e2c009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.904+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== osd.6 v2:172.21.15.104:6826/3199393218 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e24025e50 con 0x7f0e18079720 2024-09-17T05:10:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 --> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0e3c14e210 con 0x7f0e18079720 2024-09-17T05:10:34.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e32a00640 1 -- 172.21.15.16:0/1579381817 <== osd.6 v2:172.21.15.104:6826/3199393218 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0e3c14e210 con 0x7f0e18079720 2024-09-17T05:10:34.920 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:34.920 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:34.920 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f0e18079720 msgr2=0x7f0e1807bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6826/3199393218,v1:172.21.15.104:6827/3199393218] conn(0x7f0e18079720 0x7f0e1807bbb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0e18073230 msgr2=0x7f0e18075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f0e18073230 0x7f0e18075740 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c000f70 tx=0x7f0e2c009040 comp rx=0 tx=0).stop 2024-09-17T05:10:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3c0ce400 msgr2=0x7f0e3c14da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.916+0000 7f0e40e00640 1 --2- 172.21.15.16:0/1579381817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e3c0ce400 0x7f0e3c14da10 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c400 tx=0x7f0e2400c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.920+0000 7f0e3a800640 1 -- 172.21.15.16:0/1579381817 reap_dead start 2024-09-17T05:10:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.920+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 shutdown_connections 2024-09-17T05:10:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.920+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 >> 172.21.15.16:0/1579381817 conn(0x7f0e3c073ea0 msgr2=0x7f0e3c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.920+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 shutdown_connections 2024-09-17T05:10:34.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:34.920+0000 7f0e40e00640 1 -- 172.21.15.16:0/1579381817 wait complete. 2024-09-17T05:10:34.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T05:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 -- 172.21.15.16:0/1497858903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 msgr2=0x7f955c15d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 --2- 172.21.15.16:0/1497858903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c15d140 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f954c009840 tx=0x7f954c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 -- 172.21.15.16:0/1497858903 shutdown_connections 2024-09-17T05:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 --2- 172.21.15.16:0/1497858903 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c15d680 0x7f955c160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 --2- 172.21.15.16:0/1497858903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c15d140 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 -- 172.21.15.16:0/1497858903 >> 172.21.15.16:0/1497858903 conn(0x7f955c0b2250 msgr2=0x7f955c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 -- 172.21.15.16:0/1497858903 shutdown_connections 2024-09-17T05:10:35.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.096+0000 7f9560c00640 1 -- 172.21.15.16:0/1497858903 wait complete. 2024-09-17T05:10:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 Processor -- start 2024-09-17T05:10:35.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 -- start start 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c0aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15d680 0x7f955c13db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c160670 0x7f955c13e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f955c073970 con 0x7f955c0733e0 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f955c0737f0 con 0x7f955c15d680 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9560c00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f955c16a2d0 con 0x7f955c160670 2024-09-17T05:10:35.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9559e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c0aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9559e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c0aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54042/0 (socket says 172.21.15.16:54042) 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9559e00640 1 -- 172.21.15.16:0/3157621497 learned_addr learned my addr 172.21.15.16:0/3157621497 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f9551400640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15d680 0x7f955c13db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.100+0000 7f955a800640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c160670 0x7f955c13e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9559e00640 1 -- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c160670 msgr2=0x7f955c13e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9559e00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c160670 0x7f955c13e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9559e00640 1 -- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15d680 msgr2=0x7f955c13db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9559e00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15d680 0x7f955c13db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9559e00640 1 -- 172.21.15.16:0/3157621497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c152ab0 con 0x7f955c0733e0 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9551400640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15d680 0x7f955c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9559e00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c0aeb90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f954c009790 tx=0x7f954c0a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c0bb070 con 0x7f955c0733e0 2024-09-17T05:10:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f954c004430 con 0x7f955c0733e0 2024-09-17T05:10:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c15bef0 con 0x7f955c0733e0 2024-09-17T05:10:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c0b2040 con 0x7f955c0733e0 2024-09-17T05:10:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c15c4b0 con 0x7f955c0733e0 2024-09-17T05:10:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c152c80 con 0x7f955c0733e0 2024-09-17T05:10:35.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.104+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f954c0be070 con 0x7f955c0733e0 2024-09-17T05:10:35.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.108+0000 7f9552a00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9538073230 0x7f9538075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.108+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f954c12dac0 con 0x7f955c0733e0 2024-09-17T05:10:35.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.108+0000 7f9551400640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9538073230 0x7f9538075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.108+0000 7f9551400640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9538073230 0x7f9538075740 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9544005fd0 tx=0x7f9544005f20 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.112+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f954c016410 con 0x7f955c0733e0 2024-09-17T05:10:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.288+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f955c15d140 con 0x7f9538073230 2024-09-17T05:10:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.288+0000 7f9552a00640 1 -- 172.21.15.16:0/3157621497 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f955c15d140 con 0x7f9538073230 2024-09-17T05:10:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9538073230 msgr2=0x7f9538075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9538073230 0x7f9538075740 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9544005fd0 tx=0x7f9544005f20 comp rx=0 tx=0).stop 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 msgr2=0x7f955c0aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c0aeb90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f954c009790 tx=0x7f954c0a57c0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 shutdown_connections 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f9538073230 0x7f9538075740 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f955c160670 0x7f955c13e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f955c15d680 0x7f955c13db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 --2- 172.21.15.16:0/3157621497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f955c0733e0 0x7f955c0aeb90 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 >> 172.21.15.16:0/3157621497 conn(0x7f955c0b2250 msgr2=0x7f955c0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 shutdown_connections 2024-09-17T05:10:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.292+0000 7f9560c00640 1 -- 172.21.15.16:0/3157621497 wait complete. 2024-09-17T05:10:35.312 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T05:10:35.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T05:10:35.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 -- 172.21.15.16:0/3697464604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00cb790 msgr2=0x7f96d00d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 --2- 172.21.15.16:0/3697464604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00cb790 0x7f96d00d0310 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009930 tx=0x7f96bc01c790 comp rx=0 tx=0).stop 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 -- 172.21.15.16:0/3697464604 shutdown_connections 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 --2- 172.21.15.16:0/3697464604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00cb790 0x7f96d00d0310 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 --2- 172.21.15.16:0/3697464604 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96d006f590 0x7f96d00d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 -- 172.21.15.16:0/3697464604 >> 172.21.15.16:0/3697464604 conn(0x7f96d009a9f0 msgr2=0x7f96d00b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 -- 172.21.15.16:0/3697464604 shutdown_connections 2024-09-17T05:10:35.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.472+0000 7f96d6600640 1 -- 172.21.15.16:0/3697464604 wait complete. 2024-09-17T05:10:35.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 Processor -- start 2024-09-17T05:10:35.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 -- start start 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96d006f590 0x7f96d00b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96d00cb790 0x7f96d0155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00d0850 0x7f96d01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96d016c320 con 0x7f96d00d0850 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f96d016c1a0 con 0x7f96d006f590 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96d6600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f96d016c4a0 con 0x7f96d00cb790 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00d0850 0x7f96d01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00d0850 0x7f96d01560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54060/0 (socket says 172.21.15.16:54060) 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 -- 172.21.15.16:0/2358994467 learned_addr learned my addr 172.21.15.16:0/2358994467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cf400640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96d006f590 0x7f96d00b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cea00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96d00cb790 0x7f96d0155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 -- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96d00cb790 msgr2=0x7f96d0155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96d00cb790 0x7f96d0155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 -- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96d006f590 msgr2=0x7f96d00b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96d006f590 0x7f96d00b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 -- 172.21.15.16:0/2358994467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d00affc0 con 0x7f96d00d0850 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cf400640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f96d006f590 0x7f96d00b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cea00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f96d00cb790 0x7f96d0155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:35.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96cfe00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00d0850 0x7f96d01560f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f96c800be90 tx=0x7f96c800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.476+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8019070 con 0x7f96d00d0850 2024-09-17T05:10:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96c8014440 con 0x7f96d00d0850 2024-09-17T05:10:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c80148d0 con 0x7f96d00d0850 2024-09-17T05:10:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d00b02a0 con 0x7f96d00d0850 2024-09-17T05:10:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d00aeaa0 con 0x7f96d00d0850 2024-09-17T05:10:35.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96d00d1790 con 0x7f96d00d0850 2024-09-17T05:10:35.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96c801c070 con 0x7f96d00d0850 2024-09-17T05:10:35.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96b0073230 0x7f96b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f96c80949e0 con 0x7f96d00d0850 2024-09-17T05:10:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f96b0079720 0x7f96b007bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 --> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96c80091a0 con 0x7f96b0079720 2024-09-17T05:10:35.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f96c8094d90 con 0x7f96d00d0850 2024-09-17T05:10:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.480+0000 7f96cf400640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96b0073230 0x7f96b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.484+0000 7f96cea00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f96b0079720 0x7f96b007bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.484+0000 7f96cf400640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96b0073230 0x7f96b0075740 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f96c0005fd0 tx=0x7f96c000a540 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.484+0000 7f96cea00640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f96b0079720 0x7f96b007bbb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.484+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== osd.7 v2:172.21.15.104:6818/2403343602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96c80091a0 con 0x7f96b0079720 2024-09-17T05:10:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.496+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 --> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96d01544e0 con 0x7f96b0079720 2024-09-17T05:10:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.496+0000 7f96c7400640 1 -- 172.21.15.16:0/2358994467 <== osd.7 v2:172.21.15.104:6818/2403343602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96d01544e0 con 0x7f96b0079720 2024-09-17T05:10:35.501 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-17T05:10:35.501 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T05:10:35.501 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-17T05:10:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f96b0079720 msgr2=0x7f96b007bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6818/2403343602,v1:172.21.15.104:6819/2403343602] conn(0x7f96b0079720 0x7f96b007bbb0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96b0073230 msgr2=0x7f96b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f96b0073230 0x7f96b0075740 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f96c0005fd0 tx=0x7f96c000a540 comp rx=0 tx=0).stop 2024-09-17T05:10:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00d0850 msgr2=0x7f96d01560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 --2- 172.21.15.16:0/2358994467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96d00d0850 0x7f96d01560f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f96c800be90 tx=0x7f96c800bf80 comp rx=0 tx=0).stop 2024-09-17T05:10:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96cfe00640 1 -- 172.21.15.16:0/2358994467 reap_dead start 2024-09-17T05:10:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 shutdown_connections 2024-09-17T05:10:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 >> 172.21.15.16:0/2358994467 conn(0x7f96d009a9f0 msgr2=0x7f96d00cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 shutdown_connections 2024-09-17T05:10:35.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.500+0000 7f96d6600640 1 -- 172.21.15.16:0/2358994467 wait complete. 2024-09-17T05:10:35.520 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T05:10:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 -- 172.21.15.16:0/1522815969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 msgr2=0x7fe91c1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 --2- 172.21.15.16:0/1522815969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 0x7fe91c1590a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009840 tx=0x7fe90c01c7f0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 -- 172.21.15.16:0/1522815969 shutdown_connections 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 --2- 172.21.15.16:0/1522815969 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe91c0ca6b0 0x7fe91c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 --2- 172.21.15.16:0/1522815969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 0x7fe91c1590a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 -- 172.21.15.16:0/1522815969 >> 172.21.15.16:0/1522815969 conn(0x7fe91c0b2250 msgr2=0x7fe91c0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 -- 172.21.15.16:0/1522815969 shutdown_connections 2024-09-17T05:10:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.676+0000 7fe922200640 1 -- 172.21.15.16:0/1522815969 wait complete. 2024-09-17T05:10:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 Processor -- start 2024-09-17T05:10:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 -- start start 2024-09-17T05:10:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe91c0ca6b0 0x7fe91c15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 0x7fe91c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 0x7fe91c1f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c168c90 con 0x7fe91c15f100 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c168e10 con 0x7fe91c0d0940 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe922200640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fe91c168f90 con 0x7fe91c0ca6b0 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe91be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 0x7fe91c1f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe91b400640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe91c0ca6b0 0x7fe91c15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe91be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 0x7fe91c1f8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54066/0 (socket says 172.21.15.16:54066) 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.680+0000 7fe91be00640 1 -- 172.21.15.16:0/3507207521 learned_addr learned my addr 172.21.15.16:0/3507207521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe913e00640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 0x7fe91c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe91be00640 1 -- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe91c0ca6b0 msgr2=0x7fe91c15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe91be00640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe91c0ca6b0 0x7fe91c15a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe91be00640 1 -- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 msgr2=0x7fe91c1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe91be00640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 0x7fe91c1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe91be00640 1 -- 172.21.15.16:0/3507207521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c1f95a0 con 0x7fe91c15f100 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe91be00640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 0x7fe91c1f8ae0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe91400caa0 tx=0x7fe91400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914016020 con 0x7fe91c15f100 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe91400a930 con 0x7fe91c15f100 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c1f9880 con 0x7fe91c15f100 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91400adc0 con 0x7fe91c15f100 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c0a9990 con 0x7fe91c15f100 2024-09-17T05:10:35.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c1f9310 con 0x7fe91c15f100 2024-09-17T05:10:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.684+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe914019020 con 0x7fe91c15f100 2024-09-17T05:10:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.688+0000 7fe913400640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8f8073200 0x7fe8f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.688+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fe9140942a0 con 0x7fe91c15f100 2024-09-17T05:10:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.688+0000 7fe91b400640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8f8073200 0x7fe8f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:35.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.688+0000 7fe91b400640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8f8073200 0x7fe8f8075710 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0af000 tx=0x7fe90c01b030 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:35.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.692+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe914020020 con 0x7fe91c15f100 2024-09-17T05:10:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe91c152db0 con 0x7fe8f8073200 2024-09-17T05:10:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe913400640 1 -- 172.21.15.16:0/3507207521 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe91c152db0 con 0x7fe8f8073200 2024-09-17T05:10:35.869 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T05:10:35.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8f8073200 msgr2=0x7fe8f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8f8073200 0x7fe8f8075710 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0af000 tx=0x7fe90c01b030 comp rx=0 tx=0).stop 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 msgr2=0x7fe91c1f8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 0x7fe91c1f8ae0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fe91400caa0 tx=0x7fe91400cb90 comp rx=0 tx=0).stop 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 shutdown_connections 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fe8f8073200 0x7fe8f8075710 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe91c15f100 0x7fe91c1f8ae0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe91c0d0940 0x7fe91c1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 --2- 172.21.15.16:0/3507207521 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fe91c0ca6b0 0x7fe91c15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 >> 172.21.15.16:0/3507207521 conn(0x7fe91c0b2250 msgr2=0x7fe91c15f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 shutdown_connections 2024-09-17T05:10:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:35.868+0000 7fe922200640 1 -- 172.21.15.16:0/3507207521 wait complete. 2024-09-17T05:10:35.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 -- 172.21.15.16:0/4266699517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8003c40 msgr2=0x7f19d8004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 --2- 172.21.15.16:0/4266699517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8003c40 0x7f19d8004090 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001830 tx=0x7f19e401c880 comp rx=0 tx=0).stop 2024-09-17T05:10:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 -- 172.21.15.16:0/4266699517 shutdown_connections 2024-09-17T05:10:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 --2- 172.21.15.16:0/4266699517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d8003c40 0x7f19d8004090 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 -- 172.21.15.16:0/4266699517 >> 172.21.15.16:0/4266699517 conn(0x7f19f406f900 msgr2=0x7f19f40b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 -- 172.21.15.16:0/4266699517 shutdown_connections 2024-09-17T05:10:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.048+0000 7f19f8a00640 1 -- 172.21.15.16:0/4266699517 wait complete. 2024-09-17T05:10:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 Processor -- start 2024-09-17T05:10:36.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 -- start start 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19d8003c40 0x7f19f415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 0x7f19f413fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19f413bdf0 0x7f19f4140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19f40b51f0 con 0x7f19f413b9e0 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f19f40b5070 con 0x7f19f413bdf0 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f8a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f19f4160170 con 0x7f19d8003c40 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 0x7f19f413fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 0x7f19f413fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54090/0 (socket says 172.21.15.16:54090) 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 -- 172.21.15.16:0/2344235794 learned_addr learned my addr 172.21.15.16:0/2344235794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:36.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f1e00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19d8003c40 0x7f19f415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f2800640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19f413bdf0 0x7f19f4140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 -- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19d8003c40 msgr2=0x7f19f415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19d8003c40 0x7f19f415a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 -- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19f413bdf0 msgr2=0x7f19f4140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19f413bdf0 0x7f19f4140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19ebe00640 1 -- 172.21.15.16:0/2344235794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f41582e0 con 0x7f19f413b9e0 2024-09-17T05:10:36.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.052+0000 7f19f2800640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19f413bdf0 0x7f19f4140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T05:10:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ebe00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 0x7f19f413fff0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f19dc011ab0 tx=0x7f19dc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc01e070 con 0x7f19f413b9e0 2024-09-17T05:10:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19dc00fd30 con 0x7f19f413b9e0 2024-09-17T05:10:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc019670 con 0x7f19f413b9e0 2024-09-17T05:10:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f41585c0 con 0x7f19f413b9e0 2024-09-17T05:10:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f4158b80 con 0x7f19f413b9e0 2024-09-17T05:10:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f41605b0 con 0x7f19f413b9e0 2024-09-17T05:10:36.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19dc021070 con 0x7f19f413b9e0 2024-09-17T05:10:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ea000640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19d0073230 0x7f19d0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.056+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f19dc094ad0 con 0x7f19f413b9e0 2024-09-17T05:10:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.060+0000 7f19f1e00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19d0073230 0x7f19d0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.060+0000 7f19f1e00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19d0073230 0x7f19d0075740 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f19f40afb90 tx=0x7f19e40b1040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:36.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.064+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19dc05f570 con 0x7f19f413b9e0 2024-09-17T05:10:36.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.236+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19f4158060 con 0x7f19d0073230 2024-09-17T05:10:36.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.236+0000 7f19ea000640 1 -- 172.21.15.16:0/2344235794 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13357 (secure 0 0 0) 0x7f19f4158060 con 0x7f19d0073230 2024-09-17T05:10:36.239 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:36.240 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19d0073230 msgr2=0x7f19d0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19d0073230 0x7f19d0075740 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f19f40afb90 tx=0x7f19e40b1040 comp rx=0 tx=0).stop 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 msgr2=0x7f19f413fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 0x7f19f413fff0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f19dc011ab0 tx=0x7f19dc011f70 comp rx=0 tx=0).stop 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 shutdown_connections 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f19d0073230 0x7f19d0075740 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19f413bdf0 0x7f19f4140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19f413b9e0 0x7f19f413fff0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 --2- 172.21.15.16:0/2344235794 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f19d8003c40 0x7f19f415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 >> 172.21.15.16:0/2344235794 conn(0x7f19f406f900 msgr2=0x7f19f415aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:36.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 shutdown_connections 2024-09-17T05:10:36.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:36.240+0000 7f19f8a00640 1 -- 172.21.15.16:0/2344235794 wait complete. 2024-09-17T05:10:36.262 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-17T05:10:35.618505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282256,"kb_used_data":16104,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692464,"statfs":{"total":773094113280,"available":772805083136,"internally_reserved":0,"allocated":16490496,"data_stored":8266576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.425188"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":364,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T05:10:25.197252+0000","last_change":"2024-09-17T05:00:59.480019+0000","last_active":"2024-09-17T05:10:25.197252+0000","last_peered":"2024-09-17T05:10:25.197252+0000","last_clean":"2024-09-17T05:10:25.197252+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:25.197252+0000","last_undegraded":"2024-09-17T05:10:25.197252+0000","last_fullsized":"2024-09-17T05:10:25.197252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T05:00:58.418771+0000","last_clean_scrub_stamp":"2024-09-17T05:00:58.418771+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:00.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33064,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33056,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509560,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37796,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334044,"statfs":{"total":96636764160,"available":96598061056,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:36.265 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T05:00:58.418771+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=5, tm_min=0, tm_sec=58, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=5, tm_min=10, tm_sec=30, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T05:10:36.265 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T05:10:56.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T05:10:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 -- 172.21.15.16:0/665351928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa3406f2f0 msgr2=0x7ffa3406f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 --2- 172.21.15.16:0/665351928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa3406f2f0 0x7ffa3406f700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009bf0 tx=0x7ffa2c01f2f0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 -- 172.21.15.16:0/665351928 shutdown_connections 2024-09-17T05:10:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 --2- 172.21.15.16:0/665351928 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa3406f2f0 0x7ffa3406f700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 -- 172.21.15.16:0/665351928 >> 172.21.15.16:0/665351928 conn(0x7ffa340d2240 msgr2=0x7ffa340d2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 -- 172.21.15.16:0/665351928 shutdown_connections 2024-09-17T05:10:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 -- 172.21.15.16:0/665351928 wait complete. 2024-09-17T05:10:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 Processor -- start 2024-09-17T05:10:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.440+0000 7ffa3c600640 1 -- start start 2024-09-17T05:10:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa3406f2f0 0x7ffa341453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 0x7ffa3414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa340b58e0 0x7ffa3414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3406fc40 con 0x7ffa340b1c60 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3406e2b0 con 0x7ffa3406f2f0 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3406fdc0 con 0x7ffa340b58e0 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 0x7ffa3414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 0x7ffa3414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53056/0 (socket says 172.21.15.16:53056) 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 -- 172.21.15.16:0/277194295 learned_addr learned my addr 172.21.15.16:0/277194295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3a600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa340b58e0 0x7ffa3414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39c00640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa3406f2f0 0x7ffa341453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 -- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa340b58e0 msgr2=0x7ffa3414ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa340b58e0 0x7ffa3414ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 -- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa3406f2f0 msgr2=0x7ffa341453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa3406f2f0 0x7ffa341453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 -- 172.21.15.16:0/277194295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3414bfb0 con 0x7ffa340b1c60 2024-09-17T05:10:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa39200640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 0x7ffa3414b530 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c680 tx=0x7ffa2000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20016020 con 0x7ffa340b1c60 2024-09-17T05:10:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa2000a930 con 0x7ffa340b1c60 2024-09-17T05:10:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa341f6750 con 0x7ffa340b1c60 2024-09-17T05:10:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000ad80 con 0x7ffa340b1c60 2024-09-17T05:10:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3414a420 con 0x7ffa340b1c60 2024-09-17T05:10:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.444+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3406f700 con 0x7ffa340b1c60 2024-09-17T05:10:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.448+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa20019020 con 0x7ffa340b1c60 2024-09-17T05:10:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.448+0000 7ffa31600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa14073230 0x7ffa14075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.448+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7ffa200940e0 con 0x7ffa340b1c60 2024-09-17T05:10:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.448+0000 7ffa39c00640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa14073230 0x7ffa14075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.448+0000 7ffa39c00640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa14073230 0x7ffa14075740 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007fd0 tx=0x7ffa28009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.452+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffa20020020 con 0x7ffa340b1c60 2024-09-17T05:10:56.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 --> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa3414e710 con 0x7ffa14073230 2024-09-17T05:10:56.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa31600640 1 -- 172.21.15.16:0/277194295 <== mgr.4132 v2:172.21.15.104:6800/2105036570 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7ffa3414e710 con 0x7ffa14073230 2024-09-17T05:10:56.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:56.627 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-17T05:10:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa14073230 msgr2=0x7ffa14075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa14073230 0x7ffa14075740 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffa28007fd0 tx=0x7ffa28009040 comp rx=0 tx=0).stop 2024-09-17T05:10:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 msgr2=0x7ffa3414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 0x7ffa3414b530 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c680 tx=0x7ffa2000cb40 comp rx=0 tx=0).stop 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 shutdown_connections 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7ffa14073230 0x7ffa14075740 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7ffa340b58e0 0x7ffa3414ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa340b1c60 0x7ffa3414b530 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.624+0000 7ffa3c600640 1 --2- 172.21.15.16:0/277194295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa3406f2f0 0x7ffa341453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.628+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 >> 172.21.15.16:0/277194295 conn(0x7ffa340d2240 msgr2=0x7ffa340701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.628+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 shutdown_connections 2024-09-17T05:10:56.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.628+0000 7ffa3c600640 1 -- 172.21.15.16:0/277194295 wait complete. 2024-09-17T05:10:56.649 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-17T05:10:55.623735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282252,"kb_used_data":16104,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692468,"statfs":{"total":773094113280,"available":772805087232,"internally_reserved":0,"allocated":16490496,"data_stored":8266576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003056"},"pg_stats":[{"pgid":"1.0","version":"152'74","reported_seq":374,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-17T05:10:32.675084+0000","last_change":"2024-09-17T05:10:32.675084+0000","last_active":"2024-09-17T05:10:32.675084+0000","last_peered":"2024-09-17T05:10:32.675084+0000","last_clean":"2024-09-17T05:10:32.675084+0000","last_became_active":"2024-09-17T05:00:59.477651+0000","last_became_peered":"2024-09-17T05:00:59.477651+0000","last_unstale":"2024-09-17T05:10:32.675084+0000","last_undegraded":"2024-09-17T05:10:32.675084+0000","last_fullsized":"2024-09-17T05:10:32.675084+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-17T05:10:32.675013+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-17T05:10:32.675013+0000","last_clean_scrub_stamp":"2024-09-17T05:10:32.675013+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:40.910834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33060,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33064,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33056,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509564,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37792,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334048,"statfs":{"total":96636764160,"available":96598065152,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37216,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334624,"statfs":{"total":96636764160,"available":96598654976,"internally_reserved":0,"allocated":1912832,"data_stored":885730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33632,"kb_used_data":2448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338208,"statfs":{"total":96636764160,"available":96602324992,"internally_reserved":0,"allocated":2506752,"data_stored":1476098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T05:10:56.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 -- 172.21.15.16:0/2307109066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 msgr2=0x7f964015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 --2- 172.21.15.16:0/2307109066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f964015efe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9640156030 tx=0x7f963001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 -- 172.21.15.16:0/2307109066 shutdown_connections 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 --2- 172.21.15.16:0/2307109066 >> v2:172.21.15.104:3300/0 conn(0x7f964015f520 0x7f964015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 --2- 172.21.15.16:0/2307109066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f964015efe0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 -- 172.21.15.16:0/2307109066 >> 172.21.15.16:0/2307109066 conn(0x7f96400b2250 msgr2=0x7f96400d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:56.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 -- 172.21.15.16:0/2307109066 shutdown_connections 2024-09-17T05:10:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.804+0000 7f9647a00640 1 -- 172.21.15.16:0/2307109066 wait complete. 2024-09-17T05:10:56.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 Processor -- start 2024-09-17T05:10:56.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 -- start start 2024-09-17T05:10:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f96401fcf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640074e30 0x7f96400b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964015f520 0x7f9640151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9640168ed0 con 0x7f9640074a20 2024-09-17T05:10:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9640168d50 con 0x7f9640074e30 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9647a00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7f9640169050 con 0x7f964015f520 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9645000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f96401fcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9645000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f96401fcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53064/0 (socket says 172.21.15.16:53064) 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9645000640 1 -- 172.21.15.16:0/1905239359 learned_addr learned my addr 172.21.15.16:0/1905239359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9637e00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640074e30 0x7f96400b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9645a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964015f520 0x7f9640151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9645000640 1 -- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964015f520 msgr2=0x7f9640151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.808+0000 7f9645000640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964015f520 0x7f9640151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9645000640 1 -- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640074e30 msgr2=0x7f96400b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9645000640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640074e30 0x7f96400b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9645000640 1 -- 172.21.15.16:0/1905239359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96401fd4b0 con 0x7f9640074a20 2024-09-17T05:10:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9645000640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f96401fcf70 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f96300b0040 tx=0x7f96300a5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300b2040 con 0x7f9640074a20 2024-09-17T05:10:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96300043b0 con 0x7f9640074a20 2024-09-17T05:10:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9640165f10 con 0x7f9640074a20 2024-09-17T05:10:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630005280 con 0x7f9640074a20 2024-09-17T05:10:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964014f970 con 0x7f9640074a20 2024-09-17T05:10:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964015efe0 con 0x7f9640074a20 2024-09-17T05:10:56.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9630005420 con 0x7f9640074a20 2024-09-17T05:10:56.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.812+0000 7f963d600640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f961c073230 0x7f961c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:56.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.816+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7f963012dce0 con 0x7f9640074a20 2024-09-17T05:10:56.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.816+0000 7f9637e00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f961c073230 0x7f961c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.816+0000 7f9637e00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f961c073230 0x7f961c075740 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9628007fd0 tx=0x7f9628009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.820+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f96401fd4b0 con 0x7f9640074a20 2024-09-17T05:10:56.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.992+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f96400b03b0 con 0x7f9640074a20 2024-09-17T05:10:56.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:56.996+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v71)=0 v71) ==== 123+0+0 (secure 0 0 0) 0x7f96300164b0 con 0x7f9640074a20 2024-09-17T05:10:57.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f963d600640 1 -- 172.21.15.16:0/1905239359 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f9630133020 con 0x7f9640074a20 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f961c073230 msgr2=0x7f961c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f961c073230 0x7f961c075740 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9628007fd0 tx=0x7f9628009040 comp rx=0 tx=0).stop 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 msgr2=0x7f96401fcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f96401fcf70 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f96300b0040 tx=0x7f96300a5580 comp rx=0 tx=0).stop 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 shutdown_connections 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7f961c073230 0x7f961c075740 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7f964015f520 0x7f9640151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9640074e30 0x7f96400b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 --2- 172.21.15.16:0/1905239359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9640074a20 0x7f96401fcf70 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 >> 172.21.15.16:0/1905239359 conn(0x7f96400b2250 msgr2=0x7f9640152a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:57.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 shutdown_connections 2024-09-17T05:10:57.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.000+0000 7f9647a00640 1 -- 172.21.15.16:0/1905239359 wait complete. 2024-09-17T05:10:57.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T05:10:57.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T05:10:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1778842543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c80b4980 msgr2=0x7fb3c80cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1778842543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c80b4980 0x7fb3c80cde00 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ccd00 tx=0x7fb3b001c8c0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1778842543 shutdown_connections 2024-09-17T05:10:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1778842543 >> v2:172.21.15.104:3300/0 conn(0x7fb3c806de60 0x7fb3c80ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1778842543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c80b4980 0x7fb3c80cde00 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1778842543 >> 172.21.15.16:0/1778842543 conn(0x7fb3c806f900 msgr2=0x7fb3c80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1778842543 shutdown_connections 2024-09-17T05:10:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1778842543 wait complete. 2024-09-17T05:10:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 Processor -- start 2024-09-17T05:10:57.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.180+0000 7fb3cdc00640 1 -- start start 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 --2- >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3c806da50 0x7fb3c80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 0x7fb3c80ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3c80accb0 0x7fb3c80ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8167f60 con 0x7fb3c806de60 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8167de0 con 0x7fb3c80accb0 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 -- --> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c81680e0 con 0x7fb3c806da50 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 0x7fb3c80ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 0x7fb3c80ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39768/0 (socket says 172.21.15.16:39768) 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 -- 172.21.15.16:0/1081617547 learned_addr learned my addr 172.21.15.16:0/1081617547 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3c7400640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3c80accb0 0x7fb3c80ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3c6a00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3c806da50 0x7fb3c80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 -- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3c806da50 msgr2=0x7fb3c80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3c806da50 0x7fb3c80adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 -- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3c80accb0 msgr2=0x7fb3c80ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3c80accb0 0x7fb3c80ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 -- 172.21.15.16:0/1081617547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c80c9720 con 0x7fb3c806de60 2024-09-17T05:10:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bfe00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 0x7fb3c80ac770 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00b0040 tx=0x7fb3b00a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b00b2040 con 0x7fb3c806de60 2024-09-17T05:10:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fb3b00041d0 con 0x7fb3c806de60 2024-09-17T05:10:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c80c9a00 con 0x7fb3c806de60 2024-09-17T05:10:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0003600 con 0x7fb3c806de60 2024-09-17T05:10:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c80c9fc0 con 0x7fb3c806de60 2024-09-17T05:10:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.184+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c80cde00 con 0x7fb3c806de60 2024-09-17T05:10:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.188+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb3b00bf030 con 0x7fb3c806de60 2024-09-17T05:10:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.188+0000 7fb3bea00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb3a4073230 0x7fb3a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T05:10:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.188+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5335+0+0 (secure 0 0 0) 0x7fb3b012dce0 con 0x7fb3c806de60 2024-09-17T05:10:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.188+0000 7fb3c6a00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb3a4073230 0x7fb3a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T05:10:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.188+0000 7fb3c6a00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb3a4073230 0x7fb3a4075740 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8007fd0 tx=0x7fb3b8009040 comp rx=0 tx=0).ready entity=mgr.4132 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T05:10:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.192+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb3b00164b0 con 0x7fb3c806de60 2024-09-17T05:10:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3c80c94b0 con 0x7fb3c806de60 2024-09-17T05:10:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3bea00640 1 -- 172.21.15.16:0/1081617547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+3916 (secure 0 0 0) 0x7fb3b00f88a0 con 0x7fb3c806de60 2024-09-17T05:10:57.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-17T05:10:57.406 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":112,"btime":"2024-09-17T05:10:23:182395+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/404154638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6835","nonce":404154638},{"type":"v1","addr":"172.21.15.104:6837","nonce":404154638}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7954,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6836/1582784156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1582784156},{"type":"v1","addr":"172.21.15.104:6836","nonce":1582784156}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":8523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3609351509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3609351509},{"type":"v1","addr":"172.21.15.16:6834","nonce":3609351509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8733,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/741359711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":741359711},{"type":"v1","addr":"172.21.15.16:6835","nonce":741359711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[]} 2024-09-17T05:10:57.406 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 112 2024-09-17T05:10:57.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb3a4073230 msgr2=0x7fb3a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb3a4073230 0x7fb3a4075740 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8007fd0 tx=0x7fb3b8009040 comp rx=0 tx=0).stop 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 msgr2=0x7fb3c80ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 0x7fb3c80ac770 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00b0040 tx=0x7fb3b00a5690 comp rx=0 tx=0).stop 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 shutdown_connections 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:6800/2105036570,v1:172.21.15.104:6801/2105036570] conn(0x7fb3a4073230 0x7fb3a4075740 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3c80accb0 0x7fb3c80ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3c806de60 0x7fb3c80ac770 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 --2- 172.21.15.16:0/1081617547 >> [v2:172.21.15.104:3301/0,v1:172.21.15.104:6790/0] conn(0x7fb3c806da50 0x7fb3c80adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 >> 172.21.15.16:0/1081617547 conn(0x7fb3c806f900 msgr2=0x7fb3c80b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 shutdown_connections 2024-09-17T05:10:57.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-17T05:10:57.404+0000 7fb3cdc00640 1 -- 172.21.15.16:0/1081617547 wait complete. 2024-09-17T05:10:57.427 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T05:10:57.428 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T05:10:57.428 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T05:10:57.428 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:57.448 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T05:10:57.448 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T05:10:57.448 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:57.504 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T05:10:57.505 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T05:10:57.505 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:57.519 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T05:10:57.520 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T05:10:57.520 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:57.578 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T05:10:57.578 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T05:10:57.578 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T05:10:57.578 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:57.661 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T05:10:57.661 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T05:10:57.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:10:57.779 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T05:10:57.780 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T05:10:57.780 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T05:10:57.780 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-17T05:11:01.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:11:01.656 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:11:03.764 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T05:11:03.764 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T05:11:03.764 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:03.844 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T05:11:03.844 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T05:11:03.844 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:03.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-17T05:11:03.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T05:11:03.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T05:11:03.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T05:11:03.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-17T05:11:09.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-17T05:11:09.796 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-17T05:11:09.946 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T05:11:09.946 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T05:11:09.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:10.027 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T05:11:10.027 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T05:11:10.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:15.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-17T05:11:15.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-17T05:11:16.128 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T05:11:16.129 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T05:11:16.129 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.206 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T05:11:16.207 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T05:11:16.207 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.291 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T05:11:16.292 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T05:11:16.292 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T05:11:16.292 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.355 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T05:11:16.356 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T05:11:16.356 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.419 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T05:11:16.420 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T05:11:16.420 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T05:11:16.420 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.441 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T05:11:16.441 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T05:11:16.441 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.457 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T05:11:16.458 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T05:11:16.458 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T05:11:16.518 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T05:11:16.519 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T05:11:16.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2024-09-17T05:11:16.563 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:11:16.563 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T05:11:16.813 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:11:16.814 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T05:11:16.989 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:11:16.989 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T05:11:17.186 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:11:17.186 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T05:11:17.488 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:11:17.488 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T05:11:17.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:11:17.726 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T05:11:17.904 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:11:17.904 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T05:11:18.114 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:11:18.115 DEBUG:teuthology.orchestra.run.smithi104:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T05:11:18.289 INFO:tasks.ceph:Archiving mon data... 2024-09-17T05:11:18.291 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/data/mon.a.tgz 2024-09-17T05:11:18.291 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-17T05:11:18.300 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.u0uEsOKHpF 2024-09-17T05:11:18.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.u0uEsOKHpF 2024-09-17T05:11:18.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.u0uEsOKHpF 2024-09-17T05:11:18.533 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.u0uEsOKHpF is 443KB 2024-09-17T05:11:18.598 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.u0uEsOKHpF 2024-09-17T05:11:18.607 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/data/mon.b.tgz 2024-09-17T05:11:18.607 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-09-17T05:11:18.615 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.eT0feMlW3F 2024-09-17T05:11:18.616 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.eT0feMlW3F 2024-09-17T05:11:18.765 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.eT0feMlW3F 2024-09-17T05:11:18.840 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.eT0feMlW3F is 452KB 2024-09-17T05:11:18.915 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.eT0feMlW3F 2024-09-17T05:11:18.926 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/data/mon.c.tgz 2024-09-17T05:11:18.926 DEBUG:teuthology.orchestra.run.smithi104:> mktemp 2024-09-17T05:11:18.976 INFO:teuthology.orchestra.run.smithi104.stdout:/tmp/tmp.y9zec9eBFc 2024-09-17T05:11:18.976 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.y9zec9eBFc 2024-09-17T05:11:19.120 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0666 /tmp/tmp.y9zec9eBFc 2024-09-17T05:11:19.197 DEBUG:teuthology.orchestra.remote:smithi104:/tmp/tmp.y9zec9eBFc is 452KB 2024-09-17T05:11:19.267 DEBUG:teuthology.orchestra.run.smithi104:> rm -fr /tmp/tmp.y9zec9eBFc 2024-09-17T05:11:19.275 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T05:11:19.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T05:11:19.278 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T05:11:19.349 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T05:11:19.384 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T05:11:19.384 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T05:11:19.386 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/remote/smithi016/crash 2024-09-17T05:11:19.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T05:11:19.413 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/remote/smithi104/crash 2024-09-17T05:11:19.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T05:11:19.449 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T05:11:19.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:11:19.458 DEBUG:teuthology.orchestra.run.smithi104:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T05:11:19.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T05:11:19.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T05:11:19.579 INFO:tasks.ceph:Compressing logs... 2024-09-17T05:11:19.579 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T05:11:19.611 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T05:11:19.647 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35462.log 2024-09-17T05:11:19.647 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34820.log 2024-09-17T05:11:19.648 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28969.log 2024-09-17T05:11:19.648 INFO:teuthology.orchestra.run.smithi104.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35462.log.gz 2024-09-17T05:11:19.649 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34820.log.gz 2024-09-17T05:11:19.649 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35227.log 2024-09-17T05:11:19.650 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.28969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34942.log 2024-09-17T05:11:19.650 INFO:teuthology.orchestra.run.smithi104.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28969.log.gz 2024-09-17T05:11:19.651 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35227.log.gz 2024-09-17T05:11:19.651 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T05:11:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40573.log 2024-09-17T05:11:19.652 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34942.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.1.34942.log.gz -- 2024-09-17T05:11:19.652 INFO:teuthology.orchestra.run.smithi104.stderr: /var/log/ceph/ceph-client.1.34708.log 2024-09-17T05:11:19.652 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35584.log 2024-09-17T05:11:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46293.log 2024-09-17T05:11:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49963.log 2024-09-17T05:11:19.655 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40573.log.gz 2024-09-17T05:11:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42923.log 2024-09-17T05:11:19.655 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46293.log.gz 2024-09-17T05:11:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51172.log 2024-09-17T05:11:19.656 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34708.log.gz 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35876.log 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35584.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35584.log.gz 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35876.log.gz 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35222.log 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35222.log.gz 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35739.log 2024-09-17T05:11:19.657 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35085.log 2024-09-17T05:11:19.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35739.log.gz 2024-09-17T05:11:19.660 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34481.log 2024-09-17T05:11:19.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35085.log.gz 2024-09-17T05:11:19.660 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29048.log 2024-09-17T05:11:19.660 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35349.log 2024-09-17T05:11:19.660 INFO:teuthology.orchestra.run.smithi104.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34481.log.gz 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49963.log.gz 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41252.log 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42923.log.gz 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42975.log 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51172.log.gz 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.41252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41252.log.gz 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49506.log 2024-09-17T05:11:19.663 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42975.log.gz 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49909.log 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49506.log.gz 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33929.log 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38358.log 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49909.log.gz 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49805.log 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33929.log.gz 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34005.log 2024-09-17T05:11:19.664 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38358.log.gz 2024-09-17T05:11:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-09-17T05:11:19.668 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49805.log.gz 2024-09-17T05:11:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34005.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T05:11:19.668 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34005.log.gz 2024-09-17T05:11:19.668 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-09-17T05:11:19.668 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-09-17T05:11:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40102.log 2024-09-17T05:11:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39526.log 2024-09-17T05:11:19.669 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-09-17T05:11:19.669 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.40102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2024-09-17T05:11:19.669 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40102.log.gz 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.29048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35589.log 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29048.log.gz 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35349.log.gz 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35457.log 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35589.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35589.log.gz 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35457.log.gz 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T05:11:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34565.log 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29072.log 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.3.34565.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.3.34565.log.gz -- 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr: /var/log/ceph/ceph-client.1.35080.log 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.29072.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29072.log.gz 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35080.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.1.35080.log.gz -5 2024-09-17T05:11:19.671 INFO:teuthology.orchestra.run.smithi104.stderr: --verbose -- /var/log/ceph/ceph-client.admin.28945.log 2024-09-17T05:11:19.673 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34463.log 2024-09-17T05:11:19.673 INFO:teuthology.orchestra.run.smithi104.stderr: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T05:11:19.674 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.28945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29000.log 2024-09-17T05:11:19.674 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28945.log.gz 2024-09-17T05:11:19.674 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34463.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34463.log.gz 2024-09-17T05:11:19.674 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29097.log 2024-09-17T05:11:19.676 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47547.log 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39526.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51460.log 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38410.log 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35498.log 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51460.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46691.log 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38410.log.gz 2024-09-17T05:11:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44087.log 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46691.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35498.log.gz 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46691.log.gz 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51224.log 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44087.log.gz 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29418.log 2024-09-17T05:11:19.678 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.29000.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29000.log.gz 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.29097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34947.log 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29097.log.gz 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29121.log 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34947.log.gz 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.29121.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29121.log.gz 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T05:11:19.682 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34825.log 2024-09-17T05:11:19.685 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mds.d.log: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T05:11:19.685 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34825.log.gz 2024-09-17T05:11:19.686 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T05:11:19.686 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T05:11:19.686 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51224.log.gz 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41319.log 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29418.log.gz 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34976.log 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38726.log 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41319.log.gz 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39870.log 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34976.log.gz 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38726.log.gz 2024-09-17T05:11:19.687 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34790.log 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39870.log.gz 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36738.log 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35193.log 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34790.log.gz 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39240.log 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36738.log.gz 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40864.log 2024-09-17T05:11:19.688 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35193.log.gz 2024-09-17T05:11:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2024-09-17T05:11:19.689 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39240.log.gz 2024-09-17T05:11:19.690 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34570.log 2024-09-17T05:11:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50252.log 2024-09-17T05:11:19.692 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40864.log.gz 2024-09-17T05:11:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35181.log 2024-09-17T05:11:19.692 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2024-09-17T05:11:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49598.log 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50252.log.gz 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40521.log 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35181.log.gz 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51642.log 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49598.log.gz 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40521.log.gz 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28905.log 2024-09-17T05:11:19.693 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51642.log.gz 2024-09-17T05:11:19.695 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.3.34570.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T05:11:19.695 INFO:teuthology.orchestra.run.smithi104.stderr: 0.0%/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34703.log 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48491.log 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28905.log.gz 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48491.log.gz 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2024-09-17T05:11:19.696 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2024-09-17T05:11:19.697 INFO:teuthology.orchestra.run.smithi104.stderr: -- replaced with /var/log/ceph/ceph-client.3.34570.log.gz 2024-09-17T05:11:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44547.log 2024-09-17T05:11:19.698 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-09-17T05:11:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39739.log 2024-09-17T05:11:19.698 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2024-09-17T05:11:19.698 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-09-17T05:11:19.698 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.44547.log.gz 2024-09-17T05:11:19.700 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48608.log 2024-09-17T05:11:19.700 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39739.log.gz 2024-09-17T05:11:19.700 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50681.log 2024-09-17T05:11:19.700 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-09-17T05:11:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47365.log 2024-09-17T05:11:19.701 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48608.log.gz 2024-09-17T05:11:19.701 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.50681.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.50681.log.gz -5 2024-09-17T05:11:19.702 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40391.log 2024-09-17T05:11:19.702 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41471.log 2024-09-17T05:11:19.702 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47365.log.gz 2024-09-17T05:11:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-09-17T05:11:19.703 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40391.log.gz 2024-09-17T05:11:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46639.log 2024-09-17T05:11:19.704 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41471.log.gz 2024-09-17T05:11:19.705 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.34703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34703.log.gz 2024-09-17T05:11:19.705 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35734.log 2024-09-17T05:11:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T05:11:19.706 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-09-17T05:11:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43625.log 2024-09-17T05:11:19.706 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46639.log.gz 2024-09-17T05:11:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50171.log 2024-09-17T05:11:19.707 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37725.log 2024-09-17T05:11:19.707 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43625.log.gz 2024-09-17T05:11:19.708 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40658.log 2024-09-17T05:11:19.708 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50171.log.gz 2024-09-17T05:11:19.709 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35196.log 2024-09-17T05:11:19.709 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37725.log.gz 2024-09-17T05:11:19.710 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48517.log 2024-09-17T05:11:19.711 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35734.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35734.log.gz 2024-09-17T05:11:19.711 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35344.log 2024-09-17T05:11:19.711 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40658.log.gz 2024-09-17T05:11:19.711 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50074.log 2024-09-17T05:11:19.711 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35196.log.gz 2024-09-17T05:11:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40547.log 2024-09-17T05:11:19.712 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48517.log.gz 2024-09-17T05:11:19.712 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41200.log 2024-09-17T05:11:19.713 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50074.log.gz 2024-09-17T05:11:19.713 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29024.log 2024-09-17T05:11:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-09-17T05:11:19.714 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40547.log.gz 2024-09-17T05:11:19.714 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44413.log 2024-09-17T05:11:19.714 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41200.log.gz 2024-09-17T05:11:19.715 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29010.log 2024-09-17T05:11:19.715 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2024-09-17T05:11:19.716 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49754.log 2024-09-17T05:11:19.716 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44413.log.gz 2024-09-17T05:11:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29010.log.gz 2024-09-17T05:11:19.717 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2024-09-17T05:11:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50629.log 2024-09-17T05:11:19.719 INFO:teuthology.orchestra.run.smithi016.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.49754.log.gz 2024-09-17T05:11:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44326.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2024-09-17T05:11:19.719 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-09-17T05:11:19.719 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35871.log 2024-09-17T05:11:19.719 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.admin.29024.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29024.log.gz 2024-09-17T05:11:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50676.log 2024-09-17T05:11:19.720 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50629.log.gz 2024-09-17T05:11:19.720 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44495.log 2024-09-17T05:11:19.720 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-09-17T05:11:19.721 INFO:teuthology.orchestra.run.smithi104.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35344.log.gz 2024-09-17T05:11:19.722 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.50676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-09-17T05:11:19.722 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50676.log.gz 2024-09-17T05:11:19.722 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34786.log 2024-09-17T05:11:19.722 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44495.log.gz 2024-09-17T05:11:19.723 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/ceph-client.1.35871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.35871.log.gz 2024-09-17T05:11:19.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-09-17T05:11:19.723 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-09-17T05:11:19.724 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2024-09-17T05:11:19.724 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34786.log.gz 2024-09-17T05:11:19.724 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz --verbose 2024-09-17T05:11:19.725 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.41783.log 2024-09-17T05:11:19.725 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35808.log 2024-09-17T05:11:19.725 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2024-09-17T05:11:19.726 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44443.log 2024-09-17T05:11:19.727 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41783.log.gz 2024-09-17T05:11:19.727 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.35808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35808.log.gz 2024-09-17T05:11:19.727 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38674.log 2024-09-17T05:11:19.728 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48413.log 2024-09-17T05:11:19.728 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44443.log.gz 2024-09-17T05:11:19.729 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40313.log 2024-09-17T05:11:19.729 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38674.log.gz 2024-09-17T05:11:19.729 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41861.log 2024-09-17T05:11:19.730 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48413.log.gz 2024-09-17T05:11:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40313.log: gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40313.log.gz -- 2024-09-17T05:11:19.730 INFO:teuthology.orchestra.run.smithi016.stderr: /var/log/ceph/ceph-client.admin.34796.log 2024-09-17T05:11:19.731 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2024-09-17T05:11:19.731 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41861.log.gz 2024-09-17T05:11:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34796.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34796.log.gz 2024-09-17T05:11:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42400.log 2024-09-17T05:11:19.733 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45652.log 2024-09-17T05:11:19.733 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-09-17T05:11:19.734 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49454.log 2024-09-17T05:11:19.734 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42400.log.gz 2024-09-17T05:11:19.735 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43134.log 2024-09-17T05:11:19.735 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45652.log.gz 2024-09-17T05:11:19.736 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48361.log 2024-09-17T05:11:19.736 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49454.log.gz 2024-09-17T05:11:19.736 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2024-09-17T05:11:19.737 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43134.log.gz 2024-09-17T05:11:19.737 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39552.log 2024-09-17T05:11:19.737 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48361.log.gz 2024-09-17T05:11:19.738 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2024-09-17T05:11:19.739 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2024-09-17T05:11:19.739 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41653.log 2024-09-17T05:11:19.739 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39552.log.gz 2024-09-17T05:11:19.740 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-09-17T05:11:19.740 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2024-09-17T05:11:19.741 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39791.log 2024-09-17T05:11:19.741 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41653.log.gz 2024-09-17T05:11:19.742 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2024-09-17T05:11:19.742 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-09-17T05:11:19.742 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49343.log 2024-09-17T05:11:19.742 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39791.log.gz 2024-09-17T05:11:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49172.log 2024-09-17T05:11:19.744 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2024-09-17T05:11:19.744 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log 2024-09-17T05:11:19.744 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49343.log.gz 2024-09-17T05:11:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.49172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49172.log.gz 2024-09-17T05:11:19.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39687.log 2024-09-17T05:11:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-09-17T05:11:19.746 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2024-09-17T05:11:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39687.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39687.log.gz 2024-09-17T05:11:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39065.log 2024-09-17T05:11:19.748 INFO:teuthology.orchestra.run.smithi104.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T05:11:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34300.log 2024-09-17T05:11:19.749 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-09-17T05:11:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41887.log 2024-09-17T05:11:19.749 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39065.log.gz 2024-09-17T05:11:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41020.log 2024-09-17T05:11:19.750 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34300.log.gz 2024-09-17T05:11:19.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48102.log 2024-09-17T05:11:19.750 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41887.log.gz 2024-09-17T05:11:19.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45322.log 2024-09-17T05:11:19.751 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41020.log.gz 2024-09-17T05:11:19.752 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43480.log 2024-09-17T05:11:19.752 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48102.log.gz 2024-09-17T05:11:19.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40097.log 2024-09-17T05:11:19.753 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45322.log.gz 2024-09-17T05:11:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33611.log 2024-09-17T05:11:19.754 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43480.log.gz 2024-09-17T05:11:19.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.40097.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.40097.log.gz -5 2024-09-17T05:11:19.755 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42322.log 2024-09-17T05:11:19.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-09-17T05:11:19.756 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33611.log.gz 2024-09-17T05:11:19.756 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2024-09-17T05:11:19.756 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42322.log.gz 2024-09-17T05:11:19.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49167.log 2024-09-17T05:11:19.758 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-09-17T05:11:19.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37472.log 2024-09-17T05:11:19.759 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2024-09-17T05:11:19.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.49167.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.49167.log.gz 2024-09-17T05:11:19.759 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-09-17T05:11:19.760 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2024-09-17T05:11:19.760 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37472.log.gz 2024-09-17T05:11:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33877.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-09-17T05:11:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44298.log 2024-09-17T05:11:19.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47112.log 2024-09-17T05:11:19.762 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2024-09-17T05:11:19.762 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42070.log 2024-09-17T05:11:19.763 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44298.log.gz 2024-09-17T05:11:19.763 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38149.log 2024-09-17T05:11:19.764 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47112.log.gz 2024-09-17T05:11:19.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39922.log 2024-09-17T05:11:19.765 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42070.log.gz 2024-09-17T05:11:19.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42535.log 2024-09-17T05:11:19.765 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38149.log.gz 2024-09-17T05:11:19.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38440.log 2024-09-17T05:11:19.766 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39922.log.gz 2024-09-17T05:11:19.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-09-17T05:11:19.767 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.42535.log.gz 2024-09-17T05:11:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2024-09-17T05:11:19.768 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38440.log.gz 2024-09-17T05:11:19.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-09-17T05:11:19.769 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-09-17T05:11:19.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2024-09-17T05:11:19.770 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2024-09-17T05:11:19.770 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2024-09-17T05:11:19.770 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-09-17T05:11:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37249.log 2024-09-17T05:11:19.771 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2024-09-17T05:11:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40806.log 2024-09-17T05:11:19.772 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2024-09-17T05:11:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35472.log 2024-09-17T05:11:19.773 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.37249.log.gz 2024-09-17T05:11:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-09-17T05:11:19.774 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40806.log.gz 2024-09-17T05:11:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36764.log 2024-09-17T05:11:19.775 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35472.log.gz 2024-09-17T05:11:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42188.log 2024-09-17T05:11:19.776 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-09-17T05:11:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44677.log 2024-09-17T05:11:19.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36764.log.gz 2024-09-17T05:11:19.777 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2024-09-17T05:11:19.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42188.log.gz 2024-09-17T05:11:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-09-17T05:11:19.778 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44677.log.gz 2024-09-17T05:11:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51067.log 2024-09-17T05:11:19.779 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2024-09-17T05:11:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41920.log 2024-09-17T05:11:19.781 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-09-17T05:11:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38776.log 2024-09-17T05:11:19.781 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51067.log.gz 2024-09-17T05:11:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41809.log 2024-09-17T05:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41920.log.gz 2024-09-17T05:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-09-17T05:11:19.782 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38776.log.gz 2024-09-17T05:11:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2024-09-17T05:11:19.783 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41809.log.gz 2024-09-17T05:11:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43806.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T05:11:19.784 INFO:teuthology.orchestra.run.smithi016.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-09-17T05:11:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-09-17T05:11:19.785 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2024-09-17T05:11:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36326.log 2024-09-17T05:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42162.log 2024-09-17T05:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-09-17T05:11:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39765.log 2024-09-17T05:11:19.788 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36326.log.gz 2024-09-17T05:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-09-17T05:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42162.log.gz 2024-09-17T05:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41679.log 2024-09-17T05:11:19.789 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39765.log.gz 2024-09-17T05:11:19.790 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51434.log 2024-09-17T05:11:19.791 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-09-17T05:11:19.791 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37223.log 2024-09-17T05:11:19.791 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41679.log.gz 2024-09-17T05:11:19.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33585.log 2024-09-17T05:11:19.793 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51434.log.gz 2024-09-17T05:11:19.793 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34248.log 2024-09-17T05:11:19.793 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37223.log.gz 2024-09-17T05:11:19.794 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37532.log 2024-09-17T05:11:19.794 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33585.log.gz 2024-09-17T05:11:19.795 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42949.log 2024-09-17T05:11:19.795 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34248.log.gz 2024-09-17T05:11:19.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28953.log 2024-09-17T05:11:19.797 INFO:teuthology.orchestra.run.smithi016.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37532.log.gz 2024-09-17T05:11:19.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38825.log 2024-09-17T05:11:19.797 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42949.log.gz 2024-09-17T05:11:19.798 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-09-17T05:11:19.798 INFO:teuthology.orchestra.run.smithi016.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28953.log.gz 2024-09-17T05:11:19.799 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.38825.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.36686.log 2024-09-17T05:11:19.799 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.0.38825.log.gz 2024-09-17T05:11:19.800 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46943.log 2024-09-17T05:11:19.800 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-09-17T05:11:19.801 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49702.log 2024-09-17T05:11:19.801 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36686.log.gz 2024-09-17T05:11:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40152.log 2024-09-17T05:11:19.805 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46943.log.gz 2024-09-17T05:11:19.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36514.log 2024-09-17T05:11:19.805 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49702.log.gz 2024-09-17T05:11:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39266.log 2024-09-17T05:11:19.806 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40152.log.gz 2024-09-17T05:11:19.806 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50892.log 2024-09-17T05:11:19.806 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36514.log.gz 2024-09-17T05:11:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 87.5% 2024-09-17T05:11:19.807 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39266.log.gz 2024-09-17T05:11:19.807 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-09-17T05:11:19.807 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50892.log.gz 2024-09-17T05:11:19.808 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-09-17T05:11:19.808 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-09-17T05:11:19.809 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47796.log 2024-09-17T05:11:19.809 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-09-17T05:11:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-17T05:11:19.810 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-09-17T05:11:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47796.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47796.log.gz 2024-09-17T05:11:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40754.log 2024-09-17T05:11:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41731.log 2024-09-17T05:11:19.811 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-17T05:11:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40754.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40754.log.gz 2024-09-17T05:11:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35420.log 2024-09-17T05:11:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33637.log 2024-09-17T05:11:19.814 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41731.log.gz 2024-09-17T05:11:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35420.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T05:11:19.815 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35420.log.gz 2024-09-17T05:11:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46743.log 2024-09-17T05:11:19.815 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33637.log.gz 2024-09-17T05:11:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-09-17T05:11:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46743.log: 82.8% -- replaced with /var/log/ceph/ceph-client.admin.46743.log.gz 2024-09-17T05:11:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45916.log 2024-09-17T05:11:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43108.log 2024-09-17T05:11:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36136.log: /var/log/ceph/ceph-client.admin.45916.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-09-17T05:11:19.819 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45916.log.gz 2024-09-17T05:11:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2024-09-17T05:11:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36660.log 2024-09-17T05:11:19.821 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43108.log.gz 2024-09-17T05:11:19.822 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47744.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2024-09-17T05:11:19.822 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48180.log 2024-09-17T05:11:19.822 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36660.log: -- /var/log/ceph/ceph-client.admin.51538.log 2024-09-17T05:11:19.823 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36660.log.gz 2024-09-17T05:11:19.824 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34582.log 2024-09-17T05:11:19.824 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48180.log.gz 2024-09-17T05:11:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40599.log 2024-09-17T05:11:19.825 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51538.log.gz 2024-09-17T05:11:19.825 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.34582.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43832.log 2024-09-17T05:11:19.826 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34582.log.gz 2024-09-17T05:11:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29034.log 2024-09-17T05:11:19.827 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40599.log.gz 2024-09-17T05:11:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-09-17T05:11:19.828 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43832.log.gz 2024-09-17T05:11:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37197.log 2024-09-17T05:11:19.829 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29034.log.gz 2024-09-17T05:11:19.830 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34218.log 2024-09-17T05:11:19.830 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-09-17T05:11:19.831 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42782.log 2024-09-17T05:11:19.831 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37197.log.gz 2024-09-17T05:11:19.832 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39344.log 2024-09-17T05:11:19.833 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34218.log.gz 2024-09-17T05:11:19.833 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42840.log 2024-09-17T05:11:19.833 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42782.log.gz 2024-09-17T05:11:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39344.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39344.log.gz 2024-09-17T05:11:19.834 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39500.log 2024-09-17T05:11:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39581.log 2024-09-17T05:11:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42840.log: /var/log/ceph/ceph-client.admin.39500.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42840.log.gz 2024-09-17T05:11:19.837 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28827.log 2024-09-17T05:11:19.837 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39500.log.gz 2024-09-17T05:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29046.log 2024-09-17T05:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39581.log.gz 2024-09-17T05:11:19.838 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29022.log 2024-09-17T05:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28827.log.gz 2024-09-17T05:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29046.log.gz 2024-09-17T05:11:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47443.log 2024-09-17T05:11:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51276.log 2024-09-17T05:11:19.841 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29022.log.gz 2024-09-17T05:11:19.842 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48862.log 2024-09-17T05:11:19.842 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47443.log.gz 2024-09-17T05:11:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-09-17T05:11:19.843 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2024-09-17T05:11:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40261.log 2024-09-17T05:11:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48862.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48862.log.gz 2024-09-17T05:11:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40209.log 2024-09-17T05:11:19.845 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2024-09-17T05:11:19.846 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T05:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40469.log 2024-09-17T05:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40261.log.gz 2024-09-17T05:11:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2024-09-17T05:11:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40209.log: 87.6%/var/log/ceph/ceph-client.admin.40469.log: -- replaced with /var/log/ceph/ceph-client.admin.40209.log.gz 2024-09-17T05:11:19.848 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51041.log 2024-09-17T05:11:19.848 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40469.log.gz 2024-09-17T05:11:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45505.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2024-09-17T05:11:19.849 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47933.log 2024-09-17T05:11:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-09-17T05:11:19.850 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51041.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51041.log.gz 2024-09-17T05:11:19.851 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41601.log 2024-09-17T05:11:19.851 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47933.log.gz 2024-09-17T05:11:19.852 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45042.log 2024-09-17T05:11:19.852 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-09-17T05:11:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2024-09-17T05:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41601.log.gz 2024-09-17T05:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42483.log 2024-09-17T05:11:19.854 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45042.log.gz 2024-09-17T05:11:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-09-17T05:11:19.855 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2024-09-17T05:11:19.856 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42483.log: /var/log/ceph/ceph-client.0.35680.log 2024-09-17T05:11:19.857 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42483.log.gz 2024-09-17T05:11:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37366.log 2024-09-17T05:11:19.858 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gz 2024-09-17T05:11:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.35680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35680.log.gz 2024-09-17T05:11:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39188.log 2024-09-17T05:11:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35993.log 2024-09-17T05:11:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37366.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37366.log.gz 2024-09-17T05:11:19.860 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2024-09-17T05:11:19.861 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39188.log.gz 2024-09-17T05:11:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37393.log 2024-09-17T05:11:19.862 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35993.log.gz 2024-09-17T05:11:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-09-17T05:11:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36607.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2024-09-17T05:11:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-17T05:11:19.864 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37393.log.gz 2024-09-17T05:11:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48888.log 87.4% 2024-09-17T05:11:19.865 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-09-17T05:11:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47339.log 2024-09-17T05:11:19.866 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-17T05:11:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43929.log 2024-09-17T05:11:19.867 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48888.log.gz 2024-09-17T05:11:19.868 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-09-17T05:11:19.868 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47339.log.gz 2024-09-17T05:11:19.869 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46117.log 2024-09-17T05:11:19.869 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43929.log.gz 2024-09-17T05:11:19.870 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39117.log 2024-09-17T05:11:19.870 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-09-17T05:11:19.871 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.46117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46117.log.gz 2024-09-17T05:11:19.871 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36818.log 2024-09-17T05:11:19.872 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38265.log 2024-09-17T05:11:19.872 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39117.log.gz 2024-09-17T05:11:19.874 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38648.log 2024-09-17T05:11:19.874 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36818.log.gz 2024-09-17T05:11:19.875 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39214.log 2024-09-17T05:11:19.876 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38265.log.gz 2024-09-17T05:11:19.876 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49044.log 2024-09-17T05:11:19.876 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38648.log.gz 2024-09-17T05:11:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39214.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39214.log.gz 2024-09-17T05:11:19.877 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47417.log 2024-09-17T05:11:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46122.log 2024-09-17T05:11:19.878 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49044.log.gz 2024-09-17T05:11:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35368.log 2024-09-17T05:11:19.879 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47417.log.gz 2024-09-17T05:11:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-09-17T05:11:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.46122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46122.log.gz 2024-09-17T05:11:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39435.log 2024-09-17T05:11:19.881 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35368.log.gz 2024-09-17T05:11:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51668.log 2024-09-17T05:11:19.882 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-09-17T05:11:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34456.log 2024-09-17T05:11:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39435.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39435.log.gz 2024-09-17T05:11:19.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38084.log 2024-09-17T05:11:19.884 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51668.log.gz 2024-09-17T05:11:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34456.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34456.log.gz 2024-09-17T05:11:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34775.log 2024-09-17T05:11:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-09-17T05:11:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38084.log: /var/log/ceph/ceph-client.admin.34775.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38084.log.gz 2024-09-17T05:11:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51250.log 2024-09-17T05:11:19.887 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34775.log.gz 2024-09-17T05:11:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43729.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-09-17T05:11:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50697.log 2024-09-17T05:11:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.51250.log: /var/log/ceph/ceph-client.admin.36352.log 2024-09-17T05:11:19.890 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51250.log.gz 2024-09-17T05:11:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-09-17T05:11:19.890 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50697.log.gz 2024-09-17T05:11:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37589.log 2024-09-17T05:11:19.891 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36352.log.gz 2024-09-17T05:11:19.893 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47230.log 2024-09-17T05:11:19.893 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-09-17T05:11:19.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34989.log 2024-09-17T05:11:19.894 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37589.log.gz 2024-09-17T05:11:19.895 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36248.log 2024-09-17T05:11:19.895 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47230.log.gz 2024-09-17T05:11:19.896 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38845.log 2024-09-17T05:11:19.896 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34989.log.gz 2024-09-17T05:11:19.897 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-09-17T05:11:19.898 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36248.log.gz 2024-09-17T05:11:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38845.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38845.log.gz 2024-09-17T05:11:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29582.log 2024-09-17T05:11:19.898 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34326.log 2024-09-17T05:11:19.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43703.log: /var/log/ceph/ceph-client.admin.29582.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-09-17T05:11:19.900 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49624.log 2024-09-17T05:11:19.900 INFO:teuthology.orchestra.run.smithi016.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29582.log.gz 2024-09-17T05:11:19.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47677.log 2024-09-17T05:11:19.901 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34326.log.gz 2024-09-17T05:11:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51720.log 2024-09-17T05:11:19.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49624.log: /var/log/ceph/ceph-client.0.47677.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49624.log.gz 2024-09-17T05:11:19.902 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47677.log.gz 2024-09-17T05:11:19.903 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41267.log 2024-09-17T05:11:19.904 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51720.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51720.log.gz 2024-09-17T05:11:19.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42637.log 2024-09-17T05:11:19.906 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43651.log 2024-09-17T05:11:19.906 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41267.log.gz 2024-09-17T05:11:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.42637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42637.log.gz 2024-09-17T05:11:19.907 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49831.log 2024-09-17T05:11:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47874.log 2024-09-17T05:11:19.908 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43651.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43651.log.gz 2024-09-17T05:11:19.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48258.log 2024-09-17T05:11:19.909 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49831.log.gz 2024-09-17T05:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49650.log 2024-09-17T05:11:19.910 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47874.log.gz 2024-09-17T05:11:19.911 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48439.log 2024-09-17T05:11:19.911 INFO:teuthology.orchestra.run.smithi016.stderr: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.48258.log.gz 2024-09-17T05:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44938.log 2024-09-17T05:11:19.912 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49650.log.gz 2024-09-17T05:11:19.913 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35000.log 2024-09-17T05:11:19.913 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48439.log.gz 2024-09-17T05:11:19.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34791.log 2024-09-17T05:11:19.914 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44938.log.gz 2024-09-17T05:11:19.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39896.log 2024-09-17T05:11:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35000.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35000.log.gz 2024-09-17T05:11:19.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2024-09-17T05:11:19.916 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-09-17T05:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40050.log 2024-09-17T05:11:19.917 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39896.log.gz 2024-09-17T05:11:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.50397.log: --verbose -- /var/log/ceph/ceph-client.admin.46070.log 2024-09-17T05:11:19.919 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2024-09-17T05:11:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40050.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T05:11:19.920 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40050.log.gz 2024-09-17T05:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37419.log 2024-09-17T05:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46070.log.gz 2024-09-17T05:11:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39292.log 2024-09-17T05:11:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-09-17T05:11:19.923 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37419.log.gz 2024-09-17T05:11:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39292.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.39292.log.gz --verbose 2024-09-17T05:11:19.924 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.44361.log 2024-09-17T05:11:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46969.log 2024-09-17T05:11:19.926 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-09-17T05:11:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39661.log 2024-09-17T05:11:19.926 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44361.log.gz 2024-09-17T05:11:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51616.log 2024-09-17T05:11:19.927 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46969.log.gz 2024-09-17T05:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34430.log 2024-09-17T05:11:19.928 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39661.log.gz 2024-09-17T05:11:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34572.log 2024-09-17T05:11:19.929 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51616.log.gz 2024-09-17T05:11:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-09-17T05:11:19.930 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34430.log.gz 2024-09-17T05:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.34572.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-09-17T05:11:19.931 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34572.log.gz 2024-09-17T05:11:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40625.log 2024-09-17T05:11:19.932 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-09-17T05:11:19.933 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36274.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-09-17T05:11:19.933 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.38830.log 2024-09-17T05:11:19.934 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29074.log 2024-09-17T05:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40625.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40625.log.gz 2024-09-17T05:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.38830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38830.log.gz 2024-09-17T05:11:19.936 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2024-09-17T05:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40365.log 0.0% 2024-09-17T05:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29074.log.gz 2024-09-17T05:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33665.log 2024-09-17T05:11:19.937 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2024-09-17T05:11:19.938 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5/var/log/ceph/ceph-client.admin.40365.log: --verbose -- /var/log/ceph/ceph-client.admin.50918.log 2024-09-17T05:11:19.939 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40365.log.gz 2024-09-17T05:11:19.940 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28890.log 2024-09-17T05:11:19.940 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33665.log.gz 2024-09-17T05:11:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37351.log 2024-09-17T05:11:19.941 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50918.log.gz 2024-09-17T05:11:19.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2024-09-17T05:11:19.942 INFO:teuthology.orchestra.run.smithi016.stderr: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.28890.log.gz 2024-09-17T05:11:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.37351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37351.log.gz 2024-09-17T05:11:19.943 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-09-17T05:11:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41046.log 2024-09-17T05:11:19.944 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2024-09-17T05:11:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35889.log 2024-09-17T05:11:19.945 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-09-17T05:11:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44703.log 2024-09-17T05:11:19.946 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41046.log.gz 2024-09-17T05:11:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-09-17T05:11:19.947 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35889.log.gz 2024-09-17T05:11:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33955.log 2024-09-17T05:11:19.949 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44703.log.gz 2024-09-17T05:11:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42348.log 2024-09-17T05:11:19.950 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-09-17T05:11:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38961.log 2024-09-17T05:11:19.951 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.33955.log.gz 2024-09-17T05:11:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47692.log 2024-09-17T05:11:19.952 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42348.log.gz 2024-09-17T05:11:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36195.log 2024-09-17T05:11:19.953 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38961.log.gz 2024-09-17T05:11:19.954 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44831.log 2024-09-17T05:11:19.954 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47692.log.gz 2024-09-17T05:11:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36195.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36195.log.gz 2024-09-17T05:11:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47906.log 2024-09-17T05:11:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41150.log 2024-09-17T05:11:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.44831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44831.log.gz 2024-09-17T05:11:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-09-17T05:11:19.957 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47906.log.gz 2024-09-17T05:11:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29495.log 2024-09-17T05:11:19.959 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41150.log.gz 2024-09-17T05:11:19.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37067.log 2024-09-17T05:11:19.959 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gz 2024-09-17T05:11:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38323.log 2024-09-17T05:11:19.961 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29495.log.gz 2024-09-17T05:11:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41757.log 2024-09-17T05:11:19.962 INFO:teuthology.orchestra.run.smithi016.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37067.log.gz 2024-09-17T05:11:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46891.log 2024-09-17T05:11:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38323.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38323.log.gz 2024-09-17T05:11:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37751.log 2024-09-17T05:11:19.964 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41757.log.gz 2024-09-17T05:11:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-09-17T05:11:19.965 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46891.log.gz 2024-09-17T05:11:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2024-09-17T05:11:19.966 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37751.log.gz 2024-09-17T05:11:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39948.log 2024-09-17T05:11:19.967 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-09-17T05:11:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37299.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2024-09-17T05:11:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44729.log 2024-09-17T05:11:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50814.log 2024-09-17T05:11:19.969 INFO:teuthology.orchestra.run.smithi016.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.39948.log.gz 2024-09-17T05:11:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2024-09-17T05:11:19.971 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2024-09-17T05:11:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50814.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T05:11:19.971 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50814.log.gz 2024-09-17T05:11:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log/var/log/ceph/ceph-client.admin.41705.log: 2024-09-17T05:11:19.973 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2024-09-17T05:11:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41352.log 2024-09-17T05:11:19.975 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29058.log 2024-09-17T05:11:19.975 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-09-17T05:11:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49676.log 2024-09-17T05:11:19.976 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41352.log.gz 2024-09-17T05:11:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29058.log.gz 2024-09-17T05:11:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49676.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49676.log.gz 2024-09-17T05:11:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51015.log 2024-09-17T05:11:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44033.log 2024-09-17T05:11:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50579.log 2024-09-17T05:11:19.981 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51015.log.gz 2024-09-17T05:11:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44033.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44033.log.gz 2024-09-17T05:11:19.983 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-09-17T05:11:19.984 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48011.log 2024-09-17T05:11:19.985 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50579.log.gz 2024-09-17T05:11:19.986 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40690.log 2024-09-17T05:11:19.986 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-09-17T05:11:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48011.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48011.log.gz 2024-09-17T05:11:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-09-17T05:11:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40690.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40690.log.gz 2024-09-17T05:11:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46189.log 2024-09-17T05:11:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43081.log 2024-09-17T05:11:19.990 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-09-17T05:11:19.991 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46189.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46189.log.gz 2024-09-17T05:11:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43081.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43081.log.gz 2024-09-17T05:11:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2024-09-17T05:11:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36456.log 2024-09-17T05:11:19.995 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41124.log 2024-09-17T05:11:19.995 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2024-09-17T05:11:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38987.log 2024-09-17T05:11:19.997 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36456.log.gz 2024-09-17T05:11:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41124.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41124.log.gz 2024-09-17T05:11:19.998 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34134.log 2024-09-17T05:11:19.999 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38987.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38987.log.gz 2024-09-17T05:11:20.000 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37145.log 2024-09-17T05:11:20.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34134.log: /var/log/ceph/ceph-client.admin.37145.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37145.log.gz 2024-09-17T05:11:20.003 INFO:teuthology.orchestra.run.smithi016.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.34134.log.gz 2024-09-17T05:11:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-09-17T05:11:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47204.log 2024-09-17T05:11:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38384.log 2024-09-17T05:11:20.005 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-09-17T05:11:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47204.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47204.log.gz 2024-09-17T05:11:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-09-17T05:11:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38384.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38384.log.gz 2024-09-17T05:11:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50970.log 2024-09-17T05:11:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42509.log 2024-09-17T05:11:20.009 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-09-17T05:11:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50970.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50970.log.gz 2024-09-17T05:11:20.011 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42509.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42509.log.gz 2024-09-17T05:11:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-17T05:11:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-09-17T05:11:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45589.log 2024-09-17T05:11:20.014 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-17T05:11:20.016 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51486.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-09-17T05:11:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2024-09-17T05:11:20.017 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45589.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45589.log.gz 2024-09-17T05:11:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49291.log 2024-09-17T05:11:20.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35813.log 2024-09-17T05:11:20.018 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2024-09-17T05:11:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39635.log 2024-09-17T05:11:20.020 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49291.log.gz 2024-09-17T05:11:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.35813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.35813.log.gz 2024-09-17T05:11:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45734.log 2024-09-17T05:11:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46020.log 2024-09-17T05:11:20.022 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39635.log.gz 2024-09-17T05:11:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2024-09-17T05:11:20.023 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45734.log.gz 2024-09-17T05:11:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46020.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46020.log.gz 2024-09-17T05:11:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35342.log 2024-09-17T05:11:20.025 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2024-09-17T05:11:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44061.log 2024-09-17T05:11:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40916.log 2024-09-17T05:11:20.028 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35342.log.gz 2024-09-17T05:11:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44061.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44061.log.gz 2024-09-17T05:11:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44847.log 2024-09-17T05:11:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40916.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40916.log.gz 2024-09-17T05:11:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43567.log 2024-09-17T05:11:20.033 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T05:11:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45296.log 2024-09-17T05:11:20.034 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44847.log.gz 2024-09-17T05:11:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33903.log 2024-09-17T05:11:20.034 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43567.log.gz 2024-09-17T05:11:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45296.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45296.log.gz 2024-09-17T05:11:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24003.log 2024-09-17T05:11:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33903.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33903.log.gz 2024-09-17T05:11:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47770.log 2024-09-17T05:11:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.24003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34352.log 2024-09-17T05:11:20.037 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24003.log.gz 2024-09-17T05:11:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-09-17T05:11:20.038 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47770.log.gz 2024-09-17T05:11:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43215.log 2024-09-17T05:11:20.039 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34352.log.gz 2024-09-17T05:11:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39403.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-09-17T05:11:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50367.log 2024-09-17T05:11:20.042 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43215.log.gz 2024-09-17T05:11:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2024-09-17T05:11:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41247.log 2024-09-17T05:11:20.044 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50367.log.gz 2024-09-17T05:11:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49239.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2024-09-17T05:11:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.41247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.41247.log.gz 2024-09-17T05:11:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39844.log 2024-09-17T05:11:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-09-17T05:11:20.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45968.log 2024-09-17T05:11:20.049 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39844.log.gz 2024-09-17T05:11:20.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42457.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-09-17T05:11:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45968.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45968.log.gz 2024-09-17T05:11:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-09-17T05:11:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42632.log 2024-09-17T05:11:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48309.log 2024-09-17T05:11:20.054 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-09-17T05:11:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.42632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42632.log.gz 2024-09-17T05:11:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48309.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48309.log.gz 2024-09-17T05:11:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49369.log 2024-09-17T05:11:20.058 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47080.log 2024-09-17T05:11:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50730.log 2024-09-17T05:11:20.059 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49369.log.gz 2024-09-17T05:11:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34482.log 2024-09-17T05:11:20.061 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47080.log.gz 2024-09-17T05:11:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50730.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50730.log.gz 2024-09-17T05:11:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33851.log 2024-09-17T05:11:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34482.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34482.log.gz 2024-09-17T05:11:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40942.log 2024-09-17T05:11:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-09-17T05:11:20.065 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33851.log.gz 2024-09-17T05:11:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40942.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40942.log.gz 2024-09-17T05:11:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-09-17T05:11:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36300.log 2024-09-17T05:11:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-09-17T05:11:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36300.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T05:11:20.070 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36300.log.gz 2024-09-17T05:11:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37620.log 2024-09-17T05:11:20.071 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-09-17T05:11:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40495.log 2024-09-17T05:11:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37620.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37620.log.gz 2024-09-17T05:11:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40495.log: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T05:11:20.075 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40495.log.gz 2024-09-17T05:11:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37805.log 2024-09-17T05:11:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43781.log 2024-09-17T05:11:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43755.log 2024-09-17T05:11:20.078 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37805.log.gz 2024-09-17T05:11:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43781.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.43781.log.gz 2024-09-17T05:11:20.080 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T05:11:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43755.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43755.log.gz 2024-09-17T05:11:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48044.log 2024-09-17T05:11:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34984.log 2024-09-17T05:11:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51746.log 2024-09-17T05:11:20.083 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48044.log.gz 2024-09-17T05:11:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50553.log 2024-09-17T05:11:20.085 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34984.log.gz 2024-09-17T05:11:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51746.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51746.log.gz 2024-09-17T05:11:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34991.log 2024-09-17T05:11:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50553.log.gz 2024-09-17T05:11:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44599.log 2024-09-17T05:11:20.088 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34991.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34991.log.gz 2024-09-17T05:11:20.088 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-09-17T05:11:20.089 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44599.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44599.log.gz 2024-09-17T05:11:20.090 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49213.log 2024-09-17T05:11:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44007.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-09-17T05:11:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37346.log 2024-09-17T05:11:20.092 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49213.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.49213.log.gz 2024-09-17T05:11:20.092 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35394.log 2024-09-17T05:11:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.37346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37346.log.gz 2024-09-17T05:11:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35394.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35394.log.gz 2024-09-17T05:11:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34274.log 2024-09-17T05:11:20.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46241.log 2024-09-17T05:11:20.097 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36963.log 2024-09-17T05:11:20.098 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34274.log.gz 2024-09-17T05:11:20.099 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46241.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46241.log.gz 2024-09-17T05:11:20.100 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36963.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36963.log.gz 2024-09-17T05:11:20.100 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37041.log 2024-09-17T05:11:20.101 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43903.log 2024-09-17T05:11:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36633.log 2024-09-17T05:11:20.103 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37041.log.gz 2024-09-17T05:11:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48940.log 2024-09-17T05:11:20.104 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43903.log.gz 2024-09-17T05:11:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36633.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36633.log.gz 2024-09-17T05:11:20.106 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36378.log 2024-09-17T05:11:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48940.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48940.log.gz 2024-09-17T05:11:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33719.log 2024-09-17T05:11:20.108 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36378.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36378.log.gzgzip 2024-09-17T05:11:20.108 INFO:teuthology.orchestra.run.smithi016.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42897.log 2024-09-17T05:11:20.109 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33719.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33719.log.gz 2024-09-17T05:11:20.110 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28870.log 2024-09-17T05:11:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42897.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42897.log.gz 2024-09-17T05:11:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42756.log 2024-09-17T05:11:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42127.log 2024-09-17T05:11:20.112 INFO:teuthology.orchestra.run.smithi016.stderr: 74.5% -- replaced with /var/log/ceph/ceph-client.admin.28870.log.gz 2024-09-17T05:11:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48699.log 2024-09-17T05:11:20.113 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42756.log.gz 2024-09-17T05:11:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40000.log 2024-09-17T05:11:20.114 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42127.log.gz 2024-09-17T05:11:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48699.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48699.log.gz 2024-09-17T05:11:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-09-17T05:11:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40000.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40000.log.gz 2024-09-17T05:11:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39974.log 2024-09-17T05:11:20.118 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-09-17T05:11:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-17T05:11:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45218.log 2024-09-17T05:11:20.120 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39974.log.gz 2024-09-17T05:11:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33773.log 2024-09-17T05:11:20.122 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-17T05:11:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45218.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45218.log.gz 2024-09-17T05:11:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36790.log 2024-09-17T05:11:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33773.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33773.log.gz 2024-09-17T05:11:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49540.log 2024-09-17T05:11:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39318.log 2024-09-17T05:11:20.126 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36790.log.gz 2024-09-17T05:11:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49540.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49540.log.gz 2024-09-17T05:11:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39318.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2024-09-17T05:11:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38700.log 2024-09-17T05:11:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-09-17T05:11:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42585.log 2024-09-17T05:11:20.131 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38700.log.gz 2024-09-17T05:11:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50527.log 2024-09-17T05:11:20.132 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-09-17T05:11:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42585.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42585.log.gz 2024-09-17T05:11:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43003.log 2024-09-17T05:11:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50527.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50527.log.gz 2024-09-17T05:11:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39162.log 2024-09-17T05:11:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28986.log 2024-09-17T05:11:20.137 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43003.log.gz 2024-09-17T05:11:20.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39162.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39162.log.gz 2024-09-17T05:11:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28986.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28986.log.gz 2024-09-17T05:11:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38213.log 2024-09-17T05:11:20.140 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45760.log 2024-09-17T05:11:20.142 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-09-17T05:11:20.142 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38213.log.gz 2024-09-17T05:11:20.143 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44573.log 2024-09-17T05:11:20.143 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45760.log.gz 2024-09-17T05:11:20.144 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37673.log 2024-09-17T05:11:20.144 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-09-17T05:11:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41575.log 2024-09-17T05:11:20.145 INFO:teuthology.orchestra.run.smithi016.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44573.log.gz 2024-09-17T05:11:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2024-09-17T05:11:20.147 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37673.log.gz 2024-09-17T05:11:20.148 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41575.log.gz 2024-09-17T05:11:20.149 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50280.log 2024-09-17T05:11:20.149 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2024-09-17T05:11:20.150 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44651.log 2024-09-17T05:11:20.151 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33825.log 2024-09-17T05:11:20.151 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50280.log.gz 2024-09-17T05:11:20.152 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-09-17T05:11:20.153 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44651.log.gz 2024-09-17T05:11:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33825.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33825.log.gz 2024-09-17T05:11:20.154 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38032.log 2024-09-17T05:11:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-09-17T05:11:20.155 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-09-17T05:11:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38032.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38032.log.gz 2024-09-17T05:11:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45244.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz 2024-09-17T05:11:20.157 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28941.log 2024-09-17T05:11:20.159 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40443.log 2024-09-17T05:11:20.160 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-09-17T05:11:20.160 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28941.log.gz 2024-09-17T05:11:20.161 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40443.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40443.log.gz 2024-09-17T05:11:20.162 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35722.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-09-17T05:11:20.163 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50041.log 2024-09-17T05:11:20.164 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42426.log 2024-09-17T05:11:20.165 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48387.log 2024-09-17T05:11:20.165 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50041.log.gz 2024-09-17T05:11:20.166 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42426.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42426.log.gz 2024-09-17T05:11:20.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48387.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48387.log.gz 2024-09-17T05:11:20.168 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33324.log 2024-09-17T05:11:20.169 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41098.log 2024-09-17T05:11:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35564.log 2024-09-17T05:11:20.171 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33324.log.gz 2024-09-17T05:11:20.171 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41098.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41098.log.gz 2024-09-17T05:11:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35564.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35564.log.gz 2024-09-17T05:11:20.173 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37093.log 2024-09-17T05:11:20.174 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43295.log 2024-09-17T05:11:20.176 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-09-17T05:11:20.176 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37093.log.gz 2024-09-17T05:11:20.177 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37954.log 2024-09-17T05:11:20.177 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43295.log.gz 2024-09-17T05:11:20.178 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35200.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-09-17T05:11:20.178 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42214.log 2024-09-17T05:11:20.179 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37954.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37954.log.gz 2024-09-17T05:11:20.179 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39039.log 2024-09-17T05:11:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-09-17T05:11:20.181 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42214.log.gz 2024-09-17T05:11:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39039.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39039.log.gz 2024-09-17T05:11:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48810.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-09-17T05:11:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42652.log 2024-09-17T05:11:20.185 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46717.log 2024-09-17T05:11:20.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37119.log 2024-09-17T05:11:20.186 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42652.log.gz 2024-09-17T05:11:20.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46717.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46717.log.gz 2024-09-17T05:11:20.188 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34792.log 2024-09-17T05:11:20.189 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37119.log.gz 2024-09-17T05:11:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-17T05:11:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35191.log 2024-09-17T05:11:20.191 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34792.log.gz 2024-09-17T05:11:20.192 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2024-09-17T05:11:20.192 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-17T05:11:20.193 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35191.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35191.log.gz 2024-09-17T05:11:20.194 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43373.log 2024-09-17T05:11:20.194 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46382.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz /var/log/ceph/ceph-client.admin.51382.log 2024-09-17T05:11:20.194 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-17T05:11:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38878.log 2024-09-17T05:11:20.196 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43373.log.gz 2024-09-17T05:11:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51382.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51382.log.gz 2024-09-17T05:11:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38878.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38878.log.gz 2024-09-17T05:11:20.198 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2024-09-17T05:11:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36907.log 2024-09-17T05:11:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48465.log 2024-09-17T05:11:20.201 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2024-09-17T05:11:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36907.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36907.log.gz 2024-09-17T05:11:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-09-17T05:11:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48465.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48465.log.gz 2024-09-17T05:11:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44880.log 2024-09-17T05:11:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36071.log 2024-09-17T05:11:20.205 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-09-17T05:11:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44880.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44880.log.gz 2024-09-17T05:11:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36071.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36071.log.gz 2024-09-17T05:11:20.208 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35160.log 2024-09-17T05:11:20.209 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41413.log 2024-09-17T05:11:20.211 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41523.log 2024-09-17T05:11:20.211 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35160.log.gz 2024-09-17T05:11:20.212 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39091.log 2024-09-17T05:11:20.212 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41413.log.gz 2024-09-17T05:11:20.213 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41523.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41523.log.gz 2024-09-17T05:11:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40890.log 2024-09-17T05:11:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39091.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39091.log.gz 2024-09-17T05:11:20.214 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39013.log 2024-09-17T05:11:20.216 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34534.log 2024-09-17T05:11:20.216 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40890.log.gz 2024-09-17T05:11:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39013.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39013.log.gz 2024-09-17T05:11:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34534.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34534.log.gz 2024-09-17T05:11:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-09-17T05:11:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36482.log 2024-09-17T05:11:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43677.log 2024-09-17T05:11:20.222 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2024-09-17T05:11:20.222 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36482.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36482.log.gz 2024-09-17T05:11:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43677.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43677.log.gz 2024-09-17T05:11:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T05:11:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39370.log 2024-09-17T05:11:20.228 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34782.log 2024-09-17T05:11:20.228 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39370.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39370.log.gz 2024-09-17T05:11:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34782.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34782.log.gz 2024-09-17T05:11:20.229 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T05:11:20.230 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T05:11:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44387.log 2024-09-17T05:11:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45890.log 2024-09-17T05:11:20.233 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44387.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44387.log.gz 2024-09-17T05:11:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45890.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45890.log.gz 2024-09-17T05:11:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34570.log 2024-09-17T05:11:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45557.log 2024-09-17T05:11:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34570.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34570.log.gz 2024-09-17T05:11:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45557.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45557.log.gz 2024-09-17T05:11:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T05:11:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36165.log 2024-09-17T05:11:20.244 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.36165.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36165.log.gz 2024-09-17T05:11:20.244 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50788.log 2024-09-17T05:11:20.245 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43426.log 2024-09-17T05:11:20.246 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50788.log.gz 2024-09-17T05:11:20.248 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36430.log 2024-09-17T05:11:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36430.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36430.log.gz 2024-09-17T05:11:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-09-17T05:11:20.253 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43426.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29086.log 2024-09-17T05:11:20.253 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-17T05:11:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35941.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-09-17T05:11:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29086.log.gz 2024-09-17T05:11:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2024-09-17T05:11:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51590.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2024-09-17T05:11:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-09-17T05:11:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46412.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-09-17T05:11:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38239.log 2024-09-17T05:11:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33745.log 2024-09-17T05:11:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38239.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38239.log.gz 2024-09-17T05:11:20.266 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47575.log 2024-09-17T05:11:20.266 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33745.log.gz 2024-09-17T05:11:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47575.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47575.log.gz 2024-09-17T05:11:20.270 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37699.log 2024-09-17T05:11:20.273 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37699.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37699.log.gz 2024-09-17T05:11:20.273 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28868.log 2024-09-17T05:11:20.275 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28868.log.gz 2024-09-17T05:11:20.275 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33693.log 2024-09-17T05:11:20.276 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43877.log 2024-09-17T05:11:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33693.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33693.log.gz 2024-09-17T05:11:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43877.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43877.log.gz 2024-09-17T05:11:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28917.log 2024-09-17T05:11:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28917.log.gz 2024-09-17T05:11:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44521.log 2024-09-17T05:11:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44521.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44521.log.gz 2024-09-17T05:11:20.287 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49187.log 2024-09-17T05:11:20.292 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49187.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49187.log.gz 2024-09-17T05:11:20.292 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-09-17T05:11:20.292 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46440.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-09-17T05:11:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45120.log 2024-09-17T05:11:20.295 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42018.log 2024-09-17T05:11:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46613.log 2024-09-17T05:11:20.297 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45120.log.gz 2024-09-17T05:11:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42018.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42018.log.gz 2024-09-17T05:11:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47672.log 2024-09-17T05:11:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46613.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46613.log.gz 2024-09-17T05:11:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.47672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47672.log.gz 2024-09-17T05:11:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50944.log 2024-09-17T05:11:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50944.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50944.log.gz 2024-09-17T05:11:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34586.log 2024-09-17T05:11:20.308 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-09-17T05:11:20.309 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34586.log.gz 2024-09-17T05:11:20.311 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45191.log 2024-09-17T05:11:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48673.log: gzip -5 87.5% --verbose -- /var/log/ceph/ceph-client.admin.28998.log 2024-09-17T05:11:20.314 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45191.log: -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-09-17T05:11:20.315 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45191.log.gz 2024-09-17T05:11:20.317 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2024-09-17T05:11:20.318 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28998.log.gz 2024-09-17T05:11:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33799.log 2024-09-17T05:11:20.320 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49728.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2024-09-17T05:11:20.321 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43347.log 2024-09-17T05:11:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33799.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33799.log.gz 2024-09-17T05:11:20.324 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43347.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43347.log.gz 2024-09-17T05:11:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48154.log 2024-09-17T05:11:20.327 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-09-17T05:11:20.329 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47287.log 2024-09-17T05:11:20.329 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48154.log.gz 2024-09-17T05:11:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48836.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-09-17T05:11:20.331 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48754.log 2024-09-17T05:11:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47287.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47287.log.gz 2024-09-17T05:11:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48754.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48754.log.gz 2024-09-17T05:11:20.336 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-09-17T05:11:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37831.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-09-17T05:11:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34508.log 2024-09-17T05:11:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2024-09-17T05:11:20.341 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47959.log 2024-09-17T05:11:20.341 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34508.log.gz 2024-09-17T05:11:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45942.log: /var/log/ceph/ceph-client.admin.47959.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47959.log.gz 2024-09-17T05:11:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38570.log 2024-09-17T05:11:20.349 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38570.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38570.log.gz 2024-09-17T05:11:20.349 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45704.log 2024-09-17T05:11:20.350 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2024-09-17T05:11:20.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34776.log 2024-09-17T05:11:20.353 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45704.log.gz 2024-09-17T05:11:20.356 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40780.log 2024-09-17T05:11:20.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40780.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40780.log.gz 2024-09-17T05:11:20.359 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34776.log.gz 2024-09-17T05:11:20.360 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50761.log 2024-09-17T05:11:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50761.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50761.log.gz 2024-09-17T05:11:20.363 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45068.log 2024-09-17T05:11:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45068.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45068.log.gz 2024-09-17T05:11:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47718.log 2024-09-17T05:11:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47718.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47718.log.gz 2024-09-17T05:11:20.369 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41444.log 2024-09-17T05:11:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41444.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41444.log.gz 2024-09-17T05:11:20.372 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34584.log 2024-09-17T05:11:20.376 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33486.log 2024-09-17T05:11:20.376 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34584.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34584.log.gz 2024-09-17T05:11:20.376 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45270.log 2024-09-17T05:11:20.377 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33486.log.gz 2024-09-17T05:11:20.378 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46163.log 2024-09-17T05:11:20.380 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35446.log 2024-09-17T05:11:20.381 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46163.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46163.log.gz 2024-09-17T05:11:20.382 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45270.log.gz 2024-09-17T05:11:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45094.log 2024-09-17T05:11:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35446.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35446.log.gz 2024-09-17T05:11:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45094.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2024-09-17T05:11:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42808.log 2024-09-17T05:11:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42808.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42808.log.gz 2024-09-17T05:11:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49857.log 2024-09-17T05:11:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49857.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49857.log.gz 2024-09-17T05:11:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34995.log 2024-09-17T05:11:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34995.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34995.log.gz 2024-09-17T05:11:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47848.log 2024-09-17T05:11:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40994.log 2024-09-17T05:11:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47848.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47848.log.gz 2024-09-17T05:11:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40994.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40994.log.gz 2024-09-17T05:11:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-09-17T05:11:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43188.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-09-17T05:11:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49480.log 2024-09-17T05:11:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49480.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49480.log.gz 2024-09-17T05:11:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51564.log 2024-09-17T05:11:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51564.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51564.log.gz 2024-09-17T05:11:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49571.log 2024-09-17T05:11:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44625.log 2024-09-17T05:11:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49571.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49571.log.gz 2024-09-17T05:11:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44625.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44625.log.gz 2024-09-17T05:11:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-09-17T05:11:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45016.log 2024-09-17T05:11:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44964.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz 2024-09-17T05:11:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37647.log 2024-09-17T05:11:20.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45016.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45016.log.gz 2024-09-17T05:11:20.425 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-09-17T05:11:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37647.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37647.log.gz 2024-09-17T05:11:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43955.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-09-17T05:11:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40417.log 2024-09-17T05:11:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40417.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40417.log.gz 2024-09-17T05:11:20.432 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-09-17T05:11:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37876.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-09-17T05:11:20.435 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47822.log 2024-09-17T05:11:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47822.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47822.log.gz 2024-09-17T05:11:20.438 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47047.log 2024-09-17T05:11:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-09-17T05:11:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47047.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47047.log.gz 2024-09-17T05:11:20.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36404.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-09-17T05:11:20.444 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-17T05:11:20.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36404.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36404.log.gz 2024-09-17T05:11:20.445 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2024-09-17T05:11:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46587.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2024-09-17T05:11:20.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46215.log 2024-09-17T05:11:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46215.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46215.log.gz 2024-09-17T05:11:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-09-17T05:11:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47021.log 2024-09-17T05:11:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40117.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-09-17T05:11:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47021.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47021.log.gz 2024-09-17T05:11:20.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-09-17T05:11:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45350.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-09-17T05:11:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40287.log 2024-09-17T05:11:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40287.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40287.log.gz 2024-09-17T05:11:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48543.log 2024-09-17T05:11:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48543.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48543.log.gz 2024-09-17T05:11:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46319.log 2024-09-17T05:11:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46319.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46319.log.gz 2024-09-17T05:11:20.469 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47521.log 2024-09-17T05:11:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37502.log 2024-09-17T05:11:20.472 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47521.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47521.log.gz 2024-09-17T05:11:20.474 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37502.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37502.log.gz 2024-09-17T05:11:20.475 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-09-17T05:11:20.477 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51512.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-09-17T05:11:20.478 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38492.log 2024-09-17T05:11:20.480 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44910.log 2024-09-17T05:11:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38492.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38492.log.gz 2024-09-17T05:11:20.481 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44152.log 2024-09-17T05:11:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44910.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44910.log.gz 2024-09-17T05:11:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44152.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44152.log.gz 2024-09-17T05:11:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29442.log 2024-09-17T05:11:20.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29442.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29442.log.gz 2024-09-17T05:11:20.488 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48232.log 2024-09-17T05:11:20.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48232.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48232.log.gz 2024-09-17T05:11:20.490 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-09-17T05:11:20.491 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42730.log 2024-09-17T05:11:20.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42871.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-09-17T05:11:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42730.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42730.log.gz 2024-09-17T05:11:20.496 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47313.log 2024-09-17T05:11:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47313.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47313.log.gz 2024-09-17T05:11:20.499 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48725.log 2024-09-17T05:11:20.500 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46529.log 2024-09-17T05:11:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48725.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48725.log.gz 2024-09-17T05:11:20.505 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-17T05:11:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2024-09-17T05:11:20.506 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46529.log.gz 2024-09-17T05:11:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28973.log 2024-09-17T05:11:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50423.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2024-09-17T05:11:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28973.log.gz 2024-09-17T05:11:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2024-09-17T05:11:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38058.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2024-09-17T05:11:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-09-17T05:11:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50449.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-09-17T05:11:20.517 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-09-17T05:11:20.518 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2024-09-17T05:11:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35837.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-09-17T05:11:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46794.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2024-09-17T05:11:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51093.log 2024-09-17T05:11:20.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49883.log 2024-09-17T05:11:20.526 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51093.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51093.log.gz 2024-09-17T05:11:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45165.log 2024-09-17T05:11:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49883.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49883.log.gz 2024-09-17T05:11:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41072.log 2024-09-17T05:11:20.530 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45165.log.gz 2024-09-17T05:11:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41072.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41072.log.gz 2024-09-17T05:11:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36846.log 2024-09-17T05:11:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36846.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36846.log.gz 2024-09-17T05:11:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44120.log 2024-09-17T05:11:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44120.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44120.log.gz 2024-09-17T05:11:20.539 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41952.log 2024-09-17T05:11:20.544 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38935.log 2024-09-17T05:11:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41952.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41952.log.gz 2024-09-17T05:11:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38935.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38935.log.gz 2024-09-17T05:11:20.545 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41835.log 2024-09-17T05:11:20.547 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40836.log 2024-09-17T05:11:20.548 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41835.log.gz 2024-09-17T05:11:20.550 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-09-17T05:11:20.551 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40836.log.gz 2024-09-17T05:11:20.553 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42296.log 2024-09-17T05:11:20.554 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-09-17T05:11:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39713.log 2024-09-17T05:11:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42296.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42296.log.gz 2024-09-17T05:11:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38622.log 2024-09-17T05:11:20.562 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36019.log 2024-09-17T05:11:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38622.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38622.log.gz 2024-09-17T05:11:20.564 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz 2024-09-17T05:11:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36019.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36019.log.gz 2024-09-17T05:11:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38117.log 2024-09-17T05:11:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38117.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38117.log.gz 2024-09-17T05:11:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46351.log 2024-09-17T05:11:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46351.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46351.log.gz 2024-09-17T05:11:20.573 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34378.log 2024-09-17T05:11:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34378.log.gz 2024-09-17T05:11:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-09-17T05:11:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51356.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-09-17T05:11:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38295.log 2024-09-17T05:11:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38295.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38295.log.gz 2024-09-17T05:11:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44469.log 2024-09-17T05:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38518.log 2024-09-17T05:11:20.585 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44469.log.gz 2024-09-17T05:11:20.586 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-09-17T05:11:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38518.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38518.log.gz 2024-09-17T05:11:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42244.log 2024-09-17T05:11:20.591 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-09-17T05:11:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42244.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42244.log.gz 2024-09-17T05:11:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49401.log 2024-09-17T05:11:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46137.log 2024-09-17T05:11:20.597 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49401.log.gz 2024-09-17T05:11:20.598 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-09-17T05:11:20.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46137.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46137.log.gz 2024-09-17T05:11:20.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49018.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-09-17T05:11:20.602 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44826.log 2024-09-17T05:11:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.44826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44826.log.gz 2024-09-17T05:11:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50315.log 2024-09-17T05:11:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50315.log.gz 2024-09-17T05:11:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41380.log 2024-09-17T05:11:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41380.log.gz 2024-09-17T05:11:20.611 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44779.log 2024-09-17T05:11:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44779.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44779.log.gz 2024-09-17T05:11:20.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36045.log 2024-09-17T05:11:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34576.log 2024-09-17T05:11:20.617 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36045.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36045.log.gz 2024-09-17T05:11:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.34576.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34576.log.gz 2024-09-17T05:11:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39817.log 2024-09-17T05:11:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.39817.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39817.log.gz 2024-09-17T05:11:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37779.log 2024-09-17T05:11:20.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37779.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37779.log.gz 2024-09-17T05:11:20.626 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46768.log 2024-09-17T05:11:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-09-17T05:11:20.632 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46768.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46768.log.gz 2024-09-17T05:11:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49070.log: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50341.log 2024-09-17T05:11:20.633 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-09-17T05:11:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41497.log 2024-09-17T05:11:20.637 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50341.log.gz 2024-09-17T05:11:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41497.log: -- /var/log/ceph/ceph-client.admin.40235.log 2024-09-17T05:11:20.641 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41497.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49317.log 2024-09-17T05:11:20.641 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-17T05:11:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40235.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40235.log.gz 2024-09-17T05:11:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-09-17T05:11:20.645 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49317.log.gz 2024-09-17T05:11:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log/var/log/ceph/ceph-client.admin.45376.log: 2024-09-17T05:11:20.648 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-09-17T05:11:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48075.log 2024-09-17T05:11:20.649 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2024-09-17T05:11:20.650 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2024-09-17T05:11:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48075.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48075.log.gz 2024-09-17T05:11:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47469.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2024-09-17T05:11:20.656 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42374.log 2024-09-17T05:11:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42374.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42374.log.gz 2024-09-17T05:11:20.659 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-09-17T05:11:20.661 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37015.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-09-17T05:11:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42044.log 2024-09-17T05:11:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50223.log 2024-09-17T05:11:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42044.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42044.log.gz 2024-09-17T05:11:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50223.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50223.log.gz 2024-09-17T05:11:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50015.log 2024-09-17T05:11:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50015.log.gz 2024-09-17T05:11:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41293.log 2024-09-17T05:11:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41293.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41293.log.gz 2024-09-17T05:11:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38596.log 2024-09-17T05:11:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38596.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38596.log.gz 2024-09-17T05:11:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2024-09-17T05:11:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45838.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2024-09-17T05:11:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29105.log 2024-09-17T05:11:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.29105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29105.log.gz 2024-09-17T05:11:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46865.log 2024-09-17T05:11:20.685 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35698.log 2024-09-17T05:11:20.685 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46865.log: /var/log/ceph/ceph-client.0.35698.log: /var/log/ceph/ceph-client.admin.45441.log 2024-09-17T05:11:20.692 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35698.log.gz 2024-09-17T05:11:20.692 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46865.log.gz 2024-09-17T05:11:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45441.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45441.log.gz 2024-09-17T05:11:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38544.log 2024-09-17T05:11:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.38544.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38544.log.gz 2024-09-17T05:11:20.692 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36881.log 2024-09-17T05:11:20.694 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36881.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36881.log.gz 2024-09-17T05:11:20.695 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33519.log 2024-09-17T05:11:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.33519.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33519.log.gz 2024-09-17T05:11:20.698 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46560.log 2024-09-17T05:11:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49779.log 2024-09-17T05:11:20.701 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46560.log.gz 2024-09-17T05:11:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40968.log 2024-09-17T05:11:20.705 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50197.log 2024-09-17T05:11:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40968.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49779.log.gz 2024-09-17T05:11:20.707 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40968.log.gz 2024-09-17T05:11:20.709 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46917.log 2024-09-17T05:11:20.710 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50197.log.gz 2024-09-17T05:11:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28894.log 2024-09-17T05:11:20.712 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46917.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46917.log.gz 2024-09-17T05:11:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36105.log 2024-09-17T05:11:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.28894.log: 0.0%/var/log/ceph/ceph-client.admin.36105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51330.log 2024-09-17T05:11:20.717 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36105.log.gz 2024-09-17T05:11:20.720 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28894.log.gz 2024-09-17T05:11:20.721 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35316.log 2024-09-17T05:11:20.723 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.51330.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51330.log.gz 2024-09-17T05:11:20.726 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.35316.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35316.log.gz 2024-09-17T05:11:20.738 INFO:teuthology.orchestra.run.smithi016.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T05:11:21.097 INFO:teuthology.orchestra.run.smithi104.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T05:11:21.179 INFO:teuthology.orchestra.run.smithi016.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T05:11:21.195 INFO:teuthology.orchestra.run.smithi104.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T05:11:21.197 INFO:teuthology.orchestra.run.smithi016.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T05:11:21.276 INFO:teuthology.orchestra.run.smithi104.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T05:11:21.486 INFO:teuthology.orchestra.run.smithi104.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T05:11:21.495 INFO:teuthology.orchestra.run.smithi104.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T05:11:21.497 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-09-17T05:11:21.497 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m1.871s 2024-09-17T05:11:21.498 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m8.487s 2024-09-17T05:11:21.498 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m0.774s 2024-09-17T05:11:21.524 INFO:teuthology.orchestra.run.smithi016.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T05:11:21.769 INFO:teuthology.orchestra.run.smithi016.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T05:11:21.961 INFO:teuthology.orchestra.run.smithi016.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T05:11:22.067 INFO:teuthology.orchestra.run.smithi016.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T05:11:22.069 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-17T05:11:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m2.455s 2024-09-17T05:11:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m10.701s 2024-09-17T05:11:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m1.861s 2024-09-17T05:11:22.070 INFO:tasks.ceph:Archiving logs... 2024-09-17T05:11:22.070 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/remote/smithi016/log 2024-09-17T05:11:22.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T05:11:23.655 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/remote/smithi104/log 2024-09-17T05:11:23.656 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T05:11:24.596 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T05:11:24.605 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T05:11:24.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T05:11:24.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T05:11:24.686 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T05:11:24.694 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T05:11:24.695 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:11:24.698 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T05:11:24.711 INFO:teuthology.orchestra.run.smithi016.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:11:24.711 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================== 2024-09-17T05:11:24.711 INFO:teuthology.orchestra.run.smithi016.stdout:*hv01.front.sepi 67.205.162.81 3 u 62 64 377 0.082 -2.301 1.220 2024-09-17T05:11:24.711 INFO:teuthology.orchestra.run.smithi016.stdout:+hv02.front.sepi 63.231.80.2 3 u 57 64 377 0.108 -3.025 1.216 2024-09-17T05:11:24.711 INFO:teuthology.orchestra.run.smithi016.stdout:+hv03.front.sepi 74.6.168.72 3 u - 64 377 0.083 +1.381 1.378 2024-09-17T05:11:24.712 INFO:teuthology.orchestra.run.smithi016.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-17T05:11:24.713 INFO:teuthology.orchestra.run.smithi104.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T05:11:24.713 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================== 2024-09-17T05:11:24.713 INFO:teuthology.orchestra.run.smithi104.stdout:*hv01.front.sepi 67.205.162.81 3 u 63 64 377 0.090 -2.314 0.241 2024-09-17T05:11:24.713 INFO:teuthology.orchestra.run.smithi104.stdout:+hv02.front.sepi 63.231.80.2 3 u 61 64 377 0.069 -0.853 0.356 2024-09-17T05:11:24.714 INFO:teuthology.orchestra.run.smithi104.stdout:+hv03.front.sepi 74.6.168.72 3 u 59 64 377 0.081 +1.782 0.346 2024-09-17T05:11:24.714 INFO:teuthology.orchestra.run.smithi104.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-17T05:11:24.714 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T05:11:24.722 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T05:11:24.722 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T05:11:24.754 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T05:11:24.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T05:11:24.821 INFO:teuthology.task.internal:Duration was 1366.153268 seconds 2024-09-17T05:11:24.822 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T05:11:24.855 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T05:11:24.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T05:11:24.859 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T05:11:24.941 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T05:11:24.941 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-17T05:11:24.941 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T05:11:24.966 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-09-17T05:11:24.966 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T05:11:24.986 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T05:11:24.986 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T05:11:25.016 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T05:11:25.065 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T05:11:25.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T05:11:25.108 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T05:11:25.213 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T05:11:26.208 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T05:11:26.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T05:11:26.232 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T05:11:26.255 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T05:11:26.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:11:26.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:11:26.729 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-17T05:11:26.732 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-09-17T05:11:26.751 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:11:26.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:11:26.800 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T05:11:26.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T05:11:26.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T05:11:26.817 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T05:11:26.818 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/remote/smithi016 2024-09-17T05:11:26.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T05:11:26.873 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908571/remote/smithi104 2024-09-17T05:11:26.873 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T05:11:26.901 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T05:11:26.902 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T05:11:26.914 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T05:11:26.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T05:11:26.964 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T05:11:26.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T05:11:26.989 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T05:11:26.989 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T05:11:26.992 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T05:11:26.998 INFO:teuthology.orchestra.run.smithi016.stdout: 658143 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 05:11 /home/ubuntu/cephtest 2024-09-17T05:11:27.004 INFO:teuthology.orchestra.run.smithi104.stdout: 655441 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 05:11 /home/ubuntu/cephtest 2024-09-17T05:11:27.008 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T05:11:27.038 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T05:11:27.109 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/client-limits} duration: 1366.1532680988312 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-17T05:11:27.109 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T05:11:27.196 INFO:teuthology.run:pass